Hey folks,
Quick questions about best practices.
Let's say my team is using a flow to pull data from a vendor API daily at 8am. Typically the data for that day will be available by 8am, but sometimes there's a delay on the vendor's side and our API call gets an empty response. The vendor data might not be available for another 30m - 1hr after the expected 8am pull.
In this case, would it still be appropriate to add a
retry_delay
to the task and have it wait an hour, or would it be better practice to add some sort of handler to reschedule the flow for an hour later? Is the
retry_delay
basically a sleep in the backend, or does it handle scheduling the task?
k
Kevin Kho
04/05/2022, 2:04 PM
In my opinion,
retry_delay
is the easiest solution and people do this to poll for an event happening by giving a high number of
max_retries
.
I think it’s harder to track the retry number if you use a subflow and reschedule. I wonder what would not an infinite loop, whereas the retries will be capped with a max
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.