Hello everyone, I have a question about scheduling flow launches. Let's say I expect that the execution of one run will take 2 hours. Using the UI. I set a schedule with a launch every two hours. What happens if for some reason the average execution time of my flow starts taking three hours? Let's say we have a plan:
1. Launch at 12:00
2. Launch at 14:00
3. Launch at 16:00
The launch (1) took three hours. Will the scheduled launch be completed at 14:00 (2)?
If all scheduled launches are completed. Can I set up my flow so that startup (2) is skipped?
k
Kevin Kho
01/06/2022, 2:48 PM
You can on Prefect Cloud with a combination of two things. First is setting a Flow Run Concurrency Limit so that only one of these flows runs at a given time. The second is using an Automation to cancel Flow 2 if it’s late by a certain amount of time.
r
rilshok
01/06/2022, 2:54 PM
@Kevin Kho, thanks for your reply. In the case of the prefect's local server, all launches will be performed according to the schedule, right?
k
Kevin Kho
01/06/2022, 2:58 PM
Server takes more work but a community member gave their code here . You need to use a state handler to hit the GraphQL at the beginning of the Flow to check for concurrent runs. If there are, then cancel the next one.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.