Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

If I have a flow running every hour but something causes the flow to take more than an hour, can Prefect delay or cancel an execution so that they are not overlapping?

not afaik, we’ve rolled our own solution. their task or flow run concurrency limit works the opposite way where the new run would be delayed until the existing run exits.

oh maybe that concurrency is what i’m looking for.

is the concurrency limit for a particular task/flow or is it global?

i.e., i can only globally run 1 flow at a time or multiple flows can be running at the same time but only 1 instance of each flow will run at a time?

task level <https://docs.prefect.io/latest/concepts/tasks/#task-run-concurrency-limits> and flow level <https://docs.prefect.io/latest/concepts/work-pools/#work-pool-concurrency>

What is stopping you from using a new work queue?