Thomas DeBroux
09/11/2025, 8:22 PMschedules:
- cron: "30-59 8 * * 1-5"
timezone: America/Chicago
- cron: "* 9-14 * * 1-5"
timezone: America/Chicago
- cron: "0-30 15 * * 1-5"
timezone: America/ChicagoMarvin
09/11/2025, 8:22 PMMarvin
09/11/2025, 8:35 PMprefect work-queue create my-queue --limit 1
- Or set a limit on an existing queue:
prefect work-queue set-concurrency-limit <queue-name-or-id> 1
- Assign your deployment to that queue (via the deployment’s work queue setting or in the UI).
2) Use a task-level concurrency “mutex” with tags
- Concurrency limits in 2.x are task-level. Wrap the body of your flow in a single task tagged with a shared lock and set that tag’s limit to 1. Even if two flow runs start, one will wait at the critical section.
from prefect import flow, task
@task(tags=["my-lock"])
def do_work():
# your critical work here
@flow
def my_flow():
do_work()
Then create the limit:
prefect concurrency-limit create my-lock 1
- Note: This doesn’t stop the second flow run from starting, but it will block it at the task boundary so your critical work doesn’t overlap.
- Docs: Global concurrency limits (2.x)
3) Adjust schedule or runtime expectations
- If possible, schedule less frequently (e.g., every 2–5 minutes) or ensure the flow completes in <60s consistently.
- You could also nudge the boundary minutes (e.g., start the afternoon window at 15:01) to avoid transitions, if that fits your needs.
Other considerations
- Timezone and DST: You’re setting America/Chicago, which is good. DST transitions won’t affect 900/1500 most days, but be mindful around clock changes.
- Prefect 2.0.0 is quite old — the scheduler and work-queue behavior improved substantially in later 2.x. If you can upgrade within 2.x (or to 3.x), you’ll benefit from more reliable scheduling and better worker/queue controls.
Relevant docs
- Schedules (2.x)
- Global concurrency limits (2.x)
If you can share:
- Rough average/95th percentile runtime of the flow
- Whether this deployment is alone on its work queue
- How many workers you have polling that queue
…I can recommend a precise setup (queue limits vs. task lock vs. schedule tweak).