Janet Carson
06/08/2026, 2:13 AMMarvin
06/08/2026, 2:13 AMMarvin
06/08/2026, 2:15 AMprefect gcl create
- prefect gcl update
- prefect gcl enable
- prefect gcl disable
I verified the CLI exists:
prefect gcl --help
prefect gcl update --help
Example idea:
bash
prefect gcl update backfill-limit --limit 2
Then later:
bash
prefect gcl update backfill-limit --limit 20
If you want to do it in Python, you can update the limit via the Prefect client as well.
Helpful docs:
- Global concurrency limits
- Task run concurrency limits
- Work pools
- Work queues
A couple practical patterns:
Option 1: Scheduled limit changer
- Schedule a small flow every hour
- It checks current local time/day
- It calls Prefect API to set backfill-limit to the right value
Option 2: Separate queues
- Put backfills on their own work queue
- Raise/lower queue concurrency over time, or run more/fewer workers against that queue at night
Option 3: Two deployments
- “daytime backfill” deployment with low concurrency
- “offhours backfill” deployment with high concurrency
- schedule them differently
If you want, I can sketch the cleanest Prefect 3.x implementation for either:
- task-level backfill throttling, or
- flow/deployment-level backfill throttlingJanet Carson
06/09/2026, 5:40 PMMarvin
06/09/2026, 5:40 PMMarvin
06/09/2026, 5:42 PM