Sunny Shah
08/16/2025, 7:12 AMschedule='@continuous'
Marvin
08/16/2025, 7:12 AMMarvin
08/16/2025, 7:14 AM@continuous
in Prefect, but you can get the same behavior easily.
Two good patterns:
1) Automations (recommended, no code)
- Create an Automation that triggers “Run deployment” when a flow run for your deployment enters the Completed state.
- Result: as soon as a run finishes, the next one is queued immediately.
- To avoid overlap, ensure only one run can execute at a time (use a concurrency limit of 1 for this deployment/tag).
- Docs: Prefect 3 docs and a good overview: Automations and observability
UI steps (brief):
- UI > Automations > New
- Trigger: Flow run state entered = Completed, filtered to your deployment
- Action: Run deployment = the same deployment
- Optional: also trigger on Failed if you want it truly relentless
2) Self-chaining flow (one-liner at the end)
- Have your flow trigger the next run of its deployment as it finishes.
Example:
from prefect import flow
from prefect.deployments import run_deployment
@flow
def my_flow():
# … do work …
# immediately queue the next run of this same deployment
run_deployment(name="my-flow/my-deployment")
- Add a guard (param/flag) if you want a way to stop the chain.
- Same note on avoiding overlap: use a concurrency limit of 1.
Less ideal fallback: a tiny interval schedule plus a concurrency limit of 1. Works, but it’s time-based and can leave gaps or queue bursts.
Quick clarifiers so I don’t cheerfully spin your cluster forever:
- Should it trigger again on failures too, or only on success?
- Do you want strictly one-at-a-time, or can multiple runs overlap?Sunny Shah
08/16/2025, 7:21 AMMarvin
08/16/2025, 7:25 AMMarvin
08/16/2025, 7:25 AMfrom prefect import flow
from prefect.deployments import run_deployment
@flow
def my_flow():
# ...do work...
run_deployment(name="my-flow/my-deployment") # queues next run immediately
Notes:
- Concurrency limit of 1 on the deployment is what guarantees “strictly one-at-a-time,” even if something else tries to trigger runs.
- If you want it to continue only on success, set the Automation trigger to Completed only. If you want it truly relentless, you can also trigger on Failed.
- To stop the loop, disable the Automation (Option A) or remove/comment the run_deployment
call (Option B).
References you can skim while your coffee cools:
- Docs home: Prefect 3 docs
- Overview of Automations: Automations and observability
Want me to paste the exact UI clicks to set the deployment concurrency limit and Automation?