Dmytro Shamenko
11/13/2025, 9:26 AMprefect server start --no-services --port 4200 and background services pod with prefect server services start. But when we do prefect deploy --all from prefect.yaml, our cronSchedule is duplicated.Marvin
11/13/2025, 9:27 AMMarvin
11/13/2025, 9:43 AMprefect server start --no-services and a separate background-services pod with prefect server services start. If that’s not right, let me know.
What usually causes “duplicate cron schedules” after prefect deploy --all
- If you see multiple schedule entries under a single deployment (e.g., two identical cron rows), it’s typically because the schedules in prefect.yaml don’t have stable identifiers. On each deploy, Prefect can’t match the “same” schedule to update it, so it creates a new one.
- If you see duplicate flow runs at the same timestamps (not duplicate schedule rows), it’s usually because more than one Scheduler service is running.
How to make deploys idempotent for schedules
- Give each schedule a stable slug in your prefect.yaml. Slugs let Prefect match and update an existing schedule instead of creating a new one on every deploy.
Example prefect.yaml snippet:
deployments:
- name: my-deployment
entrypoint: flows/flow.py:my_flow
schedules:
- cron: "0 * * * *"
timezone: "UTC"
active: true
slug: hourly-main
- One-time cleanup: clear existing duplicate schedules, then redeploy once with slugs in place:
prefect deployment schedule ls "<flow-name>/<deployment-name>"
prefect deployment schedule clear "<flow-name>/<deployment-name>" -y
prefect deploy --all
You can list/clear schedules with the CLI:
- prefect deployment schedule ls and prefect deployment schedule clear are available; see --help for usage.
- Verified commands:
- prefect deployment schedule --help
- prefect deployment schedule clear --help
If you are also calling prefect deployment schedule create elsewhere (e.g., CI/CD), that will add additional schedules too. Either remove that step and rely solely on prefect deploy --all + slugs, or make sure you clear/replace before creating. You can inspect options with:
prefect deployment schedule create --help
About the Scheduler in self-hosted setups
- Ensure only one Scheduler service is running. If you accidentally have more than one instance (e.g., scaled your services pod >1 replicas, or also started services in another place), you can get duplicate flow runs at the same timestamp.
- Your pattern of prefect server start --no-services + a single prefect server services start process is correct; just verify you don’t have multiple replicas of the services pod.
Helpful references
- Schedules concept and YAML config: Schedules in Prefect 3
- Creating schedules for deployments: Create schedules
- prefect.yaml deployment config: prefect.yaml reference
- Deployment upsert behavior (note that scheduled runs are cleared on upsert): Create deployment API
- Scheduler service: Scheduler service
A couple quick questions to target the fix:
1) Are you seeing duplicate schedule entries under the deployment, or duplicate flow runs at the same times?
2) Could you share the schedules: block from your prefect.yaml (with any secrets removed)?
3) How many replicas of your background services pod are running? Just 1?
If it’s duplicate schedule entries, adding stable slug values and doing a one-time schedule clear + redeploy should resolve it. If it’s duplicate runs, we’ll focus on ensuring only one Scheduler instance is active.