Hey, <@ULVA73B9P>, I face that when I run self-hos...
# marvin-ai
d
Hey, @Marvin, I face that when I run self-hosted Prefect with
prefect server start --no-services --port 4200
and background services pod with
prefect server services start
. But when we do prefect deploy --all from prefect.yaml, our cronSchedule is duplicated.
m
thought for 995.0 seconds
Thanks for the details — a couple things to unpack here. First, to confirm assumptions: I’m assuming you’re on Prefect 3.x and that you’re running the webserver with
prefect server start --no-services
and a separate background-services pod with
prefect server services start
. If that’s not right, let me know. What usually causes “duplicate cron schedules” after
prefect deploy --all
- If you see multiple schedule entries under a single deployment (e.g., two identical cron rows), it’s typically because the schedules in
prefect.yaml
don’t have stable identifiers. On each deploy, Prefect can’t match the “same” schedule to update it, so it creates a new one. - If you see duplicate flow runs at the same timestamps (not duplicate schedule rows), it’s usually because more than one Scheduler service is running. How to make deploys idempotent for schedules - Give each schedule a stable slug in your
prefect.yaml
. Slugs let Prefect match and update an existing schedule instead of creating a new one on every deploy. Example
prefect.yaml
snippet:
Copy code
deployments:
  - name: my-deployment
    entrypoint: flows/flow.py:my_flow
    schedules:
      - cron: "0 * * * *"
        timezone: "UTC"
        active: true
        slug: hourly-main
- One-time cleanup: clear existing duplicate schedules, then redeploy once with slugs in place:
Copy code
prefect deployment schedule ls "<flow-name>/<deployment-name>"
prefect deployment schedule clear "<flow-name>/<deployment-name>" -y
prefect deploy --all
You can list/clear schedules with the CLI: -
prefect deployment schedule ls
and
prefect deployment schedule clear
are available; see
--help
for usage. - Verified commands: -
prefect deployment schedule --help
-
prefect deployment schedule clear --help
If you are also calling
prefect deployment schedule create
elsewhere (e.g., CI/CD), that will add additional schedules too. Either remove that step and rely solely on
prefect deploy --all
+ slugs, or make sure you clear/replace before creating. You can inspect options with:
Copy code
prefect deployment schedule create --help
About the Scheduler in self-hosted setups - Ensure only one Scheduler service is running. If you accidentally have more than one instance (e.g., scaled your services pod >1 replicas, or also started services in another place), you can get duplicate flow runs at the same timestamp. - Your pattern of
prefect server start --no-services
+ a single
prefect server services start
process is correct; just verify you don’t have multiple replicas of the services pod. Helpful references - Schedules concept and YAML config: Schedules in Prefect 3 - Creating schedules for deployments: Create schedules -
prefect.yaml
deployment config: prefect.yaml reference - Deployment upsert behavior (note that scheduled runs are cleared on upsert): Create deployment API - Scheduler service: Scheduler service A couple quick questions to target the fix: 1) Are you seeing duplicate schedule entries under the deployment, or duplicate flow runs at the same times? 2) Could you share the
schedules:
block from your
prefect.yaml
(with any secrets removed)? 3) How many replicas of your background services pod are running? Just 1? If it’s duplicate schedule entries, adding stable
slug
values and doing a one-time
schedule clear
+ redeploy should resolve it. If it’s duplicate runs, we’ll focus on ensuring only one Scheduler instance is active.