<@ULVA73B9P> I'm running prefect self-hosted using docker compose, connected to an external (not par...
n
@Marvin I'm running prefect self-hosted using docker compose, connected to an external (not part of the docker compose environment) postgresql database. The docker compose environment contains a docker container where all deployments are being created in code using
<http://flow_name.to|flow_name.to>_deployment()
and then served using
prefect.flows.serve()
. I am having an issue where deployment Schedules that have been created via the UI are being lost anytime the docker-compose is recreated. I can confirm that there are rows in the
deployment_schedule
table, then I restart the docker-compose environment, and those rows have been automatically deleted. Any idea what might cause this?
m
thought for 300.8 seconds
Yes — the most likely cause is that
serve()
is reconciling/updating the deployment on startup, and in that update it may be effectively treating schedules as “empty”, which can wipe UI-created schedules. What’s happening - Your container starts - Your code calls
flow.to_deployment(...)
and/or
flow.serve()
- Prefect checks whether that deployment already exists - If it exists, Prefect updates it to match the code-defined deployment - If no schedules are defined in code, that update path can result in the deployment’s schedules being cleared So even though the schedules were created in the UI and persisted in Postgres, the next
serve()
startup can overwrite that deployment record and remove them. This lines up very closely with your symptom: - rows exist in
deployment_schedule
- restart compose - startup code runs again - rows disappear Why this is especially likely in your setup Because you are recreating the serving container, your deployment-registration code runs every time. If those deployments are always recreated/updated from code and the code version does not include schedules, UI-managed schedules are not “authoritative”. Important distinction If you want schedules to survive restarts, they should generally be managed in the same source of truth as the deployment definition: - either define schedules in code / deployment config - or avoid re-applying the deployment definition on every container startup With
serve()
, the deployment definition is part of startup behavior, so UI edits can be overwritten. What to do Option 1: Define schedules in code If these are meant to be durable, this is the safest route. Option 2: Stop re-registering/reconciling the deployment on every container restart If possible, separate: - deployment creation/update - long-running serving/execution Option 3: Use
prefect deploy
for deployment management instead of
serve()
for this case
serve()
is convenient for local/dev and simple long-running setups, but if you want the UI-managed deployment object to remain stable across infra restarts, a worker + deployed deployment model is usually a better fit in Prefect 3. If you go the CLI route, the relevant command is `prefect deploy`: Deployments docs Recommended architecture For self-hosted production setups, I’d strongly recommend: - create deployments separately - run workers separately - let schedules belong to the deployment definition That avoids “container startup rewrote my deployment” problems. Docs - Prefect 3 deployments - Create deployments - Run flows with local process work pools Practical check A good way to confirm this is the cause: 1. Create the UI schedule 2. Do not restart compose yet — verify it exists 3. Restart only the container that runs
serve()
4. Watch logs during startup 5. Check whether the deployment is being updated right before the
deployment_schedule
rows disappear If you want, I can help you: - refactor your current
serve()
setup so schedules are preserved, or - convert this docker-compose setup to a worker/work-pool deployment model.