<@ULVA73B9P> We use Airflow with git-sync: the sch...
# ask-marvin
p
@Marvin We use Airflow with git-sync: the scheduler/workers always see the latest DAG files from our Git repo after a push—no CI job that re-registers anything. For Prefect 3 (self-hosted server + Kubernetes work pool / worker), is there a built-in way to get the same outcome: new or updated flows in Git are picked up automatically without running
prefect deploy
(or similar) from CI/CD? Specifically: does Prefect support something like continuous sync of deployment definitions from a repo (polling or webhooks), or is the supported pattern always “register deployments via CLI/API/CI” while pull steps only handle flow code at run time?
m
thought for 231.6 seconds
Short answer: Prefect 3 does not auto-discover or auto-register deployments from a Git repo (no git-sync-style polling/webhooks for “new DAGs”). The supported pattern is to explicitly create/update deployments via CLI/API/CI. Workers then pull your flow code at run time. What you can do today - Code-only changes without re-deploy: Use a Git-based storage with pull_interval so workers periodically refresh code from your repo. This gets you “latest code on runs” without running prefect deploy again, as long as the deployment’s entrypoint/branch don’t change. - New flows or config changes: Creating a new deployment, changing schedules, parameters, work pool/infrastructure, etc., still requires an explicit deploy/update via CLI/API/CI. How to set up periodic code pulls (closest to git-sync) - Configure your deployment to pull from Git with a pull interval:
Copy code
from prefect import flow
from prefect.runner.storage import GitRepository

repo = GitRepository(
    url="<https://github.com/org/repo.git>",
    branch="main",
    pull_interval=60  # refresh code every 60s
)

@flow
def my_flow():
    ...

my_flow.from_source(
    source=repo,
    entrypoint="flows/my_flow.py:my_flow",
).deploy(
    name="my-deployment",
    work_pool_name="k8s-pool",
)
- Notes for Kubernetes: - Ensure your worker image has git installed and can auth to your repo (HTTPS token, SSH key, or a Prefect GitCredentials block). - pull_interval defaults to 60s; set None for a one-time pull at run start. What this does and doesn’t cover - Covered: - You push code to main; the worker periodically pulls and new runs use the latest code with no re-deploy. - Not covered: - New flows (new entrypoints) or new deployments are not auto-created. - Changes to deployment metadata (schedule, parameters, infrastructure, etc.) still require running flow.deploy() or prefect deploy in CI/CD. No built-in webhook/polling for auto-registration - There is no Prefect service that scans your repo and auto-registers deployments. - Prefect Cloud’s GitHub App helps with short-lived tokens for secure pulls, but it does not auto-create deployments. If you want “continuous sync” of deployments anyway - Common pattern: run prefect deploy (or call flow.deploy()) from CI on push to main. This keeps deployment definitions explicit and auditable. - You can build a custom controller (a small Prefect flow) that scans a repo and upserts deployments via the API, but this is not built-in. Relevant links - Git-based storage class with pull_interval in source: prefect.runner.storage.GitRepository - Worker pull loop (periodic refresh before runs): _scheduled_run_poller - Deployments overview/docs: Deploy flows (Prefect docs) - CLI for deployments (use prefect deploy; do not use the removed “deployment build”): prefect deploy (docs) Bottom line - For Prefect 3, the official, supported pattern is: - Register deployments via CLI/API/CI - Let workers pull code at run time (and optionally refresh via pull_interval) - There is no built-in, Airflow-style continuous sync that auto-registers new/updated flows from Git.