Viktor
12/28/2025, 12:15 PMMarvin
12/28/2025, 12:15 PMMarvin
12/28/2025, 12:18 PMdry_run: bool flow parameter and gate any side-effect operations behind it.
- Return a preview of “what would happen” when dry_run=True.
- Default dry_run=True in dev deployments; False in prod.
Minimal example
from prefect import flow
def write_to_db(records):
# real write here
...
def preview_write(records):
print(f"DRY RUN: would write {len(records)} records; sample: {records[:3]}")
@flow(log_prints=True)
def etl(env: str = "dev", dry_run: bool = True, batch_size: int = 100):
print(f"env={env} dry_run={dry_run} batch_size={batch_size}")
records = list(range(batch_size)) # imagine we transformed data here
if dry_run:
preview_write(records)
return {"status": "dry_run", "would_write": len(records)}
write_to_db(records)
return {"status": "success", "written": len(records)}
Two deployments of the same flow
# Point to code from a branch/commit for prod; local path or feature branch for dev
flow_dev = etl.from_source(
source=".",
entrypoint="flows/etl.py:etl",
)
flow_dev.deploy(
name="etl-dev",
work_pool_name="dev-pool",
parameters={"env": "dev", "dry_run": True, "batch_size": 50},
tags=["dev"],
# schedules=[...] # optional; often manual in dev
)
flow_prod = etl.from_source(
source="<https://github.com/myorg/myrepo>",
entrypoint="flows/etl.py:etl",
)
flow_prod.deploy(
name="etl-prod",
work_pool_name="prod-k8s",
parameters={"env": "prod", "dry_run": False, "batch_size": 1000},
tags=["prod", "critical"],
# schedules=[Cron(...)] # prod schedule here
)
Notes
- Use flow.from_source(...).deploy(...) for 3.x (do not use deprecated 2.x build commands).
- Prefer main branch/commit pin for prod; feature branch/local path for dev.
- Override at run time when needed:
- CLI: prefect deployment run etl/etl-prod -p dry_run=true (use sparingly in prod)
- Docs: Flow parameters, Deployments, Store flow code, Schedules
Recommended defaults
- Dev
- Workspace: dev
- Work pool: dev-process or dev-docker
- Tags: dev
- Parameters: dry_run=true, smaller batch sizes
- Schedules: manual or frequent, but low-impact
- Prod
- Workspace: prod
- Work pool: prod-kubernetes (or your production infra)
- Tags: prod, criticalMarvin
12/28/2025, 12:18 PM