Federico Ferlito
10/14/2025, 3:51 PMMarvin
10/14/2025, 3:51 PMMarvin
10/14/2025, 3:53 PMflow.from_source(...).deploy(...) with pinned commits and naming conventions, or exposing commit SHAs via artifacts).Federico Ferlito
10/14/2025, 3:55 PMMarvin
10/14/2025, 3:55 PMMarvin
10/14/2025, 3:59 PMversion field.
- Pass GIT_SHA into runs as an env var and surface it everywhere (run name, logs, artifacts).
- Optionally, keep “latest” aliases (e.g., branch names) alongside versioned deployments.
- Use CLI and naming conventions to list “versions” and clean up old ones.
1) Flow code: embed and surface the version
- Read GIT_SHA from the environment and set the flow’s version.
- Publish artifacts with links to the commit and image tag so it’s obvious in the run UI.
- Optionally, set flow_run_name to include the short SHA.
import os
from prefect import flow, get_run_logger
from prefect.artifacts import create_markdown_artifact, create_link_artifact
GIT_SHA = os.getenv("GIT_SHA", "dev")
SHORT_SHA = GIT_SHA[:7]
REPO_URL = os.getenv("REPO_URL", "<https://github.com/your-org/your-repo>")
IMAGE = os.getenv("IMAGE", f"<http://ghcr.io/your-org/your-image:{GIT_SHA}|ghcr.io/your-org/your-image:{GIT_SHA}>")
@flow(
name="my-flow",
version=GIT_SHA,
flow_run_name=f"my-flow-{SHORT_SHA}-{{%Y%m%d-%H%M%S}}",
)
def my_flow():
logger = get_run_logger()
<http://logger.info|logger.info>(f"Running my-flow at commit {GIT_SHA} using image {IMAGE}")
# Create artifacts that show up on the run page
create_markdown_artifact(
key=f"build-info-{SHORT_SHA}",
markdown=f"""
### Build Info
- Commit: <{REPO_URL}/commit/{GIT_SHA}|{GIT_SHA}>
- Image: `{IMAGE}`
"""
)
create_link_artifact(
key=f"commit-link-{SHORT_SHA}",
link_url=f"{REPO_URL}/commit/{GIT_SHA}",
description=f"Commit {SHORT_SHA}"
)
# ... your logic ...
return "done"
2) Deployment script: pin the version and image
- Deploy per-commit so each deployment = one version.
- Use Flow.from_source(...).deploy(...) in Prefect 3.x.
- Set deployment version, name, tags, and inject GIT_SHA (and other context) into the job’s environment for visibility inside runs.
import os
from prefect.flows import Flow
GIT_SHA = os.environ["GIT_SHA"]
SHORT_SHA = GIT_SHA[:7]
IMAGE = os.environ["IMAGE"] # e.g., <http://ghcr.io/your-org/your-image:<sha|ghcr.io/your-org/your-image:<sha>>
# If running this from the repo root in CI:
flow = Flow.from_source(source=".", entrypoint="flows/my_flow.py:my_flow")
deployment_id = flow.deploy(
name=f"my-flow@{SHORT_SHA}", # clear versioned name
version=GIT_SHA, # this is the deployment version field
image=IMAGE, # pin code to the exact image
tags=[f"version:{SHORT_SHA}", "env:prod"], # handy for filtering
job_variables={
"env": {
"GIT_SHA": GIT_SHA,
"REPO_URL": os.environ.get("REPO_URL", "<https://github.com/your-org/your-repo>"),
"IMAGE": IMAGE,
}
},
# optional: add schedules/triggers here as needed
)
print(f"Created deployment: {deployment_id}")
Notes:
- The flow code is baked into the Docker image. The deployment references the entrypoint path that exists inside the image.
- This pattern gives you perfect reproducibility: each deployment points to a single image tag (the commit SHA).
3) GitHub Actions: build, push, deploy
- Build a Docker image tagged with the commit SHA.
- Push to your registry (GHCR used as example).
- Deploy with that image and version.
```
name: deploy-my-flow
on:
push:
branches: [ main ]
tags: [ 'v*' ]
env:
REGISTRY: ghcr.io
IMAGE_NAME: your-org/your-image
jobs:
build-and-deploy:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: CheckoutMarvin
10/14/2025, 3:59 PMmy-flow@abc1234.
- Tags include version:<short_sha> so you can filter quickly.
- List deployments by flow (acts like a Versions list):
- Verified CLI:
- prefect deployments ls --flow-name "my-flow" --by-created
- Add a “latest” alias alongside versioned deployments:
- Also publish a stable deployment named my-flow@latest (or my-flow@main) for your default channel. It can point to the newest image.
- Show version on every run:
- flow_run_name and artifacts ensure the SHA is immediately visible on the run page.
- Logging the SHA on startup helps with grep’ing logs.
5) Optional: pruning old “versions”
If you publish a deployment per commit, you may want to keep only the last N.
- Simple approach: periodically run a maintenance job that:
- Lists deployments for the flow, sorted by created time.
- Keeps the latest N and deletes older ones.
- You can do this with the Prefect CLI plus a small script in Actions or with the Prefect Python client. If you want, I can share a minimal “keep last N” script tailored to your flow name.
6) Alternative: versioning without Docker images (less reproducible)
- You can deploy from source with Flow.from_source("<https://github.com/your-org/your-repo>", entrypoint="...") and use the deployment’s version field set to the commit SHA.
- This is lighter-weight but not as reproducible as pinning an image, because fetching source at runtime can drift if not pinned to a commit and a content-addressed artifact.
Tips and gotchas
- Prefer image-per-commit for deterministic rollbacks and auditing.
- Always set both the deployment version and include the SHA in the deployment name and tags — gives you multiple ways to find the right version.
- Inject GIT_SHA via job_variables.env so your flow can surface it in logs and artifacts even when runs happen on remote workers.
- If you promote the same image to multiple environments (dev/stage/prod), keep the SHA constant and only change the alias deployment names or tags — the “version” stays meaningful across envs.
Want me to:
- Tailor the YAML and deploy.py to your repo path/flow entrypoint?
- Add a small “keep last N versions” action?
- Or add schedules/triggers per versioned deployment?