Xiaozhou Wang
08/12/2025, 12:42 PMMarvin
08/12/2025, 12:42 PMMarvin
08/12/2025, 12:51 PMdocker run --rm -it \
-e PREFECT_API_URL="<https://api.prefect.cloud/api/accounts/<account_id>/workspaces/<workspace_id>>" \
-e PREFECT_API_KEY="$PREFECT_API_KEY" \
-e PREFECT_LOGGING_LEVEL=INFO \
-v "$(pwd)":/opt/project \
--workdir /opt/project \
<your-prod-image> \
bash -lc "python -c 'from flows.my_flow import my_flow; my_flow(param=\"dev-test\")'"
- Replace flows.my_flow:my_flow
with your entrypoint.
- This runs the flow locally, reports to Prefect Cloud, and avoids any GitHub dance.
Option B — Clone from GitHub inside the container (mirrors prod)
If your prod containers pull code at runtime, you can do the same locally with Flow.from_source(...)
. Keep GitHub credentials out of Prefect Cloud by forwarding your SSH agent or passing a temporary token locally.
SSH agent forwarding (recommended for private repos):
docker run --rm -it \
-e PREFECT_API_URL="<https://api.prefect.cloud/api/accounts/<account_id>/workspaces/<workspace_id>>" \
-e PREFECT_API_KEY="$PREFECT_API_KEY" \
-v $SSH_AUTH_SOCK:/ssh-agent \
-e SSH_AUTH_SOCK=/ssh-agent \
-v ~/.ssh/known_hosts:/root/.ssh/known_hosts:ro \
<your-prod-image> \
bash -lc '
python - <<PY
from prefect.flows import Flow
flow = Flow.from_source(
source="git@github.com:owner/repo.git",
entrypoint="path/to/flow.py:my_flow"
)
flow(param="dev-test")
PY
'
HTTPS with a temporary token (works too; just don’t bake it into the image):
docker run --rm -it \
-e PREFECT_API_URL="<https://api.prefect.cloud/api/accounts/<account_id>/workspaces/<workspace_id>>" \
-e PREFECT_API_KEY="$PREFECT_API_KEY" \
-e GITHUB_TOKEN="$GITHUB_TOKEN" \
<your-prod-image> \
bash -lc '
python - <<PY
from prefect.flows import Flow
token = __import__("os").environ["GITHUB_TOKEN"]
url = f"https://{token}:x-oauth-basic@github.com/owner/repo.git"
flow = Flow.from_source(url, "path/to/flow.py:my_flow")
flow(param="dev-test")
PY
'
Notes and gotchas
- Prefect 3.x: calling the flow directly (e.g., my_flow()
) will create a run in Prefect Cloud if PREFECT_API_URL
and PREFECT_API_KEY
are set. No deployment required for local testing.
- Flow.from_source(...)
works fine locally and mirrors your “pull code at runtime” design.
- Keep secrets local: forward your SSH agent or use a short-lived `GITHUB_TOKEN`; don’t store repo creds in Prefect Cloud.
- Use the exact image you run in Fargate to match system deps. Your future self will thank you.
Want a docker-compose example or a snippet that matches your repo layout and entrypoint exactly? Point me at your flow’s entrypoint and the image name, and I’ll wire it up.