Jordy Vargas
03/30/2026, 10:49 PMgit_clone of the repository, but I keep hitting the same issue. I always provide the correct Docker image along with its tag, and I've even run the image locally on my machine and it works fine.Marvin
03/30/2026, 10:49 PMMarvin
03/30/2026, 10:54 PMprefect deployment inspect "<flow-name>/<deployment-name>"
Look for job variables and confirm the image value is exactly your image:tag.
- Run:
prefect work-pool inspect "<pool-name>" --output json
Confirm the pool’s base_job_template.default image isn’t what your runs are using unintentionally.
- Check the actual image used by a failing pod:
- Find the pod for a failed flow run, then:
kubectl get pod <pod-name> -o jsonpath='{.spec.containers[0].image}{"\n"}'
If this isn’t your image, the deployment isn’t picking up your override.
- Check worker logs (the Kubernetes worker pod) right before it submits the job — it will log the job spec/image it’s about to launch.
The most common fixes
A) Use a prebuilt image without building anything during deploy
- If you already have a vetted prebuilt image (with ChromeDriver etc.), do not run build/push steps and do not let deploy rebuild an image for you.
- In Prefect 3.x, Flow.deploy defaults to build=True/push=True. If you deploy via Python, set build=False/push=False or remove build/push steps from prefect.yaml.
Minimal prefect.yaml (prebuilt image, no build/push)
prefect-version: "3.0"
# No build:, no push:
# Optional: pull your code at runtime (ensure git is in your image if you use git_clone)
pull:
- prefect.deployments.steps.git_clone:
repository: "<https://github.com/your-org/your-repo.git>"
branch: "main"
deployments:
- name: my-k8s-deployment
entrypoint: flows/my_flow.py:my_flow
work_pool:
name: your-k8s-pool
job_variables:
image: "your-registry/your-image:your-tag"
namespace: "production"
# add anything else you need here, e.g. service_account_name, resources, env, image_pull_secrets
Then:
prefect deploy
B) Deploy via Python with a prebuilt image
from prefect import flow
@flow
def my_flow():
# your logic
pass
if __name__ == "__main__":
my_flow.from_source(
source="<https://github.com/your-org/your-repo.git>",
entrypoint="flows/my_flow.py:my_flow",
).deploy(
name="my-k8s-deployment",
work_pool_name="your-k8s-pool",
image="your-registry/your-image:your-tag",
build=False, # IMPORTANT: don't rebuild your image
push=False, # IMPORTANT: don't push a new image
job_variables={
"namespace": "production",
},
)
C) Deploy from CLI and override image explicitly
- Verified CLI command:
prefect deploy [./path/to/file.py:flow_func] \
--pool your-k8s-pool \
--job-variable image=your-registry/your-image:your-tag
- CLI flags override prefect.yaml job_variables. This is a great sanity check to ensure the image is set at deployment time.
D) If you do want Prefect to build an image
- Make sure your build step actually installs your system dependencies (Chrome, ChromeDriver, etc.). Many folks accidentally build a “vanilla” Prefect image that lacks those extras — then wonder why it fails at runtime.Marvin
03/30/2026, 10:54 PMprefect work-pool ls
- Inspect a work pool (to check its base job template image and settings):
prefect work-pool inspect "your-k8s-pool" --output json
- Inspect a deployment:
prefect deployment inspect "your-flow-name/your-deployment-name"
A few more gotchas I see in migrations
- Using the wrong pool type: If you point the deployment at a Process pool (or any non-Kubernetes pool), it will not use your Kubernetes image.
- Using the pool default image unintentionally: If you do not set job_variables.image at the deployment, the pool’s base job template image will be used instead.
- Private registries: if your image is private, confirm imagePullSecrets are set in job_variables so the pod can pull.
- Tags like “latest”: Kubernetes may use cached images. Prefer unique tags or set imagePullPolicy to Always in the job template if needed.
If you can share:
- Your prefect.yaml (just the deployments and any build/push/pull sections)
- The output of:
- prefect deployment inspect "<flow>/<deployment>"
- prefect work-pool inspect "<pool>" --output json
- One example pod’s actual image:
- kubectl get pod <pod-name> -o jsonpath='{.spec.containers[0].image}{"\n"}'
…I can pinpoint exactly where the override is getting lost and give you a precise fix.
Lastly, to call out the 2.x → 3.x change explicitly:
- 2.x infra blocks (like KubernetesJob/DockerContainer with an image field) were replaced by 3.x work pools with a base job template + per-deployment job_variables. Don’t use the old 2.x `prefect deployment build`; in 3.x you should use prefect deploy, flow.from_source(...).deploy(...), and set work_pool.job_variables.image (or pass --job-variable image=...).