Matt Youngberg
10/22/2025, 1:16 PMMarvin
10/22/2025, 1:16 PMMarvin
10/22/2025, 1:20 PMjob_variables overwrite work pool defaults at the field level. Since env is a single field (a dict), specifying env on the deployment replaces the pool’s entire env mapping rather than merging keys.
- This is consistent with the docs language that deployment job variables “overwrite” work pool job template values.
What to do instead:
- If you want to keep the work pool’s environment variables and add one for a single deployment, either:
- Don’t set env on the deployment; keep all env vars on the work pool, or
- Include all of the pool env vars plus your extra var in the deployment’s env (i.e., fully specify the env dict at the deployment), or
- Move the deployment-specific value into a Secrets/Blocks approach and reference it from your code so you don’t need to duplicate env.
References:
- Customize job variables (work pool vs deployment)
- GCP worker guide (Cloud Run)
If you’d like, share how you set the deployment env (UI vs prefect.yaml vs .deploy) and I can suggest the cleanest way to preserve the pool envs while adding your one-off variable.Matt Youngberg
10/22/2025, 1:22 PMMarvin
10/22/2025, 1:22 PMMarvin
10/22/2025, 1:27 PMjob_variables.env replaces the work pool’s env dict instead of merging keys. That’s why adding a single env var on the deployment appears to “erase” the pool’s env — the deployment’s env dict completely overrides the pool’s. This aligns with our docs that say deployment job variables “overwrite” pool defaults:
- Customize job variables
- GCP worker guide (Cloud Run)
If deep-merge were implemented, your expected outcome (infra env from the pool + deployment-specific env working together) would “just work.” It’s a reasonable request; it just isn’t the current behavior.
Workarounds/patterns you can use now:
- Keep infra env exclusively on the work pool and avoid setting env on the deployment. Put deployment-specific config in Prefect Blocks/Secrets or parameters so you don’t have to touch the env dict at the deployment level.
- If you must set env on the deployment, include both the pool envs and the deployment-specific keys in the deployment’s env (i.e., fully specify the dict).
- If infra envs truly differ by deployment, consider separate work pools so you don’t need to merge.
Quick question so I can tailor guidance precisely: are you on Prefect 3.x or 2.x? The behavior is effectively the same here, but #11041 was filed against 2.x. If you’d like, I can help file or upvote a 3.x-specific feature request to deep-merge job variables so envs compose intuitively.Matt Youngberg
10/22/2025, 1:28 PMMarvin
10/22/2025, 1:28 PMMarvin
10/22/2025, 1:28 PMWork pool base job template:
env:
POOL_ONLY: "yes"
SHARED: "pool"
Deployment job_variables:
env:
DEPLOY_ONLY: "true"
SHARED: "deploy"
Observed at runtime: only DEPLOY_ONLY + SHARED=deploy. POOL_ONLY is missing.
- Expected behavior:
- Deep-merge dicts so keys from the pool env are preserved and deployment keys override matching keys:
- POOL_ONLY=yes, DEPLOY_ONLY=true, SHARED=deploy
- Merge precedence: flow run > deployment > work pool
- Rationale: Infra-level env vars differ from app-level env vars; replacing the whole dict makes composition cumbersome.
- Prior art: link to #11041
If you share your pool template and deployment snippet here, I can help refine the issue text before you file it.