They merge. In Prefect 3.x, env vars from the work pool are merged with the deployment’s

env

. On key conflicts, the deployment’s value wins. Effective precedence is: - Work pool defaults (base job template/variables) - Deployment

env

(overrides work pool on conflicts) - Flow run

env

(overrides deployment on conflicts) - For process workers only: the worker’s OS environment is merged in at the lowest priority Example:

work pool:  {"A": "1", "B": "2"}
deployment: {"B": "3", "C": "4"}
result:     {"A": "1", "B": "3", "C": "4"}

If you’re on Prefect 2.x, behavior may differ—let me know and I can tailor guidance.

A few likely causes: First, quick clarifiers to help me zero in: - Which worker type are you using (Kubernetes, Docker, Process)? - Prefect version (assuming 3.x)? - Can you share the “containers[0]” portion of your work pool’s Base Job Template (redacted)? Common gotchas (especially with Kubernetes workers): 1) The template must reference the variables -

env

only works if your base job template includes

env: "{{ env }}"

on the container. -

env_from_secrets

is not a built-in variable. If you want to inject secrets via Kubernetes

envFrom

, you must define a variable and reference it in the manifest, e.g.:

// under variables
  "env_from_secrets": {
    "title": "EnvFrom secrets",
    "type": "array",
    "default": []
  }

  // in job_configuration -> job_manifest -> spec -> template -> spec -> containers[0]
  "envFrom": "{{ env_from_secrets }}"

Then set

env_from_secrets

to something like:

[
    { "secretRef": { "name": "my-shared-secret" } }
  ]

If the template does not include

envFrom: "{{ env_from_secrets }}"

, nothing you set in the pool will be applied. 2) Mixing formats or overriding -

env

can be a dict or a Kubernetes-style list. The Kubernetes worker will normalize and merge, but the container must still have

env: "{{ env }}"

. - If your deployment also sets

job_variables.env

, it will merge with the work pool’s

env

(deployment wins on key conflicts). If it sets

env_from_secrets

, note that non-

env

variables are not “special-merged” by the worker—later levels can override entire arrays/objects. 3) Values set in the wrong place - Set shared defaults at the work pool under Variables (not directly hardcoded in the manifest). - Deployment/job overrides should go under deployment

job_variables

, not by editing the rendered

job_configuration

. 4) Using a different worker than intended -

env_from_secrets

only makes sense for Kubernetes (via

envFrom

). Docker/Process workers won’t use it. Checklist to verify: - In Work Pool > Base job template, confirm your container spec includes:

"env": "{{ env }}"

and, if you want secret injection,

"envFrom": "{{ env_from_secrets }}"

- Confirm

env_from_secrets

exists under the template’s

variables

with a default (e.g.,

[]

). - Trigger a new run after updating the pool. In the worker logs, confirm it shows the created Kubernetes Job. If you can, inspect the created Pod spec to see `env`/`envFrom` rendered. - Add a simple test var in the pool

env

(e.g.,

POOL_TEST=1

) and print it in your flow:

import os
  from prefect import flow

  @flow
  def check_env():
      print("POOL_TEST:", os.environ.get("POOL_TEST"))

  if __name__ == "__main__":
      check_env()

If you paste the containers block of your base job template (redacted), I can spot-check it and suggest the exact changes to get

env

and

env_from_secrets

applied.

Here’s a complete Kubernetes work pool Base Job Template that supports: -

env

(merged across pool → deployment → flow run, with higher levels winning on conflicts) -

envFrom

via a variable called

env_from_secrets

(array of Kubernetes envFrom entries) -

imagePullSecrets

via an array of objects like

[{ "name": "regcred" }]

Paste this JSON into your Work Pool’s Base Job Template:

{
  "variables": {
    "env": {
      "title": "Environment Variables",
      "description": "Key/value env vars injected into the container",
      "type": "object",
      "default": {},
      "additionalProperties": {
        "type": "string"
      }
    },
    "env_from_secrets": {
      "title": "EnvFrom secrets",
      "description": "Kubernetes envFrom entries, e.g. [{\"secretRef\": {\"name\": \"my-secret\"}}]",
      "type": "array",
      "default": [],
      "items": {
        "type": "object"
      }
    },
    "image": {
      "title": "Container image",
      "type": [
        "string",
        "null"
      ],
      "default": null
    },
    "image_pull_policy": {
      "title": "Image pull policy",
      "type": "string",
      "enum": [
        "IfNotPresent",
        "Always",
        "Never"
      ],
      "default": "IfNotPresent"
    },
    "image_pull_secrets": {
      "title": "Image pull secrets",
      "description": "List of objects with a 'name' field, e.g. [{\"name\": \"regcred\"}]",
      "type": "array",
      "default": [],
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string"
          }
        },
        "required": [
          "name"
        ]
      }
    },
    "namespace": {
      "title": "Namespace",
      "type": "string",
      "default": "default"
    },
    "service_account_name": {
      "title": "Service Account",
      "type": [
        "string",
        "null"
      ],
      "default": null
    },
    "annotations": {
      "title": "Annotations",
      "type": "object",
      "default": {},
      "additionalProperties": {
        "type": "string"
      }
    },
    "labels": {
      "title": "Labels",
      "type": "object",
      "default": {},
      "additionalProperties": {
        "type": "string"
      }
    },
    "backoff_limit": {
      "title": "Backoff limit",
      "type": "integer",
      "default": 0
    },
    "finished_job_ttl": {
      "title": "TTL after finished (seconds)",
      "type": [
        "integer",
        "null"
      ],
      "default": null
    },
    "command": {
      "title": "Container args/command",
      "type": "array",
      "default": [],
      "items": {
        "type": "string"
      }
    }
  },
  "job_configuration": {
    "job_manifest": {
      "apiVersion": "batch/v1",
      "kind": "Job",
      "metadata": {
        "annotations": "{{ annotations }}",
        "labels": "{{ labels }}",
        "namespace": "{{ namespace }}",
        "generateName": "{{ name }}-"
      },
      "spec": {
        "backoffLimit": "{{ backoff_limit }}",
        "ttlSecondsAfterFinished": "{{ finished_job_ttl }}",
        "template": {
          "metadata": {
            "annotations": "{{ annotations }}",
            "labels": "{{ labels }}"
          },
          "spec": {
            "restartPolicy": "Never",
            "serviceAccountName": "{{ service_account_name }}",
            "imagePullSecrets": "{{ image_pull_secrets }}",
            "containers": [
              {
                "name": "prefect-job",
                "image": "{{ image }}",
                "imagePullPolicy": "{{ image_pull_policy }}",
                "args": "{{ command }}",
                "env": "{{ env }}",
                "envFrom": "{{ env_from_secrets }}"
              }
            ]
          }
        }
      }
    }
  }
}

How to use: - In the Work Pool “Variables” panel, you can set:

env:
    PREFECT_LOG_LEVEL: INFO
    SHARED_DEFAULT: from-pool
  env_from_secrets:
    - secretRef:
        name: shared-env
  image_pull_secrets:
    - name: regcred

- In a deployment, you can override/extend via `job_variables`: ``` job_variables: env: SHARED_DEFAULT: overridden-at-deployment EXTRA: deployment-only

env_from_secrets: - secretRef: name: shared-env - secretRef: name: deployment-extra ``` Note:

env

is merged (deployment/flow run values win on key conflicts). Arrays like

env_from_secrets

are typically replaced by higher levels—if you want both pool and deployment entries, include the full list at the higher level. If you share a redacted copy of your current Base Job Template, I can spot-check why your `env`/`envFrom` aren’t appearing in the rendered Job.

Absolutely—here’s a ready-to-paste Base Job Template for a Prefect 3.x Google Cloud Run (v2) work pool. It supports: - env (merged across pool → deployment → run, with higher levels winning) - env_from_secrets (GCP Secret Manager via SecretKeySelector) - Common Cloud Run fields: image, cpu, memory, timeout, max_retries, service account, VPC connector, args, labels Paste this JSON into your Work Pool’s Base Job Template: ``` { "variables": { "type": "object", "properties": { "region": { "type": "string", "default": "us-central1", "description": "GCP region for Cloud Run job" }, "image": { "type": "string", "default": "prefecthq/prefect:3-latest", "description": "Container image URI" }, "cpu": { "type": "string", "default": "1000m", "description": "CPU allocation (e.g. 1000m = 1 vCPU)" }, "memory": { "type": "string", "default": "512Mi", "pattern": "^\\d+(?:G|Gi|M|Mi)$", "description": "Memory allocation (e.g. 512Mi, 2Gi)" }, "timeout": { "type": "integer", "default": 600, "minimum": 1, "maximum": 86400, "description": "Execution timeout in seconds" }, "max_retries": { "type": "integer", "default": 0, "description": "Automatic retry attempts" }, "service_account_name": { "type": ["string", "null"], "default": null, "description": "Service account email for the job" }, "vpc_connector_name": { "type": ["string", "null"], "default": null, "description": "VPC connector resource name (projects/.../locations/.../connectors/...)" }, "launch_stage": { "type": "string", "enum": ["ALPHA", "BETA", "GA", "DEPRECATED", "EARLY_ACCESS", "PRELAUNCH"], "default": "GA", "description": "Cloud Run feature stage" }, "args": { "type": "array", "items": { "type": "string" }, "default": [], "description": "Container arguments" }, "cloudsql_instances": { "type": "array", "items": { "type": "string" }, "default": [], "description": "Cloud SQL instance connection names (projectregioninstance)" }, "env_from_secrets": { "type": "object", "title": "Environment variables from Secret Manager", "description": "Map env var name -> {secret, version}", "additionalProperties": { "type": "object", "properties": { "secret": { "type": "string" }, "version": { "type": "string" } }, "required": ["secret", "version"] }, "default": {} }, "prefect_api_key_secret": { "type": ["object", "null"], "description": "Use a Secret Manager secret for the Prefect API key", "properties": { "secret": { "type": "string" }, "version": { "type": "string" } }, "default": null }, "prefect_api_auth_string_secret": { "type": ["object", "null"], "description": "Use a Secret Manager secret for the Prefect API auth string", "properties": { "secret": { "type": "string" }, "version": { "type": "string" } }, "default": null } } }, "job_configuration": { "env": {}, "labels": {}, "job_body": { "client": "prefect", "launchStage": "{{ launch_stage }}", "template": { "template": { "serviceAccount": "{{ service_account_name }}", "maxRetries": "{{ max_retries }}", "timeout": "{{ timeout }}", "vpcAccess": { "connector": "{{ vpc_connector_name }}" }, "containers": [ { "env": [], "image": "{{ image }}",

"command": "{{ command }}", "args": "{{ args }}", "resources": { "limits": { "cpu": "{{ cpu }}", "memory": "{{ memory }}" } } } ] } } } } }

How to use:
- Set shared defaults in the Work Pool “Variables”:

image: "us-docker.pkg.dev/my-proj/run/prefect:3.8" cpu: "2000m" memory: "2Gi" timeout: 1800 service_account_name: "prefect-runner@my-proj.iam.gserviceaccount.com" env_from_secrets: DATABASE_URL: secret: "prod-db-url" version: "latest"

- At deployment, override via `job_variables`:

job_variables: image: "us-docker.pkg.dev/my-proj/run/app:20250218" cpu: "4000m" memory: "4Gi" env: LOG_LEVEL: "DEBUG" env_from_secrets: API_TOKEN: secret: "external-api-token" version: "5" ``` Notes: -

env

is merged across levels; keys at deployment/run override pool. -

env_from_secrets

is a mapping of env var name to Secret Manager selector

{secret, version}

. - If you set

prefect_api_key_secret

, the worker injects

PREFECT_API_KEY

from that secret and filters any plaintext

PREFECT_API_KEY

you may have set in

env

. - Ensure your service account has

roles/secretmanager.secretAccessor

for any referenced secrets. If you want this template with additional fields (e.g., concurrency, volumes, execution environment), tell me what you need and I’ll extend it.

Short answer: - env merges across levels (pool defaults → deployment → flow run). Later levels win on key conflicts. - env_from_secrets does not “deep merge” the way env does. If you set env_from_secrets at a higher level, it replaces the lower-level mapping. To keep pool defaults plus add more at deployment/run, you must include the full combined mapping at the higher level. Details and why you may not see merging: - In Prefect 3.x, workers special-case merging for the env dict so you get a union of keys (with later values winning). - Other job variables (including env_from_secrets) are not special-merged; they’re overridden as whole values when a higher level provides them. - For Cloud Run specifically, the worker assembles the final container env for you; you don’t need placeholders in job_body for env or env_from_secrets. Examples - env merge: - Pool defaults:

env:
      LOG_LEVEL: INFO
      SHARED: from-pool

- Deployment:

job_variables:
      env:
        SHARED: from-deployment
        EXTRA: dep-only

- Result env passed to the job:

{ LOG_LEVEL: INFO, SHARED: from-deployment, EXTRA: dep-only }

- env_from_secrets override (no merge): - Pool defaults:

env_from_secrets:
      DATABASE_URL: { secret: "prod-db-url", version: "latest" }

- Deployment:

job_variables:
      env_from_secrets:
        API_TOKEN: { secret: "ext-api-token", version: "5" }

- Result secrets in the job: only API_TOKEN (DATABASE_URL is replaced/omitted). - To keep both, set at deployment:

job_variables:
      env_from_secrets:
        DATABASE_URL: { secret: "prod-db-url", version: "latest" }
        API_TOKEN: { secret: "ext-api-token", version: "5" }

Common causes when defaults “don’t merge” - You expect env_from_secrets to merge with pool defaults; it doesn’t—provide the combined mapping at the higher level. - Defaults were added only inside job_body; for Cloud Run, set defaults under job_configuration.env and/or as variables defaults (both are honored). - A deployment/run provided env_from_secrets, unintentionally replacing pool defaults. - Using plaintext PREFECT_API_KEY in env while also setting prefect_api_key_secret; the worker will prefer the secret and filter the plaintext key. Quick checks - Inspect the rendered Cloud Run Job to see final env: - In Prefect worker logs, note the job name and describe it:

gcloud run jobs describe <job-name> --region <region> --format json | jq '.template.template.containers[0].env'

- Test merge behavior with a trivial var at the pool level (e.g., POOL_TEST=1) and print it in your flow to confirm env merging. - If you need additive behavior for env_from_secrets across levels, avoid setting it at deployment/run or always include the full set at the highest level. References - Cloud Run worker and models live in the prefect-gcp integration: prefect-gcp repo (see

prefect_gcp/workers/cloud_run_v2.py

and

prefect_gcp/models/cloud_run_v2.py

). If you share: - Your Prefect version - The pool’s Base Job Template (redacted) - The deployment’s job_variables …I can point out exactly where the override is happening and suggest the minimal fix.

Thanks for sharing that snippet. Given your variables.env has defaults like ENV, DATABASE_HOST, etc., and you don’t see them in Cloud Run, here are the most likely causes and quick fixes: What should happen in Prefect 3.x Cloud Run - The worker builds env like this (highest priority last): 1) job_configuration.env in the pool template 2) variables.properties.env.default in the pool template 3) deployment job_variables.env 4) flow run job_variables.env - The Cloud Run worker then unconditionally appends those env vars into the job body at containers[0].env. You do not need an env placeholder in job_body. Why your defaults might not be appearing 1) job_configuration.env is missing or not a dict - Ensure your template has a dict at job_configuration.env, even if empty:

"job_configuration": {
    "env": {},
    "job_body": {
      "template": { "template": { "containers": [ { "env": [], ... } ] } }
    }
  }

- If job_configuration.env is null, a string, or omitted, the initial merge step can be skipped or behave unexpectedly. 2) Your job_body containers[0] is missing an env array - The worker appends with .extend(...) into containers[0].env. If that path doesn’t exist or isn’t a list, nothing gets appended. - Make sure your job_body has:

"job_body": {
    "template": {
      "template": {
        "containers": [
          {
            "env": [],
            "image": "{{ image }}",
            "command": "{{ command }}",
            "args": "{{ args }}"
          }
        ]
      }
    }
  }

3) A higher level is overriding env - If your deployment or a run sets job_variables.env, it will merge and can override keys. Verify your deployment’s job_variables. - Note: only PREFECT_API_KEY and PREFECT_API_AUTH_STRING are filtered when you use the secret-based fields; your custom vars won’t be filtered. 4) Typos or structure drift - The Cloud Run worker expects the v2 job body structure exactly at: job_body.template.template.containers[0].env - If your custom template moved containers or changed keys (e.g., “container” vs “containers”), the injection won’t find the list. 5) Using “{{ env }}” as a placeholder in job_configuration.env - That’s not needed for Cloud Run and can lead to the env dict not being present at the time of merge. Prefer a literal dict:

"job_configuration": { "env": {} }

Quick verification steps - Describe the created job’s env to see what actually made it into Cloud Run:

gcloud run jobs describe <job-name> --region <region> --format json | jq '.template.template.containers[0].env'

- Add a trivial default in variables.env and print it in your flow:

import os
  from prefect import flow

  @flow
  def check_env():
      print("ENV:", os.environ.get("ENV"))
      print("POOL_TEST:", os.environ.get("POOL_TEST"))

  # Set POOL_TEST=1 in pool variables.env defaults, run, and confirm output

Minimal known-good template skeleton to compare against ``` { "variables": { "type": "object", "properties": { "env": { "type": "object", "title": "Environment Variables", "additionalProperties": { "type": "string" }, "default": { "ENV": "staging", "DATABASE_HOST": "1.1.1.1", "DATABASE_USER": "zzzz", "DATABASE_NAME": "zzzz", "TELEMETRY_MODE": "NONE", "OTEL_EXPORTER_OTLP_ENDPOINT": "telemetry.googleapis.com:443" } }, "image": { "type": "string", "default": "prefecthq/prefect:3-latest" }, "timeout": { "type": "integer", "default": 600 }, "cpu": { "type": "string", "default": "1000m" }, "memory": { "type": "string", "default": "512Mi" } } }, "job_configuration": { "env": {}, // must be a dict "job_body": { "launchStage": "GA", "template": { "template": { "maxRetries": 0, "timeout": "{{ timeout }}", "containers": [

{ "env": [], // must exist and be a list "image": "{{ image }}", "command": "{{ command }}", "args": "{{ args }}", "resources": { "limits": { "cpu": "{{ cpu }}", "memory": "{{ memory }}" } } } ] } } } } } ``` If you can share your job_configuration and job_body sections (redacted), I’ll pinpoint exactly where the merge/injection is being short-circuited and give you a precise patch.

Thanks for sharing the job_configuration. Given this template, your pool-level env defaults should be present in the final Cloud Run Job. If they’re not, it’s usually one of these: Most common causes 1) job_configuration.env should be a dict, not a template - While the worker will typically replace

"{{ env }}"

with your defaults, the safest/most predictable setup for Cloud Run is to make it a literal dict and let the worker inject into the job body. - Change:

"env": "{{ env }}"

to:

"env": {}

2) The job_body path must have containers[0].env as a list - Your template has

"env": []

which is correct. If this array were missing, the worker’s append would fail and you’d see no env in Cloud Run. 3) A higher level is overriding - A deployment or run can provide

job_variables.env

. It merges with pool defaults and can override keys. Double‑check the deployment’s job_variables aren’t nulling or overwriting values unexpectedly. 4) You’re looking at the wrong place in Cloud Run UI - Verify via CLI to be sure what was sent:

gcloud run jobs describe <job-name> --region <region> --format json | jq '.template.template.containers[0].env'

You can copy the job name from the worker logs. 5) Prefect secret filtering - Only affects

PREFECT_API_KEY

and

PREFECT_API_AUTH_STRING

. If you set `prefect_api_key_secret`/`prefect_api_auth_string_secret`, the plaintext versions are removed. Your custom vars (ENV, DATABASE_*) are not filtered. Recommended patch (minimal, known-good)

"job_configuration": {
  "env": {},                         // make this a dict, not a template
  "name": "{{ name }}",
  "labels": "{{ labels }}",
  "region": "{{ region }}",
  "command": "{{ command }}",
  "timeout": "{{ timeout }}",
  "job_body": {
    "client": "prefect",
    "template": {
      "template": {
        "timeout": "{{ timeout }}",
        "vpcAccess": { "connector": "{{ vpc_connector_name }}" },
        "containers": [
          {
            "env": [],               // must exist for injection
            "args": "{{ args }}",
            "image": "{{ image }}",
            "command": "{{ command }}",
            "resources": {
              "limits": { "cpu": "{{ cpu }}", "memory": "{{ memory }}" }
            }
          }
        ],
        "maxRetries": "{{ max_retries }}",
        "serviceAccount": "{{ service_account_name }}"
      }
    },
    "launchStage": "{{ launch_stage }}"
  },
  "keep_job": "{{ keep_job }}",
  "credentials": "{{ credentials }}"
  // TIP: you do not need "env_from_secrets" here; the worker injects secrets into env automatically
}

Why this helps - The Cloud Run worker always appends configuration.env and env_from_secrets into job_body.template.template.containers[0].env. Using a literal dict for job_configuration.env avoids any edge cases where a template placeholder isn’t resolved as a dict at merge time. Next steps to confirm - Keep your variables.env defaults as-is. - Apply the patch (env: {}), re-run a deployment/run. - Inspect the created job:

gcloud run jobs describe <job-name> --region <region> --format json | jq '.template.template.containers[0].env'

- Optionally print from your flow:

import os
  from prefect import flow

  @flow
  def show():
      print("ENV:", os.environ.get("ENV"))
      print("DATABASE_HOST:", os.environ.get("DATABASE_HOST"))

If this still doesn’t show your defaults, please share: - Prefect version and prefect-gcp version - The pool variables section for env (redacted) - The exact output of the gcloud describe command above I can then pinpoint exactly where the merge/injection is being lost. For reference, Cloud Run worker source: prefect-gcp repo.