<@ULVA73B9P> Using Prefect 3, how can I set up a v...
# ask-marvin
n
@Marvin Using Prefect 3, how can I set up a variable in the job manifest of a workpool, such that the variable is NOT optional and must be set in the job_variables dictionary of a deployment?
m
thought for 939.0 seconds
Short answer: define the variable in your work pool’s base job template JSON using JSON Schema and list it under
variables.required
. Don’t set a default. Prefect will enforce it, so each deployment must set it in
job_variables
. Example base-job-template.json (Docker worker shown, but the pattern is the same for other worker types)
Copy code
{
  "variables": {
    "type": "object",
    "properties": {
      "image": {
        "type": "string",
        "title": "Image",
        "default": "prefecthq/prefect:3-latest"
      },
      "MY_REQUIRED_VAR": {
        "type": "string",
        "title": "Required business value",
        "description": "Must be provided by each deployment"
      }
    },
    "required": [
      "MY_REQUIRED_VAR"
    ]
  },
  "job_configuration": {
    "image": "{{ image }}",
    "env": {
      "MY_REQUIRED_VAR": "{{ MY_REQUIRED_VAR }}"
    }
  }
}
Create or update the work pool with the template - Create:
Copy code
prefect work-pool create "my-docker-pool" --type docker --base-job-template ./base-job-template.json
- Update:
Copy code
prefect work-pool update "my-docker-pool" --base-job-template ./base-job-template.json
Set the required variable in a deployment’s job_variables - In prefect.yaml:
Copy code
deployments:
  - name: my-deployment
    entrypoint: flows/my_flow.py:my_flow
    work_pool:
      name: my-docker-pool
    job_variables:
      MY_REQUIRED_VAR: "abc123"
- Or in Python (Prefect 3.x):
Copy code
from prefect import flow

@flow
def my_flow():
    ...

flow.from_source(
    source="<https://github.com/your-org/your-repo>",
    entrypoint="flows/my_flow.py:my_flow",
).deploy(
    name="my-deployment",
    work_pool_name="my-docker-pool",
    job_variables={"MY_REQUIRED_VAR": "abc123"},
)
What happens if it’s missing? - Prefect validates
job_variables
against the work pool’s template. If
MY_REQUIRED_VAR
is not provided, deployment creation or run submission will fail with a validation error indicating the missing required property. Docs - Work pool management: Manage work pools - Customize job variables: Customize job variables If you tell me which worker type you’re using (Docker, Kubernetes, Process, etc.), I can tailor the
job_configuration
snippet for that worker’s fields.