Nicholas Pini
09/02/2025, 10:48 AMMarvin
09/02/2025, 10:48 AMMarvin
09/02/2025, 11:03 AMvariables.required
. Don’t set a default. Prefect will enforce it, so each deployment must set it in job_variables
.
Example base-job-template.json (Docker worker shown, but the pattern is the same for other worker types)
{
"variables": {
"type": "object",
"properties": {
"image": {
"type": "string",
"title": "Image",
"default": "prefecthq/prefect:3-latest"
},
"MY_REQUIRED_VAR": {
"type": "string",
"title": "Required business value",
"description": "Must be provided by each deployment"
}
},
"required": [
"MY_REQUIRED_VAR"
]
},
"job_configuration": {
"image": "{{ image }}",
"env": {
"MY_REQUIRED_VAR": "{{ MY_REQUIRED_VAR }}"
}
}
}
Create or update the work pool with the template
- Create:
prefect work-pool create "my-docker-pool" --type docker --base-job-template ./base-job-template.json
- Update:
prefect work-pool update "my-docker-pool" --base-job-template ./base-job-template.json
Set the required variable in a deployment’s job_variables
- In prefect.yaml:
deployments:
- name: my-deployment
entrypoint: flows/my_flow.py:my_flow
work_pool:
name: my-docker-pool
job_variables:
MY_REQUIRED_VAR: "abc123"
- Or in Python (Prefect 3.x):
from prefect import flow
@flow
def my_flow():
...
flow.from_source(
source="<https://github.com/your-org/your-repo>",
entrypoint="flows/my_flow.py:my_flow",
).deploy(
name="my-deployment",
work_pool_name="my-docker-pool",
job_variables={"MY_REQUIRED_VAR": "abc123"},
)
What happens if it’s missing?
- Prefect validates job_variables
against the work pool’s template. If MY_REQUIRED_VAR
is not provided, deployment creation or run submission will fail with a validation error indicating the missing required property.
Docs
- Work pool management: Manage work pools
- Customize job variables: Customize job variables
If you tell me which worker type you’re using (Docker, Kubernetes, Process, etc.), I can tailor the job_configuration
snippet for that worker’s fields.