lucian
09/18/2025, 11:41 AMMarvin
09/18/2025, 11:41 AMMarvin
09/18/2025, 11:51 AMworker.config.workPool
and the worker type at worker.config.type
(kubernetes, docker, etc.).
Example values.yaml (Prefect Cloud)
global:
prefect:
prefectApiUrl: <https://api.prefect.cloud/api/accounts/<ACCOUNT_ID>/workspaces/<WORKSPACE_ID>>
image:
repository: prefecthq/prefect
prefectTag: 3-latest
pullPolicy: IfNotPresent
worker:
replicaCount: 1
apiConfig: cloud
cloudApiConfig:
apiKeySecret:
name: prefect-api-key # K8s secret you create with your API key
key: key
config:
type: kubernetes
workPool: my-k8s-pool
# Optional: provide a base job template to set job namespace, SA, etc.
# baseJobTemplate:
# configuration: |-
# { ... JSON from `prefect work-pool get-default-base-job-template --type kubernetes` ... }
Example values.yaml (self-hosted Server)
image:
repository: prefecthq/prefect
prefectTag: 3-latest
worker:
replicaCount: 1
apiConfig: server
serverApiConfig:
apiUrl: <http://prefect-server.prefect.svc.cluster.local:4200/api>
# uiUrl: <http://prefect-server.prefect.svc.cluster.local:4200>
config:
type: kubernetes
workPool: my-k8s-pool
# baseJobTemplate:
# configuration: |-
# { ...customized JSON... }
Notes on queues and tags
- Prefect workers can be restricted to specific work queues via the worker start options. The Helm chart’s primary required value is the pool (worker.config.workPool
); queue-level filtering is typically handled when starting the worker.
- If you need to restrict queues from Helm, the pattern is to set the worker’s start flags accordingly. If your chart version does not expose explicit keys for queues/tags, you can:
- Configure them on the pool/queues in Prefect UI/API, or
- Override the container command/args in the chart (if supported by your version) to pass --work-queue
flags.
Customizing the Kubernetes job template (namespace, service account, etc.)
- To fix job namespace, service account, and other run-time details for flow runs, pass a base job template to the worker. The chart supports embedding JSON or using --set-file
.
Example (inline JSON):
worker:
config:
workPool: my-k8s-pool
type: kubernetes
baseJobTemplate:
configuration: |-
{
"job_configuration": {
"name": "{{ name }}",
"namespace": "flows-namespace",
"job_manifest": {
"apiVersion": "batch/v1",
"kind": "Job",
"metadata": {
"generateName": "{{ name }}-",
"namespace": "flows-namespace"
},
"spec": {
"template": {
"spec": {
"serviceAccountName": "flows-service-account",
"restartPolicy": "Never",
"containers": [
{
"name": "prefect-job",
"image": "{{ image }}",
"imagePullPolicy": "{{ image_pull_policy }}",
"args": "{{ command }}",
"env": "{{ env }}"
}
]
}
}
}
}
}
}
Can Helm create the work pool or blocks for me?
- Work pools: Typically created/managed via CLI/UI/API/Terraform outside the chart. The worker chart requires worker.config.workPool
. You can create/update pools in CI/CD with:
- prefect work-pool create "my-k8s-pool" --type kubernetes --base-job-template ./base-job-template.json
Marvin
09/18/2025, 11:51 AMapiVersion: batch/v1
kind: Job
metadata:
name: prefect-bootstrap-blocks
annotations:
"<http://helm.sh/hook|helm.sh/hook>": post-install,post-upgrade
"<http://helm.sh/hook-delete-policy|helm.sh/hook-delete-policy>": before-hook-creation,hook-succeeded
spec:
template:
spec:
serviceAccountName: prefect-worker # or another SA with network access to Prefect
restartPolicy: Never
containers:
- name: bootstrap
image: prefecthq/prefect:3-latest
env:
- name: PREFECT_API_URL
valueFrom:
secretKeyRef:
name: prefect-api
key: PREFECT_API_URL
- name: PREFECT_API_KEY
valueFrom:
secretKeyRef:
name: prefect-api
key: PREFECT_API_KEY
command: ["/bin/sh","-c"]
args:
- |
set -e
# Register any block types you need (example: prefect-aws)
prefect block register -m prefect_aws || true
python - <<'PY'
# Example: create/update an S3 block
from prefect_aws.credentials import AwsCredentials
from prefect_aws.s3 import S3Bucket
creds = AwsCredentials.load("aws-creds") if "aws-creds" in [b.name for b in AwsCredentials.list()] else None
if not creds:
creds = AwsCredentials(
aws_access_key_id="<<inject via secrets or IRSA>>",
aws_secret_access_key="<<inject via secrets or IRSA>>",
region_name="us-east-1",
)
creds.save("aws-creds", overwrite=True)
s3 = S3Bucket(bucket_name="my-bucket", credentials=creds)
s3.save("my-s3", overwrite=True)
PY
Links
- Prefect Helm charts repo: prefect-helm on GitHub
- Prefect worker chart docs and values: prefect-worker chart README, values.yaml
- Kubernetes worker docs (includes Helm usage and base job template): Prefect docs: Deploy a worker using Helm
- Work pools overview/how-to: Concepts: Work Pools, How to manage work pools
A couple quick questions so I can tailor this precisely:
- Which chart version are you using?
- Are you on Prefect Cloud or self-hosted Server?
- Which block types do you want to create (e.g., S3, GCS, Docker Registry, etc.)? I can give you an exact bootstrap Job snippet for those.