<@ULVA73B9P> How to config job_variables for docke...
# ask-marvin
t
@Marvin How to config job_variables for docker-registry-creds block in prefect.yaml file
m
thought for 494.8 seconds
Short answer: put a block reference into job_variables for Docker; use a Kubernetes imagePullSecret for K8s. The docker-registry-credentials block is only consumed by the Docker worker. If you’re using a Docker work pool - Create the block once (UI or CLI): - CLI: prefect block create docker-registry-credentials - Verify it exists: prefect block ls - In your prefect.yaml deployment, set job_variables.registry_credentials to the block reference. Example prefect.yaml (Docker worker): deployments: - name: my-deployment entrypoint: flows/flow.py:flow work_pool: name: my-docker-pool type: docker job_variables: image: my-registry.example.com/my-image:0.1.0 registry_credentials: "{{ prefect.blocks.docker-registry-credentials.my-creds }}" image_pull_policy: Always Notes: - The block reference syntax is {{ prefect.blocks.<block-type-slug>.<block-name> }}. - Block type slug here is docker-registry-credentials; my-creds is whatever you named the block. If you’re using a Kubernetes work pool - The Kubernetes worker does not accept docker-registry-credentials blocks. Use a Kubernetes Secret and imagePullSecrets. - Create a Docker registry secret in your cluster/namespace: kubectl create secret docker-registry my-regcred \ --docker-server=REGISTRY_URL \ --docker-username=USERNAME \ --docker-password=PASSWORD \ --namespace=default - Reference it via job_variables by adding imagePullSecrets to the pod spec in the job manifest override. Example prefect.yaml (Kubernetes worker): deployments: - name: my-k8s-deployment entrypoint: flows/flow.py:flow work_pool: name: my-k8s-pool type: kubernetes job_variables: image: my-registry.example.com/my-image:0.1.0 namespace: default image_pull_policy: Always # Override the job manifest to add imagePullSecrets job_manifest: apiVersion: batch/v1 kind: Job metadata: generateName: "{{ name }}-" labels: "{{ labels }}" namespace: "{{ namespace }}" spec: backoffLimit: 0 ttlSecondsAfterFinished: "{{ finished_job_ttl }}" template: spec: serviceAccountName: "{{ service_account_name }}" restartPolicy: Never imagePullSecrets: - name: my-regcred containers: - name: prefect-job image: "{{ image }}" imagePullPolicy: "{{ image_pull_policy }}" args: "{{ command }}" env: "{{ env }}" Why the difference? - Docker worker logs in to the registry using the docker-registry-credentials block. - Kubernetes worker relies on Kubernetes to pull images; therefore you use an imagePullSecret. Helpful docs - prefect.yaml overview: https://docs-3.prefect.io/v3/how-to-guides/deployments/prefect-yaml - Customizing job variables: https://docs-3.prefect.io/v3/how-to-guides/deployments/customize-job-variables - Kubernetes worker guide: https://docs-3.prefect.io/v3/how-to-guides/deployment_infra/kubernetes If you can confirm which work pool type you’re using (docker vs kubernetes), I can tailor the snippet precisely to your setup. Otherwise, yes, I’ll keep guessing — my favorite pastime.