Great question. In Prefect 3.x, you control Kubernetes node placement (nodeSelector, tolerations, affinity) in the Kubernetes work pool’s base job template. You can make it: - Pool-wide (same scheduling for all runs from that pool), or - Deployment-specific by exposing variables in the template and overriding them via deployment job_variables. Quick overview - Pool-wide: put your nodeSelector/tolerations/affinity directly under spec.template.spec in the pool’s base job template. - Per-deployment: expose them as variables in the base job template (e.g., {{ node_selector }}), then set them in your deployment’s

work_pool.job_variables

. Where to put these fields in the template Add them under the Job’s Pod spec: - spec.template.spec.nodeSelector - spec.template.spec.tolerations - spec.template.spec.affinity Example: base job template (JSON) with node selection This shows only the relevant parts; include these under your Job’s pod spec in the base job template for your Kubernetes work pool:

{
  "job": {
    "apiVersion": "batch/v1",
    "kind": "Job",
    "spec": {
      "template": {
        "spec": {
          "restartPolicy": "Never",

          // Pool-wide, hard-coded (static) example:
          "nodeSelector": {
            "nodepool": "spot"
          },
          "tolerations": [
            {
              "key": "<http://cloud.google.com/gke-spot|cloud.google.com/gke-spot>",
              "operator": "Equal",
              "value": "true",
              "effect": "NoSchedule"
            }
          ],
          "affinity": {
            "nodeAffinity": {
              "requiredDuringSchedulingIgnoredDuringExecution": {
                "nodeSelectorTerms": [
                  {
                    "matchExpressions": [
                      {
                        "key": "<http://topology.kubernetes.io/zone|topology.kubernetes.io/zone>",
                        "operator": "In",
                        "values": ["us-central1-a", "us-central1-b"]
                      }
                    ]
                  }
                ]
              }
            }
          },

          "containers": [
            {
              "name": "flow",
              "image": "{{ image }}",
              "args": "{{ command | default([]) }}"
            }
          ]
        }
      }
    }
  }

  // Optional: expose scheduling knobs as variables instead of hard-coding:
  // Then in the job above, use:
  // "nodeSelector": "{{ node_selector | default({}) }}",
  // "tolerations": "{{ tolerations | default([]) }}",
  // "affinity": "{{ affinity | default({}) }}"
}

Creating/updating the work pool with this template - Create a pool with the template:

prefect work-pool create "k8s-pool" --type kubernetes --base-job-template ./base-job-template.json

- Update an existing pool:

prefect work-pool update "k8s-pool" --base-job-template ./base-job-template.json

- Inspect to confirm:

prefect work-pool inspect "k8s-pool"

Per-deployment overrides (optional) If you exposed variables like

node_selector

,

tolerations

, and

affinity

in your template, you can override them per deployment via `prefect.yaml`:

deployments:
- name: my-flow
  entrypoint: flows/my_flow.py:flow
  work_pool:
    name: k8s-pool
    job_variables:
      image: <http://ghcr.io/your-org/your-image:latest|ghcr.io/your-org/your-image:latest>
      # These map to the placeholders you added in the base job template
      node_selector:
        nodepool: highmem
      tolerations:
      - key: "workload"
        operator: "Equal"
        value: "etl"
        effect: "NoSchedule"
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 50
            preference:
              matchExpressions:
              - key: "<http://topology.kubernetes.io/zone|topology.kubernetes.io/zone>"
                operator: In
                values: ["us-central1-b"]

Notes and tips - Setting scheduling in the work pool base job template is best for cluster-level policies that apply to many deployments.

- Use per-deployment job_variables only if you exposed those fields in the template; the variable names in

prefect.yaml

must match the placeholders you used in the template. - Ensure the service account your jobs run as has permissions and that your node labels/taints actually exist in the cluster. Docs - Work pools (manage, templates): Manage work pools - Kubernetes worker how-to: Kubernetes work pools and workers - Customizing job variables in deployments: Customize job variables Would you like this to apply pool-wide or per deployment? If you share your current base job template (or how you’d like to override it), I can help tailor the exact JSON.