I’m trying to set environment variables in a kubernetes flow run by using the job template but the environment variable value is still using the one that is set when I create my docker Image. How do I override the environment variable set when creating the image with one on a job template to dynamically change how the flow will run?

Hi @Josh have you tried setting the

env

on the

KubernetesRun

runconfig?

@Nate I would like to set it dynamically based on ENV values in the agent. For example: If I have 1 agent per customer, I’d like to be able to have the customer ID as an ENV variable on the agent and then any flows that run on that agent will check the env value CUSTOMER=A or CUSTOMER=B and do different things accordingly.

Can you share (with no secrets please) your agent manifest?

agent manifest looks like this

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: prefect-agents
  name: prefect-agents
  namespace: prefect-agents
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prefect-agents
  template:
    metadata:
      labels:
        app: prefect-agents
    spec:
      nodeSelector:
        <http://cloud.google.com/compute-class|cloud.google.com/compute-class>: Balanced
        <http://kubernetes.io/arch|kubernetes.io/arch>: amd64
      containers:
        - name: agent-because
          image: prefecthq/prefect:1.2.4-python3.9
          resources:
            requests:
              cpu: "256m"
              memory: "256Mi"
            limits:
              cpu: "512m"
              memory: "512Mi"
          command:
            [
              "prefect",
              "agent",
              "kubernetes",
              "start",
              "--job-template",
              "<gcs://k8s_prefect_job_templates/job-template-because.yaml>",
            ]
          imagePullPolicy: IfNotPresent
          env:
            - name: PREFECT__CLOUD__AGENT__AUTH_TOKEN
              value: ""
            - name: PREFECT__CLOUD__API
              value: <https://api.prefect.io>
            - name: NAMESPACE
              value: prefect-agents
            - name: IMAGE_PULL_SECRETS
              value: ""
            - name: PREFECT__CLOUD__AGENT__LABELS
              value: "['k8s_customer:because', 'k8s_amazon_sp_api__orders:because', 'k8s_cache_generation:because']"
            - name: JOB_MEM_REQUEST
              value: ""
            - name: JOB_MEM_LIMIT
              value: ""
            - name: JOB_CPU_REQUEST
              value: ""
            - name: JOB_CPU_LIMIT
              value: ""
            - name: IMAGE_PULL_POLICY
              value: ""
            - name: SERVICE_ACCOUNT_NAME
              value: ""
            - name: PREFECT__BACKEND
              value: cloud
            - name: PREFECT__CLOUD__AGENT__AGENT_ADDRESS
              value: http://:8080
            - name: PREFECT__CLOUD__API_KEY
              value: xxxxxxxxxxxxxxxxxxxxxxx
            - name: PREFECT__CLOUD__TENANT_ID
              value: ""
            - name: GOOGLE_APPLICATION_CREDENTIALS
              value: "/secretmount/gcp_key"
          volumeMounts:
            - name: secretmount
              mountPath: "/secretmount/"
              readOnly: true

        - name: agent-boosted-commerce
          image: prefecthq/prefect:1.2.4-python3.9
          resources:
            requests:
              cpu: "256m"
              memory: "256Mi"
            limits:
              cpu: "512m"
              memory: "512Mi"
          command:
            [
              "prefect",
              "agent",
              "kubernetes",
              "start",
              "--job-template",
              "<gcs://k8s_prefect_job_templates/job-template-boosted-commerce.yaml>",
            ]
          imagePullPolicy: IfNotPresent
          env:
            - name: PREFECT__CLOUD__AGENT__AUTH_TOKEN
              value: ""
            - name: PREFECT__CLOUD__API
              value: <https://api.prefect.io>
            - name: NAMESPACE
              value: prefect-agents
            - name: IMAGE_PULL_SECRETS
              value: ""
            - name: PREFECT__CLOUD__AGENT__LABELS
              value: "['k8s_customer:boosted-commerce', 'k8s_amazon_sp_api__orders:boosted-commerce']"
            - name: JOB_MEM_REQUEST
              value: ""
            - name: JOB_MEM_LIMIT
              value: ""
            - name: JOB_CPU_REQUEST
              value: ""
            - name: JOB_CPU_LIMIT
              value: ""
            - name: IMAGE_PULL_POLICY
              value: ""
            - name: SERVICE_ACCOUNT_NAME
              value: ""
            - name: PREFECT__BACKEND
              value: cloud
            - name: PREFECT__CLOUD__AGENT__AGENT_ADDRESS
              value: http://:8081
            - name: PREFECT__CLOUD__API_KEY
              value: xxxxxxxxxxxxxxxxxxxxxxxxxxx
            - name: PREFECT__CLOUD__TENANT_ID
              value: ""
            - name: GOOGLE_APPLICATION_CREDENTIALS
              value: "/secretmount/gcp_key"
          volumeMounts:
            - name: secretmount
              mountPath: "/secretmount/"
              readOnly: true

Got it so the question is, is there higher precedence than the variable set within the container

gotcha thanks for helping. So the precedence will be 1. jobtemplate 2. agent manifest 3. docker image Is that right?

Agent manifest is completely separate, so just 1 and 3