Hi all, I was wondering if there's a way of settin...
# ask-community
g
Hi all, I was wondering if there's a way of setting environments for my jobs on the Kubernetes Agent. I'm trying that by using the Helm chart, and my
agent
section of the
values.yaml
file looks like this:
Copy code
agent:
  enabled: true
  prefectLabels:
    - mylabel
  ...
  job:
    ...
    envFrom:
      - secretRef:
          name: gcp-credentials
The secret
gcp-credentials
exists and is correct. Unfortunately, this doesn't seem to work
a
What do you mean by an environment in this context? Do you mean a distinction between dev and prod? Or do you mean a containerized flow’s execution environment?
g
I mean environment variables, such as
PATH
or
PREFECT__BACKEND
a
if #1, you could have a dev agent and prod agent. They both could exist in different Kubernetes namespaces and as long as each of them have labels attached to them (e.g. “dev” and “prod” labels), you could this way distinguish your flows running in dev vs. prod. if #2, you can provide a custom Docker image to your
KubernetesRun
, ensuring that your flow has all dependencies it needs.
g
it's none of these two
a
So you only want to set environment variables? You can attach those directly to KubernetesRun, or set them in the Kubernetes job template YAML file:
Copy code
env:
        - name: PREFECT__CLOUD__API
          value: <https://api.prefect.io>
        - name: PREFECT__BACKEND
          value: cloud
g
shouldn't this work on my
values.yaml
file, considering it already has an
agent.job
section containing
resources
definition?
https://github.com/PrefectHQ/server/blob/master/helm/prefect-server/templates/agent/deployment.yaml#L61 I see, you don't use anything else other than
resources
under
agent.job
. I'll try that with the job template YAML file, thanks!
a
you can also set it there as well, env section works the same way in values.yaml:
Copy code
env:          
 - name: "var_name"
   value: "value"
 - name: "another"
   value: "one"
g
so I can use
env
but not
envFrom
?
a
I can only see “env” in values.yaml in the agent section: https://github.com/PrefectHQ/server/blob/master/helm/prefect-server/values.yaml#L355
g
exactly, that's the point. then I can't use neither
env
nor
envFrom
for my jobs directly in
values.yaml
, right?
a
env seems possible. But this is for the agent, not for jobs. For a job, I think the easiest would be when you define your template separately, store it e.g. on S3 and provide this S3 file on
KubernetesRun
. This has worked for many people from the community. https://docs.prefect.io/orchestration/flow_config/run_configs.html#kubernetesrun
Copy code
flow.run_config = KubernetesRun(job_template_path="<s3://bucket/path/to/spec.yaml>")
g
ok, I'll try that out, thank you very much!
👍 1
https://github.com/PrefectHQ/server/pull/322 I've added custom job templates for the helm chart, hope it makes sense for Prefect. it fits perfectly for our use case
🙌 1