https://prefect.io
Join Slack
hey all, how do we consume env vars in the prefect.yaml file? not a prefect variable
# best-practices
j
hey all, how do we consume env vars in the prefect.yaml file? not a prefect variable
n
j
thank you!
n
catjam
j
Copy code
name: pipelines
prefect-version: 2.14.3

pull:
  - prefect.deployments.steps.set_working_directory:
      directory: /app/projects/pipelines

definitions:
  work_pools:
    dc_work_pool: &dc_work_pool
      name: "{{ $WORK_POOL_NAME }}"
      job_variables:
        image: "{{ $REGISTRY }}/{{ $IMAGE_NAME }}:{{ $TAG }}"

deployments:
  - name: my_flow_docker_pool_2
    entrypoint: dc/pipelines/test_flow.py:my_flow
    work_pool: *dc_work_pool
for some reason, reading an env var for the work pool name does not work
if i use that variable elsewhere, the yaml file is able to read it
any idea what's going on here?
Copy code
This deployment configuration references work pool '{{ $WORK_POOL_NAME }}' which does not exist. This
means no worker will be able to pick up its runs. You can create a work pool in the Prefect UI.
fyi @Nate -- looks like someone else bumped into the same issue i have here: https://github.com/PrefectHQ/prefect/issues/10751 perhaps there's a bug where the work pool name field can't read from env vars?
n
hmm not sure - i’ll try to see what’s going on with that today
thanks for raising!
j
yup, np!
@Nate -- in the meantime, do you have a workaround for this? we want to be able to swap the work pool name out depending on the environment.
Copy code
definitions:
  work_pools:
    work_pool: &work_pool
      name: "{{ $WORK_POOL_NAME }}-{{ $ENV }}" # ideally we do something like this
n
mm i tried this out and it looks like im not able to template into the 
work_pool
name
field for some reason, which seems like bug. so i agree with your (I presume) comment
ill take a look at that
j
yup, my comment
@Nate -- following up here 🙂
n
hi jon i dont have time to look at this now but my hunch is that we'll need an issue / someone to pick it up here, bc it does appear we cannot template into the work pool name etc if you could create a ticket that specifically addresses this problem (if not exists) i would appreciate it! we'll get on it asap
j
will do, thank you!
n
🙏 thank you
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by