Has anyone had an issue with env vars not getting passed fro Prefect Community #prefect-community

Has anyone had an issue with env vars not getting ...

Alex Shea

08/12/2022, 7:10 PM

Has anyone had an issue with env vars not getting passed from the agent to the job with a kubernetes run leveraging

PREFECT__CLOUD__AGENT__ENV_VARS

agent variable?

Bianca Hoch

08/12/2022, 7:26 PM

Hi Alex, here are a few follow up questions: • Are you getting a specific error? • What is your use case?

Alex Shea

08/12/2022, 7:35 PM

I am trying to pass environment variables whos contents are managed within IaC to also be passed into the flow execution that way. I don't want my data engineers to need to know anything about IaC when working within our systems

Alex Shea

08/12/2022, 7:36 PM

The agent is being deployed as version 1.0.0 as we are migrating from ecs to kubernetes. I am am trying not to change compatibility. I am using this variable as documented in the help text

Copy code

-e, --env TEXT                 Environment variables to set on each
                                 submitted flow run.

Alex Shea

08/12/2022, 7:38 PM

the command I used to generate the deployment was the

prefect agent kubernetes install

with the -env. There wasn't any documentation and I used the errors from the flow deployment to help me understand. I had variables getting passed in, but I did not check to see how they were getting passed. Now I have an error when trying to use ENV variables in a dbt metadata file.

Bianca Hoch

08/12/2022, 9:34 PM

Can you provide and example of how you're using the --env call? Also, here's an article that outlines how to use --env for setting secrets, which is done in a similar fashion.

Bianca Hoch

08/12/2022, 9:37 PM

Also the syntax in the yaml for the agent deployment will appear something like this:

Copy code

- name: PREFECT__CLOUD__AGENT__ENV_VARS
          value: '{"test": "test2"}'

5 Views

Open in Slack

Previous Next