Any idea why my environment variables are missing ...
# prefect-communitya
Adam Roderick
07/10/2020, 12:49 PMAny idea why my environment variables are missing during a pipeline execution? I'm using Docker storage, and locally I can spin up a container and confirm that the env vars are indeed there
n
nicholas
07/10/2020, 1:24 PM
Hi @Adam Roderick - can you give a little more info about your setup? Are these Prefect-prefixed environment variables?
a
Adam Roderick
07/10/2020, 2:06 PMNo, they are the dictionary I pass into Docker storage
n
nicholas
07/10/2020, 2:48 PM
Got it @Adam Roderick - I've not tried it personally, but I'd imagine those variables still need to be prefixed with
PREFECT__a
Adam Roderick
07/10/2020, 2:50 PMNo, these are variables that are injected into the docker image. It was working until this morning when I redeployed, and I'm not sure what is different
Adam Roderick
07/10/2020, 2:51 PMTrying to track it down. The build step even shows that they are applied,
Step 21/22 : ENV DBT_EXECUTABLE=/datateer/venv/dbt/bin/dbt DBT_PROFILES_DIR=...printenvn
nicholas
07/10/2020, 2:52 PM
Interesting, let me ping someone on this
a
Adam Roderick
07/10/2020, 2:56 PMThanks, I'll keep looking too
👍 1
j
josh
07/10/2020, 3:00 PM@Adam Roderick how are you attempting to access the env vars during your flow run?
a
Adam Roderick
07/10/2020, 3:02 PMActually I'm accessing them at build time... that might be the issue.
os.environ['MELTANO_EXECUTABLE']j
josh
07/10/2020, 3:03 PMYeah there’s a chance they aren’t set during build time. Where exactly during the build are you attempting to retrieve them?
a
Adam Roderick
07/10/2020, 3:04 PMin the constructor of a custom Task
Adam Roderick
07/10/2020, 7:03 PMI think it's an order of operations thing. Thanks for checking on it with me
👍 1
2 Views