Hi, does anyone know if the env vars set on the prefect agent (deployed in k8s) get injected into a dask cluster automatically? Or do we also have to duplicate them when making the pod spec for dask? Our executor is defined like:
Hi @Kevin Kho, we’re using the cloud plus an agent in our GKE cluster
k
Kevin Kho
09/02/2021, 4:55 PM
I went through the code and I don’t think they are injected. What env variables these?
b
Brad I
09/02/2021, 6:13 PM
I’m talking about any env vars that are found when you run a flow, under ‘Advanced’ -> ‘Environment Variables’. We set a few of them when we register our flows, but now are trying to move from
LocalDaskExecutor
to the
DaskExecutor
.
k
Kevin Kho
09/02/2021, 6:46 PM
So env vars are not carried over but Prefect specific env vars are carried over through the context.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.