When running prefect docker agents it seems that the a running docker container does not automatically inherit the context from the agent. Is that intended behavior? How best to set the context inside the docker container?

@Florian Kühnlenz by context, do you mean environment variables, or what other settings haven’t been inherited from the agent?

I mean the prefect context.

@Florian Kühnlenz you don’t have to do anything to explicitly set Prefect context. Prefect context is a thread-safe object created at flow runtime to store flow’s and task’s specific information such as flow name, task name, parameters, scheduled start time, etc. Afaik it doesn’t inherit any information from the agent other than the configuration from config.toml 1. Is config.toml what you are referring to? 2. If so, what information isn’t inherited correctly from the agent? 3. Did you set that information before starting the agent?

Yes. We set the value in the config.toml and wanted to use it in the container that is started by the agent. We restarted the agent after setting the value. We could not see get the value inside the container, and has to resort to setting it via an environment variable passed in, when starting the agent.

@Florian Kühnlenz hmm interesting. How did you set your configuration and how do you access it within the flow?

We set it by editing the config.toml and access it via import prefect and then using prefect.context.get. One thing I can try to check is if the agent is looking in the right place for the config. So just the be sure the context inside the container should have all the values the docker agent sees in the config.toml?

@Florian Kühnlenz can you share the syntax that you use? I think you may need to change the syntax like so:

prefect.context.config.get("your_config_variable_name")

or:

prefect.config.get("your_config_variable_name")

Aha! I will try this. We put the variable inside the context section of the toml.

@Florian Kühnlenz I see, I thought you are setting it directly in the config.toml. Are you on Prefect Cloud? If so, there is actually a much more convenient way of storing key value pairs - the KV Store, it’s described here: https://docs.prefect.io/orchestration/concepts/kv_store.html#kv-store

We are on cloud. But we would like to have a value that tells us wether we are local, in ci or in prod. Not sure if the kv Store is the right solution for this.

I think it could be if you are on a multi-tenant infrastructure, but this is possible only if you are on an Enterprise Cloud plan. Many users who don’t have Enterprise use projects to differentiate between dev and prod.

That was a misunderstanding, we only want prefect cloud to be prod. But we need a way to detect when a flow is executed there.

Got it - just if you are interested, we do offer multi-tenant options that would allow you to have both dev and prod in Prefect Cloud - if you would like to give it a try, you can write to sales@prefect.io. To differentiate between backend-supported flow run vs. local run, there should be a flag in the context telling whether it is triggered from a backend or just running with Prefect Core, have a look at this thread: https://prefect-community.slack.com/archives/CL09KU1K7/p1637074773202100?thread_ts=1637058721.182400&cid=CL09KU1K7