Is it possible to run a docker image with differen...
# ask-community
j
Is it possible to run a docker image with different environment variables in the container as specified by a Flow parameter? Locally, I can do this via
docker run -env FOO=BAR <my_docker_image>
I would like to be able to execute a flow on this image with a Docker Agent by calling the flow with the parameters
Copy code
{
  "FOO": "BAR"
}
Is this possible with Prefect? Or is there a Prefect idiom for such a concept? Use cases for changing environment variables on container create/flow execution would be 1. defining which customer code path we want to trigger (database, configs, special methods and permissions) 2. Setting how we want to persist data (which database type to use, how to cache)
z
Hey Josh, you can set the
RunConfig
per flow run and it has an environment variables section.
k
It would be
DockerRun(...,env={"FOO":"BAR"})
j
But I wouldn’t be able to change that on a per-flow basis by setting a parameter would I? The flows would be forced to have one permanent environment variable
z
You can change the run config per flow run the same way you'd change parameters
j
For instance if I wanted to have a
run_models
flow and I wanted to set the databases used based on the environment variable, I wouldn’t be able to do that with the Prefect Flow Parameters would I?
How do I do that?
z
How are you creating your flow runs?
j
via schedule or UI
z
The UI has a run config menu while creating a flow run
j
Got it. I see it now in the “advanced” section
Are there docs you could point me to for how to do it via
Schedule
or by calling the API for ad-hoc runs?
z
I'm not sure we actually provide it via schedules yet
The
Client.create_flow_run
method takes a
run_config
https://docs.prefect.io/api/latest/client/client.html#client-2
If you're calling GraphQL directly, it's just a JSON blob. You can see an example of the blob by calling
run_config.serialize()
Is there a reason you're not just toggling your settings based on the parameter values directly instead of environment variables?
j
I’m calling some methods of a legacy service that is configured via ENV variables. Was originally built to be a web-server and has logic in there we would like to call ad-hoc via Prefect
z
Hmm.. You could inject things into
os.environ
based on parameters if it's a Python service.
🤔 1