Hi folks. We’re still trying to move to github sto...
# ask-community
b
Hi folks. We’re still trying to move to github storage from Docker, but have run into a new snag. It seems like our toml config files are no longer being picked up. We’ve always configured them at registration time as
PREFECT__USER_CONFIG_PATH=prod.toml
but
prefect.config
no longer seems to have the values from the toml file. Any idea how we can work around this?
k
Hi @Brian Mesick, did you store the toml inside the Docker image?
b
It’s unclear to me where we’d do that. We’ve been using the default Prefect images not building our own dockerfiles. Is it not possible to pull it in alongside the flow script?
k
Oh I was just asking where the tomy config files live. Are they on the agent already?
b
The live in github alongside the flow python files.
z
Hey @Brian Mesick -- I think current best practice here would be to • Set all the config values in the
RunConfig
via environment variables or • Copy your config into a docker image based on the prefect base image then specify that as your image via a
DockerRun
config
We're exploring support for copying things alongside your flow script in the
Git
storage class (distinct from
GitHub
) but we're not sure what the best way to make that work with all of the different execution environments is
b
Yeah, that’s pretty rough since we went to this specifically to avoid having to make the image changes so frequently and this loses the ability to configure things per-environment without re-baking the image. On the upside we’re not currently doing that anyway so we can probably set these things as env vars in the
KubernetesRun
z
Hmm. I will set a reminder for myself to dig into this later today if I have time or tomorrow and I'll see if there's a good way to handle this.
Re the run config env var method though: you can probably generate the environment variables from your prod.toml file pretty easily as well so you can still spec them out that way.
b
Thanks Michael
z
Hey Brian, I haven't had time to look into this yet and I'm going to open an issue on Github so it doesn't get forgotten.
👍 1
Internally, we use environment variables so I think that's probably the most feasible way for now. @Marvin open "Document recommended pattern to pass Prefect config via RunConfig"