Thread
#prefect-community
    Pedro Machado

    Pedro Machado

    1 year ago
    What is the best way to pass several environment variables to a docker agent? I have them in a
    .env
    file but I don't see a way to pass this file in the cli. Alternatively, how do I tell the agent to use my
    config.toml
    ? I have one at
    ~/.prefect/config.toml
    that is used when using
    flow.run()
    Can I tell the agent running on the same machine to use this config file?
    nicholas

    nicholas

    1 year ago
    Hi @Pedro Machado ! You can pass environment variables to the agent from the CLI using the
    -e
    or
    --env
    flags as detailed here in the docs.
    Pedro Machado

    Pedro Machado

    1 year ago
    I have 12 of them. I was wondering if there was a way to pass them via a file. What about the context approach? How can I pass context to the docker agent?
    nicholas

    nicholas

    1 year ago
    Hm I don't think they're passable via a file but perhaps context is a good way to go because you can read the variables you'd like into the context at build time
    Pedro Machado

    Pedro Machado

    1 year ago
    Is it possible to set the context in a config.toml for the agent to use?
    nicholas

    nicholas

    1 year ago
    So I think it's important to note the agent's purpose in Prefect, which is as an orchestrator/manager for disparate, independent workflows. As such, it usually doesn't make sense to pass flow-specific context to an agent, but instead to define them on the flow's context or environment directly. For example, you can build a flow with docker storage that contains and
    env_vars
    dict in which you can pass your desired environment variables from the build context
    Pedro Machado

    Pedro Machado

    1 year ago
    I see what you are saying. I could pass
    env_vars
    at build time for flow-specific configuration. What do you recommend for things like db credentials? I am currently passing them to the agent as env variables and reading them via
    EnvVarSecret
    . Is this the optimal way if we don't have access to a secret store? Do you have any recommended patterns to set up the code so that it runs locally with
    flow.run()
    and in a docker container? Unfortunately, we'll have to use the local file system for results and output and the paths are different than in the local environment. I got it to work using env variables like
    PREFECT_BASE_DIR
    which points to the directory where the flows to read some sql templates and
    PREFECT__CONTEXT__LOCAL_RESULTS_DIR
    to specify where the results will be written. Does this sound reasonable or can you think of a better approach?
    nicholas

    nicholas

    1 year ago
    Definitely @Pedro Machado -
    EnvVarSecret
    is the way to go in that case. You can always set environment variables during your docker's build step as well.
    flow.run
    will run in your local process so for that to happen you'd need to call that from within the container itself. Otherwise that sounds like a viable approach