Join Slack

<@ULVA73B9P> When serving a flow (ex my_flow.serve(…)), where is the best point to define any ENV co...

# ask-marvin

Blake

05/18/2024, 9:44 PM

@Marvin When serving a flow (ex my_flow.serve(…)), where is the best point to define any ENV configs?

Marvin

05/18/2024, 9:44 PM

The best point to define any environment configurations for a flow such as when using

my_flow.serve(…)

is in the deployment YAML or when using the

.deploy()

method. You can set environment variables in the

job_variables

configuration block which will be used by the flow run. Here's how: 1. In the deployment YAML: You can define environment variables using the

{{ $ENV_VAR_NAME }}

syntax within the

job_variables

block of your

prefect.yaml

file. Below is an example:

Copy code

yaml
deployments:
- name: demo-deployment
  entrypoint: demo_project/demo_flow.py:some_work
  work_pool:
    name: local
  job_variables:
    env:
      EXECUTION_ENVIRONMENT: "{{ $EXECUTION_ENVIRONMENT }}"
      MY_NOT_SO_SECRET_CONFIG: "{{ $MY_NOT_SO_SECRET_CONFIG }}"
  schedule: null

The environment variables would typically be set on the machine where

prefect deploy

is executed. 2. Using the
.deploy()
method: If you're using the

.deploy()

method for deploying your flow, you can pass the environment variables as a dictionary to the

job_variables

argument. For example:

Copy code

python
if __name__ == "__main__":
    flow.from_source(
        source="<https://github.com/zzstoatzz/prefect-monorepo.git>",
        entrypoint="src/demo_project/demo_flow.py:some_work"
    ).deploy(
        name="demo-deployment",
        work_pool_name="local",
        job_variables={
          'env': {
            'EXECUTION_ENVIRONMENT': 'staging',
            'MY_NOT_SO_SECRET_CONFIG': 'plumbus'
          }
        }
    )

Ensure you have these environment variables set before running the deployment command. Read more about overriding job variables in the Prefect documentation.

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.