<@ULVA73B9P> how to ensure that .env file is loade...
# ask-marvin
i
@Marvin how to ensure that .env file is loaded in deployments?
m
thought for 278.4 seconds
Short answer: Prefect does not automatically load your app’s
.env
into deployment runs. It only auto-loads
.env
for
PREFECT_*
settings. To ensure your own variables are present at runtime, pass them via job variables or start your workers with the
.env
loaded. How to do it, depending on your setup: General (deploy-time templating in prefect.yaml) - Use
{{ $VAR_NAME }}
to pull from your shell environment when you run
prefect deploy
. - Make sure your
.env
is loaded into the shell first. Example prefect.yaml:
Copy code
deployments:
  - name: my-deployment
    entrypoint: flows.py:my_flow
    work_pool:
      name: my-pool
      job_variables:
        env:
          DATABASE_URL: "{{ $DATABASE_URL }}"
          API_KEY: "{{ $API_KEY }}"
Run:
Copy code
set -a && source .env && set +a
prefect deploy -n my-deployment
Docs: - Define deployments with YAML (templating) - Customize job variables Process worker (local subprocess) - Process workers inherit the worker process environment. Start the worker with your
.env
loaded:
Copy code
set -a && source .env && set +a
prefect worker start --pool my-process-pool
- Alternatively, if you have a Python entrypoint to start the worker, call
python-dotenv
before starting it:
Copy code
from dotenv import load_dotenv
load_dotenv()
# then start the worker or call flow.run()
Docker worker - Pass environment via
job_variables.env
(either in prefect.yaml or via
flow.deploy(...)
). - Load
.env
before deploying so templated
{{ $VAR }}
values resolve, or set the env directly in your shell.
Copy code
deployments:
  - name: my-deployment
    entrypoint: flows.py:my_flow
    work_pool:
      name: my-docker-pool
      job_variables:
        env:
          DATABASE_URL: "{{ $DATABASE_URL }}"
          LOG_LEVEL: "{{ $LOG_LEVEL }}"
Kubernetes worker - Same pattern with
job_variables.env
. You can use simple key/value or full Kubernetes
valueFrom
for Secrets/ConfigMaps.
Copy code
deployments:
  - name: my-deployment
    entrypoint: flows.py:my_flow
    work_pool:
      name: my-k8s-pool
      job_variables:
        env:
          - name: DATABASE_URL
            valueFrom:
              secretKeyRef:
                name: app-secrets
                key: database-url
          - name: LOG_LEVEL
            value: "{{ $LOG_LEVEL }}"
Deploy via Python (flow.deploy) - Load
.env
in the deploy script, then pass values into `job_variables.env`:
Copy code
from dotenv import load_dotenv
load_dotenv()

from prefect import flow
import os

@flow
def my_flow():
    pass

if __name__ == "__main__":
    my_flow.deploy(
        name="my-deployment",
        work_pool_name="my-pool",
        job_variables={
            "env": {
                "DATABASE_URL": os.environ.get("DATABASE_URL"),
                "API_KEY": os.environ.get("API_KEY"),
            }
        }
    )
Important notes - Prefect auto-loads
.env
only for Prefect settings (
PREFECT_*
). Your app env vars must be provided as above. - Values templated into
prefect.yaml
get stored with the deployment. For secrets, prefer K8s Secrets, a secrets manager, or Prefect Blocks instead of hardcoding. - To simply “inherit” your local
.env
, use the Process worker and start it with
.env
loaded; for Docker/K8s you must configure environment in the job variables or the underlying platform (Secrets/ConfigMaps). If you share which worker type you’re using, I can tailor an example to your exact setup.
i
@Marvin How would I specify a given PYTHONPATH to some deployments?
m
thought for 33.0 seconds
You set it as an environment variable on the deployment. In Prefect 3.x, put
PYTHONPATH
under
work_pool.job_variables.env
for just the deployments you want, or set it at the work-pool default if you want it for all runs. Options: - Per-deployment in prefect.yaml
Copy code
deployments:
  - name: ingest
    entrypoint: flows/ingest.py:ingest
    work_pool:
      name: docker-pool
      job_variables:
        env:
          PYTHONPATH: "/opt/shared:/app/src"   # Linux/macOS (use ; on Windows)
  - name: transform
    entrypoint: flows/transform.py:transform
    work_pool:
      name: k8s-pool
      job_variables:
        env:
          PYTHONPATH: "{{ $PYTHONPATH }}"      # pull from your shell at deploy time
- Per-deployment via Python
Copy code
from prefect import flow

@flow
def my_flow():
    ...

my_flow.deploy(
    name="with-pythonpath",
    work_pool_name="my-pool",
    job_variables={"env": {"PYTHONPATH": "/repo/libs:/repo/app"}}
)
- Process worker (local subprocess) - If you want all runs handled by a single worker to inherit it:
Copy code
PYTHONPATH=/path1:/path2 prefect worker start --pool my-process-pool
- For only some deployments, prefer setting it in
job_variables.env
as above. - Docker worker - Either set via deployment
job_variables.env
(above), or bake it into the image:
Copy code
# Dockerfile
ENV PYTHONPATH="/opt/shared:/app/src:${PYTHONPATH}"
- Kubernetes worker - Simple value:
Copy code
work_pool:
  name: my-k8s-pool
  job_variables:
    env:
      PYTHONPATH: "/mnt/shared:/app"
- Or full K8s-style list:
Copy code
job_variables:
  env:
    - name: PYTHONPATH
      value: "/mnt/shared:/app"
Notes - Use
:
on Linux/macOS and
;
on Windows for
PYTHONPATH
. - If you want to source from a
.env
, load it into your shell first, then use
{{ $PYTHONPATH }}
in `prefect.yaml`:
Copy code
set -a && source .env && set +a
prefect deploy -n my-deployment
Docs - Customize job variables - Define deployments with YAML (templating)