Short answer: yes. In Prefect 3.x,

prefect.yaml

supports placeholder-based templating so you can inject values at deploy time. You can source values from: - Environment variables (via your shell or a local

.env

file) - Prefect Variables (workspace-level, set in the UI or via API) - Block references (for secrets/credentials you’ve saved as Blocks) Key points: - Placeholders are written with double curly braces and are resolved by Prefect when you run

prefect deploy

. - The most common pattern is to feed values via env vars (export them or put them in a

.env

file next to your project) and reference them in

prefect.yaml

. - There is no special

--var

style flag on

prefect deploy

to inject arbitrary variables; use env vars, Prefect Variables, or Blocks instead. Example

name: my-project
prefect-version: ">=3.0.0"

deployments:
  - name: my-deployment
    entrypoint: flows/my_flow.py:flow
    work_pool:
      # Inject a work pool name from an environment variable
      name: "{{ env.WORK_POOL_NAME }}"
    # Inject a schedule from an env var too
    schedule:
      cron: "{{ env.DEPLOY_CRON }}"
    # Use Prefect Variables for non-secret config
    parameters:
      source_table: "{{ variables.SOURCE_TABLE }}"
    # Example: pass job variables (consumed by your work pool’s base job template)
    job_variables:
      image: "<http://ghcr.io/acme/myflow:{{|ghcr.io/acme/myflow:{{> env.IMAGE_TAG }}"
      # For secrets/credentials, prefer a Block reference instead of plaintext
      # e.g. a saved Secret block—see docs for the exact reference syntax
      # db_password: "{{ block('system.secret', 'my-db-password').value }}"

How to provide values at deploy time - Export env vars in your shell and run deploy:

WORK_POOL_NAME=prod-pool DEPLOY_CRON="0 2 * * *" IMAGE_TAG=v1.2.3 prefect deploy

- Or create a

.env

file in your project directory:

WORK_POOL_NAME=prod-pool
  DEPLOY_CRON=0 2 * * *
  IMAGE_TAG=v1.2.3

Then run `prefect deploy`; Prefect will read env vars from the environment (and

.env

for settings), and placeholders will resolve. Docs to reference - prefect.yaml guide - Customize job variables - Settings, profiles, and .env files If you can share what you’re trying to parameterize (e.g., image tags, pool names, schedules, secrets), I can suggest an exact

prefect.yaml

snippet tailored to your setup.

You’re very close — the placeholder syntax in Prefect 3 for environment variables is not

{{ env.VAR }}

. Use

{{ $VAR }}

. Update your prefect.yaml like this:

work_pool:
  name: "{{ $WORK_POOL_NAME }}"

Then deploy with an environment variable set:

WORK_POOL_NAME=toto uv run prefect deploy

Two more important notes: - The work pool must already exist in your workspace. If

toto

doesn’t exist yet, create it first (via UI or CLI), e.g.:

prefect work-pool create toto --type process

- The interpolation happens at deploy time. If you still see the literal

{{ $WORK_POOL_NAME }}

in the error, the env var wasn’t present in that process. If you prefer centrally managed values instead of shell env vars, you can also reference Prefect Variables:

work_pool:
  name: "{{ prefect.variables.WORK_POOL_NAME }}"

Then set the variable in your Prefect workspace (UI or API). This is handy when you reuse the same

prefect.yaml

across environments. Docs: - prefect.yaml guide - Customize job variables If you share the other fields you want to parameterize (e.g., image tags, schedules, secrets), I can suggest exact placeholders for those too.

Yes — you can use environment variables in pull steps. A couple of specifics: - Use the env var placeholder syntax with a leading dollar:

{{ $MY_ENV_VAR }}

(not

{{ env.MY_ENV_VAR }}

). - Pull steps run on the worker at flow runtime, so those placeholders resolve against the worker’s environment. Make sure the env var is set where the worker runs. Your example adjusted:

pull:
  - *git_configuration

  - prefect.deployments.steps.git_clone:
      id: clone-step
      repository: git@github.com:my-org/prefect-pipelines.git
      commit_sha: "{{ $GIT_COMMIT_SHA_IUEP_PIPELINE }}"

  - prefect.deployments.steps.set_working_directory:
      directory: "{{ clone-step.directory }}"

  - *install_dependencies

Notes: -

{{ clone-step.directory }}

is valid: step outputs can be referenced by

{{ <step-id>.<key> }}

in later steps. - Ensure

GIT_COMMIT_SHA_IUEP_PIPELINE

is present in the worker’s environment (e.g., export before starting a Process worker, or add it to your work pool’s base job template/env via the UI if you’re using Docker/Kubernetes). Related docs: - prefect.yaml guide - Customize job variables If you share how your worker is running (process, Docker, Kubernetes), I can suggest the cleanest place to set that env var so it’s available during pull.

Great questions — here’s how redeploys work in Prefect 3.x: What happens if you run prefect deploy again for the same deployment? - It upserts. If a deployment with the same flow name + deployment name already exists,

prefect deploy

updates it; otherwise it creates it. - Any fields you changed in

prefect.yaml

(or via env vars/variables/placeholders) are applied to the existing deployment: parameters, work_pool, storage/source, schedules, tags, job_variables, etc. - If you change the deployment name, you’ll create a new deployment (the old one remains). If you redeploy with different values - The new values are saved to the deployment. Future runs will use the updated config. - Example: change the schedule → future auto-scheduled runs will follow the new schedule. Already-created runs are not retroactively changed. - Example: change the work pool on the deployment → future runs will target the new pool; existing scheduled runs will still target whatever pool they were created with. What about work pools? - Work pools are separate resources;

prefect deploy

doesn’t create or modify them. - If you point a deployment at a pool that doesn’t exist, Prefect warns and no workers will pick up runs until you create the pool. - Creating/updating pools is idempotent:

prefect work-pool create my-pool --type process

If

my-pool

exists, this updates it (e.g., with a new base job template when provided). See

prefect work-pool create --help

. Tips for reusing the same prefect.yaml with different values - Keep placeholders in

prefect.yaml

and inject values at deploy time via environment variables (e.g.,

{{ $WORK_POOL_NAME }}

,

{{ $IMAGE_TAG }}

) or Prefect Variables/Blocks. - Re-running

prefect deploy

with different env vars will update the deployment with those resolved values. Docs - prefect.yaml guide - Customize job variables If you want, share what you plan to change between environments (pool, schedule, image, git commit, etc.), and I can suggest a pattern to make your

prefect.yaml

reusable and safe.

Short answer: to route runs to a client’s own infrastructure (i.e., their work pool), each deployment must be bound to that client’s pool. So, with 3 flows and N clients, the common pattern is 3 × N deployments. Why: in Prefect 3.x, a deployment has a fixed

work_pool.name

. Workers only poll their pool. There isn’t a runtime flag to switch a run to another pool; job variables/parameters can change behavior inside a run but cannot change which pool executes it. Practical approaches to manage this cleanly - Reuse one prefect.yaml with placeholders and deploy per client

deployments:
    - name: "flow-a-{{ $CLIENT }}"
      entrypoint: flows/flow_a.py:flow_a
      work_pool:
        name: "client-{{ $CLIENT }}-pool"

    - name: "flow-b-{{ $CLIENT }}"
      entrypoint: flows/flow_b.py:flow_b
      work_pool:
        name: "client-{{ $CLIENT }}-pool"

    - name: "flow-c-{{ $CLIENT }}"
      entrypoint: flows/flow_c.py:flow_c
      work_pool:
        name: "client-{{ $CLIENT }}-pool"

Then for each client:

# ensure the client pool exists
  CLIENT=acme prefect work-pool create "client-${CLIENT}-pool" --type process

  # deploy all three flows for that client
  CLIENT=acme prefect deploy

Repeat with

CLIENT=beta

, etc. This gives one deployment per flow per client, all sharing the same codebase. - Programmatic deployment loop (helpful if clients change frequently)

from prefect import flow
  clients = ["acme", "beta", "gamma"]

  for c in clients:
      flow.from_source(
          "<https://github.com/your-org/your-repo>",
          entrypoint="flows/flow_a.py:flow_a",
      ).deploy(
          name=f"flow-a-{c}",
          work_pool_name=f"client-{c}-pool",
          parameters={"client": c},
      )
      # repeat for flow_b, flow_c

This upserts deployments so you can rerun it as clients are added/removed. What about work pools? - You’ll need one pool per client and at least one worker per pool, running in the client’s infrastructure. - Creating/updating pools is idempotent:

prefect work-pool create client-acme-pool --type process

If it already exists, this updates it when you provide new options (e.g., a new base job template for Docker/Kubernetes pools). - A worker watches exactly one pool; if you have many client pools, run one worker per pool (or more for scale/HA). Other notes - Re-deploying the same deployment name updates it (applies new schedule, parameters, job variables, storage/source, and pool), it does not create a duplicate. - You can use additional structure like work queues within a pool, but queues do not let a single deployment span multiple pools. Docs - Prefect YAML and templating: prefect.yaml guide - Job variables (useful for per-client runtime config inside the same pool): Customize job variables If you share your worker type (process, Docker, Kubernetes) and how many clients you’re targeting, I can sketch an exact YAML + script to generate all deployments and pools automatically.

Short answer: don’t write to /root. Use a writable path like the working directory, $HOME, or /tmp — and if you need persistence in containers, mount a volume and write there. Details - Pull steps run on the worker. If your worker is a container (Docker/Kubernetes), it often runs as a non-root user and/or with a read-only root filesystem. That’s why mkdir under /root fails. - Prefer: - The directory returned by your git_clone step (and set via set_working_directory) - $HOME (the user’s home dir) — note: $USER is a username, not a directory; use $HOME - /tmp for ephemeral scratch space - A mounted volume path (Docker/Kubernetes) if you need persistence across runs Examples - Use the git clone output and create a subfolder relative to it:

pull:
  - prefect.deployments.steps.git_clone:
      id: clone
      repository: git@github.com:my-org/prefect-pipelines.git
      commit_sha: "{{ $GIT_COMMIT_SHA_IUEP_PIPELINE }}"
  - prefect.deployments.steps.set_working_directory:
      directory: "{{ clone.directory }}"
  - prefect.deployments.steps.utility.run_shell_script:
      script: |
        mkdir -p data/cache
        echo "Ready in $(pwd)"

- Use /tmp explicitly:

- prefect.deployments.steps.utility.run_shell_script:
    script: |
      mkdir -p /tmp/my-dir

- Use $HOME (more reliable than $USER):

- prefect.deployments.steps.utility.run_shell_script:
    script: |
      mkdir -p "$HOME/my-dir"

If you’re on Docker/Kubernetes and need a guaranteed writable/persistent path: - Docker work pool: mount a volume and set a working dir via job variables

deployments:
  - name: my-deployment
    work_pool:
      name: my-docker-pool
    job_variables:
      volumes:
        - "myvolume:/workspace"   # docker volume named 'myvolume'
      working_dir: "/workspace"

- Kubernetes work pool: add an emptyDir or PVC in the pool’s base job template and mount it at e.g. /workspace, then set working_dir to /workspace. Your pull steps can write there. If you tell me which work pool type you’re using (process, Docker, Kubernetes) I can give you an exact snippet for a writable working directory in your environment. Docs: - prefect.yaml guide - Customize job variables

You’re hitting two issues at once: 1) Your script isn’t expanding $HOME - The

run_shell_script

step does not expand environment variables by default. Set

expand_env_vars: true

, or use Prefect’s templating

{{ $HOME }}

. - Also, don’t prefix with a literal slash and a literal

$HOME

in the string that’s not being expanded; use

$HOME

(no leading slash required), or

{{ $HOME }}

via Prefect templating. 2) Your root filesystem is read-only - Many Docker/Kubernetes base job templates run with a read-only root filesystem, so writing to

/root

or even some home paths may fail unless you mount a writable volume and/or set a writable working directory. Recommended fixes Option A: Expand env vars and write to $HOME (if $HOME is writable)

- prefect.deployments.steps.utility.run_shell_script:
    expand_env_vars: true
    script: |
      mkdir -p "$HOME/.ssh"
      echo "OK: $HOME/.ssh"

Or using Prefect templating:

- prefect.deployments.steps.utility.run_shell_script:
    script: |
      mkdir -p "{{ $HOME }}/.ssh"

If this still fails with “read-only file system”, your $HOME is on the read-only root. In that case, use Option B or C. Option B: Use a writable mount/working directory and avoid $HOME entirely - For Docker or Kubernetes pools, mount a volume and set

working_dir

via job variables, then place keys in that mount. - Docker example job variables:

job_variables:
      volumes:
        - "myvolume:/workspace"
      working_dir: "/workspace"

- Then:

- prefect.deployments.steps.utility.run_shell_script:
        script: |
          mkdir -p /workspace/.ssh
          # use GIT_SSH_COMMAND to point git at your key in /workspace/.ssh

Option C: Avoid writing SSH keys by using git_clone credentials - The

git_clone

step accepts credentials so you don’t need ~/.ssh: - Use a token over HTTPS:

- prefect.deployments.steps.git_clone:
        repository: <https://github.com/my-org/prefect-pipelines.git>
        access_token: "{{ prefect.variables.GH_TOKEN }}"

Set

GH_TOKEN

as a Prefect Variable or env var. - Or provide a credentials Block (e.g., a Git credentials block) and reference it in

credentials:

so the step handles auth without writing to ~/.ssh. Why your current command failed - You used:

mkdir -p /$HOME/.ssh

and saw:

mkdir: /$HOME: Read-only file system

Because

expand_env_vars

was false,

$HOME

wasn’t expanded, so it literally tried to create “/$HOME/.ssh” on the root filesystem, which is read-only. If you share which work pool type you’re using (process, Docker, Kubernetes) I can send an exact snippet to ensure a writable path for keys in your environment. Docs - Prefect YAML and steps: prefect.yaml guide - Job variables (set working dir, mounts, env): Customize job variables