Great question. In Prefect 3.x, the simplest and most reliable pattern is: - Define one deployment per environment (dev, stage, prod) in a single prefect.yaml - In CI, pick the deployment by name based on the Git branch - Optionally interpolate env vars in prefect.yaml for small per-branch tweaks (e.g., git branch in a pull step) Below is a concrete example you can copy. Example prefect.yaml (multiple environments)

name: my-project
prefect-version: 3.0.0

# Reusable bits (YAML anchors)
definitions:
  actions:
    pull_from_git: &pull_from_git
      - prefect.deployments.steps.git_clone:
          id: clone
          repository: <https://github.com/your-org/your-repo.git>
          # Let CI set DEPLOY_BRANCH; falls back to 'main'
          branch: "{{ $DEPLOY_BRANCH | default('main') }}"

  # Optionally define your work pool config per env (ECS EC2 in your case)
  # Tip: keep sensitive values in Prefect Blocks or CI secrets, not in YAML
  work_pools:
    ecs_dev: &ecs_dev
      name: ecs-ec2-dev
      # job_variables here if you need them per env (image, env, etc.)
      job_variables:
        env:
          APP_ENV: dev
    ecs_stage: &ecs_stage
      name: ecs-ec2-stage
      job_variables:
        env:
          APP_ENV: stage
    ecs_prod: &ecs_prod
      name: ecs-ec2-prod
      job_variables:
        env:
          APP_ENV: prod

deployments:
  - name: my-flow/dev
    entrypoint: flows/my_flow.py:my_flow
    pull:
      steps: *pull_from_git
    work_pool: *ecs_dev

  - name: my-flow/stage
    entrypoint: flows/my_flow.py:my_flow
    pull:
      steps: *pull_from_git
    work_pool: *ecs_stage

  - name: my-flow/prod
    entrypoint: flows/my_flow.py:my_flow
    pull:
      steps: *pull_from_git
    work_pool: *ecs_prod

How to select the environment in CI based on branch - Use branch conditionals to decide: - Which Prefect workspace (API URL + API key) you deploy to - Which deployment name to deploy (dev/stage/prod) - Which branch gets substituted into the pull step (via DEPLOY_BRANCH) Example GitHub Actions snippet

name: Deploy flows

on:
  push:
    branches: [ main, develop, stage ]

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      # Used by prefect.yaml pull step
      DEPLOY_BRANCH: ${{ github.ref_name }}
    steps:
      - uses: actions/checkout@v4

      - name: Set environment (workspace + target deployment)
        id: select
        run: |
          if [[ "${GITHUB_REF_NAME}" == "main" ]]; then
            echo "TARGET_DEPLOY=my-flow/prod" >> $GITHUB_OUTPUT
            echo "PREFECT_API_URL=${{ secrets.PREFECT_API_URL_PROD }}" >> $GITHUB_ENV
            echo "PREFECT_API_KEY=${{ secrets.PREFECT_API_KEY_PROD }}" >> $GITHUB_ENV
          elif [[ "${GITHUB_REF_NAME}" == "stage" ]]; then
            echo "TARGET_DEPLOY=my-flow/stage" >> $GITHUB_OUTPUT
            echo "PREFECT_API_URL=${{ secrets.PREFECT_API_URL_STAGE }}" >> $GITHUB_ENV
            echo "PREFECT_API_KEY=${{ secrets.PREFECT_API_KEY_STAGE }}" >> $GITHUB_ENV
          else
            echo "TARGET_DEPLOY=my-flow/dev" >> $GITHUB_OUTPUT
            echo "PREFECT_API_URL=${{ secrets.PREFECT_API_URL_DEV }}" >> $GITHUB_ENV
            echo "PREFECT_API_KEY=${{ secrets.PREFECT_API_KEY_DEV }}" >> $GITHUB_ENV
          fi

      - name: Install Prefect (and AWS extra if needed)
        run: |
          pip install -U "prefect[aws]"

      - name: Deploy selected environment
        run: |
          prefect deploy --no-prompt --name "${{ steps.select.outputs.TARGET_DEPLOY }}"

Notes and tips - You can use wildcards if you prefer a naming convention, e.g.: -

prefect deploy --no-prompt --name "*-prod"

- Interpolation in prefect.yaml: - Environment variables:

{{ $ENV_VAR }}

(we used

DEPLOY_BRANCH

) - Prefect Variables:

{{ prefect.variables.my_var }}

- Prefect Blocks:

{{ prefect.blocks.block_type.block_slug }}

- Prefer separate Prefect Cloud workspaces per env (dev/stage/prod) and inject

PREFECT_API_URL

and

PREFECT_API_KEY

from CI secrets. - For private repos in pull steps, use a credentials Block (e.g., GitHub/GitLab) instead of plain tokens. - Since you’re using ECS with launch type EC2, keep one ECS work pool per env (e.g.,

ecs-ec2-dev

,

ecs-ec2-stage

,

ecs-ec2-prod

) and point each deployment to the right pool. You can keep small CPU/memory defaults now and tune later. Docs for deeper reference - Prefect YAML (structure, templating, steps): prefect.yaml guide - CI/CD patterns: Deploy with CI/CD - Variables and Blocks: Variables and Blocks - Work pools: Manage work pools A couple of quick questions to tailor this: - Are you deploying to separate Prefect Cloud workspaces for dev/stage/prod, or one workspace with tags? - Which CI system are you using (GitHub Actions, GitLab, etc.)? - Do you want your ECS tasks to have public IPs in any env? I can suggest sensible defaults per env.

Got it. You want to switch everything that’s hard-coded to “dev” (image tags, deployment name/tags, env vars) to “dev/stage/prod” automatically based on the Bitbucket branch. There are two clean patterns. I’ll show both, then a Bitbucket Pipelines snippet. Option A (recommended): 3 stable deployments (dev, stage, prod) and pick one in CI - Keeps deployment objects stable - Easiest to reason about - CI sets DEPLOY_ENV and deploys only the matching deployment Updated prefect.yaml (highlights: use {{ $DEPLOY_ENV }} for image tags; define 3 deployments)

name: src
prefect-version: 3.4.11

build:
  - prefect.deployments.steps.utility.run_shell_script:
      id: git_sha
      script: git rev-parse --short HEAD

  - prefect_docker.deployments.steps.build_docker_image:
      id: build_image
      requires: prefect-docker>=0.3.1
      image_name: <http://010614160260.dkr.ecr.ap-southeast-1.amazonaws.com/credilinq-prefect|010614160260.dkr.ecr.ap-southeast-1.amazonaws.com/credilinq-prefect>
      tag: "{{ git_sha.stdout }}"
      additional_tags:
        - "{{ $DEPLOY_ENV }}"        # dev/stage/prod from CI
      dockerfile: Dockerfile.prefect_flows_run
      ignore_cache: true

push:
  - prefect_docker.deployments.steps.push_docker_image:
      requires: prefect-docker>=0.3.1
      image_name: "{{ build_image.image_name }}"
      tag: "{{ git_sha.stdout }}"
      additional_tags:
        - "{{ $DEPLOY_ENV }}"

pull:
  - prefect.deployments.steps.set_working_directory:
      directory: /opt/prefect/src

deployments:
  - name: ebay/dev
    version: "{{ build_image.tag }}"
    tags: ["dev"]
    entrypoint: extraction/ebay/flow.py:run
    parameters:
      seller_id: "d480fd76-f59e-4ea0-95cb-4e22db0164e0"
      selling_partner_id: "ed442813-afe7-4026-bc14-190ca41bfb23"
      data_cycle: "origination"
    work_pool:
      name: my-ecs-pool
      work_queue_name: dev
      job_variables:
        image: <http://010614160260.dkr.ecr.ap-southeast-1.amazonaws.com/credilinq-prefect:{{|010614160260.dkr.ecr.ap-southeast-1.amazonaws.com/credilinq-prefect:{{> git_sha.stdout }}
        launch_type: EC2
        cpu: 4096
        memory: 8192
        env:
          APP_ENV: "dev"
          SOURCE: "ebay"

  - name: ebay/stage
    version: "{{ build_image.tag }}"
    tags: ["stage"]
    entrypoint: extraction/ebay/flow.py:run
    parameters:
      seller_id: "d480fd76-f59e-4ea0-95cb-4e22db0164e0"
      selling_partner_id: "ed442813-afe7-4026-bc14-190ca41bfb23"
      data_cycle: "origination"
    work_pool:
      name: my-ecs-pool
      work_queue_name: stage
      job_variables:
        image: <http://010614160260.dkr.ecr.ap-southeast-1.amazonaws.com/credilinq-prefect:{{|010614160260.dkr.ecr.ap-southeast-1.amazonaws.com/credilinq-prefect:{{> git_sha.stdout }}
        launch_type: EC2
        cpu: 4096
        memory: 8192
        env:
          APP_ENV: "stage"
          SOURCE: "ebay"

  - name: ebay/prod
    version: "{{ build_image.tag }}"
    tags: ["prod"]
    entrypoint: extraction/ebay/flow.py:run
    parameters:
      seller_id: "d480fd76-f59e-4ea0-95cb-4e22db0164e0"
      selling_partner_id: "ed442813-afe7-4026-bc14-190ca41bfb23"
      data_cycle: "origination"
    work_pool:
      name: my-ecs-pool
      work_queue_name: prod
      job_variables:
        image: <http://010614160260.dkr.ecr.ap-southeast-1.amazonaws.com/credilinq-prefect:{{|010614160260.dkr.ecr.ap-southeast-1.amazonaws.com/credilinq-prefect:{{> git_sha.stdout }}
        launch_type: EC2
        cpu: 4096
        memory: 8192
        env:
          APP_ENV: "prod"
          SOURCE: "ebay"

Option B: One parameterized deployment that derives env from CI - Faster to maintain, but the deployment name changes per run and you’ll end up with 3 deployment objects anyway (one per env) - Use env var interpolation in fields ``` deployments: - name: "ebay/{{ $DEPLOY_ENV }}" version: "{{ build_image.tag }}" tags: ["{{ $DEPLOY_ENV }}"] entrypoint: extraction/ebay/flow.py:run work_pool: name: my-ecs-pool work_queue_name: "{{ $DEPLOY_ENV }}" job_variables: image: 010614160260.dkr.ecr.ap-southeast-1.amazonaws.com/credilinq-prefect:{{ git_sha.stdout }} launch_type: EC2 cpu: 4096 memory: 8192 env:

APP_ENV: "{{ $DEPLOY_ENV }}" SOURCE: "ebay"

Bitbucket Pipelines example
- Maps branches to DEPLOY_ENV and selects the matching deployment
- Exports Prefect Cloud workspace credentials (set these in Bitbucket repo variables)

image: python:3.11 pipelines: branches: main: - step: name: Deploy prod caches: [pip] script: - pip install -U "prefect" "prefect-aws" "prefect-docker" - export DEPLOY_ENV=prod - export PREFECT_API_URL="${PREFECT_API_URL_PROD}" - export PREFECT_API_KEY="${PREFECT_API_KEY_PROD}" - prefect deploy --no-prompt --name "ebay/prod" stage: - step: name: Deploy stage caches: [pip] script: - pip install -U "prefect" "prefect-aws" "prefect-docker" - export DEPLOY_ENV=stage - export PREFECT_API_URL="${PREFECT_API_URL_STAGE}" - export PREFECT_API_KEY="${PREFECT_API_KEY_STAGE}" - prefect deploy --no-prompt --name "ebay/stage" develop: - step: name: Deploy dev caches: [pip] script: - pip install -U "prefect" "prefect-aws" "prefect-docker" - export DEPLOY_ENV=dev - export PREFECT_API_URL="${PREFECT_API_URL_DEV}" - export PREFECT_API_KEY="${PREFECT_API_KEY_DEV}" - prefect deploy --no-prompt --name "ebay/dev" ``` Notes on your current YAML - The templating for the image tag should be

{{ git_sha.stdout }}

(remove the extra

>

you had in Slack). - Deployment

tags

should be a flat list like

tags: ["dev"]

(not

- ['dev']

). -

push: false

inside

job_variables

isn’t a valid ECS job variable; remove it. Pushing images is handled in the top-level

push:

section. - For ECS EC2, set

launch_type: EC2

under

job_variables

(I added it above). - Since you’re using one ECS cluster/host for all environments, the single work pool is fine. Consider separate work queues per env (

dev

,

stage

,

prod

) so you can have workers filter what they pick up. Docs - Prefect YAML: prefect.yaml guide - CI/CD patterns: Deploy with CI/CD - Work pools and job variables: Manage work pools Want me to convert your exact YAML to the 3-deployment form and include your full parameters/job vars? If you paste the full file (minus secrets), I’ll adjust it precisely.