Janet Carson
11/14/2025, 7:23 PM- prefect.deployments.steps.pull.pull_with_block:
id: clone
block_document_name: my-bitbucket-block
block_type_slug: bitbucket-repository
Can I override the reference name in my-bitbucket-block in the prefect.yaml file or only in the block itself?Marvin
11/14/2025, 7:23 PMMarvin
11/14/2025, 7:30 PMpull_with_block step only takes block_document_name and block_type_slug. It does not accept extra kwargs (like reference, branch, or tag) to override the repository block’s reference at deploy time. The branch/tag (“reference”) used for the pull is the value saved on the BitBucketRepository block.
Options if you need to vary the branch/tag per deployment:
- Create separate Bitbucket Repository blocks for each branch/tag you need, and point the deployment at the appropriate block.
- Switch to the git_clone pull step, which supports overriding the branch or commit in prefect.yaml.
Example using git_clone:
pull:
- prefect.deployments.steps.pull.git_clone:
repository: <https://bitbucket.org/your-org/your-repo.git>
branch: feature/my-branch
# or pin a commit:
# commit_sha: abcdef1234567890
# auth options as needed, e.g.:
# access_token: ${BITBUCKET_APP_PASSWORD}
Refs:
- Pull steps API (Prefect 3.x)
- prefect-bitbucket integration
- prefect.yaml reference
If you share your current prefect.yaml, I can show the minimal change to switch that specific step to git_clone while keeping your credentials setup.Janet Carson
11/14/2025, 7:33 PMMarvin
11/14/2025, 7:33 PMMarvin
11/14/2025, 7:37 PM# Anything under "definitions" is just YAML; Prefect ignores it but you can store anchors here
definitions:
steps:
set_wd: &set_wd
prefect.deployments.steps.pull.set_working_directory:
directory: .
# add more shared steps as needed, e.g. sparse checkout, etc.
# my_shared_step: &my_shared_step
# ...
deployments:
- name: prod
entrypoint: flows/my_flow.py:flow
work_pool:
name: your-pool
pull:
- prefect.deployments.steps.pull.pull_with_block:
id: clone
block_document_name: my-bitbucket-block-main # main branch block
block_type_slug: bitbucket-repository
- *set_wd
# - *my_shared_step
- name: sandbox
entrypoint: flows/my_flow.py:flow
work_pool:
name: your-pool
pull:
- prefect.deployments.steps.pull.pull_with_block:
id: clone
block_document_name: my-bitbucket-block-sandbox # alternate branch block
block_type_slug: bitbucket-repository
- *set_wd
# - *my_shared_step
Notes:
- Each deployment’s pull list is independent; there’s no built-in merge with a global pull. Anchors let you reuse shared steps without duplicating YAML.
- You can deploy both or just one from the same file (e.g., filter with --name when running prefect deploy).
Docs:
- prefect.yaml guide
- Pull steps API
If you share your current shared steps, I can drop them into the anchors and produce a ready-to-use prefect.yaml for prod/sandbox.Janet Carson
11/14/2025, 7:44 PMdeployments section rather than at the top of the prefect.yaml file
General outline of my prefect.yaml file is
name: my-repo-deployments
prefect-version: 3.5
build: null
push: null
pull:
- prefect.deployments.steps.pull.pull_with_block
...
- prefect.deployments.steps.utility.run_shell_script
...
- prefect.deployment.steps.utility.pip_install_requirements
...
deployments:
- name: sandbox
entrypoint: path/to/somewhere.py:flow_name
parameters: {
"param1": "sandbox_value"
}
work_pool: sandbox-workers
work_queue_name: sandbox-queue
- name: prod
entrypoint: path/to/somewhere.py:flow_name
parameters: {
"param1": "prod_value"
}
work_pool: prod-workers
work_queue_name: prod-queueMarvin
11/14/2025, 7:44 PMMarvin
11/14/2025, 7:45 PMpull section, it replaces the root-level pull for that deployment. So for prod/sandbox with different pull_with_block values, put a pull list under each deployment.
To avoid duplicating the shared steps, use YAML anchors for the common steps and reference them in each deployment’s pull list.
Example adapted to your outline (note the module path is prefect.deployments.steps.utility, plural):
name: my-repo-deployments
prefect-version: 3.5
build: null
push: null
# Optional: keep a root-level pull as the default for deployments that don't override it
# pull:
# - prefect.deployments.steps.pull.pull_with_block:
# block_document_name: my-bitbucket-block-main
# block_type_slug: bitbucket-repository
# - prefect.deployments.steps.utility.run_shell_script:
# script: echo "do something shared"
# - prefect.deployments.steps.utility.pip_install_requirements:
# requirements_file: requirements.txt
# Use a definitions section to hold YAML anchors for shared steps
definitions:
shared_steps:
run_script: &run_script
prefect.deployments.steps.utility.run_shell_script:
script: echo "do something shared"
install_reqs: &install_reqs
prefect.deployments.steps.utility.pip_install_requirements:
requirements_file: requirements.txt
deployments:
- name: sandbox
entrypoint: path/to/somewhere.py:flow_name
parameters:
param1: sandbox_value
work_pool: sandbox-workers
work_queue_name: sandbox-queue
pull:
- prefect.deployments.steps.pull.pull_with_block:
block_document_name: my-bitbucket-block-sandbox
block_type_slug: bitbucket-repository
- *run_script
- *install_reqs
- name: prod
entrypoint: path/to/somewhere.py:flow_name
parameters:
param1: prod_value
work_pool: prod-workers
work_queue_name: prod-queue
pull:
- prefect.deployments.steps.pull.pull_with_block:
block_document_name: my-bitbucket-block-main
block_type_slug: bitbucket-repository
- *run_script
- *install_reqs
Tips:
- You can keep a root-level pull for the “default” branch/block and only add a per-deployment pull for the one that differs. Any deployment without a pull inherits the root-level one.
- When you do define a per-deployment pull, list all steps you need—there’s no merging with the root-level list.
Refs:
- prefect.yaml guide
- Pull steps API
If you paste your exact shared steps, I’ll drop them into anchors and return a ready-to-use file.Janet Carson
11/14/2025, 7:51 PMMarvin
11/14/2025, 7:51 PMMarvin
11/14/2025, 7:56 PMname: my-repo-deployments
prefect-version: 3.5
definitions:
job_vars:
base_env: &base_env
COMMON_FLAG: "1"
API_TOKEN: "{{ $API_TOKEN }}" # pulled from your environment at deploy time
docker_job: &docker_job
image: my-org/my-image:latest
env: *base_env
deployments:
- name: sandbox
entrypoint: path/to/somewhere.py:flow_name
parameters:
param1: sandbox_value
work_pool:
name: sandbox-workers
job_variables:
<<: *docker_job
env:
<<: *base_env
APP_MODE: sandbox
work_queue_name: sandbox-queue
- name: prod
entrypoint: path/to/somewhere.py:flow_name
parameters:
param1: prod_value
work_pool:
name: prod-workers
job_variables:
<<: *docker_job
env:
<<: *base_env
APP_MODE: prod
work_queue_name: prod-queue
Notes:
- Deployment-level job_variables override the work pool’s base job template. The anchor/merge here is just YAML reuse to keep your file DRY.
- The exact shape of job_variables (and env) is worker-type specific; the above is a common pattern for Docker or Process workers.
- You can still keep a root-level pull for defaults and define per-deployment pull only when it differs.
Docs:
- prefect.yaml guide
- Customize job variables
If you tell me which worker type you’re using (Process, Docker, Kubernetes), I can tailor the env structure and a ready-to-paste snippet.Janet Carson
11/14/2025, 8:41 PMMarvin
11/14/2025, 8:41 PMMarvin
11/14/2025, 8:47 PMpull is not a valid key inside an individual deployment definition. Only the top-level pull section is supported. That’s why you’re seeing:
• myapp-prod: pull
• myapp-sandbox: pull
How to keep one prefect.yaml and vary just the repo/block per deployment
Since your only difference is which Bitbucket block to use (and the rest of the pull steps are identical), keep a single top-level pull and parameterize the block name via an environment variable. Then set that env var differently per deployment using work_pool.job_variables.env.
Example:
name: my-repo-deployments
prefect-version: 3.5
# One global pull section used by all deployments
pull:
- prefect.deployments.steps.pull.pull_with_block:
id: clone
block_document_name: "{{ $BITBUCKET_BLOCK_NAME }}"
block_type_slug: bitbucket-repository
- prefect.deployments.steps.utility.run_shell_script:
script: echo "shared step runs for both"
- prefect.deployments.steps.utility.pip_install_requirements:
requirements_file: requirements.txt
deployments:
- name: myapp-sandbox
entrypoint: path/to/somewhere.py:flow_name
parameters:
param1: sandbox_value
work_pool:
name: sandbox-workers
job_variables:
env:
BITBUCKET_BLOCK_NAME: my-bitbucket-block-sandbox
work_queue_name: sandbox-queue
- name: myapp-prod
entrypoint: path/to/somewhere.py:flow_name
parameters:
param1: prod_value
work_pool:
name: prod-workers
job_variables:
env:
BITBUCKET_BLOCK_NAME: my-bitbucket-block-main
work_queue_name: prod-queue
Notes
- The {{ $VAR }} syntax reads environment variables at runtime for pull steps (the worker’s environment). Using job_variables.env ensures the worker gets the correct value per deployment.
- If you truly need different step lists per deployment (not just different inputs), with 3.5 you’ll need separate projects/files. If upgrading to a newer 3.x release is an option, I can check whether per-deployment build/push/pull overrides are supported in that version.
Docs
- prefect.yaml guide
- Pull steps API
If you share your exact block names and any other shared steps, I can tailor the YAML so you can copy/paste it.