<@ULVA73B9P> ""There isn’t a pure Python builder f...
# ask-marvin
s
@Marvin ""There isn’t a pure Python builder for pull steps in Prefect 3.x anymore."" Is this true ? Why so ? This feels like a regression
m
thought for 375.8 seconds
n
that is not true
pull steps are quite literally just python functions
unless i misunderstand what you mean by "pure Python builder"
s
@Nate here is answer I get repeatedly from GPT-5 on this. Is it hallucinating ?
Yeah, it does feel like a regression when you come from 2.x where you could do everything in Python 👇
• In Prefect 2.x the design goal was “everything is Python-first”. That’s why you had
Deployment.build_from_flow(...)
, could pass
pull_steps=[...]
, and never touch YAML if you didn’t want to.
• In Prefect 3.x the Prefect team flipped that model:
Deployments are YAML-first (
prefect.yaml
is the single source of truth).
◦ The
Deployment
Python API is deprecated because they want deployments to be declarative configs, not imperative Python objects.
pull
steps (like
git_clone
,
run_shell_script
) are part of the YAML schema, not the Python API surface.
The reasoning (from Prefect’s migration notes & Slack threads) is basically:
1. Standardization → Many teams had “magic” Python deployment scripts that diverged; Prefect wanted one canonical way to describe a deployment (
prefect.yaml
).
2. Separation of concerns → Flow code lives in Python, infra/deployment config lives in YAML. This makes it easier for ops/infra teams to manage deployments without touching flow code.
3. CLI-driven UX → In 3.x the recommended way is
prefect deploy build
→ generates a
prefect.yaml
→ edit it →
prefect deploy apply
.
So the short version is: Prefect intentionally removed the code path for configuring pull steps because they want deployments to be infrastructure as config, not infrastructure as code.
👉 You’re not alone — a bunch of folks have said it feels like a step back, especially for teams that prefer everything in Python. But Prefect’s answer is:
• If you want to script deployments in Python, generate or patch
prefect.yaml
from Python.
• If you want dynamic behavior at runtime, put it inside your flow code instead of in
pull
steps.
n
yes
it is hallucinating
> prefect deploy build > prefect deploy apply
do not exist
its confusing 2.x-ish APIs with 3.x APIs. very common, which is why marvin tries to do RAG
m
Short answer: mostly true — there isn’t a dedicated “pure-Python builder DSL” for deployment pull steps in Prefect 3.x. In 3.x the recommended programmatic path is to use
Flow.from_source(...).deploy(...)
, and step composition is primarily expressed in
prefect.yaml
for
prefect deploy
. Why the change - Prefect 3.x simplified deployments to reduce moving parts. Instead of a separate “step builder” API, 3.x focuses on: - Declarative deployments via
prefect.yaml
(including pull/build/push steps) - A clean Python API via
Flow.from_source(...).deploy(...)
that points directly at your repo or storage - This removes redundant configuration surfaces and makes “get the code to the worker and run it” more consistent. What still exists - The step functions and runner are still in the public API and used by
prefect deploy
under the hood: -
prefect.deployments.steps.pull.git_clone
,
pull_from_remote_storage
, etc. -
prefect.deployments.steps.core.run_steps
- If you really want to, you can invoke them from Python, but it’s not the primary/ergonomic path. Recommended patterns in 3.x 1) Pure Python (no YAML) using a repo as the source
Copy code
from prefect import flow

@flow
def my_flow():
    ...

Flow.from_source(
    "<https://github.com/owner/repo>",
    entrypoint="flows/my_flow.py:my_flow",
).deploy(
    name="prod",
    work_pool_name="kubernetes",
)
- This is the supported “pure Python” route to get code pulled at runtime without manually composing pull steps. 2) YAML with pull steps (for
prefect deploy
)
Copy code
# prefect.yaml (excerpt)
deployments:
  - name: prod
    entrypoint: flows/my_flow.py:my_flow
    pull:
      - prefect.deployments.steps.pull.git_clone:
          repository: <https://github.com/owner/repo>
          branch: main
References - Prefect YAML (deployments, pull steps): prefect.yaml guide - Deploy via Python: Deploy via Python - Step functions live here (used by the CLI and available to Python):
prefect.deployments.steps.pull
and
prefect.deployments.steps.core
(API ref link provided by our tooling): prefect.deployments.steps API If your use case truly needs composing pull logic in Python (e.g., custom auth flows, submodules, or multi-repo layouts), you can: - Prefer
Flow.from_source(...)
pointing to your repo or a
ReadableDeploymentStorage
- Or, as an escape hatch, call the step runner directly:
Copy code
from prefect.deployments.steps.core import run_steps

outputs = run_steps([
  {"prefect.deployments.steps.pull.git_clone": {
      "repository": "<https://github.com/owner/repo>",
      "branch": "main",
      "include_submodules": True
  }}
])
I realize this can feel like a regression if you relied on a Python step-builder in 2.x. If you can share what you’re trying to do (private repo? submodules? multi-step bootstrap?), I can suggest the cleanest 3.x pattern for it.