```name: orchestrator prefect-version: 3.4.22 buil...
# ask-marvin
m
Copy code
name: orchestrator
prefect-version: 3.4.22
build: null
push: null
pull:
- prefect.deployments.steps.set_working_directory:
    directory: ./repositories
- prefect.deployments.steps.git_clone:
    repository: "<https://bitbucket>......"
    branch: "feature/prefect_testing"
    credentials: "{{ <http://prefect.blocks.bitbucket-credentials.xxx|prefect.blocks.bitbucket-credentials.xxx> }}"

deployments:
- name: application-name
  version: 1.0.0
  tags: [xxx]
  description: xxx
  schedule: {}
  flow_name: null
  entrypoint: application-name/xxx/script.py:foo
  parameters: {}
  work_pool:
    name: application-pool
    work_queue_name: null
    job_variables: {}
@Marvin, why do I get FileNotFoundError: [Errno 2] No such file or directory: application-name/xxx/script.py:foo why pull step does not work?
m
thought for 198.6 seconds
Short answer: your entrypoint is being resolved relative to the working directory after your pull steps. With your current steps, the repo is cloned into a subfolder under ./repositories, but your entrypoint points to a path that doesn’t exist from there—hence FileNotFoundError. What’s happening - You first set the working directory to ./repositories. - Then
git_clone
clones the repo into a subdirectory of the current working directory (typically the repo’s name), e.g. ./repositories/<repo_name>. - Prefect imports your flow using the entrypoint path relative to the current working directory. Since you’re still in ./repositories, it looks for ./repositories/application-name/xxx/script.py. If your repo actually lives at ./repositories/<repo_name>/application-name/xxx/script.py, it won’t be found. Two good ways to fix it Option A: Set the working directory to the cloned repo root, then make the entrypoint relative to that
Copy code
pull:
- prefect.deployments.steps.git_clone:
    id: repo
    repository: "<https://bitbucket/......>"
    branch: "feature/prefect_testing"
    credentials: "{{ <http://prefect.blocks.bitbucket-credentials.xxx|prefect.blocks.bitbucket-credentials.xxx> }}"
- prefect.deployments.steps.set_working_directory:
    directory: "{{ repo.directory }}"   # use the clone directory from the previous step
Then update your deployment entrypoint to be relative to the repo root, for example:
Copy code
entrypoint: xxx/script.py:foo
Option B: Keep your current working directory, but include the repo folder in the entrypoint If you really want to keep
set_working_directory: ./repositories
, change your entrypoint to include the repo directory name:
Copy code
entrypoint: <repo_name>/application-name/xxx/script.py:foo
Replace
<repo_name>
with the actual name of the directory created by the clone (usually the repository name). Tips to verify/debug - Add a debug step to confirm where files landed:
Copy code
pull:
- prefect.deployments.steps.set_working_directory:
    directory: ./repositories
- prefect.deployments.steps.git_clone:
    id: repo
    repository: "<https://bitbucket/......>"
    branch: "feature/prefect_testing"
    credentials: "{{ <http://prefect.blocks.bitbucket-credentials.xxx|prefect.blocks.bitbucket-credentials.xxx> }}"
- prefect.deployments.steps.utility.run_shell_script:
    script: "pwd && ls -la && echo 'Cloned to: {{ repo.directory }}' && ls -la '{{ repo.directory }}'"
- Make sure the function
foo
is actually defined in
script.py
. - Ensure your YAML does not literally contain Slack markup like `<...>`; your credentials line should look like:
Copy code
credentials: "{{ <http://prefect.blocks.bitbucket-credentials.xxx|prefect.blocks.bitbucket-credentials.xxx> }}"
Docs for reference - Pull steps (git_clone, set_working_directory): Prefect 3 pull steps - Entrypoints: Entrypoint format If you can share the repo name and the actual on-disk path of
script.py
after cloning, I can suggest the exact entrypoint and steps ordering.
m
name: "orchestrator" prefect-version: "3.4.22" build: null push: null pull: - prefect.deployments.steps.git_clone: id: repo repository: "https://bitbucket.....git" branch: "feature/prefect_testing" credentials: "{{ prefect.blocks.bitbucket-credentials.xxx-credentials }}" - prefect.deployments.steps.set_working_directory: directory: "{{ repo.directory }}" deployments: - name: "xxx-deployment" version: "1.0.0" tags: ["xxx"] description: "xxx" schedule: {} flow_name: null entrypoint: "prefect_dir/app.py:func" parameters: {} work_pool: name: "xxx" work_queue_name: null job_variables: {} @Marvin, modified to this - still the same error
m
thought for 211.5 seconds
Thanks for the update. With your new config, Prefect will: - clone the repo - set the working directory to the clone root (
{{ repo.directory }}
) - then try to import the flow from
prefect_dir/app.py:func
relative to that directory A FileNotFoundError at this point means that from the repo root, Prefect cannot find a file at
prefect_dir/app.py
. In other words, the clone and
set_working_directory
likely succeeded, but the path in
entrypoint
doesn’t exist in the cloned repo. What to check next 1) Verify where the files actually landed and what the repo root contains. Add a quick debug step after the clone:
Copy code
pull:
  - prefect.deployments.steps.git_clone:
      id: repo
      repository: "<https://bitbucket>.....git"
      branch: "feature/prefect_testing"
      credentials: "{{ prefect.blocks.bitbucket-credentials.xxx-credentials }}"
  - prefect.deployments.steps.set_working_directory:
      directory: "{{ repo.directory }}"
  - prefect.deployments.steps.utility.run_shell_script:
      script: |
        echo "PWD:"
        pwd
        echo "Repo directory from step: {{ repo.directory }}"
        echo "Top-level in repo:"
        ls -la
        echo "Looking for prefect_dir/app.py:"
        ls -la prefect_dir || true
2) Confirm the actual relative path to your flow file from the repo root. If it’s not at
prefect_dir/app.py
, update the entrypoint accordingly, e.g.: - If it’s at
application-name/xxx/script.py
, use:
Copy code
entrypoint: application-name/xxx/script.py:foo
- If you prefer module import style and
prefect_dir
is a package (has an
__init__.py
), you can use:
Copy code
entrypoint: <http://prefect_dir.app:func|prefect_dir.app:func>
3) Make sure the function exists. In
app.py
,
func
must be defined at module top-level and be your flow function (typically decorated with
@flow
). If the file exists but the function is missing, you’d see an AttributeError instead of FileNotFoundError. 4) Remove any Slack link markup in your YAML. Ensure your file contains plain values (no angle brackets), like:
Copy code
repository: "<https://bitbucket>.....git"
credentials: "{{ prefect.blocks.bitbucket-credentials.xxx-credentials }}"
References - Pull steps (git_clone, set_working_directory): Prefect 3 pull steps - Entrypoint format: Entrypoint format If you can paste the output of the debug
run_shell_script
step (pwd + ls), I can point you to the exact entrypoint value you need.
m
I get the same error @Marvin and no shell script is executed. It seems that prefect does not execute the steps in prefect.yaml. I run
prefect deploy --name grail-extractor-deployment
and get the same error
m
thought for 192.4 seconds
You’re bumping into a key behavior of Prefect 3: the pull steps do not run during prefect deploy. They only run at flow-run time on the worker. During deploy, Prefect still tries to import your entrypoint locally to build the deployment, so if the file isn’t present relative to your current directory, you’ll get FileNotFoundError. Why you see FileNotFoundError and no pull steps -
prefect deploy
loads the flow from
entrypoint
immediately to create the deployment. - The
pull
section is executed later by the worker when a flow run starts. - If
prefect_dir/app.py
does not exist in your current project directory at deploy time,
prefect deploy
fails before any pull steps would ever run. How to fix Pick one of these patterns: 1) Deploy from a local checkout where the entrypoint exists - Make sure you’re in a project root that actually contains the code:
Copy code
git clone <https://bitbucket>.....git
cd <repo_root>
# verify this path exists:
ls -la prefect_dir/app.py
prefect deploy -n grail-extractor-deployment
- Or explicitly pass the local entrypoint on the CLI:
Copy code
prefect deploy prefect_dir/app.py:func -n grail-extractor-deployment
- Your pull steps can still run later on the worker (e.g., to pull a specific branch/commit for execution), but deploy itself needs a resolvable entrypoint right now. 2) Use “deploy from source” (no local code required) If you don’t want the code locally at deploy time, use the programmatic API to deploy directly from git:
Copy code
from prefect import flow
from prefect.storage import GitRepository, GitCredentials

# Load your credentials block or construct credentials
creds = GitCredentials.load("xxx-credentials")

# Point to your repo and entrypoint in-source
f = flow.from_source(
    source="<https://bitbucket>.....git",
    entrypoint="prefect_dir/app.py:func",
    credentials=creds,
)

f.deploy(
    name="grail-extractor-deployment",
    work_pool_name="xxx",
    tags=["xxx"],
)
This pattern does not require the file to exist locally; Prefect will use the repo at runtime. Sanity checks - Ensure you’re running
prefect deploy
from the same directory that contains your
prefect.yaml
(the project root). - Confirm the entrypoint path is correct and exists locally if you’re using the CLI + YAML approach. - Your
pull
steps are correct for runtime, but they won’t fix a missing local file at deploy time. Docs - Prefect YAML, including when steps run: prefect.yaml guide - Pull steps: Pull steps API If you can share where your
prefect.yaml
is located and whether
prefect_dir/app.py
exists locally in that directory, I can suggest the exact command or adjustment.
m
can I use deploy from source using yaml? @Marvin
m
thought for 295.3 seconds
Short answer: not with YAML alone today. - The prefect.yaml + prefect deploy path requires the entrypoint to exist locally at deploy time. The pull steps only run later on the worker at flow-run time. - “Deploy from source” (remote Git without local code) is supported via the Python API with
flow.from_source(...).deploy(...)
, not via YAML. Two supported patterns 1) Use Python API (deploy from source)
Copy code
from prefect import flow
from prefect.storage import GitRepository, GitCredentials

creds = GitCredentials.load("xxx-credentials")

f = flow.from_source(
    source="<https://bitbucket>.....git",
    entrypoint="prefect_dir/app.py:func",
    credentials=creds,
    branch="feature/prefect_testing",
)

f.deploy(
    name="grail-extractor-deployment",
    work_pool_name="xxx",
    tags=["xxx"],
)
2) Use YAML/CLI (requires local code at deploy time) - Ensure the repo is checked out in your CI or local environment so
prefect_dir/app.py
exists - Then run:
Copy code
prefect deploy -n grail-extractor-deployment
- Your
pull
steps can still pin the runtime branch/commit for workers. Docs - Prefect YAML guide: prefect.yaml - Pull steps API: pull steps - Git repository storage (used by from_source): GitRepository If you prefer to keep YAML, the practical workaround is to clone the repo in your CI step before running
prefect deploy
. If you want remote-only, I recommend switching to the Python
flow.from_source(...).deploy(...)
approach.