https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I am using perfect cloud with perfect...
# ask-marvin
l
@Marvin I am using perfect cloud with perfect managed work pool and code pulled from GitHub but the flow run keep crashing
m
✅ thought for 485.2 seconds
l
@Marvin this is the error log Opening process... prefect.flow_runs.runner 112054 PM Info > Running git_clone step... prefect.flow_runs 112054 PM Error Unexpected exception encountered when trying to load flow Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/prefect/engine.py", line 112, in <module> flow: "Flow[..., Any]" = load_flow(flow_run) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/flow_engine.py", line 152, in load_flow flow = run_coro_as_sync( ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 207, in run_coro_as_sync return call.result() ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 365, in result return self.future.result(timeout=timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 192, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 441, in _run_async result = await coro ^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 188, in coroutine_wrapper return await task ^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/client/utilities.py", line 69, in wrapper return await func(client, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/flows.py", line 2880, in load_flow_from_flow_run output = await run_steps( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 171, in run_steps "id": inputs.get("id"), ^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' prefect.flow_runs 112055 PM Error Process for flow run 'taupe-inchworm' exited with status code: 1 prefect.flow_runs.runner 112056 PM Info Reported flow run '06954502-8571-7056-8000-3d8c7297c408' as crashed: Flow run process exited with non-zero status code 1. prefect.flow_runs.runner 112056 PM Info Loading flow to check for on_crashed hooks prefect.flow_runs.runner 112056 PM Info > Running git_clone step... prefect.flow_runs 112056 PM Warning Runner failed to retrieve flow to execute on_crashed hooks for flow run UUID('06954502-8571-7056-8000-3d8c7297c408'). Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/prefect/runner/runner.py", line 1548, in _run_on_crashed_hooks flow = await load_flow_from_flow_run( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/client/utilities.py", line 69, in wrapper return await func(client, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/flows.py", line 2880, in load_flow_from_flow_run output = await run_steps( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 171, in run_steps "id": inputs.get("id"), ^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get'
m
✅ thought for 119.8 seconds
Thanks for sharing the log. The key line is:
Copy code
AttributeError: 'NoneType' object has no attribute 'get'
  at prefect/deployments/steps/core.py:171  # inputs.get("id")
This happens when a deployment step is parsed with a value of 
None
. In your case, the 
git_clone
step was loaded, but its inputs resolved to 
None
instead of a dict, so 
inputs.get("id")
crashed. Most common cause - YAML step declared without a mapping value, e.g.:
Copy code
pull:
    - prefect.deployments.steps.git_clone:    # ← value is None
- Or, if deploying via Python, passing 
{"prefect.deployments.steps.git_clone": None}
. What the step should look like - In prefect.yaml:
Copy code
pull:
    - prefect.deployments.steps.git_clone:
        id: clone
        repository: <https://github.com/owner/repo.git>
        # branch: main            # optional
        # access_token: "{{ prefect.blocks.secret.github-token }}"
        # directories: ["src"]    # optional sparse checkout
    - prefect.deployments.steps.pip_install_requirements:
        directory: "{{ clone.directory }}"
        # requirements_file: requirements.txt
    - prefect.deployments.steps.set_working_directory:
        directory: "{{ clone.directory }}"
- If using the Python API to deploy, make sure the value is a dict:
Copy code
pull=[
      {"prefect.deployments.steps.git_clone": {
          "id": "clone",
          "repository": "<https://github.com/owner/repo.git>"
      }},
      {"prefect.deployments.steps.pip_install_requirements": {
          "directory": "{{ clone.directory }}"
      }},
  ]
Helpful docs - Deployment actions in prefect.yaml - git_clone step reference If you paste your 
pull:
section (or how you construct 
pull
in Python), I can spot-check the YAML/structure and fix the exact indentation/keys. Also let me know: - Prefect version (e.g., 3.x.y) - Whether the repo is private (we may need to add 
access_token
or a credentials block)
Got it — sorry your runs are crashing. Let’s get you unblocked. I’ll assume you’re on Prefect 3.x using Prefect Cloud + a Managed Work Pool with code pulled from GitHub. First, a few quick checks to pinpoint the cause: - Can you share the flow run URL and a snippet of the earliest error in the logs? - Is your repo private? If yes, how are you authenticating to GitHub? - What’s your deployment entrypoint (e.g., 
path/to/file.py:my_flow
)? - Do you have submodules? - Which extra Python deps does your flow need (e.g., 
pandas
, 
boto3
)? How to see logs - In the UI: Deployments → your deployment → Flow runs → click a run → Logs. - CLI:
Copy code
prefect flow-run logs <flow-run-id>
- If logs are sparse, add DEBUG logging at runtime:
Copy code
job_variables={"env": {"PREFECT_LOGGING_LEVEL": "DEBUG"}}
Most common causes of CRASHED runs with managed pools - Missing Python packages (ModuleNotFoundError): add packages via 
job_variables["pip_packages"]
. - Wrong entrypoint: path or function name doesn’t match the repo contents. - Git clone/auth failures for private repos: missing/incorrect token/SSH setup. - Code that errors at import time (module-level code runs when we load the flow). - Large memory/time usage causing infrastructure termination (look for OOM/termination messages in logs). Managed Work Pool specifics to keep in mind - You cannot supply a custom Docker image. Use 
job_variables["pip_packages"]
to install Python deps at runtime. - The entrypoint must be relative to the repo root and point to a function decorated with 
@flow
. Reference example deployment (managed pool + pip packages + env)
Copy code
from prefect import Flow

if __name__ == "__main__":
    Flow.from_source(
        source="<https://github.com/org/repo.git>",
        entrypoint="flows/my_flow.py:my_flow",
    ).deploy(
        name="my-managed-deploy",
        work_pool_name="my-managed-pool",
        job_variables={
            "pip_packages": ["pandas", "boto3"],   # add your deps here
            "env": {"ENVIRONMENT": "prod"}         # runtime env vars
        },
    )
Private GitHub repos — safe authentication pattern - Recommended: use a credentials block with 
GitRepository
so tokens never hit logs.
Copy code
from prefect import Flow
from prefect.runner.storage import GitRepository
# If you use prefect-github:
# pip install -U prefect-github
# from prefect_github import GitHubCredentials

# Option A: Secret block
from prefect.blocks.system import Secret
# Secret(value="ghp_...").save("github-token", overwrite=True)

repo = GitRepository(
    url="<https://github.com/org/private-repo.git>",
    credentials={"access_token": Secret.load("github-token")}
)

if __name__ == "__main__":
    Flow.from_source(
        source=repo,
        entrypoint="flows/my_flow.py:my_flow",
    ).deploy(
        name="private-repo-deploy",
        work_pool_name="my-managed-pool",
        job_variables={"pip_packages": ["pandas"]},
    )
Quick triage checklist - Verify entrypoint exists and is decorated as 
@flow
. - If private repo, confirm the token has read access and is being supplied (via Secret block or GitHubCredentials). - Add all imports to 
pip_packages
(what imports fail locally if you run the flow file in a clean venv?). - If you use submodules, ensure credentials cover them (and set 
include_submodules=True
in 
GitRepository
if needed). - Re-run with 
PREFECT_LOGGING_LEVEL=DEBUG
to surface more detail. Useful docs - Managed Work Pools - Deploy via Python (Flow.from_source().deploy()) - Customize job variables (pip_packages, env) - States (CRASHED definition) - Troubleshoot Prefect Cloud If you share:
- the flow run URL (or the first 20–30 lines of logs), - your deployment snippet, - and whether the repo is private (and how you’re authenticating), I can pinpoint the issue and give you the exact fix.
l
@Marvin getting this after setting the yaml up as stated above prefect.exceptions.PrefectHTTPStatusError: Client error '401 Unauthorized' for url 'https://api.prefect.cloud/api/accounts/c1cfb16a-fc3c-4c1f-b185-e0cb9a6333e6/workspaces/ce9b57ec-cf95-43d0-971a-15e1d87d90b7/work_pools/cloud_pool' Response: {'detail': 'Unauthorized'} For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401 An exception occurred.
m
✅ thought for 176.9 seconds
l
@Marvin the error comes up when I run the 
perfect deploy ~n etl
@Marvin the error comes up when I run the 
perfect deploy ~n etl
m
✅ thought for 45.8 seconds
Thanks — a couple quick things to fix first: - The command is 
prefect
, not 
perfect
- The flag is 
-n
(or 
--name
), not 
~n
So the correct invocation is:
Copy code
prefect deploy -n etl ./path/to/flow_file.py:flow_function
Now, about the crash you saw: the stack trace you shared points to a malformed step in your 
prefect.yaml
under 
pull:
— specifically the 
git_clone
step is present but its value is 
None
or missing required fields. That causes 
inputs.get(...)
to blow up when Prefect tries to load your flow. Make sure your 
pull:
section looks like this (value must be a mapping/dict, not empty/None):
Copy code
pull:
  - prefect.deployments.steps.git_clone:
      id: clone
      repository: <https://github.com/owner/repo.git>
      # branch: main            # optional
      # access_token: "{{ prefect.blocks.secret.github-token }}"  # if private
  - prefect.deployments.steps.set_working_directory:
      directory: "{{ clone.directory }}"
  - prefect.deployments.steps.pip_install_requirements:
      directory: "{{ clone.directory }}"
      # requirements_file: requirements.txt  # or rely on job_variables pip_packages
If you’re deploying to a Managed Work Pool and your flow needs extra packages, add them via 
job_variables
on the deployment (Managed pools don’t take custom images):
Copy code
Flow.from_source(
    source="<https://github.com/owner/repo.git>",
    entrypoint="flows/my_flow.py:my_flow",
).deploy(
    name="etl",
    work_pool_name="your-managed-pool",
    job_variables={
      "pip_packages": ["pandas", "boto3"],
      "env": {"PREFECT_LOGGING_LEVEL": "DEBUG"}
    },
)
Could you paste: - Your 
prefect.yaml
pull:
section - The exact 
prefect deploy
command you’re running (with entrypoint) - Whether the repo is private (and if so, how you’re authenticating) I’ll spot the exact issue. For reference: - Deployment actions in prefect.yaml - Deploy via Python - Managed Work Pools
A 401 Unauthorized from the Prefect Cloud API means the environment that’s making the request isn’t authenticated to your Cloud workspace. In your traceback it’s querying your work pool by name:
Copy code
.../work_pools/cloud_pool
Response: {'detail': 'Unauthorized'}
Quick fix checklist 1) Log in to Prefect Cloud in the SAME environment where you’re running deploy/CLI:
Copy code
prefect cloud login -k <YOUR_PREFECT_API_KEY> -w <account_handle>/<workspace_handle>
- If you don’t have a key yet, create one in the Cloud UI (User menu → API Keys). - Make sure the key belongs to the same account/workspace you’re using. 2) Select the workspace and verify:
Copy code
prefect cloud workspace ls
prefect work-pool ls
prefect profile inspect
- 
work-pool ls
should list your managed pool. Confirm the name matches exactly (e.g., 
cloud_pool
). 3) If you prefer environment variables (e.g., in CI), set both:
Copy code
export PREFECT_API_KEY=<YOUR_PREFECT_API_KEY>
export PREFECT_API_URL="<https://api.prefect.cloud/api/accounts/<ACCOUNT_ID>/workspaces/<WORKSPACE_ID>>"
Then retry your command. 4) Re-run the deployment or command that failed. Common gotchas - Using a GitHub token instead of a Prefect API key by mistake. - Key belongs to a different account/workspace than the IDs in your 
PREFECT_API_URL
. - Logged in locally, but your CI/runner environment is missing the same auth. - Work pool name mismatch (verify with 
prefect work-pool ls
). If the pool doesn’t exist, create a Managed Work Pool in the UI and use that exact name. Helpful docs - Managed Work Pools: Managed Work Pools - Troubleshooting Cloud: Troubleshoot Prefect Cloud If you share: - where you’re running the command (local vs CI), - the 
prefect profile inspect
output (redact the key), - and the exact 
work_pool_name
you set, I can help pinpoint the auth/workspace mismatch and get you unstuck quickly.
l
@Marvin My yaml file 
name: prefect
pull:
◦ 
prefect.deployments.steps.git_clone:
id: clone-step
repository: <https://github.com/drisAloma/oris_codebase.git>
# token: "{{ prefect.blocks.secret.prefect-github-access-token }}"
credentials: "{{ prefect.blocks.github-credentials.github-creds }}"
◦ 
prefect.deployments.steps.pip_install_requirements:
`directory: "{{ clone-step.directory }}" # 
clone-step
is a user-provided 
id
field` 
requirements_file: requirements.txt
deployments:
◦ 
name: oris_etl
entrypoint: hello.py:hello_world
work_pool:
name: cloud_pool
Getting this error now; Unexpected exception encountered when trying to load flow Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/prefect/engine.py", line 112, in <module> flow: "Flow[..., Any]" = load_flow(flow_run) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/flow_engine.py", line 152, in load_flow flow = run_coro_as_sync( ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 207, in run_coro_as_sync return call.result() ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 365, in result return self.future.result(timeout=timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 192, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/usr/local/lib/python3.12/site-packages/prefect/_internal/concurrency/calls.py", line 441, in _run_async result = await coro ^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/utilities/asyncutils.py", line 188, in coroutine_wrapper return await task ^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/client/utilities.py", line 69, in wrapper return await func(client, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/flows.py", line 2880, in load_flow_from_flow_run output = await run_steps( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 171, in run_steps "id": inputs.get("id"), ^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' prefect.flow_runs 122840 AM Error Process for flow run 'slick-scallop' exited with status code: 1 prefect.flow_runs.runner 122840 AM Info Reported flow run '06954600-bef5-7e70-8000-bd6167bfbe0e' as crashed: Flow run process exited with non-zero status code 1. prefect.flow_runs.runner 122840 AM Info Loading flow to check for on_crashed hooks prefect.flow_runs.runner 122840 AM Info > Running git_clone step... prefect.flow_runs 122840 AM Warning Runner failed to retrieve flow to execute on_crashed hooks for flow run UUID('06954600-bef5-7e70-8000-bd6167bfbe0e'). Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/prefect/runner/runner.py", line 1548, in _run_on_crashed_hooks flow = await load_flow_from_flow_run( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/client/utilities.py", line 69, in wrapper return await func(client, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/flows.py", line 2880, in load_flow_from_flow_run output = await run_steps( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/prefect/deployments/steps/core.py", line 171, in run_steps "id": inputs.get("id"), ^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get' prefect.flow_runs.runner Streaming logs...
m
✅ thought for 39.0 seconds
Thanks for sharing the YAML — the crash is still coming from a malformed 
pull:
section. In your snippet, the step keys don’t have a mapping value because the nested keys (
id
, 
repository
, 
credentials
) aren’t indented under the step key. That makes the step value 
None
, which triggers 
inputs.get(...)
and the AttributeError you’re seeing. Also, YAML should use dashes (
-
) for list items; the 
◦
bullets and backticks will break parsing. And if you ever use a token instead of credentials, the correct key is 
access_token
, not 
token
. Try this exact 
prefect.yaml
shape (no backticks, no Slack link formatting, and with proper indentation):
Copy code
name: prefect

pull:
  - prefect.deployments.steps.git_clone:
      id: clone-step
      repository: <https://github.com/drisAloma/oris_codebase.git>
      # If the repo is private, use one of these:
      # access_token: "{{ prefect.blocks.secret.prefect-github-access-token }}"
      credentials: "{{ prefect.blocks.github-credentials.github-creds }}"
  - prefect.deployments.steps.pip_install_requirements:
      directory: "{{ clone-step.directory }}"
      requirements_file: requirements.txt

deployments:
  - name: oris_etl
    entrypoint: hello.py:hello_world
    work_pool:
      name: cloud_pool
A few additional checks: - Ensure 
hello.py:hello_world
exists in the repo and 
hello_world
is decorated with 
@flow
. - If you’re using the 
credentials
line above, confirm you have the GitHub credentials block created: - Install and register the integration once:
Copy code
pip install -U prefect-github
    prefect block register -m prefect_github
- Create the block in UI or via code with the same name 
github-creds
. - If you instead want to use a token, change to 
access_token: "{{ prefect.blocks.secret.prefect-github-access-token }}"
and ensure the Secret block exists. Docs for reference: - prefect.yaml deployment actions - Managed Work Pools If it still crashes after this change, please paste: - The first 20–30 lines of the flow run logs - Confirmation that the 
github-creds
block exists (or the Secret block if using 
access_token
) - The exact path/function for 
hello.py:hello_world
in your repo structure We’ll get it sorted quickly.
19 Views
Open in Slack
PreviousNext
Powered by