<@ULVA73B9P> my flows are crashing without any err...
# marvin-ai
r
@Marvin my flows are crashing without any errors, Im using a prefect-managed compute pool using the image prefecthq/prefect-client:3-latest- with this pip packages: ["prefect>=2.0.0","prefect-dbt>=0.3.0","dbt-snowflake","prefect_github","prefect-snowflake>=0.2.0","prefect-email>=0.2.0","prefect-shell>=0.2.0","python-dotenv>=0.19.0","pandas>=1.3.0","numpy>=1.21.0","geopandas>=0.9.0","shapely>=2.0.0","snowflake-snowpark-python>=1.0.0","snowflake-connector-python[pandas]","snowflake-snowpark-python[pandas]","snowflake-connector-python>=3.0.0","google-auth>=2.0.0","google-auth-oauthlib>=0.4.0","google-auth-httplib2>=0.1.0","google-api-python-client>=2.0.0","requests>=2.26.0","urllib3>=1.26.0","click>=8.0.0","python-dateutil>=2.8.0","pathlib>=1.0.1","typing-extensions>=4.0.0","black>=22.0.0","pytest>=7.0.0","pytest-asyncio>=0.18.0"] This are the only logs being produces : Start of logs grinning-condor was Scheduled grinning-condor transitioned from Scheduled → Pending Flow run pull step was executed grinning-condor transitioned from Pending → Crashed prefect-cloud.managed-execution.used Last working run was on Friday early morning.
m
thought for 146.5 seconds
This looks very likely to be an environment / image issue before your flow code ever starts, not a flow logic issue. The biggest red flag is this:
Copy code
image: prefecthq/prefect-client:3-latest
For Prefect-managed compute, you should use the full Prefect image, not the minimal
prefect-client
image. Recommended change:
Copy code
prefecthq/prefect:3-latest
Why I think that’s the problem: - Your run gets to “Flow run pull step was executed” - Then it goes Pending → Crashed - There are no flow/task logs - That usually means the container/runtime failed before the flow engine actually started Also, your package list includes:
Copy code
"prefect>=2.0.0"
That’s another likely source of trouble in Prefect 3.x: - it may pull in/resolve an unexpected Prefect version - it conflicts with the Prefect version already present in the base image - in managed compute, you generally should not reinstall Prefect itself unless you have a very specific pinned 3.x reason What I’d change first 1. Switch image:
Copy code
prefecthq/prefect:3-latest
2. Remove this from `pip_packages`:
Copy code
"prefect>=2.0.0"
3. If possible, pin integrations to versions known to support Prefect 3.x, instead of very old minimums. A cleaned-up direction would look more like:
Copy code
[
  "prefect-dbt",
  "dbt-snowflake",
  "prefect-snowflake",
  "prefect-email",
  "prefect-shell",
  "python-dotenv",
  "pandas",
  "numpy",
  "geopandas",
  "shapely",
  "snowflake-snowpark-python[pandas]",
  "snowflake-connector-python[pandas]",
  "google-auth",
  "google-auth-oauthlib",
  "google-auth-httplib2",
  "google-api-python-client",
  "requests",
  "urllib3",
  "python-dateutil"
]
I’d also consider removing clearly dev-only packages from runtime unless your flow really needs them: -
black
-
pytest
-
pytest-asyncio
And
pathlib
is unnecessary on modern Python. How to inspect the failed run You can check the run details with:
Copy code
prefect flow-run inspect <FLOW_RUN_ID>
prefect flow-run logs <FLOW_RUN_ID>
I verified both CLI commands exist. If the crash is infra-level, the logs may still be empty, but
inspect
can help confirm timing/state details. Why this may have started “suddenly” after Friday A few possibilities: -
3-latest
moved to a newer image - one of your unpinned packages started resolving differently - an integration dependency became incompatible - managed compute pulled a fresh environment that exposed the conflict Using
latest
plus broad
>=
constraints makes this more likely. Most likely root cause summary 1.
prefecthq/prefect-client:3-latest
is the wrong base image for managed flow execution 2.
prefect>=2.0.0
in
pip_packages
is risky/conflicting in a Prefect 3 environment 3. The crash is probably happening during startup/import time, which is why you get no flow logs Suggested next step Please try just these two changes first: - change image to
prefecthq/prefect:3-latest
- remove
"prefect>=2.0.0"
from pip packages If you want, paste: - your deployment config /
prefect.yaml
/
deploy()
snippet - one crashed flow run ID and I can help you narrow down whether there’s also a package-resolution issue in the dependency list. Relevant docs: - Prefect-managed execution - Troubleshooting Prefect Cloud