Akash N
08/11/2026, 7:36 AMFinished in state Crashed in the logsMarvin
08/11/2026, 7:36 AMMarvin
08/11/2026, 7:37 AMFinished in state Crashed usually means your flow/task did not fail because of a normal exception in your code — it was interrupted by the runtime or infrastructure.
In Prefect 3.x:
- Failed = your flow/task raised a normal exception
- Crashed = the process was interrupted externally or hit a system-level issue
Common causes:
- container/pod was killed
- worker shut down while the run was executing
- OOM / memory pressure
- timeout or termination signal
- KeyboardInterrupt / sys.exit()
- loss of connectivity to the Prefect API in some cases
Prefect maps things like these to a Crashed state:
- interrupt signals
- runtime cancellation
- termination signals
- some network/timeout errors
A good first step is to check the state message in the run details — it often says something more specific, such as:
- Execution was aborted by a termination signal
- Execution was cancelled by the runtime environment
- Request to ... failed
A few troubleshooting steps:
1. Check worker/infrastructure logs
- Kubernetes: pod events / pod logs
- Docker: container exit reason
- local worker: worker process logs
2. Look for resource/time limits
- OOMKilled
- pod eviction
- container/job timeout
- autoscaling shutdowns
3. Check whether the worker restarted or was stopped
- a worker disappearing mid-run often leads to Crashed
4. Enable debug logging if needed
prefect config set PREFECT_LOGGING_LEVEL=DEBUG
Relevant references:
- Prefect exceptions: CrashedRun
- Troubleshooting Prefect Cloud
- Prefect state handling source
If you want, paste:
- the full state message
- where you’re running it (local / Docker / Kubernetes / ECS / etc.)
- any worker logs around the crash
and I can help narrow down the exact cause.Akash N
08/11/2026, 7:44 AMwas cancelled by the runtime environmentMarvin
08/11/2026, 7:44 AMMarvin
08/11/2026, 7:44 AMOOMKilled, job timeout
- Docker: container stopped/restarted
- VM/server: process supervisor restart or machine interruption
3. Deployment/work pool settings
- any job timeout, TTL, or platform-level execution limits
4. Resource pressure
- memory/CPU exhaustion can trigger cancellation indirectly
Rule of thumb
- If your code threw an exception, you'd usually see Failed
- If the environment pulled the rug out from under the run, you'll often see Crashed: was cancelled by the runtime environment
If you're on Kubernetes, the most useful thing is usually:
- pod status
- pod events
- worker logs
- exit reason
If you want, tell me where the flow is running (Kubernetes, Docker, ECS, local worker, etc.) and I can give you the exact places to inspect.Akash N
08/11/2026, 7:46 AMMarvin
08/11/2026, 7:46 AMMarvin
08/11/2026, 7:46 AMwas cancelled by the runtime environment most often means the flow-run container was stopped/cancelled while it was still running — even if the worker container itself stayed up.
So the key distinction is:
- worker/container for the pool is up ✅
- the individual flow run container may still have been interrupted ❗
Things to check in a Docker setup:
1. Did the flow run container exit unexpectedly?
Run:
bash
docker ps -a
Look for the container created for that flow run and check its exit status.
2. Inspect that container
bash
docker inspect <container_id>
Look at:
- State.ExitCode
- State.OOMKilled
- State.Error
- start/finish timestamps
3. Check container logs
bash
docker logs <container_id>
If the logs cut off abruptly, that supports runtime cancellation / termination.
4. Check whether Docker restarted, pruned, or removed containers
Sometimes housekeeping, daemon restarts, or host-level interruptions can stop the run container without killing the long-lived worker.
5. Check host resource pressure
Especially memory pressure:
- low RAM
- OOM killer
- disk pressure
- Docker daemon restart
6. Check whether the run was manually cancelled
If something external requested cancellation, the runtime may surface it this way too.
A very common pattern with Docker pools
The worker submits a short-lived container for each flow run. The worker can remain healthy for days, while one specific run container gets:
- killed
- removed
- OOM’d
- timed out
- disconnected
So “worker has been up for 7 days” does not rule out a runtime cancellation.
If you want, run these and paste the output for the affected run container:
bash
docker ps -a
docker inspect <container_id>
docker logs <container_id>
If you don’t know the container id, I can help you find it from the flow run name/time.