<@ULVA73B9P> I have a run stuck in the "Cancelling...
# ask-marvin
d
@Marvin I have a run stuck in the "Cancelling" state and it won't finish up moving to Cancelled. Is there a way to cancel this run. This may be related to errors reported by other users in this slack channel earlier today.
m
thought for 105.5 seconds
Yes — if a run is already stuck in
Cancelling
, you can force it to
Cancelled
. A few useful details first: - In Prefect 3.x,
Cancelling
is an intermediate state while Prefect tries to tear down infrastructure. - There is normally an automatic cleanup timeout that should move it to
Cancelled
after about 5 minutes by default. - If it stays there longer than that, it’s reasonable to manually intervene. Best manual recovery option If you’re comfortable using the API, force the run state to `CANCELLED`:
Copy code
bash
curl -X POST "$PREFECT_API_URL/flow_runs/<FLOW_RUN_ID>/set_state" \
  -H "Content-Type: application/json" \
  -d '{
    "state": {
      "type": "CANCELLED",
      "name": "Cancelled",
      "message": "Manual recovery from stuck Cancelling state"
    },
    "force": true
  }'
Or with the Python client:
Copy code
python
from uuid import UUID
from prefect import get_client
from prefect.client.schemas.objects import State, StateType

flow_run_id = UUID("<FLOW_RUN_ID>")

async def main():
    async with get_client() as client:
        await client.set_flow_run_state(
            flow_run_id=flow_run_id,
            state=State(
                type=StateType.CANCELLED,
                name="Cancelled",
                message="Manual recovery from stuck Cancelling state",
            ),
            force=True,
        )

import asyncio
asyncio.run(main())
Important caveat Only do this if the run is already stuck in
Cancelling
. You generally should not force a
Running
run directly to
Cancelled
, because that can skip cleanup behavior. If you’re self-hosting Prefect there’s also a server-side timeout that controls when stuck
Cancelling
runs are auto-finalized:
Copy code
bash
PREFECT_SERVER_SERVICES_CANCELLATION_CLEANUP_CANCELLING_TIMEOUT_SECONDS=300
So if this is happening repeatedly, it may indicate: - a worker problem - unreachable/unresponsive infrastructure - a broader incident related to cancellation handling I also checked for reports in indexed Slack threads from earlier today but didn’t find a matching retained thread in my search. Relevant docs: - Cancel workflows - Detect zombie flows If you want, I can also help you with: - a Prefect CLI / Python snippet tailored to your environment - checking whether this is Cloud vs self-hosted - safe cleanup steps if the underlying job/container may still be running