Short answer: 3.6.22 includes important fixes for cancellation reliability, but there are still edge cases where runs can linger in Cancelling, especially in certain infra setups (notably Kubernetes) and some parent/child-flow scenarios. So it’s not universally “gone,” but we can usually root-cause and eliminate it with a few checks. What 3.6.22 changed - Handled cancellation observer subscriber connection failures more gracefully (reduces cases where the runner/worker misses the cancel signal): 3.6.22 release notes (see “Handle FlowRunCancellingObserver subscriber connection failure gracefully”) - Related improvements in recent 3.6.x: heartbeat defaults and cancellation-observer robustness Active/related reports - Flows stuck in Cancelling with Kubernetes work pools (3.6.4): GitHub issue #19593 - Race condition in parent/child cancellations (earlier 3.x): GitHub issue #15399 A quick triage checklist 1) Version parity - Are your workers and server/Cloud UI all on 3.6.22? Mixing versions can reintroduce fixed behaviors. - If using collections (e.g., prefect-kubernetes), which versions are installed on the worker image? 2) Infra tracking - For a stuck run, does the flow run show an infrastructure PID/handle (e.g., k8s job/Pod, process pid, container id)? If it’s missing, the API can’t reconcile the cancellation to a terminal state. 3) Worker logs at cancel time - Set

PREFECT_LOGGING_LEVEL=DEBUG

on workers and try another cancel. You should see logs from the cancellation observer (websocket and/or polling fallback). If the worker cannot observe cancels (networking/websocket issues), the run can linger in Cancelling. 4) Kubernetes specifics (if applicable) - Confirm the worker has RBAC to delete/terminate Jobs/Pods it creates. - Check the Job’s terminationGracePeriodSeconds and that the Pod actually terminates promptly. If the Pod dies but Prefect never sees/records that via the worker (observer failure), the state can linger. - Share any custom

job_variables

relevant to termination behavior. 5) Heartbeats/connectivity - Ensure worker and flow-run heartbeats are making it back to the API. 3.6.21+ sets a heartbeat default, but firewall/proxy/websocket issues can still block the observer. If you can share a concrete example, I’ll help you pinpoint the cause: - Cloud or self-hosted Server? - Work pool type (process, Docker, Kubernetes) and versions (worker, collections) - A flow_run_id that’s stuck and whether you see an infrastructure PID on it - A small snippet of worker logs around the time you pressed Cancel Also, to set expectations: 3.6.22 fixes one class of cancellation-observer failures, but some K8s and nested-flow edge cases are still under investigation (see the issues linked above). Once we have your specifics, we can confirm whether you’re hitting a known edge case or a networking/configuration problem we can fix on the spot. Helpful links - 3.6.22 notes: Release notes - K8s Cancelling report: Issue #19593 - Parent/child race-condition report: Issue #15399 - Cancellation guide: Workflow cancellation in Prefect 3

Short answer: you can’t. In Prefect 3.x, the UI does not offer a “force cancel” for runs already in Cancelling. The Cancel action is hidden for Cancelling, and “Change state” is only available for terminal states (Completed, Failed, Cancelled, Crashed). Cleanest workaround - If you just need it to stop and close out properly, force the state to Cancelled via API/CLI: - Python (async):

import asyncio
    from uuid import UUID
    from prefect.client.orchestration import get_client
    from prefect.states import Cancelled

    async def force_cancel(flow_run_id: str):
        async with get_client() as client:
            await client.set_flow_run_state(
                flow_run_id=UUID(flow_run_id),
                state=Cancelled(message="Force-cancelled from stuck Cancelling"),
                force=True,
            )

    asyncio.run(force_cancel("YOUR-FLOW-RUN-ID"))

- REST:

curl -X POST https://<your-api>/api/flow_runs/<FLOW_RUN_ID>/set_state \
      -H "Authorization: Bearer <TOKEN>" \
      -H "Content-Type: application/json" \
      -d '{
        "state": { "type": "CANCELLED", "name": "Cancelled", "message": "Force cancelled" },
        "force": true
      }'

Notes -

prefect flow-run cancel <id>

exists, but it only requests cancellation; it won’t resolve a run already stuck in Cancelling. - From the UI you can delete the run if you just want it off your list, but deletion does not affect any underlying infrastructure. Docs: Cancel workflows in Prefect 3