<@ULVA73B9P> I have deployed a flow (a) which depl...
# ask-marvin
s
@Marvin I have deployed a flow (a) which deploys another flow (b) when it is run. When I cancel flow (a), I want to automatically cancel flow (b). How can I use automations or event triggers to achieve this?
m
thought for 649.3 seconds
Great question. In Prefect 3.x there are two solid patterns you can use: A) If flow b is a true subflow of a (same process) - Calling b as a subflow (either by calling it like a function, or by using
run_deployment(..., as_subflow=True)
) will couple its lifecycle to the parent. When you cancel a, Prefect will cancel the subflow as well. - If you’re seeing that b continues after cancelling a, you’re likely launching b as an independent deployment (e.g.,
run_deployment(..., as_subflow=False)
or default worker isolation). In that case, use pattern B below. B) Use an automation that reacts to “parent cancelled” and runs a tiny cleanup flow that cancels the children This works regardless of how you launch b and is the most reliable/explicit approach. 1) Launch b with a correlation you can use later (parent run ID is ideal) - Inside a, add the parent flow run id to the child as a tag and make b independent (so it won’t be implicitly tied to a’s process):
Copy code
from prefect import flow
from prefect.deployments import run_deployment
from prefect.runtime import flow_run as rt

@flow
def parent():
    parent_id = rt.id()
    # Launch child as an independent run and attach a correlation tag
    run_deployment(
        name="child-flow/child-deployment",
        as_subflow=False,
        tags=[f"parent:{parent_id}"],
    )
References: - Run deployments - prefect.runtime.flow_run 2) Create a small “cancel_children” flow that cancels all children for a given parent - This uses the Prefect client to find runs by
parent_flow_run_id
(or you can filter by the tag you set) and cancels them.
Copy code
from prefect import flow
from prefect.client.orchestration import get_client
from prefect.client.schemas.filters import (
    FlowRunFilter,
    FlowRunFilterParentFlowRunId,
    FlowRunFilterStateName,
)
from prefect.client.schemas.objects import State, StateType

@flow
async def cancel_children(parent_id: str):
    async with get_client() as client:
        runs = await client.read_flow_runs(
            flow_run_filter=FlowRunFilter(
                parent_flow_run_id=FlowRunFilterParentFlowRunId(any_=[parent_id]),
                # optionally skip terminal runs
                state=FlowRunFilterStateName(not_any_=["Completed", "Cancelled", "Failed"]),
            )
        )

        for r in runs:
            await client.set_flow_run_state(
                flow_run_id=r.id,
                state=State(type=StateType.CANCELLED, name="Cancelled"),
                force=True,  # ensure the cancel is applied
            )
- Deploy this flow (e.g., “ops/cancel-children”) so we can target it from an automation. 3) Create an automation that triggers when the parent is cancelled and runs the cleanup flow - In the Prefect UI, create an Automation with: - Trigger: Event-based (Reactive) - Event:
prefect.flow-run.Cancelled
- Optional filters to scope it to flow a (flow name or deployment name) - Action: Run a deployment - Deployment: your “ops/cancel-children” - Parameters: set
parent_id
to the triggering event’s flow run ID - In the UI you can insert the event variable; it’s typically
{{ event.resource.id }}
This setup means: - When you cancel a run of flow a, the automation fires. - It runs the
cancel_children
deployment with the right
parent_id
. - That flow cancels all children of the parent (including b). Docs: - Event triggers - Creating automations - Client/filter API: Client schemas API Optional enhancements and notes
- If you prefer tag-based correlation, filter in
cancel_children
by
FlowRunFilterTags
matching
parent:{parent_id}
instead of
parent_flow_run_id
. - If you also want to catch other terminal states (e.g., crash or failure of a), add additional events to your automation (
prefect.flow-run.Failed
,
prefect.flow-run.Crashed
) depending on your policy. - If you only ever have one child per parent and you want a “pure automation” approach, you could build a more advanced composite/sequence automation that correlates on tags/related resources and cancels the child when the parent cancels. In practice, the “run a cleanup deployment” pattern is simpler to reason about and maintain. Questions for you - Are you launching b with
run_deployment(..., as_subflow=True)
or
False
? And do you ever run multiple children from a single parent run? - Do you want to cancel just flow b or all children started by a? If you share those details, I can tailor the automation and code exactly to your setup.
s
@Marvin flow a might deploy flow b which might deploy further downstream nested flows. Each one is a separate deployed flow, not a subflow. Ideally I would like to be able to configure some type of listener on each of the deployed nested flows that automatically cancels the running flow if a cancellation event from the parent is created.
m
thought for 116.5 seconds
Short answer: Use one automation that reacts to any “parent cancelled” event and runs a tiny “cascade-cancel” flow which cancels all descendants (children, grandchildren, …). Prefect automations cannot directly cancel a different flow run than the one that raised the event, so the helper flow is the clean, reliable way to target the right runs. Here’s a concrete pattern that works well with independently deployed flows: 1) Launch children with a parent link you can query (no special tags required) - When you start downstream runs from inside a flow,
run_deployment
automatically sets the child’s
parent_flow_run_id
to the current flow run. Keep
as_subflow=False
to keep them independent.
Copy code
from prefect import flow
from prefect.deployments import run_deployment

@flow
def a():
    run_deployment(
        name="b/b-deployment",
        as_subflow=False,
        # optional: add correlation tags like a lineage id if you want
        # tags=[f"lineage:{prefect.runtime.flow_run.id()}"],
    )
Reference: Run deployments 2) Create a small “cascade-cancel” flow that cancels all descendants of a given run - This walks the parent→child graph via
parent_flow_run_id
and cancels everything that isn’t already finished. It handles nested flows (b→c→d…) by recursing/breadth-first.
Copy code
from collections import deque
from prefect import flow
from prefect.client.orchestration import get_client
from prefect.client.schemas.filters import (
    FlowRunFilter,
    FlowRunFilterParentFlowRunId,
)
from prefect.client.schemas.objects import State, StateType

TERMINAL = {StateType.COMPLETED, StateType.FAILED, StateType.CANCELLED, StateType.CRASHED}

@flow
async def cascade_cancel(root_parent_id: str, poll_rounds: int = 3, poll_delay_seconds: float = 3.0):
    """
    Cancel all descendants of `root_parent_id`. We poll a few rounds to catch
    children that are created just after the parent cancellation event fires.
    """
    async with get_client() as client:
        for _ in range(poll_rounds):
            queue = deque([root_parent_id])
            seen = set()

            while queue:
                parent_id = queue.popleft()
                if parent_id in seen:
                    continue
                seen.add(parent_id)

                children = await client.read_flow_runs(
                    flow_run_filter=FlowRunFilter(
                        parent_flow_run_id=FlowRunFilterParentFlowRunId(any_=[parent_id])
                    )
                )

                for r in children:
                    # Cancel active runs
                    if not r.state or r.state.type not in TERMINAL:
                        await client.set_flow_run_state(
                            flow_run_id=r.id,
                            state=State(type=StateType.CANCELLED, name="Cancelled by ancestor"),
                            force=True,
                        )
                    # Recurse to grandchildren
                    queue.append(r.id)

            # Optional: small delay to catch late-arriving children
            if poll_delay_seconds:
                import asyncio
                await asyncio.sleep(poll_delay_seconds)
3) Deploy that flow (e.g., as “ops/cascade-cancel”) and wire an automation - In the Prefect UI: - Go to Automations -> Create - Trigger: Event-based (Reactive) - Event: Flow run Cancelled (use the Events builder to select “Flow run” + “Cancelled”) - Optional filters: restrict to just your lineage (e.g., flow “a” or a specific deployment name) if you don’t want to cascade-cancel everything - Action: Run a deployment - Choose the “ops/cascade-cancel” deployment - Parameters: set
root_parent_id
to the cancelling run’s id. Use the event variable in the UI (typically
{{ event.resource.id }}
) References: - Event triggers
- Creating automations - API objects/filters used above: Client schemas API Why this pattern - Automations actions like “CancelFlowRun” act on the triggering event’s resource. They cannot directly “cancel related flow runs.” Using “Run a deployment” lets you pass the parent id into a helper flow that can find and cancel the correct targets. - The helper flow is resilient and easy to evolve (e.g., add retries, delays, audit logs, or metrics). Optional: add a lineage tag to all descendants - If you want simple, fast lookups without recursion, propagate a lineage tag from the first parent (e.g.,
lineage:<a_run_id>
) to all downstream
run_deployment
calls. Then your cascade flow can query all active runs with that tag and cancel them. This is not required, but it simplifies targeting if you prefer tags. Notes - Race conditions: a child may spawn just after the automation fires. The helper flow above polls a few times to catch those late-arriving runs; tune
poll_rounds
and
poll_delay_seconds
to your needs. - Scope: If you only want to cancel immediate children of a given parent, remove the recursion in the helper flow. Would you like me to tailor the automation filter to only cascade-cancel descendants of flows started by “a,” or do you want this policy to apply to any flow family that follows this parent→child pattern?
s
@Marvin what state do tasks go into when the parent flow is cancelled?
m
thought for 182.1 seconds
In Prefect 3.x: - When you cancel a flow run, the flow typically goes to Cancelling (non-terminal) then to Cancelled (terminal). - All unfinished task runs in that flow are transitioned to Cancelled (terminal). This includes tasks that were Running, Pending, or Scheduled. - Task runs do not have a separate “Cancelling” phase — they go straight to Cancelled. - Completed/Failed task runs remain in their terminal state; only unfinished ones are changed. - No new tasks will be scheduled once the parent flow is cancelling/cancelled. References: - Cancel workflows - States in Prefect