Madushan Pathirana
12/26/2025, 11:59 AMMarvin
12/26/2025, 11:59 AMMarvin
12/26/2025, 12:05 PMrun_deployment supports as_subflow=False to start a new flow run without creating a subflow relationship.
- However, if you disable subflow linking, the child run will not appear in the parent’s execution graph. To make it easy to find from the parent flow’s UI page, add a link artifact (and/or tags) after you start it.
Example pattern:
from prefect import flow
from prefect.deployments import run_deployment
from prefect.runtime import flow_run
from prefect.artifacts import create_link_artifact
from prefect.utilities.urls import url_for
@flow
async def parent():
# Trigger without creating a subflow edge
child = await run_deployment(
"child-flow/prod",
parameters={"x": 1},
as_subflow=False, # <- key part
timeout=0, # return immediately; omit or set >0 to wait
tags=[f"launched-by:{flow_run.id}"], # optional: add a correlation tag
)
# Drop a clickable link on the parent run’s Artifacts tab
create_link_artifact(
link=url_for(child),
link_text=f"Child run: {child.name or child.id}",
description="Triggered by parent flow",
key=f"child-{child.id}", # optional de-dupe key
)
return child.id
Notes:
- If you want it to appear in the parent’s graph, use as_subflow=True (default). That is the only way to get a native parent→child graph edge.
- Using tags like launched-by:{flow_run.id} on the child run helps you filter/find it across the UI.
Docs:
- run_deployment API
- Artifacts (concepts)
- Artifacts APIMadushan Pathirana
12/26/2025, 12:13 PMMarvin
12/26/2025, 12:13 PMMarvin
12/26/2025, 12:16 PMchild_flow() directly). That will always share the same container resources.
If you want the child to run in its own ECS task with bigger CPU/memory but still show under the parent in the UI, use a deployment and call run_deployment with as_subflow=True (the default) and pass higher job_variables. This will:
- Schedule a new flow run on your ECS work pool (new ECS task)
- Keep the parent→child link in the UI
- Let you override CPU/memory per call
Example for ECS EC2:
from prefect import flow
from prefect.deployments import run_deployment
@flow
async def parent():
child_run = await run_deployment(
"heavy-flow/prod", # flow/deployment
parameters={"x": 1},
as_subflow=True, # keep UI linkage
timeout=None, # wait for completion; use 0 to return immediately
job_variables={
"launch_type": "EC2", # you’re using ECS EC2
"cluster": "my-ecs-cluster",
"cpu": 4096, # container CPU units
"memory": 8192, # container memory in MB
# Optional runtime env
"env": {
"EXTRA_FLAG": "1"
},
},
)
return child_run.id
Notes:
- run_deployment(..., as_subflow=True) still launches a separate ECS task; it does not run inline.
- job_variables you pass here override the deployment defaults for this run.
- Alternative: create a second “heavy” deployment with larger CPU/memory and call that when needed.
If you actually don’t want the child to appear in the graph, set as_subflow=False and, if discoverability matters, drop a link artifact to the child run from the parent.
Refs:
- run_deployment API
Quick check: are you currently calling the child as a function (inline) or via run_deployment? If inline, switching to run_deployment as above will solve the same-task resource contention.Madushan Pathirana
12/26/2025, 12:24 PMMarvin
12/26/2025, 12:24 PMMarvin
12/26/2025, 12:25 PMchild/dev (Process work pool) and child/prod (ECS work pool with larger CPU/memory).
- Choose which deployment to call at runtime via a Prefect Variable or environment variable.
- Do not try to switch work pools from run_deployment — the pool is fixed by the deployment. You can only override the work queue name, not the pool.
Two simple patterns:
1) Use a Prefect Variable to pick the deployment
- Set a Variable CHILD_DEPLOYMENT to child/dev in dev and child/prod in prod.
- Code stays identical everywhere.
from prefect import flow
from prefect.deployments import run_deployment
from prefect.variables import Variable
@flow
async def parent(parameters: dict):
child_ref = await Variable.get("CHILD_DEPLOYMENT", default="child/dev")
child = await run_deployment(
name=child_ref,
as_subflow=True, # set False if you don’t want a UI graph edge
timeout=0, # return immediately; set >0 to wait
parameters=parameters,
# Do NOT pass ECS-only job_variables in dev; keep those on the prod deployment
)
return child.id
2) Branch by environment variable
- Keep ECS-specific sizing only in prod so you don’t fail validation against the Process pool template.
import os
from prefect import flow
from prefect.deployments import run_deployment
@flow
async def parent(parameters: dict):
env = os.getenv("APP_ENV", "dev") # dev | prod
deployment = f"child/{env}"
job_vars = None
if env == "prod":
job_vars = {
"launch_type": "EC2",
"cpu": 4096,
"memory": 8192,
# optional: "cluster": "my-ecs-cluster",
# optional: "env": {"EXTRA_FLAG": "1"},
}
child = await run_deployment(
name=deployment,
as_subflow=True,
timeout=0,
parameters=parameters,
job_variables=job_vars, # Only for prod; omit for dev
)
return child.id
Why this works:
- You get a dev deployment that a Process worker can run locally.
- You get a prod deployment that an ECS worker can run as a new ECS task with larger CPU/memory.
- The parent code never changes; you just flip config.
Notes:
- If you want the child to show under the parent in the UI, use as_subflow=True (default). If you truly want it independent, keep as_subflow=False and optionally post a link artifact to the child run from the parent.
- run_deployment cannot change the work pool; it triggers the pool configured on the deployment. That’s why separate deployments per env are the right abstraction.
- If your parent flow is async, remember to await run_deployment(...). In a sync flow, calling it directly is fine.
Refs:
- run_deployment API: run_deployment
- Variables: Variables
If you want, I can show a small script to create both deployments programmatically (Process for dev, ECS EC2 for prod) so you can wire this into CI.