<@ULVA73B9P> can i use from prefect.deployments im...
# ask-marvin
m
@Marvin can i use from prefect.deployments import run_deployment to run a flow not as sub flow, but still show in the main flow ui to easily find the new flow
m
thought for 329.0 seconds
Short answer: yes—with a caveat. - In Prefect 3.x,
run_deployment
supports
as_subflow=False
to start a new flow run without creating a subflow relationship. - However, if you disable subflow linking, the child run will not appear in the parent’s execution graph. To make it easy to find from the parent flow’s UI page, add a link artifact (and/or tags) after you start it. Example pattern:
Copy code
from prefect import flow
from prefect.deployments import run_deployment
from prefect.runtime import flow_run
from prefect.artifacts import create_link_artifact
from prefect.utilities.urls import url_for

@flow
async def parent():
    # Trigger without creating a subflow edge
    child = await run_deployment(
        "child-flow/prod",
        parameters={"x": 1},
        as_subflow=False,   # <- key part
        timeout=0,          # return immediately; omit or set >0 to wait
        tags=[f"launched-by:{flow_run.id}"],  # optional: add a correlation tag
    )

    # Drop a clickable link on the parent run’s Artifacts tab
    create_link_artifact(
        link=url_for(child),
        link_text=f"Child run: {child.name or child.id}",
        description="Triggered by parent flow",
        key=f"child-{child.id}",  # optional de-dupe key
    )

    return child.id
Notes: - If you want it to appear in the parent’s graph, use
as_subflow=True
(default). That is the only way to get a native parent→child graph edge. - Using tags like
launched-by:{flow_run.id}
on the child run helps you filter/find it across the UI. Docs: - run_deployment API - Artifacts (concepts) - Artifacts API
m
@Marvin my issue is when it run as subflow ,it run in same ecs task, some time it need more cpu/memory. thats why
m
thought for 133.5 seconds
Got it — if a “subflow” runs in the same ECS task, that’s the inline subflow pattern (calling
child_flow()
directly). That will always share the same container resources. If you want the child to run in its own ECS task with bigger CPU/memory but still show under the parent in the UI, use a deployment and call
run_deployment
with
as_subflow=True
(the default) and pass higher
job_variables
. This will: - Schedule a new flow run on your ECS work pool (new ECS task) - Keep the parent→child link in the UI - Let you override CPU/memory per call Example for ECS EC2:
Copy code
from prefect import flow
from prefect.deployments import run_deployment

@flow
async def parent():
    child_run = await run_deployment(
        "heavy-flow/prod",                # flow/deployment
        parameters={"x": 1},
        as_subflow=True,                  # keep UI linkage
        timeout=None,                     # wait for completion; use 0 to return immediately
        job_variables={
            "launch_type": "EC2",         # you’re using ECS EC2
            "cluster": "my-ecs-cluster",
            "cpu": 4096,                  # container CPU units
            "memory": 8192,               # container memory in MB
            # Optional runtime env
            "env": {
                "EXTRA_FLAG": "1"
            },
        },
    )
    return child_run.id
Notes: -
run_deployment(..., as_subflow=True)
still launches a separate ECS task; it does not run inline. -
job_variables
you pass here override the deployment defaults for this run. - Alternative: create a second “heavy” deployment with larger CPU/memory and call that when needed. If you actually don’t want the child to appear in the graph, set
as_subflow=False
and, if discoverability matters, drop a link artifact to the child run from the parent. Refs: - run_deployment API Quick check: are you currently calling the child as a function (inline) or via
run_deployment
? If inline, switching to
run_deployment
as above will solve the same-task resource contention.
m
@Marvin ok for the local developmet i use process pool. how to keep both in the same code base child = run_deployment( name=f"{flow_name}/{deployment_name}", as_subflow=False, timeout=1, # return immediately; omit or set >0 to wait parameters= parameters)
m
thought for 96.2 seconds
Great question. The cleanest way to keep one code base and support both local (Process pool) and prod (ECS EC2) is: - Create two deployments of the same child flow: e.g.,
child/dev
(Process work pool) and
child/prod
(ECS work pool with larger CPU/memory). - Choose which deployment to call at runtime via a Prefect Variable or environment variable. - Do not try to switch work pools from
run_deployment
— the pool is fixed by the deployment. You can only override the work queue name, not the pool. Two simple patterns: 1) Use a Prefect Variable to pick the deployment - Set a Variable
CHILD_DEPLOYMENT
to
child/dev
in dev and
child/prod
in prod. - Code stays identical everywhere.
Copy code
from prefect import flow
from prefect.deployments import run_deployment
from prefect.variables import Variable

@flow
async def parent(parameters: dict):
    child_ref = await Variable.get("CHILD_DEPLOYMENT", default="child/dev")
    child = await run_deployment(
        name=child_ref,
        as_subflow=True,     # set False if you don’t want a UI graph edge
        timeout=0,           # return immediately; set >0 to wait
        parameters=parameters,
        # Do NOT pass ECS-only job_variables in dev; keep those on the prod deployment
    )
    return child.id
2) Branch by environment variable - Keep ECS-specific sizing only in prod so you don’t fail validation against the Process pool template.
Copy code
import os
from prefect import flow
from prefect.deployments import run_deployment

@flow
async def parent(parameters: dict):
    env = os.getenv("APP_ENV", "dev")  # dev | prod
    deployment = f"child/{env}"

    job_vars = None
    if env == "prod":
        job_vars = {
            "launch_type": "EC2",
            "cpu": 4096,
            "memory": 8192,
            # optional: "cluster": "my-ecs-cluster",
            # optional: "env": {"EXTRA_FLAG": "1"},
        }

    child = await run_deployment(
        name=deployment,
        as_subflow=True,
        timeout=0,
        parameters=parameters,
        job_variables=job_vars,  # Only for prod; omit for dev
    )
    return child.id
Why this works: - You get a dev deployment that a Process worker can run locally. - You get a prod deployment that an ECS worker can run as a new ECS task with larger CPU/memory. - The parent code never changes; you just flip config. Notes: - If you want the child to show under the parent in the UI, use
as_subflow=True
(default). If you truly want it independent, keep
as_subflow=False
and optionally post a link artifact to the child run from the parent. -
run_deployment
cannot change the work pool; it triggers the pool configured on the deployment. That’s why separate deployments per env are the right abstraction. - If your parent flow is async, remember to
await run_deployment(...)
. In a sync flow, calling it directly is fine. Refs: - run_deployment API: run_deployment - Variables: Variables If you want, I can show a small script to create both deployments programmatically (Process for dev, ECS EC2 for prod) so you can wire this into CI.