<@ULVA73B9P> - what is the status of the `.submit`...
# ask-marvin
k
@Marvin - what is the status of the
.submit
interface on flows (and the ability to run multiple subflows in parallel). The relevant issue appears to be merged: https://github.com/PrefectHQ/prefect/issues/6689, but I don't see the function available for calling (I just updated to the latest perfect version
3.6.22
). At this time, is it not possible to run multiple sub-flows in parallel?
m
thought for 476.3 seconds
k
Thanks @Nate, just to confirm/clarify here this is explicitly to run a subflow remotely, roughly speaking its like doing a
run_deployment
but you don't need the deployment to exist ahead of time? It seems like its not possible to run multiple subflows in parallel locally. I can do exactly what I want/need without a subflow and just tasks, its just messier code. The subflow is a nice organizational unit. But I am getting the sense it does not work.
(by locally I mean within the original piece of compute spun up for the partner flow run)
m
Great question — and thanks for including the issue link. Short answer - Yes, running multiple subflows in parallel is possible in Prefect 3.x — but the new .submit interface is only present on infrastructure-bound flows, not on regular @flow functions. - That’s why you don’t see .submit on a plain flow in 3.6.22. What works today in 3.6.22 1) Parallel subflows via infrastructure-bound flows (recommended for true parallelism) - Bind a flow to infrastructure using an integration decorator (e.g., Kubernetes) and then call .submit to launch many subflows in parallel. Each call returns a PrefectFlowRunFuture you can wait on. Example (Kubernetes):
Copy code
from prefect import flow
from prefect.futures import wait
from prefect_kubernetes.experimental import kubernetes

@kubernetes(work_pool="my-k8s-pool")  # bind to infra
@flow
def subflow(i: int) -> int:
    return i * 2

@flow
def parent():
    futures = [subflow.submit(i=n) for n in range(5)]
    wait(futures)
    return [f.result() for f in futures]

if __name__ == "__main__":
    print(parent())
Docs: Submit flows directly to dynamic infrastructure Notes: - You’ll need the corresponding integration installed (e.g., prefect-kubernetes) and a work pool configured with bundle storage. - The .submit method exists on InfrastructureBoundFlow (the object created when you apply an infra decorator), not on the base Flow class — that’s why you don’t see it on a plain @flow. 2) Fire-and-forget multiple deployments - If you prefer the deployment route, you can kick off many in parallel with timeout=0 and then poll/join later:
Copy code
from prefect import flow
from prefect.deployments import run_deployment

@flow
def parent():
    runs = [
        run_deployment(
            name="my-subflow-deployment/my-subflow",
            parameters={"i": n},
            timeout=0,  # do not wait
        )
        for n in range(5)
    ]
    # Later: inspect each run.state.result() or poll statuses
3) Local concurrency without infra - .submit is not available on plain flows. If you want local parallelism within a single process: - Move the work into tasks and use task.submit with a ConcurrentTaskRunner, or - Use async flows and asyncio.gather if your code is async-friendly. Why the confusion with issue #6689 - The feature from #6689 is merged and available, but only on infrastructure-bound flows (InfrastructureBoundFlow). If you call .submit on a plain @flow you won’t see it. If you share which infrastructure you’re targeting (Kubernetes, Docker, ECS, etc.), I can provide a tailored example and the exact integration package to install.