https://prefect.io
Join Slack
Hello, is it possible to use multiple different images for tasks in a prefect flow? And with a Kuber...
# prefect-server
m
Hello, is it possible to use multiple different images for tasks in a prefect flow? And with a Kubernetes runner or Dask runner?
a
m
thanks! I’ll check it out:)
👍 1
in order to get flow of flows to work, do i have to use wait_for_flow_a? or can i just create_flow_run and define upstream_tasks as previous create_flow_run?
So an example would be:
Copy code
with Flow("Flow_of_flows", storage=STORAGE) as flow:

    child1 = create_flow_run(
        flow_name="flow_child1",
        project_name=PROJECT_NAME,
        run_config=RUNCONFIG,
    )

    child2 = create_flow_run(
        flow_name="flow_child2",
        project_name=PROJECT_NAME,
        upstream_tasks=[child1],
        run_config=RUNCONFIG,
    )
a
yes, that works - the wait task is required if you need to start the next flow run only after the first once completed - it is essentially polling for the state of other flow run to determine whether to run downstream tasks or block and not run downstream tasks if e.g. one child flow run failed
m
thanks for the clarification!
👍 1
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by