I'm trying to figure out how to run some Tasks in parallel, like something like this in the second stage. There is no data passed between the stages, the tasks in the previous stage just have to complete with Success I know how to run them sequentailly like task_2(task_1()) but unclear if I should be using .map?

Hi Eric, how you build your flow is independent of how you execute your flow. In this case, whether your Flow can be executed in parallel is determined by your choice of executor. Check out this doc for more in-depth info on parallelism: https://docs.prefect.io/core/idioms/parallel.html

ah so it needs Dask then for this second stage parallelism

Yea no worries! Our docs have grown to a point where discoverability is a big challenge; it’s something we plan to work on and improve soon!

will the flow.visualize() automatically show the parallelism once dask is the execution engine?

Hm no, flow.visualize only shows your flows structure, not its runtime properties - if you use the Prefect UI you can best detect parallelism on the gantt chart view

does 1 worker in DaskKubernetesEnvironment environment equal a single process, i.e. 1 task can be run concurrently?

Not necessarily; each worker can be started with nthreads or nprocs flags (which have sensible defaults based on machine configuration) that give each individual worker parallelism capabilities

got it, that makes sense

Yup exactly