https://prefect.io logo
f

flavienbwk

10/11/2020, 4:10 PM
Hi, about Prefect flows and agents : does 1 flow == 1 agent, or are tasks in flows parallelized over the available agents ? Should I use Dask for that ?
j

Jacob Blanco

10/12/2020, 12:47 AM
Agents are responsible for managing the launching of flows. A single agent can run multiple flows depending on your setup. Dask is for task-level parallelization like when you map over inputs.
👍 1
upvote 2
k

Krzysztof Nawara

10/12/2020, 7:36 AM
@Jacob Blanco But it's not restricted to mapped-tasks only, right? It's enough that you have independent branches in your DAG to take advantage of the speedup offered by Dask?
d

Dylan

10/12/2020, 3:23 PM
@Krzysztof Nawara that’s correct 👍
Prefect + Dask help ensure that 1) your tasks are executed as fast as infrastructure becomes available 2) your task dependencies are respected