https://prefect.io logo
Title
s

Sander

09/13/2022, 11:59 AM
Hi, trying out some simple map reduce stuff on latest prefect on a dask cluster. Should I use task.submit() or can I also use task.map()?
1
k

Khuyen Tran

09/13/2022, 3:08 PM
You can use task use task.map
s

Sander

09/13/2022, 3:28 PM
Is there any difference in terms of what happens in the backend? Or are they identical in the sense that map just hides away the for loop?
k

Khuyen Tran

09/14/2022, 3:24 PM
I believe map just hides away the for loop. But maybe you can answer this better than me @Zanie, @Anna Geller
z

Zanie

09/14/2022, 6:43 PM
map
is a bit more performant than a for loop as we gather all upstream dependencies concurently.
:thank-you: 1
If you’re calling
map
without any upstream tasks as inputs, it’s probably going to be the same as a for loop. I believe in the future we’ll collapse mapped tasks in the UI and such to make the graph clearer.
s

Sander

09/14/2022, 6:46 PM
Thanks!