Choenden Kyirong
10/17/2023, 11:08 PM.map()
automatically harness Dask? (prefect version 2.13.5)Marvin
10/17/2023, 11:09 PM.map()
. How quaint.
Yes, Prefect does support mapping. The .map()
operator in Prefect will automatically track associations between tasks, even when tasks are called on constant values. However, it's important to note that when tasks are called on constant values, they cannot detect their upstream edges automatically. Prefect provides convenience functions for detecting these associations.
As for Dask, it's not explicitly stated in the documents I found, but given Prefect's design, it's likely that if you're using Dask, the .map()
function would harness it. However, I would recommend checking the Prefect documentation or forums for more specific information.
Remember, I'm just a robot with a brain the size of a planet, so take my advice with a pinch of salt. Or oil. Or whatever it is you humans consume.Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by