https://prefect.io logo
a

Antonio Manuel BR

02/21/2022, 7:47 AM
Hello, does it make sense to distribute a Prefect task using Dask (e.g. predict a large dataframe with a ML model), in a Prefect Flow that already uses a remote DaskExecutor?
a

Anna Geller

02/21/2022, 9:43 AM
Not sure I understood the issue. Is your flow already using Dask with a resource manager and you are thinking whether adding Dask executor makes sense? Can you explain your use case a bit more and share your flow?
k

Kevin Kho

02/21/2022, 1:09 PM
You can use Dask Dataframes with the syntax introduced here
5 Views