Hello, does it make sense to distribute a Prefect task using Dask (e.g. predict a large dataframe with a ML model), in a Prefect Flow that already uses a remote DaskExecutor?
a
Anna Geller
02/21/2022, 9:43 AM
Not sure I understood the issue. Is your flow already using Dask with a resource manager and you are thinking whether adding Dask executor makes sense?
Can you explain your use case a bit more and share your flow?
k
Kevin Kho
02/21/2022, 1:09 PM
You can use Dask Dataframes with the syntax introduced here