https://prefect.io logo
b

bral

01/02/2021, 7:54 AM
hi folks. I am using a dask executor to perform tasks on multiple machines via map (). but it happens that the processing of one task takes a very long time and because I work with a pandas and I cannot parallelize the calculations already inside map (). I would like to use modin for this. has anyone tried something like this?
n

nicholas

01/02/2021, 2:02 PM
Hi @bral - I haven't personally used modin so others might want to chime in here but I see no reason you couldn't use it, particularly in conjunction with Prefect resource manager tasks (depending on how you want to manage your dask resources).
b

bral

01/02/2021, 5:08 PM
nicholas thank you! Modin has an api similar to pandas. Ok, i will try your approach ! Best regards!
😄 1
4 Views