Hi there If I pass an object to a task with the dask runtime

Question

Hi there If I pass an object to a task with the dask runtime What exactly will happen to this object I was thinking it would be serialized and copied for every task But it seems that there will also be some logic runned Is this process documented anywhere Sure I can give more details if needed

Michael Hadorn · Answer

Ok to specify my question how is cloudpickle working My object i wanna process with tasks need some data from the database first In the task the object can be used without database because all the needed properties are correctly filled Currently every worker is redo the database initialized although this is not in the init method

Dylan · Answer

Hi < Michael Hadorn> I m not sure that I totally understand your question Could you provide a short code snippet to illustrate the behavior that s confusing

Michael Hadorn · Answer

Hi Dylan Thx for your answer Sorry didn t got the notification My problem is solved now I messed up the init part of my flow thats way some code was run again on every dask worker Best wishes Michael

Dylan · Answer

Glad you got it sorted