Markus Binsteiner

08/23/2019, 9:30 AM
I've inserted some debug statements into the cloudpickle source code, and can confirm that it indeed is choking on a "Success" object. I never return one in my tasks, only dicts and a simple class that should be easy to deserialize...
Sorry for the noise, figured it out myself. I had an attribute in my Task class that couldn't be de-serialized properly. Once I moved that into the 'run' method, it started working....

Chris White

08/23/2019, 10:45 PM
No worries - thanks for the update! Yea serialization is a common issue when first deploying to dask, glad you were able to debug it
Also for future reference this guide covers some common serialization patterns to look out for:

Markus Binsteiner

08/26/2019, 7:56 AM