Aiden Price

01/14/2021, 10:59 PM
Hi everyone, I ran into the
Could not serialize object of type Success
problem that I've seen a few times on this Slack. From my reading of I suspect it's because I'm using
classes to hold database connection and HTTP connection pools while I map over everything in my Dask cluster. Is it not possible to share a connection pool across a Dask mapped pipeline (thinking about it now, it would be difficult)? If not how should I go about mapped reading & writing? Thanks.
Something I forgot to mention, I'm also trying to cache the result in this instance, which will then lead to mapping downstream of this point.