Hi guys
I’m having issues when using python’s multiprocessing to run multiple flows in parallel. I know that there’s an integration with Dask for these things, but I do wonder if the error’s source is on my end or on Prefect’s end.
The error I’m getting is:
_pickle.PicklingError: Can't pickle <function name at 0x7fb2a9b37790>: import of module '__prefect_loader__' failed
I’m getting this error only when running the deployment through Prefect Cloud (Runs on Kubernetes), but not when running the flow locally.
Thanks
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.