David Martin Calalang
07/10/2025, 8:42 PMsquare
is the name of the task that I'm passing to my flow...
Traceback (most recent call last):
File "/usr/local/lib/python3.13/site-packages/distributed/protocol/pickle.py", line 60, in dumps
result = pickle.dumps(x, **dump_kwargs)
_pickle.PicklingError: Can't pickle <function square at 0x7b68d25cd1c0>: it's not the same object as __main__.square
For more context, I'm using the dask task_runner with a pre-existing Fargate cluster I have on AWS ECS. From my understanding prefect serializes tasks prior to sending them to the dask scheduler, and that might be where my issue is. Not quite sure where to look/what to fix in my code regarding this though.
Thanks!