Hello, I’m having this error, all the solutions se...
# ask-community
j
Hello, I’m having this error, all the solutions seems outdated
Copy code
TypeError: cannot pickle '_thread.lock' object
a
Can you share more info about your flow structure? Do you use any database connections or HTTP clients? Those cannot be serialized and sent between Dask workers. If you don’t use Dask, you may look at Script-based storage. Kevin explained it here.
upvote 2
k
Hey @Joseph Oladokun, this is likely from a task having an output that can’t be serialized as Anna mentioned
j
Thanks @Anna Geller for your response, yes, i use http clients to fetch data from an API
@Kevin Kho I’m fetching data from an API and writing it to the s3 bucket
a
Did you manage to solve it? If you move the boto3 client to the task in which you interact with S3, this should solve the serialization issue.
j
Thanks @Anna Geller, I have been able to solve the issue
👍 1