Jimmy Le
10/21/2020, 10:20 PMTypeError: cannot pickle '_thread.lock' object
.
Has anyone run into a similar problem? Any suggestions would be appreciated!Dylan
Client
for something that’s not thread safe.Jimmy Le
10/22/2020, 2:41 PMDylan
Marwan Sarieddine
10/22/2020, 5:01 PMimport attr
from prefect import Flow, task
@attr.s(auto_attribs=True, kw_only=True)
class A:
size: int
@task
def get_size():
a = A(size=2)
return a.size
with Flow("test-flow") as flow:
get_size()
flow.register("test-flows")
traceback:
Traceback (most recent call last):
File "attr_flow.py", line 19, in <module>
flow.register("test-flows")
File "/Users/marwansarieddine/.pyenv/versions/etl-embs/lib/python3.8/site-packages/prefect/core/flow.py", line 1608, in register
registered_flow = client.register(
File "/Users/marwansarieddine/.pyenv/versions/etl-embs/lib/python3.8/site-packages/prefect/client/client.py", line 734, in register
serialized_flow = flow.serialize(build=build) # type: Any
File "/Users/marwansarieddine/.pyenv/versions/etl-embs/lib/python3.8/site-packages/prefect/core/flow.py", line 1451, in serialize
self.storage.add_flow(self)
File "/Users/marwansarieddine/.pyenv/versions/etl-embs/lib/python3.8/site-packages/prefect/environments/storage/local.py", line 140, in add_flow
flow_location = flow.save(flow_location)
File "/Users/marwansarieddine/.pyenv/versions/etl-embs/lib/python3.8/site-packages/prefect/core/flow.py", line 1520, in save
cloudpickle.dump(self, f)
File "/Users/marwansarieddine/.pyenv/versions/etl-embs/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 55, in dump
CloudPickler(
File "/Users/marwansarieddine/.pyenv/versions/etl-embs/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 563, in dump
return Pickler.dump(self, obj)
TypeError: cannot pickle '_thread._local' object
Dylan
stored_as_script=True
option might solve this issueMarwan Sarieddine
10/22/2020, 5:13 PMJimmy Le
10/25/2020, 7:52 PMf.storage = Local(path="path/to/flow.py", stored_as_script=True)
Registered my flow and ran my agent afterwards.
Screenshot of victory attached.Dylan
Felix Vemmer
11/07/2020, 5:17 PMstored_as_script=True
.
However, if I have for example a authenticate.py module
where If I have some basic common selenium tasks such as:
• create_driver -> returns selenium driver
• login_xyz -> returns selenium driver in logged in state
And I want to import those into another script I get again:
TypeError: cannot pickle '_thread.lock' object
So as long as they are all in one file it’s fine but importing reintroduces the old issue. Any idea on some fix, otherwise I’ll just pile all the code together into one big flow 🙂
Thanks!Chris White
Marvin
11/15/2020, 4:46 PM