Hey guys, How do I keep a database connection active through out an entire flow? I keep getting the pickling error when I try to share/use the database connection within tasks

TypeError: cannot pickle '_thread.lock' object

Right now the best way would be to use a connection pool, but in the very near future we’ll have a more first-class way of re-using connections with this PR: https://github.com/PrefectHQ/prefect/pull/3139

Thanks @Chris White is there a way to disable the picking for certain tasks? I tried the checkpoint=False, but it doesn't work

No - Prefect supports parallelism / distributed computation in a first-class way which requires objects to be shared among processes; the only way to do that for generic python objects is via pickle (or

cloudpickle

in Prefect’s case)

@ARun I haven’t tried it, but I assume that

cloudpickle

extends standard pickle methods, and you can wrap what you don’t want to pickle in a class with __setstate__ and __getstate__ defined, which drop or substitute for whatever content you don’t want to pickle. You could probably define these on a Task-derived class itself to customize how it is pickled.