Hello All, I want to one sqlalchemy engine through...
# prefect-community
Hello All, I want to one sqlalchemy engine through the flow, to avoid code duplicaton. My simple solution was to make a task which returns sqlalchemy engine object as result. It works when executing it locally, but returns error when executing it on server( cannot pickle local_object ). Is there a solution to this problem or is there a best practice how to share connection in a flow? Thanks!
🙌 1
👀 2
I've seen this suggested: https://docs.prefect.io/core/idioms/resource-manager.html My own solution was to use a flow state handler to setup the connection when the flow enters running state, storing it in what is essentially a thread-safe singleton, then disconnect when the flow enters finished state. I'm using the local executor though.
Got it working with resource_manager.