Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi! I have a flow in which I run a bunch of queries. I'd like to establish connection to the db only once, but I'm not sure how to do that in Prefect. If I use a task I cannot just put the db object in the context (as per <https://docs.prefect.io/api/latest/utilities/context.html#context> it is not recommended to mutate context in tasks and even downstream tasks do not see it). I cannot pass around the db connection as it is not serializable. Is there a recommended way to achieve this?

I like to have a function like `get_connection` , which maintains a singleton connection (only creates a connection if it hasn't already created one before, otherwise returns the created one). This avoids passing the connection around as a task input, instead you call `get_connection()`  within your `task.run`

Note that if your flow executor is distributed to multiple machines *OR* uses processes as workers, this method creates a connection for each process. There is no way to get around this afaik, database connections cannot and should not be shared between operating system processes.
threads can share connections no problem.