https://prefect.io logo
Title
k

Kamil Okáč

12/23/2019, 12:38 PM
Hi. Is there a way to reuse a resource (database connection in my case) over mapped task runs (in local threaded environment with multiple workers)?
✔️ 1
c

Chris White

12/23/2019, 6:20 PM
Hey @Kamil Okáč! This is a great question; we don’t currently have a first class way of exposing a shared resource at this moment, mainly due to the various execution environments that we support. In this case, if you understand the environment and implications, you could use a global python variable and / or a utility function such as:
CONNECTION = None

def get_connection():
    if not CONNECTION:
        CONNECTION = connection_logic()
    return CONNECTION
within your individual tasks
k

Kamil Okáč

12/24/2019, 1:57 PM
Thanks for pointing me in the right direction. I can't use global variables, as these are shared between threads, but using threading.local was the way to go.
:upvote: 1
c

Chris White

12/24/2019, 2:59 PM
@Marvin archive “How to share a resource between mapped task runs”