Hi. Is there a way to reuse a resource (database connection in my case) over mapped task runs (in local threaded environment with multiple workers)?

Hey @Kamil Okáč! This is a great question; we don’t currently have a first class way of exposing a shared resource at this moment, mainly due to the various execution environments that we support. In this case, if you understand the environment and implications, you could use a global python variable and / or a utility function such as:

CONNECTION = None

def get_connection():
    if not CONNECTION:
        CONNECTION = connection_logic()
    return CONNECTION

within your individual tasks

Thanks for pointing me in the right direction. I can't use global variables, as these are shared between threads, but using threading.local was the way to go.

@Marvin archive “How to share a resource between mapped task runs”