Hi all! I'm trying to figure out some of the subtl...
# ask-community
s
Hi all! I'm trying to figure out some of the subtleties of concurrency and task submission. We have a bunch of workers, each with a lot of memory, to do some data-intensive number crunching. I have a flow which uses about 40GB of memory, and I want to spin up a bunch of these akin to:
Copy code
@flow
def parent_flow(start: dt, end: dt):
   for date in get_dates(start, end):
        memory_intensive_flow(date)
But I want to make sure that these subflows are distributed over all the workers and not just scheduled on the worker the parent flow is executing in. I'm not sure how to quickly write a dummy flow and figure out which worker replica its executed on, so I thought I'd check and see if anyone knew the answer off hand.