Hey, I’m looking for something similar as asked here (batch mapped tasks or nested mapping). Has anyone got something like this working? The example provided by Michael does something differently than I want. What I’m after is: 1. Retrieve some dynamic list (i.e. query from database) 2. Batch the result set in 20ish items to process in parallel 3. For each item in the batch, do a branch of chained tasks 4. Wait for batch to complete, repeat for next batch until exhausted Basically I want to write something as

tasks = dynamic_list_of_tasks()
windowed_tasks = fixed_window(tasks, window_size=5)

def process_item(item):
    x = task_1()
    task_2(x)

def process_window(tasks):
    apply_map(process_item, tasks)

apply_map(process_window, windowed_tasks)

Unfortunately not...

Thanks for replying! Yes agreed, would be great if it was possible to have nested mappings (or batch process of a mapping)

Sorry that we haven't given an answer here in this thread 🙂 I think responding to the Github issue is the best bet as this is something that will take significant consideration for us to include full support for.

Thats okay 🙂. I kinda concluded that too, it seems like a nice feature. However, since it isn’t supported yet, it seems like might be tricky to implement. Everyone, rally behind this ticket: https://github.com/PrefectHQ/prefect/issues/2459 😛