Nico Aiello
02/26/2020, 9:17 PMJeremiah
02/26/2020, 9:21 PMNico Aiello
02/26/2020, 9:29 PMbatches = get_batches()
items = run_batch.map(batches)
run_item.map(items)
but this third line does not do what I want - it runs on the entire list of items (so in my example [a1, a2, a3] and [b1, b2]) instead of running individually on each of a1, a2, a3, b1, b2. Does that make sense?Jeremiah
02/26/2020, 9:36 PMflat_map
, and we’ve been talking about implementing one). Normally, I would advise you to add a task to your flow whose job was to intercept the output of your first map step, flatten it, and then have your second task map over that flattened output. However, that might not play nicely with our depth-first semantics, since the “flatten” step will block until all upstream results are complete. With the current version of Prefect my advice is what I just wrote, and I want to think more about your branching case as we write tests for depth-first execution.Nico Aiello
02/26/2020, 9:38 PMJeremiah
02/26/2020, 9:39 PMNico Aiello
02/26/2020, 9:41 PMJeremiah
02/26/2020, 9:54 PMHenrique Guarnieri
02/26/2020, 11:49 PMNico Aiello
03/10/2020, 2:41 PMJeremiah
03/10/2020, 8:10 PMNico Aiello
03/10/2020, 8:17 PM