Jake Schmidt12/18/2019, 4:33 PM
where the values are variable-length lists. I’d like to map a fanout on each key, then a fanout for each element in that key’s value, such that the operation on each value also receives the result of the “outer fanout’s” operation on its key. Would the best approach be to use a nested flow? I saw https://github.com/PrefectHQ/prefect/issues/1311 is similar...
Zachary Hughes12/18/2019, 6:00 PM
Jake Schmidt12/18/2019, 6:01 PM
tasks, followed by a
task, followed by another map.
for key, varlenlist in jobs.items(): key_result = key_task(key) item_results =  for item in varlenlist: a = task_a(item, key_result) b = task_b(item, key_result) item_results.append(a) item_results.append(b) result = join_task(item_results)
Zachary Hughes12/18/2019, 6:31 PM
Jake Schmidt12/20/2019, 2:43 PM
Zachary Hughes12/20/2019, 3:24 PM
Jake Schmidt12/20/2019, 3:52 PM
), so that we could at least see it in
? Right now
seems to only compute/display the outer flow. Is there an example of the use of
Zachary Hughes12/20/2019, 7:19 PM
will take all tasks from the subflow and add them to the parent flow. This oftentimes results in two disjoint DAGs being represented, unless you built both flows using the exact same tasks, e.g.
@task def common_task(): ... @task def task_one(x): ... @task def task_two(y): ... with Flow("one") as flow_one: a = task_one(common_task) with Flow("two") as flow_two: b = task_two(common_task) flow_two.update(flow_one)