Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

question about mapping tasks -is it possible for a task to be 'mapped' over multiple iterators? (think task with 2 parameters and I want to 'map' over both with a list of param values)

:thinking_face: Is something like this what you’re after?
```from prefect import task, Flow, Parameter

@task
def do_something(a, b):
    print(a)
    print(b)

with Flow("two-map") as flow:
    a = Parameter('a')
    b = Parameter('b')

    do_something.set_upstream(a, key="a", mapped=True)
    do_something.set_upstream(b, key="b", mapped=True)

flow.run(parameters={"a": [1, 2, 3], "b": [4, 5, 6]})```

humm, i'm not useing prefect params (don't need to set them at runtime)

```with Flow(name='backfill-flow', result=result_store) as flow:

    backfill_task_result = backfill_task.map(
        symbol=['SPY','GLUU','IHI','NVDA'],
        backfill_date=['2020-07-01','2020-07-02'],
        tick_type=unmapped('trades')
    )```

just trying to map over 2 separate lists that map unto 2 separate (normal) parameters

when I run the above flow it maps over the `backfill_date` param but not the `symbol` param. Hence my question

Hi <@UVD10RS0L> - Prefect’s map has a `zip` semantic for multiple arguments, rather than a `product` semantic (see this <https://github.com/PrefectHQ/prefect/issues/1986|related issue>). To achieve your goal, I would recommend precomputing the combinations of `symbol` and `backfill_dates` you require, and mapping over that result. For example
```backfill_task.map(
  list(itertools.product(
    ['SPY', 'GLUU', 'IHI', 'NVDA'], 
    ['2020-07-01', '2020-07-02']
  )
)```

You could also perform that product step in a Prefect task to do it dynamically (I think the linked issue has an example of that)

I do think it would be a nice to have native support for multiple mapped params, and (perhaps) for nested mappings

The product semantic implicitly requires gathering all data, since it can’t be done in parallel (in Spark it would require a `shuffle`), so I think we will always recommend using a separate task like the one in the linked issue. Otherwise it would unexpectedly break map’s parallel semantic by secretly necessitating a “reduce” step

However I do actually have a draft PR open for nested mapping: <https://github.com/PrefectHQ/prefect/pull/2898>