05/08/2020, 8:03 AM
Hi there. I really like Prefect's mapping feature and in general the approach towards being able to easily map/reduce your tasks. But AFAIK you can only do a full map or a full reduce. Do you plan on supporting reducing by key? I understand you can do it "manually" within a single task, but a parallel/distributed version could definitely be more effective.


05/08/2020, 1:06 PM
Interesting idea @Zviri! Yeah as it stands right now you would have to do something manually in a reduction task in order to filter the upstream mapped states. I’m not fully sure exactly how reduction by key would work but I like thinking about the idea. Would you mind opening an issue with this feature request and we can get more eyes on it? 😄