Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Good afternoon :slightly_smiling_face: I am on Prefect 1.0 and was asking myself - mainly out of curiosity - if it is possible to exclude a specific element while calling .map(), store that element and process it at a later point within the flow? (Not talking about unmapped)

You would need to do it as part of your task - e.g. you may store those elements that you want to process later in some DB or KV Store (if you are on Prefect Cloud, we have embedded KV store) and retrieve those later, and pass only those elements to mapping task that you want to process now

Thanks for the quick reply Anna :slightly_smiling_face: I will leave it as it is but I definitely have a look at your suggestion.

Does this help? <https://docs.prefect.io/api/latest/tasks/control_flow.html#filtertask|https://docs.prefect.io/api/latest/tasks/control_flow.html#filtertask><https://docs.prefect.io/api/latest/tasks/control_flow.html#filtertask| >Pass the mapped outputs into FilterTask and deal with the filtered tasks accordingly 

<https://docs.prefect.io/core/concepts/mapping.html#filter-map-output|https://docs.prefect.io/core/concepts/mapping.html#filter-map-output>

Yeah I saw that earlier, I think it would work like this. Thanks Chris :slightly_smiling_face:

Works like a charm :wink: Is it only possible to use lambda expressions in the filter_func parameter?

It accepts any Callable, so any function should work! 