Rainer Schülke

    Rainer Schülke

    4 months ago
    Good afternoon 🙂 I am on Prefect 1.0 and was asking myself - mainly out of curiosity - if it is possible to exclude a specific element while calling .map(), store that element and process it at a later point within the flow? (Not talking about unmapped)
    Anna Geller

    Anna Geller

    4 months ago
    You would need to do it as part of your task - e.g. you may store those elements that you want to process later in some DB or KV Store (if you are on Prefect Cloud, we have embedded KV store) and retrieve those later, and pass only those elements to mapping task that you want to process now
    Rainer Schülke

    Rainer Schülke

    4 months ago
    Thanks for the quick reply Anna 🙂 I will leave it as it is but I definitely have a look at your suggestion.
    Chris L.

    Chris L.

    4 months ago
    Does this help? https://docs.prefect.io/api/latest/tasks/control_flow.html#filtertask Pass the mapped outputs into FilterTask and deal with the filtered tasks accordingly
    Rainer Schülke

    Rainer Schülke

    4 months ago
    Yeah I saw that earlier, I think it would work like this. Thanks Chris 🙂
    Works like a charm 😉 Is it only possible to use lambda expressions in the filter_func parameter?
    Chris L.

    Chris L.

    4 months ago
    It accepts any Callable, so any function should work!