Robin

    Robin

    1 year ago
    Hey all ūüôā We got the following error when trying to map over a list of dictionaries. First just to clarify: mapping over a list of dictionaries should be fine, right? Second, what could be the reason for the message?
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Hi Robin, mapping over a list of dictionaries should be fine. The error you're getting here is happening in your task here (the actual mapping appears to be working correctly) - looks like you're getting a list where you expected a dict - perhaps you're passing in a list of lists instead?
    Robin

    Robin

    1 year ago
    Good point! Thanks for the quick feedback ūüôā
    In that case, this should work, right?
    reduce_list = task(lambda list_of_lists: reduce(list.extend, list_of_lists, []))
    @task
    def reduce_list(list_of_lists):
        out = []
        for l in list_of_lists:
            out.extend(l)
            
        return out
    did the trick ūüôā
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    That'd work, you could also make use of the new(ish)
    prefect.flatten
    , which will do that for you (but potentially in a more efficient way).
    your_task.map(flatten(list_of_lists))
    See https://docs.prefect.io/core/concepts/mapping.html#flat-mapping for more info.
    Robin

    Robin

    1 year ago
    Thanks for the tip! that looks neat ūüôā
    @Severin Ryberg [sevberg]