https://prefect.io
Join Slack
Is `flatten` expected to work on a `dict`? ```from prefect import Flow, task, context, flatten @tas...
# ask-community
m
Is 
flatten
expected to work on a 
dict
?
Copy code
from prefect import Flow, task, context, flatten

@task
def A():
    return {i: list(range(i)) for i in range(3)}

@task
def B(y):
    logger = context.get("logger")
    <http://logger.info|logger.info>(y)


with Flow('flat map') as f:
    a = A()  # {0: [], 1: [0], 2: [0,1]}
    b = B.map(flatten(a))

if __name__ == "__main__":
    f.run()
I'm getting an unexpected error:
ERROR - prefect.flat map | Unexpected error occured in FlowRunner: KeyError(3)
If not, my workaround would be:
Copy code
@task
def flatten_dict(d):
    return [(k, v) for k, v in d.items()]
s
that doesn't seem like it would do what flatten is supposed to do?
upvote 1
m
I might be missing something. I'm trying to map over all key-value pairs in the dict.
s
then i don't see that you need to "flatten" the dict, just return your list(dict.items())
k
Steve is right. Flatten is to bump nested lists into a one-level list
m
Sorry, my own example didn't do what I wanted. 🤦‍♂️ This is what I'm looking for:
Copy code
@task
def flatten_dict(d):
    return [(k, v) for k, v_list in d.items() for v in v_list]
It converts
Copy code
{0: [], 1: [0], 2: [0,1]}
to
Copy code
[(1, 0), (2, 0), (2, 1)]
k
Ahh, Are you good now?
m
Yeah, just wanted to confirm that 
flatten
doesn't cover this.
k
Yep that’s right
👍 1
14 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by