Mathijs Miermans
03/24/2022, 4:41 PMflatten
expected to work on a dict
?
from prefect import Flow, task, context, flatten
@task
def A():
return {i: list(range(i)) for i in range(3)}
@task
def B(y):
logger = context.get("logger")
<http://logger.info|logger.info>(y)
with Flow('flat map') as f:
a = A() # {0: [], 1: [0], 2: [0,1]}
b = B.map(flatten(a))
if __name__ == "__main__":
f.run()
I'm getting an unexpected error:
ERROR - prefect.flat map | Unexpected error occured in FlowRunner: KeyError(3)
@task
def flatten_dict(d):
return [(k, v) for k, v in d.items()]
Steve R
03/24/2022, 4:43 PMMathijs Miermans
03/24/2022, 4:44 PMSteve R
03/24/2022, 4:46 PMKevin Kho
03/24/2022, 4:46 PMMathijs Miermans
03/24/2022, 4:51 PM@task
def flatten_dict(d):
return [(k, v) for k, v_list in d.items() for v in v_list]
{0: [], 1: [0], 2: [0,1]}
to
[(1, 0), (2, 0), (2, 1)]
Kevin Kho
03/24/2022, 4:52 PMMathijs Miermans
03/24/2022, 4:52 PMflatten
doesn't cover this.Kevin Kho
03/24/2022, 4:53 PM