Hi Everyone, I am prototyping an ETL flow where I want to read a bunch of files, do some transformation, and write the results in parallel using a

LocalDaskExecutor

My flow looks something like this:

list_of_files = Parameter()
list_of_frames = extract_fn.map(list_of_files) # This is indeed getting executed in parallel

How can I pass both the result of the previous mapped fn and also the input parameters to my transform fn? I understand that I could probably refactor the transform_fn to avoid using the

list_of_files

but wanted to know if there is any other way of handling this.

transform_fn.map(list_of_frames, list_of_files)

As you can tell I am very new to prefect. Thanks a lot!

I think what you're asking for is

prefect.unmapped

: https://docs.prefect.io/core/concepts/mapping.html#unmapped-inputs. This lets you wrap an argument to a

.map

call , and prefect will pass that argument in directly, rather than map over it.

The later. I am trying to get the filename passed to the transform_fn

transform_fn.map(list_of_frames, list_of_files)

should work then.

that seems to give an error -

some reference tasks failed

. Will debug it