Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi, another question about steps. We'd like one task to unzip folders in gcs, and another task to process each file in the folder. Can that be done where the second task fans out (multiprocess)?

Yes. For Prefect 1, see <https://docs.prefect.io/core/concepts/mapping.html#mapping|mapping> and for Prefect 2, you can just use a for loop and call new tasks. You just need to call `.result()` on the first task to make it a Python object before you loop

using a for loop will do the tasks sequentially right?

Not quite. The default TaskRunner is ConcurrentTaskRunner so if they are independent, they will run concurrrently