I have a question about dynamically generating tasks thread

Question

Bradley Hurley · Answer

Here is some sample code ```from prefect tasks jupyter jupyter import ExecuteNotebook def generate notebook tasks flow for nb file in os listdir run nb task = ExecuteNotebook path=nb file parameters= flow add task run nb task def create flow gt Flow with Flow as flow generate notebook tasks flow=flow return flow FLOW = create flow ```

Bradley Hurley · Answer

Is it reasonable to generate tasks like above at flow generation time or is it better to have a task that returns a collection of tasks

Nate · Answer

instead of using the imperative API to create a flow like above I would just map `ExecuteNotebook` over your `nb files` like ```from prefect tasks jupyter jupyter import ExecuteNotebook execute notebook task = ExecuteNotebook add default params here with Flow my nb execute flow as flow could also do nb files = glob glob ipynb nb files = nb file for nb file in os listdir results = execute notebook task map path=nb files ``` prefect will generate an instance of `execute notebook task` for each element in your iterable `nb files`

Bradley Hurley · Answer

Thanks < Nate> The problem for mapping is that we don t end up with a schematic that shows each task right

Bradley Hurley · Answer

Ideally I would end up with something like

Nate · Answer

visually in the schematic the mapped task would be collapsed into one item but it shows how many mapped instances have been created you can click on that item and it ll show mapped task runs

Bradley Hurley · Answer

Thanks I appreciate the feedback