Hi all! Is there possibility looping of flow ? We have long running task

Do you want to loop a flow, or are you concerned that your flow is looping?

Thanks for link, but in our case we have several tasks, which depends on each other, and we want looping this group of tasks while result of first task in group is true. Example : first task get list of files from kafka , second task run as .map over dask cluster. Equivalent is while(len(files)) > 0 do staff.

There might be an easier way, but you can run a flow from a flow. So at the end of your tasks, you can check if you want to "loop", and if you do you can call the same flow again with the new parameters you want. https://docs.prefect.io/core/idioms/flow-to-flow.html This does slow things down a bit though, since it takes time to schedule a prefect flow, so if you can loop within your flow without scheduling a new flow that would be much better.

thanks! we just want to avoid starting the entire flow due to re-registration in order to reduce overhead costs. but we don't know how :)

Hey Bral, you might want to check out https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a as well

Sorry?

In your link, what did you mean specially?