Hi we ve got a flow of flows which runs every time a file is

Question

Hi we ve got a flow of flows which runs every time a file is added and then at the end runs a flow that updates a dataset in Power BI Since we re only able to refresh PowerBI 8 times a day is there a way to group the power bi refreshes together in case multiple files are added Just wondering what might be the best way to architect that or if anyone s done something similar

Anna Geller · Answer

iterating some business logic over a runtime discoverable list of files looks like a perfect use case for mapping did you try that

Anna Geller · Answer

alternatively you could have a flow that fires any time a new file is added in <https discourse prefect io t is there an equivalent to sensors in prefect how do i trigger event driven workflows 76|an event driven way> if I misunderstood your question could you explain in more detail what you are trying to do

Anna Geller · Answer

to group multiple flow runs or task runs in Prefect 2 0 there is a flexible concept of tags and the UI provides filter functionality allowing you to even build custom dashboards based on that <https discourse prefect io t how can i organize my flows based on my business logic or team structure 66|this topic may help>

Jonathan Mathews · Answer

Thanks We have it set up in an event driven way so that a new flow is triggered every time a file is added The problem is when a new file is added and then 2 mins later another one and so on

Jonathan Mathews · Answer

Ideally it would be something like merging flows or have the powerbi refresh on some kind of a delay

Jonathan Mathews · Answer

Then if the refresh flow had already been triggered and was pending it would not allow it to trigger again

Kevin Kho · Answer

I think there are things you can do there but the workload sounds too unpredictable Is this Prefect 1 or 2 For Prefect 1 you can use the state handler to check for the last flow run and if it s too recent cancel the current run The merging is slightly possible with that state handler The parent flow can also schedule the child flow with a delay But the problem is you don t know the exact delay here to use And I think going this route makes the architecture far more complicated that trying to bump up the 8 refreshes a day would be significantly less work