Hi. I am a newbie and checking out the possibiliti...
# ask-community
i
Hi. I am a newbie and checking out the possibilities of prefect for our team. I was wondering whether there is a built in de-duplication of running a flow. What I mean is - if a particular flow (with given parameters) has been scheduled for a time T, and an event executes the flow with the same parameters within at T - dt (dt > 0) the scheduled flow is skipped (de-duplicated) (or vice versa for that matter)?
b
Hey Imi! In my mind, one way to go about this would be to add a task at the beginning of your flow script which checks to see if the inputs have been processed previously. If yes, have the flow complete and skip any downstream processes. If not, continue on with execution. I think using transactions to make your flow idempotent will help you out here.
i
Thanks. That I understand that by coding means it can be achieved. Will such an approach not report false statistics/history on the flow executed?
b
Will such an approach not report false statistics/history on the flow executed?
Would you mind sharing a bit more about what you mean by false statistics/history?
i
What I mean: How will a flow's status appear in the overview? It will have to get started and then cancelled (by transaction or other distributed lock means). Is there a way to make clear to the operator that a task has not failed, but was cancelled because of the recent execution of an identical flow.