Right now, idempotency keys have a 24 hour timeout, per the docs, and based on what I’m seeing in local testing. The only time they’re freed earlier is if the flow is re-registered, in which case I think that’s just a “virtual free”, because I think the pair is really “flow_id, idempotency_key”

If I’m misunderstanding concurrency limits, okay, but I think you have to know the labels ahead of time and have agents setup and ready. I really want to use the same label so prefect can load balance (once we finally get kubernetes agents going), except I don’t want to allow more than one idempotency_key at a time and then freeing it up. But if there’s a way to do it with concurrency limits and generated labels or something, I’m all ears. Open to learning something new. 🙂

So I guess using a concurrency limit of 1 is the closest thing. You can add an extra label to this Flow and add it on the agent since the agent labels just need to a superset of flow labels

without having to know all the labels in advance?

Because the only way I can think of accomplishing this is persisting some kind of global state somewhere like the KV Store. Before the Flow begins, you can look up the state and if the state shows something else is running, just look up again after certain period of time.

Ok I think where my current thinking is if you don’t want the 2nd Flow to even start, the mechanism has to open on the orchestration level and that has to be Flow Run Concurrency Limits using labels or a rearchitecting of the problem. If you can have the second Flow run and just pause indefinitely, then it can be done in the state handler and you can do it more dynamically

Though I wouldn’t be surprised if we really did indeed hit the limitations

We have a use-case where we want to run a flow that takes a very long time to run. Given a set of parameters to the flow, we only want the running instance to run for those params. However, if a different set of params are given and with a different “mutex_key”, we would like that one to be able to run at the same time as the first one. Think of it like this: we want to run this flow for different customers against their data. It’s okay that it takes a long time, and it’s okay to run against each customer individually. However, we don’t want to allow more than one flow at a time to hit each customer, because it would compound and eventually cause a major outage on a regular basis, because we are talking about big database queries and things like that underneath it all. So, this use-case comes in very handy when you want to serialize “runs with the same parameters”. Please let me know if you need more clarification.

Yes the only thing that can help you right now is using state handlers and checking for these slots that you create yourself. It would be good to manage them with the KV Store , but I’ll bring this up to the team to talk about

@Anna Geller overall, your approach works pretty nicely. I really appreciate it. I’m having one issue though, and that’s making sure that if, let’s say, 5 of the same flow+run_name are scheduled at the same time, this handler won’t notice, because it limits its checks to checking for

Running

. Do you know if there’s any way to “serialize” those checks so that at most one check for this flow is happening at a time across the agents?

Hmm, I suppose if multiple flows do happen to come alive, we can just do an additional check and see that the original “message” to run was already captured. That’s pretty easy. Not perfect, but doable. Though, if you do have ideas for my question about serializing the state checker, I’d love some feedback. 😄

I’m gonna note down what you said for cancelling flows too and look into that in more detail in a bit. Either way, you’ve shown me a ton about what can be done with state handlers.

Can I contribute this state handler back to y’all if I get it doing what we want? This is valid use-case that I imagine a lot of people have in situations like ours.