Hey folks, is it possible to have a flow that can ...
# ask-community
a
Hey folks, is it possible to have a flow that can not run more than one parallel flow run? I saw the
idempotency_key
but not sure that it fits the purpose - maybe I’m missing something?
a
Hey Adam! We're currently evaluating different ways to implement this. There are a few different use cases that we want to make sure we address, or else intentionally do not address, e.g. should subsequent flow runs block if one is already running, or fail immediately? For the moment, you can use global concurrency limits to achieve limiting parallelism, e.g.:
Copy code
GLOBAL_CONCURRENCY_LIMIT_NAME = "only-one-flow"

@flow
async def my_flow():
    async with concurrency(GLOBAL_CONCURRENCY_LIMIT_NAME, occupy=1):
        for x, y in [(1, 2), (2, 3), (3, 4), (4, 5)]:
            await asyncio.sleep(random.randint(0, 5))
            return x + y
Just be aware that the concurrency limit in this example is implicitly created in an inactive state. You can activate it from the UI.
I'll be doing some lower-level work to get us closer to other ways of limiting parallel runs soon.
The idempotency key is a little different, and in contexts where you can use that, such as manually triggering a flow run from a deployment, that can help avoid scheduling a flow run (but not running two runs of a flow in parallel, if for some reason they have the same idempotency key).
It also isn't configurable for runs that Prefect generates from a Deployment schedule, just runs triggered via API or using helpers like
run_deployment
.
a
Thanks @Andrew Brookins!