Ankit
04/08/2024, 5:07 PMMarvin
04/08/2024, 5:07 PMRunning
or Pending
state, polling the pool for work will not allow additional runs to start until the existing ones have finished.
Here is the excerpt from the Prefect documentation about managing concurrency with work pools:
"Each work pool can optionally restrict concurrent runs of matching flows. For example, a work pool with a concurrency limit of 5 will only release new work if fewer than 5 matching runs are currently in a Running
or Pending
state. If 3 runs are Running
or Pending
, polling the pool for work will only result in 2 new."
Manage concurrency with work pools
However, this does not include functionality for canceling ongoing runs if a new run is attempted. Fully automated cancellation based on new runs would require custom logic, potentially through state handlers, listeners, or an orchestration process that uses the Prefect API to monitor and cancel runs. As of my knowledge cutoff in April 2023, Prefect Cloud does not seem to provide built-in features to cancel an ongoing flow run when a new run from the same deployment is attempted.