Another question 🙂 is it possible to have the following constraint: Schedule "flow a" to run every 10 minutes & prevent scheduling "flow a" if there is an existing invocation still running.
k
Kevin Kho
11/11/2021, 3:36 PM
For Cloud, we have Flow Run Concurrency. For server, I have this code snippet from another Server user.
Kevin Kho
11/11/2021, 3:37 PM
It gets all the flow runs of the flow and sees if there are still ongoing ones. This state handler is then attached to the Flow
d
Daniel Katz
11/11/2021, 3:37 PM
perfect, thank you!
Daniel Katz
11/11/2021, 3:38 PM
My team is very much enjoying Prefect so far, mostly thanks to your amazing community response time!
k
Kevin Kho
11/11/2021, 3:39 PM
Thank you! 🙂
k
Kevin Mullins
11/11/2021, 4:31 PM
For Prefect Cloud, with the concurrency based off the flow run config labels, does this mean that an agent has to be configured with specific labels to limit flows?
For instance, say I have
source1-database-extract
and
source2-database-extract
flows that each flow should only have a single instance per source to prevent issues. Does this mean I have to add tags for every flow to the agent to ensure they can run only once per source?
k
Kevin Kho
11/11/2021, 4:33 PM
Yes but flow labels just need to be a subset of agent labels of if you have a label for
source1
and another for
source2
, the agent can pick both up if it has both the labels.
And then yes you would apply the concurrency limit on the
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.