Hey guys I have a prefect job that is running on ...
# prefect-community
Hey guys I have a prefect job that is running on a schedule. What I want to basically achieve is that the new flow does not get run until the old one is finished. For instance consider a flow is scheduled every hour and if it takes more than an hour for it to finish then the new scheduled one should not start until the old one has finished running. I was looking at
Inspecting flow runs
from prefect.backend import FlowRunView
flow_run = FlowRunView.from_flow_run_id("4c0101af-c6bb-4b96-8661-63a5bbfb5596")
Can you please guide me on the best approach to achieve the above use case. Really appreciate your time and help.
Hi @Haseeb Ahmad, if you are on Prefect Cloud, you can look into Flow Run Concurrency Limits. If you are on Prefect Server, you need to do something like this where you delay/exit the flow is there is already a running one
Thanks @Kevin Kho very much for your response. I truly appreciate that. Just to give you more context. I am on prefect cloud. There is this one job that is scheduled every day that basically fetches documents from stfp. Sometimes the documents are in huge numbers and the job takes more than one day to finish. And as it is scheduled every day so, the new run gets instantiated. Whenever this happens it uses up my 2 slots for running jobs causing others jobs to be queued. So I wanted to add some logic into this job to check if there already is a job running and a new one does not get created. So, if I label this particular job to say 'xyz' and limit it to 1. Would this help my cause and not impact the other jobs. Again thanks for your help.
Yes exactly that is the thought
thank you @Kevin Kho
👍 1
Hey @Kevin Kho unfortunately, we right now are on legacy pricing model, until we move to upgraded plan, is there an alternative to achieve this same scenario. Appreciate your help
You can check the second link in the original response which is a Github code snippet that lets you achieve it from the state handler. This is more to cancel the second Flow though rather than wait for the first to complete but I think you can repurpose