Amogh Kulkarni

    Amogh Kulkarni

    2 months ago
    @Anna Geller - We are seeing one issue with our flow today. We have scheduled our flow every 5 minutes. We use KubernetesRun as our run config and S3 as our storage. We have scheduled the flow from the Prefect Cloud. The issue that we are seeing is that if the flow run’s run time exceeds 5 minutes, a second flow run of the flow runs in parallel. Today because of upstream long running jobs, our first flow run was running for 15 minutes and in the meanwhile there were 3 flow runs of the same flow that were running. Is there any way to ensure that a new flow run runs only when the previous flow run completes?
    Kevin Kho

    Kevin Kho

    2 months ago
    For Prefect 1, You can use flow run concurrency to make sure that only one flow runs at a time, and then you can use Automations to cancel a Flow if they are late by a certain amount of time.
    Amogh Kulkarni

    Amogh Kulkarni

    2 months ago
    Thanks @Kevin Kho. I see in the docs that flow concurrency limit is driven by labels. So far we have been adding labels only for the agents. But in this case we will have to add a separate label per flow and also ensure that the labels in the agent are a superset so that the agent picks up the flow. Is my understanding correct?
    Kevin Kho

    Kevin Kho

    2 months ago
    Yes exactly
    Amogh Kulkarni

    Amogh Kulkarni

    2 months ago
    Thanks a lot @Kevin Kho