m

    Marwan Sarieddine

    1 year ago
    Hi Folks, we ran a flow that triggered 11 downstream flows on prefect cloud using
    StartFlowRun
    , the 11 flows got “queued”, but since we are on team plan (i.e. limited by flow concurrency) - only 2 of the flows were scheduled and the rest were late runs - so far so good, then the late runs get scheduled and run, but for some unkown reason 5 of the 11 late runs disappeared on their own and I don’t see why that happened
    the 5 late runs that disappeared don’t show as cancelled or failed - there is no longer any record of them on the UI
    Also it seems now we can only schedule one flow at a time … (i.e. the flow is queued as a late run)
    nicholas

    nicholas

    1 year ago
    Hi @Marwan Sarieddine - sorry you’re running into that, when you get a chance could you DM me your team url?
    m

    Marwan Sarieddine

    1 year ago
    Thanks for following up - just sent you a DM
    nicholas

    nicholas

    1 year ago
    Hi @Marwan Sarieddine - following up on this; thank you for your patience! After some investigation, I've determined your flow did correctly kick off additional runs. It also appears that you (or someone on your team) may have re-registered the flow(s) associated with those runs; when a flow is re-registered, any runs in a
    Scheduled
    state are deleted since they won't be able to run. I understand this would be unseen behavior from your end and difficult to debug! To fix that, we've come up with a solution where any runs that weren't created by the Prefect Scheduler (the service that creates runs in the future from your flow schedules) won't be removed but will instead be put into
    Cancelled
    states with a message detailing why that action was taken.
    m

    Marwan Sarieddine

    1 year ago
    I see thanks for explaining what happened, showing the flow runs as Cancelled with an informative message would certainly be helpful
    a

    Alex Papanicolaou

    1 year ago
    Just for clarification: a flow that is already running will not be stopped?
    nicholas

    nicholas

    1 year ago
    That's correct @Alex Papanicolaou!
    Great @Marwan Sarieddine - we're already working on getting that into Server so you can expect that in Cloud before too long 🙂
    And here's the Server PR: https://github.com/PrefectHQ/server/pull/185 (big shoutout to chris for making that happen so quickly!)