Thread
#prefect-ui
    s

    Sean Caldwell

    4 months ago
    Hey guys, checked my overnight flows and they failed with an error on the databricks side; however, when I restart then they fail right away with no info in the logs on the cloud. Is there any known issues at the moment? All my agents are active - run id = 5fbc05b3-d2e3-43ee-8d5d-842aaa04e23b
    Further info - I started another run on another flow via the same agent and all is fine so far - this run id is: f3233bf6-1cdb-4260-9a03-0c3259bc0b3a
    Anna Geller

    Anna Geller

    4 months ago
    the first run ID doesn't exist on our side - is it a typo? and thanks for the follow-up - that's also something I wanted to suggest - whether you could start a new flow run instead of restarting the current one
    s

    Sean Caldwell

    4 months ago
    Anna Geller

    Anna Geller

    4 months ago
    did you configure results? this is required for restarts to work
    s

    Sean Caldwell

    4 months ago
    No this is not something I have done - however, my previous attempts at restarts usually work and I have not made any changes to my configuration in a long time....
    This what I am seeing in the logs when trying to restart:
    Anna Geller

    Anna Geller

    4 months ago
    I see no logs here other than state updates in the first one - hard to understand what is the root cause here the second run only shows that some Spark jobs failed
    exactly, I see the same - it looks like some Spark jobs are failing? did you check why this might be happening?
    s

    Sean Caldwell

    4 months ago
    Nothing is ever sent to databricks after I hit restart...
    Anna Geller

    Anna Geller

    4 months ago
    restart functionality is to restart a flow run from a failed task run - given that your work is not really happening on the Prefect side, but rather on Spark, it might be more helpful to start a new flow run rather than restart to restart your data processing process
    processing process 🤯 😄
    s

    Sean Caldwell

    4 months ago
    Okay cool - just a bit strange as it used to allow us to restart from the failed task but we have lost that functionality now 😞
    Really strange as this one restarted from some failed spark jobs: https://cloud.prefect.io/aggreko-data/flow-run/bd4a0de0-b1d7-411d-a361-090af4e047f5?overview
    Anna Geller

    Anna Geller

    3 months ago
    Checking after holidays: LMK if this is still an open issue
    s

    Sean Caldwell

    3 months ago
    Nah this was more of an observation on Prefect allowing us to restart even tho we had not changed any config when it never let us on the previous run! Thanks for coming back to me 🙂
    Anna Geller

    Anna Geller

    3 months ago
    Awesome, thanks for the update, appreciate it!