Hey guys, checked my overnight flows and they failed with an error on the databricks side; however, when I restart then they fail right away with no info in the logs on the cloud. Is there any known issues at the moment? All my agents are active - run id = 5fbc05b3-d2e3-43ee-8d5d-842aaa04e23b

the first run ID doesn't exist on our side - is it a typo? and thanks for the follow-up - that's also something I wanted to suggest - whether you could start a new flow run instead of restarting the current one

this is defo the run id - i missed the 5 sorry 😛: https://cloud.prefect.io/aggreko-data/flow-run/5fbc05b3-d2e3-43ee-8d5d-842aaa04e23b Here is another two we are having the same issue on: https://cloud.prefect.io/aggreko-data/flow-run/107a3153-ef3e-4657-a969-b6aa6396f41e https://cloud.prefect.io/aggreko-data/flow-run/324b7fc4-24e2-411e-9d39-65ead86f627a We are just unable to restart them.

did you configure results? this is required for restarts to work

No this is not something I have done - however, my previous attempts at restarts usually work and I have not made any changes to my configuration in a long time....

I see no logs here other than state updates in the first one - hard to understand what is the root cause here the second run only shows that some Spark jobs failed

Nothing is ever sent to databricks after I hit restart...

restart functionality is to restart a flow run from a failed task run - given that your work is not really happening on the Prefect side, but rather on Spark, it might be more helpful to start a new flow run rather than restart to restart your data processing process

Okay cool - just a bit strange as it used to allow us to restart from the failed task but we have lost that functionality now 😞

Checking after holidays: LMK if this is still an open issue

Nah this was more of an observation on Prefect allowing us to restart even tho we had not changed any config when it never let us on the previous run! Thanks for coming back to me 🙂

Awesome, thanks for the update, appreciate it!