On Prefect-Cloud, I had 1000 flow runs and after r...
# ask-communityj
Jack Sundberg
02/28/2021, 4:12 AMOn Prefect-Cloud, I had 1000 flow runs and after roughly 150 of them completed, the remaining all failed with "_Failed after exceeding scheduled work SLA_". Is there some unwritten limit to Prefect Cloud that I'm missing? My best guess is a limit on task duration or maybe a limit on how far "behind schedule" a flow run is allowed to be.
c
Chris White
02/28/2021, 4:22 AM
Hi jack - yes at the moment runs that are more than 48 hours late are marked as failed
j
Jack Sundberg
02/28/2021, 4:26 AMGotcha. In this case, mine were only at 24hrs late. Also is there a way to turn this off or extend the allowed time?
c
Chris White
02/28/2021, 4:28 AM
At the moment the setting isn’t configurable but we are open to adjusting it for sure - were these all ad hoc runs that you wanted to finish regardless of how late they ended up being?
j
Jack Sundberg
02/28/2021, 4:28 AMYep.
Jack Sundberg
02/28/2021, 4:29 AMThey're all ad hoc and are pretty long runs (range from 1 minute to 1 hr at the moment)
c
Chris White
02/28/2021, 4:30 AM
Gotcha that makes sense, I’ll raise an issue internally and get back to you with what the outcome is 👍👍
j
Jack Sundberg
02/28/2021, 4:32 AMThanks!
c
Chris White
03/01/2021, 4:18 PM
Hi @Jack Sundberg - we did discuss this and decided that we are going to remove this service entirely; we will be rolling out a very rich set of configurable SLA features to Standard tier users soon and feel that will be a much more “Prefect experience” than the hidden service you ran into. You can expect the change to go live in the next 24 hours!
j
Jack Sundberg
03/01/2021, 4:24 PMThat's awesome! It certainly feels off to use prefect in primarily ad-hoc way and always see warnings from late runs haha. Thank you so much for the quick turn-around!
🚀 1
👍 1
Jack Sundberg
03/01/2021, 4:29 PMWhile I have you, can I ask about two more things? Do you know if Kyle DeMerritt is out of office? From our last meeting, I wouldn't be surprised if he was taking time off for his back -- and I haven't heard from him since Thursday about setting up a paid Cloud plan. The second thing is much less urgent and if you have extra time today -- it's just checking out my pull request which has been sitting dormant https://github.com/PrefectHQ/prefect/pull/4146
Jack Sundberg
03/01/2021, 4:30 PMSorry to keep laying on the questions/requests 🥴
c
Chris White
03/01/2021, 5:09 PM
No worries, I’ll follow up with Kyle and let him know; no promises on getting to the PR today, but noted
j
Jack Sundberg
03/01/2021, 5:12 PMThank you :)
👍 1
4 Views