Channels
#prefect-community
Title
y
Yaron Levi
07/19/2023, 2:11 PMHi 👋
A Run is late for 10 hours for no reason.
Using Prefect Cloud.
Adding more details in a thread.
Would love to get help on the issue or a point of direction for investigation.
✅ 1
CleanShot 2023-07-19 at 17.11.42@2x.jpg
The backing infra is ECS tasks, but I don’t think it’s related in this case, since an ECS task wasn’t even started.
Normally, when there are problems in the ECS Task side of things, the Run is in Pending, and the AWS fails to start an ECS Task for some reason.
Here, it’s as if the Prefect Cloud itself just missing this Run and haven’t started any action from it’s side.
The late run Id:
j
Jake Kaplan
07/19/2023, 2:36 PMhey, thanks for flagging this. will take a look
y
Yaron Levi
07/19/2023, 2:38 PM@Jake Kaplan Thanks. If you need more info to help the debug, please ask.
e
Emil Christensen
07/19/2023, 2:57 PM@Yaron Levi Jut to confirm... the run never transitioned to
Latej
Jake Kaplan
07/19/2023, 3:30 PMHey @Yaron Levi if it's not a hassle could list the flow run ids you see on that page that appear stuck in scheduled?
y
Yaron Levi
07/19/2023, 3:33 PM@Jake Kaplan Yes, It was stuck on Scheduled all the time (still is) and the work queue is healthy.
One second, I’ll bring the list.
Ah, ok, so there other ones are scheduled for the future and everything is ok.
I am only talking about the single one that is stuck (marked with red border in the image I’ve sent).
j
Jake Kaplan
07/19/2023, 3:35 PMahhh got it, thank you
👍 1
y
Yaron Levi
07/19/2023, 3:47 PM@Jake Kaplan Do you want me to open an issue in Git for this?
j
Jake Kaplan
07/19/2023, 3:50 PMno need atm but thank you for the offer, I think I have found the issue on our end. A very small number of flow runs appear have gotten stuck as you described
Can you let me know though if you see your run change from
SCHEDULEDLATEPENDINGy
Yaron Levi
07/19/2023, 4:03 PMOK, great it run now, thanks!
j
Jake Kaplan
07/19/2023, 4:11 PMappreciate you reporting it 🙏
👍 1