Channels
pacc-may-31-2023
prefect-ai
pacc-clearcover-june-12-2023
marvin-in-the-wild
data-ecosystem
geo-israel
pacc-june-14-2023
geo-japan
pacc-london-2023
prefect-cloud
ppcc-may-16-2023
prefect-azure
prefect-docker
prefect-recipes
gratitude
geo-nyc
geo-bay-area
geo-boston
geo-london
geo-dc
geo-chicago
geo-berlin
geo-texas
geo-seattle
geo-colorado
prefect-community
data-tricks-and-tips
prefect-aws
prefect-gcp
introductions
find-a-prefect-job
prefect-dbt
random
events
ask-marvin
show-us-what-you-got
prefect-getting-started
prefect-integrations
prefect-contributors
best-practices-coordination-plane
announcements
prefect-server
prefect-ui
prefect-kubernetes
Title
c
Claire Scott
04/27/2023, 1:18 AMHas anyone else had any issues with the scheduler in prefect cloud this-morning? mine stopped generating new scheduled flow runs, so just quietly stopped running when it ran out of existing yellow dots . turning the schedule on the deployment off and on again re-generated a new set, but no more where added, so a couple of hours later, it just stopped running again, just had to turn the deployments off and on for the third time today
👀 1
1️⃣ 1
g
gojaeheon
04/27/2023, 1:52 AMOur service does, too. Is there any problem in scheduler~? @Prefect
c
Claire Scott
04/27/2023, 2:21 AMthanks @gojaeheon for the +1 I just had to restart the deployments again, 4th time today,
g
gojaeheon
04/27/2023, 2:23 AMMe too,, 4th time today.. We should restart the deployments all day..😂
e
Erik Amundson
04/27/2023, 2:35 AMHaving this same issue as well.
👍 1
n
Nate
04/27/2023, 3:20 AMhi all, sorry about the trouble, I will raise this - do you mind sharing whether you're on prefect cloud 1 or 2?
i assume 2, just want to be sure
:thank_you: 2
c
Claire Scott
04/27/2023, 3:21 AMPrefect cloud 2
:thank-you: 1
e
Erik Amundson
04/27/2023, 3:21 AMCloud 2
c
Claire Scott
04/27/2023, 3:21 AMI'll dm you my workspace if it helps
g
gojaeheon
04/27/2023, 3:31 AMcloud 2
n
Nate
04/27/2023, 3:38 AMthank you all - I've raised internally and will have an update as soon as we can
🙌 2
a
Alex Prejean
04/27/2023, 3:38 AMwe're having a similar problem with cloud 2
d
Dhruv
04/27/2023, 3:39 AMsame
n
Nate
04/27/2023, 4:19 AMhi all - we've identified an issue with the scheduler and have pushed a fix
flow run scheduling should now resume as normal - please let us know if your issue persists
🙌 4
:thank_you: 2
:thank-you: 1
c
Claire Scott
04/27/2023, 4:22 AMLooks promising so far, thanks Nate
t
Taran Singh
04/27/2023, 4:35 AM@Nate Thank you
g
gojaeheon
04/27/2023, 5:18 AM@Nate It doesn't seem to work again..
Are the others okay? @Claire Scott
c
Claire Scott
04/27/2023, 5:40 AMI'm project broken again too @Prefect 😞
m
Mateo Merlo
04/27/2023, 1:45 PMI’m having the same issue with the ECS agent and the deployments.
👀 1
n
Nate
04/27/2023, 2:06 PMhi @Mateo Merlo - can you confirm you are still noticing issues with flow run scheduling now? could you describe what you’re seeing?
m
Mateo Merlo
04/27/2023, 3:40 PMHi @Nate , we have re deployed the agent and now seems that flows are working. Thanks!!
:thank-you: 1
n
Nate
04/27/2023, 3:47 PMthanks for the update!
m
Mateo Merlo
05/03/2023, 6:01 AMHi @Nate, I'm having an issue that I think can be related with this previous one. At least 2 3 times a day we have a pipeline that crashed (Is not always the same pipeline but the error is the same. Image). Every time that a pipeline crashed, the ECS task keeps running indefinitely (we have to stop it manually) and the agent (running in a different ECS task) is not able to continue with other pipelines so I ends with a lot of Late pipelines.