https://prefect.io logo
Title
r

Rich Tata

11/06/2022, 9:42 PM
So DST caused some havoc in my flows today. After the time change, every flow has been running two times, once at it's normal time, and again one hour in the future. I set them using cron string in my yaml and adjusting the timezone to my local and thought based on documentation that it would auto adjust (pictured). So what's the plan of action here, is it something I can change in the UI, or do I have to re-deploy all of these flows?
1
1
Pausing and unpausing all deployments fixed my deployment schedules for all future runs. Still, did anyone else encounter this issue? Was this my error or a bug?
b

Bianca Hoch

11/07/2022, 3:10 PM
Hi Rich, thank you for reaching out. This appears to be a bug, and we are going to file a ticket for it. Can you tell us a bit more about how you applied the schedule to the flow? And would you mind sharing the yaml file where you set the cron string?
r

Rich Tata

11/07/2022, 3:21 PM
We use CLI to create and apply deployments once the code is ready. 1. prefect deployment build ./<FILENAME>.py:<FLOWFUNCTIONNAME> -n <DEPLOYMENTNAME> -q <WORKQUEUE> --cron "* * * * *" -t "<TAG1>" -o "../yaml/<CHANGETHIS>-deployment.yaml" a. We use this command template to generate a yaml; cron is specified there. b. the Timezone is set to America/Detroit in the cron file. 2. prefect deployment apply ../yaml/<CHANGETHIS>-deployment.yaml a. Once we're sure it looks right, we use this to deploy to prefect. b. verify everything is right (workqueue, schedule, tags) in the prefect UI.
@Bianca Hoch Here's one of the yamls. Note this issue was for every scheduled flow we have deployed, across all workqueues. (27 & 7 respectively)
a

Aaron

11/07/2022, 3:30 PM
I had the exact same issue, jobs ran twice an hour apart. Mine were deployed at the command line with Python, and then scheduled in the UI with cron
b

Bianca Hoch

11/07/2022, 3:56 PM
Thank you for sharing this! @Aaron, would you mind sharing the cron string that you set in the UI for the schedule? Would you also mind providing the IDs of the affected flow runs?
@Rich Tata Could you also provide some example IDs of the affected flow runs? Thank you again!
a

Aaron

11/07/2022, 3:58 PM
cron string:
Flow run IDs: e87d232b-990f-45a0-b4bf-71a21d5c2e8b ran at 8am, 867b40a4-824a-438d-b546-c5af067cc60e ran at 7am
:gratitude-thank-you: 1
r

Rich Tata

11/07/2022, 4:01 PM
@Bianca Hoch They were duplicates of all our flows, so I don't know which were the problems, and which were original. I'll share a handful of pairs. 78b27915-bc51-4b59-bc9c-8ff405c93973 0edb8d60-fd59-433c-a995-895c4e0c3f23 19eabb57-d58f-4c4e-b5af-8c790ca74f44 85c4a62b-1021-4c96-995c-98d0c4664468 5eccbbc7-dc08-4ab1-ade2-02c71bd1a203 b60d73cb-2f10-4cab-91a8-63078a0b58b7
:gratitude-thank-you: 1
b

Bianca Hoch

11/07/2022, 4:17 PM
Thank you both! Lastly, would you mind sharing the versions of prefect you are using?
a

Aaron

11/07/2022, 4:17 PM
2.2.0
r

Rich Tata

11/07/2022, 4:18 PM
b

Bianca Hoch

11/07/2022, 4:53 PM
Hey team, I have created an issue here . Feel free to add to it as you see fit, or request that I adjust some of the information in the details of the issue.
🙏 1
a

Andrew Pruchinski

11/07/2022, 5:18 PM
Hey @Bianca Hoch! FWIW this also happened in version 1.2. Thank you
e

Emil Ordoñez

11/07/2022, 6:45 PM
Same issue here. I'm using schedule type Cron and Interval, both of them are being repeated. Common thing in all scheduled flows is an the "America/New_York timezone". We are scheduling trough the Prefect Cloud UI. Prefect version on the Agent side is 2.4.0 The runs marked with an X are the ones that are repeated. It's worth notice that I didn't see them coming, they weren't showing as Upcoming Runs, in the Flow Runs tab