https://prefect.io logo
Title
m

Matt Yamkowy

04/06/2022, 8:06 PM
Hi. I'm just starting to look at Prefect 2.0. Is there a suggested way of handling a waiting period mid flow? For example if I start a flow, and after an initial task I want to wait for 5 minutes before continuing, do I just sleep for 5 minutes in a task? What about if I wanted to wait for days?
:discourse: 1
k

Kevin Kho

04/06/2022, 8:11 PM
You could do that with a sleep yeah. What is the use case for waiting days? Because maybe at that point to can trigger a sub flow scheduled for a later time
m

Matt Yamkowy

04/06/2022, 8:13 PM
In some cases we would detect an alarm, then start gathering data, days later we'd check if the gathered data met certain criteria before continuing with tasks. So are you saying I could dynamically schedule a follow up flow to check on my gathered data?
k

Kevin Kho

04/06/2022, 8:17 PM
Ah I guess not yet. We do have
Client.create_flow_run_from_deployment
here but I guess it doesn’t take in a scheduled start time yet. The equivalent in Prefect 1 does
m

Matt Yamkowy

04/06/2022, 8:19 PM
Hmm, can you suggest a way to do this? or will it be coming as 2.0 comes out of beta?
k

Kevin Kho

04/06/2022, 8:22 PM
it will be coming, for now though you’d either need to poll or use Prefect 1.0. I did leave an internal message to confirm and i’ll get back to you
m

Matt Yamkowy

04/06/2022, 8:22 PM
Thanks, I appreciate it!
a

Andrew Huang

04/06/2022, 8:26 PM
Or maybe I was wondering if you could set up a scheduled deployment every hour to check if there’s new data; if not, stop the flow run prematurely and recheck in another hour? to prevent clobbering, could also set concurrency limit = 1 https://orion-docs.prefect.io/concepts/schedules/
k

Kevin Kho

04/06/2022, 8:28 PM
I think at that point, you can just use the task level retry to achieve it can it will be simpler
👍 1
m

Matt Yamkowy

04/06/2022, 8:29 PM
It could be possible to setup a schedule to check if any jobs are complete, but we could potentially have a hundreds of different flows so we'd have to plan it out well. I was hoping prefect would have something built in to handle it.
I know temporal can do async sleep built in, I was hoping there was something similar in prefect.
k

Kevin Kho

04/06/2022, 8:38 PM
Not exactly the same issue, but this is heavily related. Definitely in the list of things to do for Prefect 2.0
m

Matt Yamkowy

04/06/2022, 8:39 PM
This is great. Is there an estimate as to when you're going to leave beta?
k

Kevin Kho

04/06/2022, 8:41 PM
No public timeline for that yet
m

Matt Yamkowy

04/06/2022, 8:41 PM
Thanks
k

Kevin Kho

04/06/2022, 8:42 PM
Of course! Prefect 1.0 can handle this for sure though if you want to give that a try
👍 1