Hi together,
Is there a way to schedule a flow to start as soon as the previous run of the same flow finished?
s
Stéphan Taljaard
03/25/2022, 7:07 AM
Hi. You can have multiple scheduled/ongoing runs for a flow at the same time. Probably you'd want to give them different values for their parameters, though.
Thank's @Anna Geller. I think this is what I was looking for 🙂
👍 1
Dennis Hinnenkamp
03/25/2022, 3:08 PM
Is it also possible to combine the logic of never ending flows with a scheduler? In other words: If the flow is successfully completed, start it again directly, but only between 05:00am - 08:00pm?
a
Anna Geller
03/25/2022, 3:29 PM
Sure it's possible, as easy as
.py
🙂
Copy code
import prefect
from prefect import task, Flow
from prefect.tasks.prefect import create_flow_run
import time
import pendulum
@task(log_stdout=True)
def hello_world():
print("Sleeping...")
time.sleep(4) # to have enough time to kill it
return "hello world"
def never_ending_state_handler(obj, old_state, new_state):
if new_state.is_successful():
logger = prefect.context.get("logger")
now = pendulum.now().time() # change to utcnow if you prefer UTC time
if pendulum.time(5) <= now <= pendulum.time(20):
<http://logger.info|logger.info>("It's between 5 AM and 8 PM - creating a new run...")
create_flow_run.run(flow_name="never-ending-flow", project_name="community")
else:
<http://logger.info|logger.info>("It's NOT between 5 AM and 8 PM - ending flow runs for today")
return new_state
with Flow("never-ending-flow", state_handlers=[never_ending_state_handler]) as flow:
hello_task = hello_world()
Anna Geller
03/25/2022, 3:30 PM
This line does the "scheduling" part:
Copy code
if pendulum.time(5) <= now <= pendulum.time(20):
d
Dennis Hinnenkamp
03/25/2022, 3:37 PM
Very nice, thanks again! I was thinking of a similar solution in Python, but I was afraid that there would already be a BP from you that is not realised in Python.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.