https://prefect.io
Join Slack
Hi everyone, what’s the better pattern to finish a looping Flow as successful that has been schedule...
# ask-community
j
Hi everyone, what’s the better pattern to finish a looping Flow as successful that has been scheduled indefinitely. As of now, the main Task of the Flow raises 
ENDRUN(Finished(message="all rows from file read"))
when it has finished reading all the lines in a big file (I’m not using the LOOP signal, the Flow is an exercise to properly use 
Result
objects). Should I create a state handler and trigger a final reference task or is there something like 
ENDRUN
but at the Flow level? Any other alternatives?
script here, just in case.
read_batch_events.py
d
Hi @Julio Venegas, That’s for sharing your script! Very helpful 😄
I might suggest raising a 
SUCCESS
signal instead of 
ENDRUN
because I believe you’re trying to communicate a successful outcome for the Task
j
Ok, thanks! But if there are more runs scheduled, it would continue running each of them right?
d
The way I’m reading this, 
read-row-batch
will run as a single Task Run for a given Flow run
So using either Signal is going to end the execution of that particular Task Run
(you won’t hit the subsequent while loop)
If you want to run another Task only if you’ve hit that conditional block, you could instead return some object that indicates that
and use control flow Tasks to enforce it
j
Niiice!! Will try out your suggestions, thanks a lot!
d
If I understand you correctly, you want to run a Task only if that conditional block where the ENDRUN Signal resides is reached, correct?
anytime!
j
If that conditional block is reached, I want to finish the Flow and not run any other downstream Tasks.
d
Got it!
j
So basically at the time ENDRUN is raised, I also want to finish the Flow.
d
Same idea applies, just in the inverse
If you return an object that says “don’t keep going”, you can use a control flow Task to end the run
🙌 1
Does that make sense?
j
Makes sense!! Will dig into the conditional logic page you sent.
d
👍
Glad I could help! 😄
j
Thanks a bunch Dylan!
@Dylan would you mind checking my script? I did the conditional case Task, and I wanted to add a reference Task with 
flow.set_referenece_tasks()
but I’m getting a 
ValueError: reference tasks must be part of the flow.
actually, since that task has no downstream dependencies, it would be a terminal task right? so no need to make it a reference task anyways
Forget about the 
ValueError
question, it’s still a bit odd that it was raised but setting the reference task was unnecessary anyways. Thanks for all the help!
d
Hey Julio, sorry to miss out here
Seems like you got this sorted?
Let me know if I can help 😄
j
All sorted out 😄 thanks!
5 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by