What is Prefect-ish way to end a flow run. Suppose one of the early tasks in a flow does not have the required data to continue the rest of the flow, how does one gracefully end a flow?
k
Kevin Kho
04/07/2021, 4:58 PM
Hi @Hawkar Mahmod, what do you mean by gracefully? You don’t want it to be displayed as
FAILED
?
h
Hawkar Mahmod
04/07/2021, 4:59 PM
It wouldn’t be failed because its valid path. It’s just not possible to run a subsequent tasks because a dataframe is empty.
k
Kevin Kho
04/07/2021, 5:01 PM
Have you looked into state signals ? I am thinking you can raise the SKIP signal based on the empty dataframe then it would skip downstream tasks
and do use it in my flow for something else but it strikes as odd that Prefect doesn’t support the ability to use native Python conditional like it does for returning results of tasks. I don’t know enough about way Prefect works to understand why this is. Thank you for the suggestion about state signals. I’ll have a look.
Hawkar Mahmod
04/08/2021, 1:12 PM
@Kevin Kho using the SKIP signal worked perfectly for my use case. Thank you. It means not having to write loads of extra guard clauses in downstream tasks and a way of the flow succeeding even when many tasks are skipped.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.