Another question, is it possible to have prefect's cli run to exit with an error if any of the tasks error out while watching the state or streaming the logs (-w, -l)?
j
Jeremiah
11/30/2020, 9:59 PM
Hi @JC Garcia - generally in Prefect we try hard not to apply semantic meaning to states, as it becomes very difficult to generalize. What constitutes a failure to one person might be an expected behavior to another, or might be handled via on_failure triggers in order to lead to an ultimately successful result. Therefore we try to focus on supplying tools for working with task states, but encourage all reactions to those states to be done via those tools (meaning, inside the Prefect library). My gut feeling is that exiting the CLI command with such an error would cross that line and become very difficult to align with broad user expectations. However, perhaps it’s worth opening a GitHub discussion to see how others feel? Perhaps with sufficient motivation it could be a flag (not trying to tip the scale here, just thinking out loud)
j
JC Garcia
12/01/2020, 1:55 AM
I totally hear you, my use case for this is a CI pipeline that waits on a prefect job running elsewhere. I'll look at the source code and perhaps create a PR for it if others find it useful
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.