https://prefect.io logo
Join Slack
Powered by
hello...newbie question...why doesn't flow failure...
# ask-community
b
hello...newbie question...why doesn't flow failure return a non zero exit code. If one were to run this self contained inside of a docker container without using prefect cloud to start...how do you get the process to exit the docker run with error
c
Hey @Braun Reyes - one of Prefect’s features is error capturing and handling; 
flow.run()
will always only ever return a 
State
object corresponding to the flow’s final state. It sounds like you want non-zero exit codes for the sake of monitoring, which is a separate business - I’d recommend placing a terminal task on your Flow with an 
any_failed
trigger that calls 
sys.exit(1)
(or 
os._exit(1)
) to achieve your goal
j
It’s a good question — sometimes we call this “failing successfully.” What that means is Prefect tries not to apply business logic to your states (failures don’t take down the system). You’re more than welcome to add that logic yourself, however!
b
is that how most companies accomplish the docker deployment use cases. In a world without prefect cloud, lets say you schedule your prefect flow as a simple docker task(ECS or Kube job), in my experience you want a non zero exit code on exception so that trickles down the typical monitoring and notification workflows
it sounds like I answered my own question...your explanations make sense.
I am pretty new to contributing to open source...I adapted the ShellTask to do exactly what the Airflow Bash Operator does...which streams the logs instead of outputting at the end. Would prefect be open to that kind of change?
Copy code
sub_process = Popen(
                [self.shell, tmp.name],
                stdout=PIPE,
                stderr=STDOUT,
                env=current_env
            )
            line = ''
            for raw_line in iter(sub_process.stdout.readline, b''):
                line = raw_line.decode('utf-8').rstrip()
                <http://self.logger.info|self.logger.info>(line)
            sub_process.wait()
            if sub_process.returncode:
                msg = "Command failed with exit code {0}: {1}".format(
                    sub_process.returncode, line
                )
                raise prefect.engine.signals.FAIL(msg) from None  # type: ignore
        return line
this was to support dbt commands in a way that feels seamless for how people currently use it
c
yea absolutely - we’d love a PR like this!
j
💯
2 Views
Open in Slack
PreviousNext
Powered by