Also there 2 more strategic questions:
1. How doesn't Prefect understand that flow fails? I have a Flow with a single Task. Task calls some external module and that module throws an exception, but Task and Flow in Prefect completes successfully?
a
Anna Geller
03/24/2022, 1:32 PM
Usually, when your code throws an exception, this is how Prefect can find out that something failed in your task run. But it could be that if your task is triggered e.g. by a
ShellTask
, the return code indicates success despite the actual exception. Could you share your flow code? This would help to determine what is happening there
i
Ievgenii Martynenko
03/24/2022, 2:19 PM
Unfortunately I cannot share the code, but you pointed me to some ideas. Also is there a common task return code indicating a failure inside a task?
a
Anna Geller
03/24/2022, 5:15 PM
I'm particularly interested in how dependencies between tasks are configured and which task is set as a reference task. If you set the failing task as a reference task, then the entire flow run will be considered as failed.
https://docs.prefect.io/core/about_prefect/next-steps.html#reference-tasks
I'm not completely agree, as customer logger is added properly. Anyway when I find a RC, I will share results here as usual. I suppose that is not connected with Prefect.
a
Anna Geller
03/24/2022, 6:06 PM
sure, keep us posted on that
Anna Geller
03/24/2022, 6:07 PM
but based on the screenshot you shared, the custom logger did not seem to be connected to Prefect which is why Prefect was not aware of the exception
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.