Peter Roelants
01/28/2021, 3:14 PMany_failed
?
I'm not sure which is the most idiomatic way of dealing with this failure, since updating a database or sending a message to Kafka itself is prone to failure (and "negative engineering" to deal with that failure). Does Prefect have an opinion/guidelines how to deal with this?Zanie
on_failure
kwarg for Task
, see https://docs.prefect.io/api/latest/core/task.html#task-2message_kafka(failed_task, failed_state) -> None
that will be called on failure of a specific task.Peter Roelants
01/28/2021, 5:31 PMon_failure
kwarg before, thanks for pointing it out.Zanie
Peter Roelants
01/28/2021, 6:37 PMold_state
's results 😞
The only other solution to get the inputs of the failed task indeed follows your suggestion: to add the task inputs to the failed state manually, similar as in this example using signals.
However, I feel like capturing the exception and re-raising them with the tasks's input is defeating a bit of Prefect's value proposition to reduce "negative engineering". Could I make a feature request somewhere to include meta-data like the failed task's input to the Failure state?Zanie
prefect
repo so we can discuss the best way to solve this.Peter Roelants
01/28/2021, 6:46 PMZanie
Peter Roelants
01/28/2021, 7:00 PMZanie
Peter Roelants
02/01/2021, 9:20 AM