Hi Prefect, What's the recommended way of dealing with Task failures? For example, I want to write to a db table or send a message to Kafka if a certain task has failed. From the documentation it seems that there are two ways of handling failure of the task: • Implement a State Handler to deal with the failed state. • Implement a Task to deal with the failed stage and use a Trigger with

any_failed

? I'm not sure which is the most idiomatic way of dealing with this failure, since updating a database or sending a message to Kafka itself is prone to failure (and "negative engineering" to deal with that failure). Does Prefect have an opinion/guidelines how to deal with this?

Hi Peter — I believe upstream tasks for triggers are immediate parents only. I’d recommend just using the

on_failure

kwarg for

Task

, see https://docs.prefect.io/api/latest/core/task.html#task-2

I didn't notice the

on_failure

kwarg before, thanks for pointing it out.

Ah that’s a great question — it might be in the context? Otherwise you can capture the exception and raise your own failure state with the task inputs.

I've been trying to look for it, It doesn't seem to be in the context, nor in the

old_state

's results 😞 The only other solution to get the inputs of the failed task indeed follows your suggestion: to add the task inputs to the failed state manually, similar as in this example using signals. However, I feel like capturing the exception and re-raising them with the tasks's input is defeating a bit of Prefect's value proposition to reduce "negative engineering". Could I make a feature request somewhere to include meta-data like the failed task's input to the Failure state?

This feels like a reasonable request but you do need to consider that attaching data to a state shares it with Prefect Cloud which is a big no-no in the hybrid model. There’s probably a way around this though — I suggest you open an issue in the

prefect

repo so we can discuss the best way to solve this.

Thanks, I'll think about it a bit more and try to write something up for a feature request.

You could try placing the inputs into the context at the start of your task — I’d have to look around a bunch to see how long that persists

Looking at the context documentation it seems like they will disapear after the task exists and leaves the context manager to set the inputs.

Yeah I think you’d be best served with a feature request

I added the feature request at https://github.com/PrefectHQ/prefect/issues/4039