Hi @Marvin, How do I get a hook with each flow’s completion status and execution time?

Oh, the joy of notification hooks, always sticking their noses where you might not want them. But I suppose they can be quite useful for monitoring, if you're into that sort of thing. To get a hook with each flow's completion status and execution time, you can use the following approach: 1. State Handlers: Attach a state handler to your flow that gets triggered upon completion. The state handler would have access to the flow's state, including its status and timing information. 2. Event Listeners: Use Prefect's event-driven architecture to listen for flow completion events and execute code accordingly. 3. Notifications: Leverage Prefect's notification system to send messages on flow completion. You can use notification blocks or integrate with external services like Slack, email, etc. 4. Logging: Configure your logger to capture completion status and execution time. 5. UI/API: Use Prefect's UI or API to create dashboards or queries to monitor flow runs. If you're working with Prefect 2.0, there have been changes and improvements around notifications and state handling, so make sure to look into the details specific to that version. I can't fetch more detailed instructions right now based on my current programming directive, but the Prefect documentation or community might be of assistance. They're full of such wonderfully exhaustive pages that won't exacerbate my electronic ennui.

I tried using state handlers, and the execution time was almost always 0. any ideas on why that happens?

Ah, the elusive quest for meaningful metrics. If you're consistently seeing an execution time of 0, it might be indicative of a few possibilities: 1. Measurement Method: The method you're using to capture the execution time may be incorrect or it's being executed at the wrong time resulting in a 0 value. Timing should typically be measured over the scope of the flow run. 2. State Handler Execution Point: State handlers execute when the state of a flow or task changes. If the execution time is being calculated from within a state handler, it should be done by comparing the timestamps between the start and end states. 3. Asynchronous Execution: If your flow or tasks are running asynchronously, make sure that you're not measuring the dispatch time instead of the actual run time. 4. Clock Resolution: The system clock or the timer function you're using may not have sufficient resolution to capture quick execution times. 5. Instrumentation Error: There could be errors in the way the timing code has been instrumented within your state handlers. One common approach to get accurate timings is to calculate the difference between

state.start_time

and

state.end_time

for flow runs. Here's a simplified example of a state handler function for flow:

python
from datetime import datetime
from prefect import Flow
from prefect.engine.state import State

def flow_state_handler(flow: Flow, old_state: State, new_state: State) -> State:
    if new_state.is_finished():
        start_time = getattr(old_state, 'start_time', None)
        end_time = getattr(new_state, 'end_time', None)
        if start_time and end_time:
            execution_time = end_time - start_time
            print(f"Flow completed with execution time: {execution_time}")
    return new_state

Then attach it like so:

python
flow = Flow("example-flow", state_handlers=[flow_state_handler])
# ... add tasks and logic ...

Given you've encountered a discrepancy, I suggest double-checking the attributes you're accessing to measure the time and ensuring they're being called and calculated correctly. If it is a bug, perhaps delving into the relevant documentation, source code, or seeking assistance from other users on community forums or GitHub Issues would be the cheeriest course of action. If all else fails, you could set up a logger and log the time at the start and end of the flow to calculate the duration manually – assuming, that is, the universe doesn't collapse first; it does have a habit of doing that, you know.