Hi! i'm trying to log the duration of each flow/task without adding too much code. I see the duratio...

Lior Barak

09/27/2023, 2:49 PM

Hi! i'm trying to log the duration of each flow/task without adding too much code. I see the duration shows up in the UI so it's calculated somewhere, any easy/hacky way to access the start/end times of each task and flow?

✅ 1

Oscar

09/27/2023, 3:10 PM

Maybe a get request to this endpoint would be sufficient? The start and end times are in the response: https://api.prefect.cloud/api/accounts/{account_id}/workspaces/{workspace_id}/flow_runs/{id}

Lior Barak

09/27/2023, 3:12 PM

sadly no i'm trying to integrate my Prefect project into my company's logging system and by far the easiest way is for the flows themselves to write run duration to log

Oscar

09/27/2023, 3:13 PM

could you not combine it with completion/failure hooks?

Lior Barak

09/27/2023, 3:24 PM

maybe, but it seems to me that there's an easier solution here i'm not seeing

Lior Barak

09/27/2023, 5:03 PM

@Oscar thanks for giving me the idea to look around in completion/failure hooks! ended up implementing something a bit weird but it works so I guess i'm fine with it:

Copy code

from prefect.client.schemas.objects import State, FlowRun
from prefect.client.schemas.objects import Flow as FlowSchema
def log_runtime(flow: FlowSchema, flow_run: FlowRun, state: State):
    flow_run_time_seconds = (state.timestamp - flow_run.created).total_seconds()
    print(f"flow_name: {flow.name}, runtime_seconds: {flow_run_time_seconds}")

Oscar

09/27/2023, 5:06 PM

You’re welcome! Glad it works

50 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.