Hi! We are shipping our Kubernetes Run logs to Datadog. Datadog (as well as most log aggregators) prefers logs in JSON format. What is the best way to get Prefect logs formatted in JSON?

Hi @Matt Liszewski! You’re in luck, they already are! Try fetching the logs for a particular Flow Run from our GraphQL API using the interactive API and a query like this:

query {
  flow_run_by_pk(id: "FLOW_RUN_ID_HERE") {
    id
    name
    state
    logs {
      id
      timestamp
      level
      message
    }
  }
}

thanks. the issue is Datadog prefers the logs in a JSON format but there is no straightforward way to change the formatter on the Prefect logs. I was able to get it to work on a "local" run by overwriting the Stream handler but that doesn't work when running in a container. I also can't seem to find a working example of attaching a custom log handler. Are there any examples out there of people successfully getting Prefect flows executing in a container to output JSON logs

Hey @Matt Liszewski -- just want to confirm that you need the Prefect logs rather than just your own personal logs? Also, is there a reason you can't pull the logs from the database, format them, and send them to datadog?

I basically want the std.out logs that appear on our containers running the jobs formatted in JSON. We have Datadog setup to pull these logs from our Kubernetes cluster where the jobs are executing. The logs show up but it requires custom configuration for us to parse them in Datadog and prevents us from easily adding custom

extra

log fields that we may want to trend or alert on in Datadog

I understand now 🙂 Dylan and I will check in with some more of the devs and see if we can find an ergonomic way to do this.

Awesome thanks

@Matt Liszewski did you manage to get this to work? We’re trying to do the same. We don’t see any logs from stdout from Kubernetes Jobs