Hey all, We are trying to create a terminal state handler to dump the flow logs at the end of a flow and save them somewhere permanently. I am assuming we need to write a graphql query to do so, but I can't figure out how to do this. Is there any documentation or example you could point me to? I see this function

*prefect.client.client.Client.write_run_logs*(logs)

- but i want to download, not upload the logs

Hey @Kathryn Klarich, I think you would need to use the graphql API to hit the logs table to retrieve the logs like:

query {
  flow_run {
    id
    logs {
      id
      message
    }
  }
}

but for large logs, this can be slow and you might hit timeouts. There is also an API limit of 5 MB so I don’t think this is recommended. It seems a bit repetitive because your process is writing logs to Prefect Cloud and then you are downloading it and then saving it elsewhere. Maybe you can write it to a FileHandler (which I have a snippet) for, and then use the state handler to upload that file to S3? If you have some other logging service like Datadog, I think you would attach a StreamHandler to send those logs to both Prefect Cloud and that other service

Hey @Kevin Kho thanks for the quick response and the FileHandler code. We are considering trying to hook our logs up to DataDog currently but are not there yet, so that may be hlpeful in the future. Will try what you suggested for now

That example does it on the task level, but you can do it on the Flow level I think if you store as a script. You can also maybe make a function or decorator to attach it per task