Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Potentially related to <https://prefect-community.slack.com/archives/CL09KU1K7/p1634766491212000> on a long-running flow, we are no longer able to receive the logs for that flow. Any time I use `prefect get logs --id uuid_of_that_flow`, it gets killed with no logs retrieved.

Is there another place to go look for logs?

```$ prefect get logs --id my_uuid
Killed```

Last night, when I could still get logs, the logs had grown to 13m. We do have debugging information on, and I can see that the flow is still operational, since I see it with `procps` tools and because it writes some transient files to disk, and it's still doing that.

Hi! `prefect get logs` doesn't support pagination yet so may not be able to download all of the logs

Although usually it'd fail with a different error. See <https://github.com/PrefectHQ/prefect/issues/4719> for a tracking issue,

<@U01CEUST9B5> Sorry for the delay. For some reason I didn't notice a notification.

This is in our prod env, and I don't have access to the UI, at least not without figuring a way to punch a hole to it.

But I'm surmising from what you've said that if I could connect to the UI, it knows how to paginate correctly somehow, and it's technically possible. Does that mean there are some python prefect library hooks I could use to write my own custom version of a CLI tool to grab logs?

<@U020D7UAFFG> you can use limit and offset in a loop to retrieve logs in chunks. There's log retrieval at <https://github.com/PrefectHQ/prefect/blob/master/src/prefect/backend/flow_run.py#L428> that does not introduce limit/offset but can be a starting point.

Here's a file with the more advanced query the UI makes <https://github.com/PrefectHQ/ui/blob/master/src/graphql/Logs/flow-run-logs.gql>

If you want to add limit/offset handling to that `get_logs` function we'd be happy to accept the contribution

Thank you, and yes, I did have the same though about maybe attempting to expand the log capabilities of the CLI and also add a way to cancel the flow runs, since I have that working locally in a 2 line piece of code