Hi everyone - is there an easy way to get the Docker container id for a flow run? We are currently get it by checking the prefect logs for the flow run, but I’m wondering if there’s a better way. We’re using the Docker agent and we are on prefect 0.15.13.

why do you need a container ID?

We’re having issues with prefect logging not capturing everything,

prefect get logs --id <id>

doesn’t return all of the output. So we set up the Docker daemon to log to AWS CloudWatch Logs, but those logs we must retrieve by container id

why don't you leverage Prefect backend for logs? the UI should display all the logs - could you explain a bit more why did you decide to retrieve logs via CLI command?

So I could view or work with the entire log file easily.

prefect get logs --id <id> | less

or ``prefect get logs --id <id> | grep` is easier than clicking thru the UI, which for me is only displaying a small portion of the log file. It could be a largish log file, 1-5 MB

it's definitely your choice how you decide to do it - if the UI doesn't show all logs that you need, this seems to be a different problem though

It’s about inspecting logs myself - the flow run failed with an error and I’d like to take a closer look at the logging

gotcha - I've never used that feature myself but if this doesn't show all logs you need, I believe this is the actual problem here, rather than logging the container ID

yeah I agree that’s the real problem here. For reference, retrieving the logs from AWS CloudWatch logs for this flow runs returns a log file of about 8 MB, but

prefect get logs

returns a log of just 412kb. Maybe

prefect get logs

has some limit of how many log statements it will return or something like that

Checking after holidays: LMK if this is still an open issue looks like the issue you had so far was that some logs did not get properly stored in the backend - if this is still an ongoing issue, you would need to cross-check how you add logs in your flow, how you configured the logger itself (e.g. log level) and ideally share some small example flow that I could reproduce that shows the issue