Hi All, question regarding logs in the prefect UI. We mostly use the RunNamespacedJob Task with the option log_level='info' to make sure the logs also show in the prefect logs. This worked fine in version 0.14.10. Recently we upgraded to 0.15.3 and I'm noticing that the logs don't show up in the prefect ui anymore. The logs do show up in the prefect-flow pod. Any idea how to fix this?

@Mathijs Schoorl I don’t know how it was handled in previous Prefect versions, but

RunNamespacedJob

task doesn’t collect its logs to the backend. For that you would have to leverage

ReadNamespacedPodLogs

task. So it’s not an issue with the UI but rather with this task. I have to examples to better explain what I mean: #1 This flow https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows_task_library/s3_kubernetes_run_RunNamespacedJob.py doesn’t collect any logs. It only triggers the pod and finishes. #2 This flow creates the K8s job, then list pods that are available to find the pod that we’ve just spun up, gets the pod name from the list, and finally gets and prints the logs from this pod: https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows_task_library/s3_kubernetes_run_RunNamespacedJob_and_get_logs.py

Hi Anna, I'll look into it your example. As far as I know the RunNamespacedJob does read logs when you enable it and in the background uses ReadNamespacedPodLogs. But like I said the logs are indeed read and visible in the job pod log but not in the ui.

You’re correct about the logs in the task, I can see it here: https://github.com/PrefectHQ/prefect/blob/ca19b43947eb16e36fe9ea777e4e65c15baaf0b8/src/prefect/tasks/kubernetes/job.py#L753 and there is an open issue about it here - if you want, feel free to chime in and describe your use case incl. which Prefect version and agent you use: https://github.com/PrefectHQ/prefect/issues/4675

Thanks! I'll check it out