Thread
#prefect-ui
    Mathijs Schoorl

    Mathijs Schoorl

    10 months ago
    Hi All, question regarding logs in the prefect UI. We mostly use the RunNamespacedJob Task with the option log_level='info' to make sure the logs also show in the prefect logs. This worked fine in version 0.14.10. Recently we upgraded to 0.15.3 and I'm noticing that the logs don't show up in the prefect ui anymore. The logs do show up in the prefect-flow pod. Any idea how to fix this?
    prefect-job logs in the pod:
    same run logs in the UI:
    Anna Geller

    Anna Geller

    10 months ago
    @Mathijs Schoorl I don’t know how it was handled in previous Prefect versions, but
    RunNamespacedJob
    task doesn’t collect its logs to the backend. For that you would have to leverage
    ReadNamespacedPodLogs
    task. So it’s not an issue with the UI but rather with this task. I have to examples to better explain what I mean: #1 This flow https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows_task_library/s3_kubernetes_run_RunNamespacedJob.py doesn’t collect any logs. It only triggers the pod and finishes. #2 This flow creates the K8s job, then list pods that are available to find the pod that we’ve just spun up, gets the pod name from the list, and finally gets and prints the logs from this pod: https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows_task_library/s3_kubernetes_run_RunNamespacedJob_and_get_logs.py
    Mathijs Schoorl

    Mathijs Schoorl

    10 months ago
    Hi Anna, I'll look into it your example. As far as I know the RunNamespacedJob does read logs when you enable it and in the background uses ReadNamespacedPodLogs. But like I said the logs are indeed read and visible in the job pod log but not in the ui.
    Anna Geller

    Anna Geller

    10 months ago
    You’re correct about the logs in the task, I can see it here: https://github.com/PrefectHQ/prefect/blob/ca19b43947eb16e36fe9ea777e4e65c15baaf0b8/src/prefect/tasks/kubernetes/job.py#L753 and there is an open issue about it here - if you want, feel free to chime in and describe your use case incl. which Prefect version and agent you use: https://github.com/PrefectHQ/prefect/issues/4675
    Mathijs Schoorl

    Mathijs Schoorl

    10 months ago
    Thanks! I'll check it out