https://prefect.io logo
Title
j

Josef Trefil

11/12/2020, 11:36 AM
Hi everyone, can anyone please help me with getting loggs from Kubernetes
RunNamespacedJob
to TASK RUN window in the UI? So far I'm getting just:
12:21:32
INFO
RunNamespacedJob
Job test-app has been created.
12:21:37
INFO
RunNamespacedJob
Job test-app has been completed.
12:21:37
INFO
RunNamespacedJob
Job test-app has been deleted.
but I'd also like to see what's going on INSIDE the job itself. Thank you in advance! 🙂👍
d

Dylan

11/12/2020, 2:07 PM
Hi @Josef Trefil! Try setting
log_stdout=True
in your task decorator 👍
j

Josef Trefil

11/12/2020, 2:15 PM
Hi @Dylan, thank you for the reply. I've been trying it this way (app_job on line 5):
namespace = 'default'
postgres_deployment_body = read_yaml('../postgres_deployment.yml')
app_job_body = read_yaml('../app_job.yml')
postgres_deployment = CreateNamespacedDeployment(body=postgres_deployment_body, namespace=namespace, kubernetes_api_key_secret=None)
app_job = RunNamespacedJob(body=app_job_body, kubernetes_api_key_secret=None, log_stdout=True)
delete_postgres_deployment = DeleteNamespacedDeployment(kubernetes_api_key_secret=None)

with Flow("first-attempt-deploy") as flow:
    # name = prefect.Parameter("name", default='John')

    patch_k8s = patch_kubernetes()
    postgres_deployment_result = postgres_deployment(upstream_tasks=[patch_k8s])
    app_job_result = app_job(upstream_tasks=[postgres_deployment_result])
    delete_postgres_deployment(deployment_name=postgres_deployment_body['metadata']['name'],
                               upstream_tasks=[app_job_result])

flow.storage = Docker()
flow.register(project_name="myproject")
but it didn't do anything. I've tried to put logger.info() , logger.error() and print() statements into the job, but niether showed up. Is this the way you mean, or is there another one?
d

Dylan

11/12/2020, 2:24 PM
That’s the way I meant
hmmm
It turns out we don’t actually have a task in the task library for getting pod logs (although we would accept a contribution). You would have to call something like 
read_namespaced_pod_log
 from the kubernetes library to get the logs from the pod that the job creates. Prefect doesn’t have access to the logs by default because the job is running in a separate container
j

Josef Trefil

11/12/2020, 5:39 PM
@Dylan Alright, thank you very much. :-) I'm afraid my sklills are not yet on such a level for me to be a valid contributor to the project, but maybe in future. Prefect looks very promissing! Thanks again! :-)
d

Dylan

11/12/2020, 5:51 PM
Anytime!
j

Joël Luijmes

11/25/2020, 12:00 PM
I also ran into this issue, soo I opened up a PR!
💯 1
d

Dylan

11/25/2020, 3:42 PM
Awesome!!