https://prefect.io logo
Join Slack
Powered by
Hi! I’m having some trouble in getting proper logs...
# ask-community
t
Hi! I’m having some trouble in getting proper logs from 
RunNamespacedJob
 task. I had set the 
log_level='info'
 option, but when it’s failed, no any logs are being sent to Prefect. Instead, I only get this from UI.
Copy code
FAIL signal raised: FAIL('Job dbt-run-from-flow failed, check Kubernetes pod logs for more information.')
After going to the k8s pod, I see exception logs:
Copy code
dbt ls --models tm_snowflake.* --profiles-dir=. --profile default --target dev
Encountered an error while reading profiles:
  ERROR Runtime Error
  Compilation Error
    Could not render {{ env_var('DBT_PASSWORD') }}: Env var required but not provided: 'DBT_PASSWORD'
Encountered an error:
Runtime Error
  Could not run dbt
make: *** [Makefile:6: ls] Error 2
How could I pass any logs from k8s pods to Prefect UI?
k
Hey @Tony Yun, I think there is also a 
ReadNamespacedPodLogs
task you have to use to pull the logs
t
oh I see. How could I miss this one.. thanks!
k
Actually no, I think it should do it for you here
Maybe you can try with debug level?
t
hmm. debug gives me too much extra which I don’t like
k
I guess you might need to list the pods, find it, then read the logs?
t
ok let me try few options
debug level doesn’t work either
k
Looks like there is a bug with that here
There is also a workaround. Feel free to chime in also there
t
i think that’s exactly what’s happened
ok then
btw, there is also another bug to me that, the Prefect code hard coded k8s job names so I’m not allowed to use 
generateName
to generate Random k8s job names: https://github.com/PrefectHQ/prefect/blob/8fcb9aa97d739fa25405c60cbb28b649c130bd3e/src/prefect/tasks/kubernetes/job.py#L706
should I report this in Issues too?
k
Won’t the name already be generated by then?
Ah I see what you mean
Yeah feel free to open a separate issue for that?
t
ok
it’s sad to me that I couldn’t make k8s pod logs available on Prefect… which the job will run blindly and requires manual intervention on CLI to view logs..
k
Hey I heard you will have an enterprise support channel. Those guys will be able to help better haha
t
hah, I’m not sure if it’s better than this channel. let me see.
k
Oh they will be because of this
2 Views
Open in Slack
PreviousNext
Powered by