https://prefect.io
Join Slack
Hello there, I am seeing my flow logs showing up intermixed with my agent logs, and am wondering why...
# ask-community
b
Hello there, I am seeing my flow logs showing up intermixed with my agent logs, and am wondering why. I'm running Prefect 2.8.0 (cloud), running on EKS with 
KubernetesJob
infrastructure.
✅ 1
For example, here's what I see when tailing stdout on the agent pod:
Copy code
│ 21:49:34.025 | INFO    | Flow run 'authentic-termite' - Created task run 'dbt_build-0' for task 'dbt_build'                                                                                                                                       
│ 21:49:34.026 | INFO    | Flow run 'authentic-termite' - Executing 'dbt_build-0' immediately...                                                                                                                                                    
│ Wrote profile to /home/grebe/.dbt/profiles.yml                                                                                                                                                                                                    
│ Running dbt command: dbt --log-format json build --select XXX --exclude XXX --profiles-dir /home/grebe/.dbt --project-dir /home/grebe/src/dbt
Snipped but it shows the rest of the logs from that flow run.
The flows are definitely running in separate jobs, I am able to inspect and tail those pods separately.
r
KubernetesJob
has output streaming enabled by default
Unless you've turned it off, this is what I'd expect to see
In the UI, you'll see a 'Stream Output' option you can edit, and if you create your blocks in Python you can set the 
stream_output
attribute to 
False
👍 1
b
No kidding, OK! We just deployed this as our first production flows on prefect 2, and I never noticed this behavior during development.
I'll try that setting, thanks for the quick tip.
👍 1
Docs say...
If set, stream output from the container to local standard output.
I still want to see the output in the job itself, just not the agent... is this still what I want to change?
r
Yes, it should still show up in the job itself. By 'local standard output' I believe it means the local output of the agent running the flow. We could probably update the docs to make that more clear, so we appreciate the feedback.
b
Sweet, that seems very logical but ambiguous at the same time. Appreciate your help!
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by