Does anyone here have a problem with prefect agent not flushing stdout or just contents getting lost when doing something like
prefect agent local start -l mylabel --show-flow-logs 2>&1 | tee /my/file/here.log
? I've got a very long-running flow, and I don't see any output. Is prefect buffering? Do I need to use
stdbuf
when running the process?
k
Kevin Kho
10/30/2021, 11:05 PM
Hey @William Grim, this is new to me, but it actually worked for me. Maybe going to DEBUG level logs will give you more logs to tell?
Copy code
prefect agent local start -l mylabel --show-flow-logs --log-level=DEBUG 2>&1 | tee here.log
This worked for me and I got an output
w
William Grim
10/30/2021, 11:17 PM
@Kevin Kho sometimes it works. I see it only output info occasionally through tee. My theory is that it buffers and doesn't output, even though I would've thought it should flush.
I'm going to try stdbuf soon and, if it doesn't work, I'll probably try to install my own handler at startup by getting in really early at startup.
I'm just on vacation now or I would probably go ahead and try.
William Grim
10/30/2021, 11:18 PM
If it's something like me theory, I'll submit some docs or something to help people that want to do this sort of thing.
k
Kevin Kho
10/30/2021, 11:20 PM
Sure that would certainly be welcome. I saw you post in the earlier thread about outputting all the logs with
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.