David Elliott
11/03/2022, 2:17 PMKubernetesJob
infra?
I’m finding that the agent creates the job + pod just fine (and it the flow + pod run through to completion) but after X seconds (per that timeout parameter) the agent logs Job 'xxxxx': Job did not complete.
per this even though the job is mid-way through running? ie it doesn’t seem to have any negative effect on the flow, it’s just telling me the job didn’t complete even when the job is very much still running..? Feels like something’s not quite right, just wanting to understand what the intention is….stream_output=False
set because I was finding the pod (flow) logs were being intermittently streamed back to the agent which felt dodgy, and stream_output=false
appears to have stopped that, but now I have this error…stream_output
= false (so as not to stream the logs back to the agent) it skips the part where it follows the pod logs (presumably until the flow is completed)
• in skipping that part, it jumps right to wait for job to complete section
• however it appears to be exiting per the else clause after the job_watch_timeout_seconds
.
• Feels like the intended behaviour is to wait until the job returns as completed, but it appears that the watch.stream
is not yielding anything after a short period and the for loop exits prematurely, resulting in this error log…
Any thoughts on why this might happen / whether it should be fixed?Zanie
11/03/2022, 4:01 PMjob_watch_timeout_seconds
so it was declared incompleteDavid Elliott
11/03/2022, 4:05 PMstream_output
also wants to be False by default as well..? seems odd to have the agent streaming out the logs of its flows/pods..? in 1.0 I was quite used to the agent just being a creator / watcher of flow runs, rather than streaming the logs outZanie
11/03/2022, 4:08 PMDavid Elliott
11/03/2022, 4:10 PMstream_output
setting to switch off the behaviour 🙂 it is there in the docs though so that’s on me!Jarvis Stubblefield
11/08/2022, 6:46 PMZanie
11/09/2022, 5:07 PM