Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Is there a way to get the output from a kubernetes `RunNamespacedJob` and use it as an input for another task?

<@U0265CL3G0L>, what output are you looking for? Does your job produce some data that you want to read?

I haven’t built anything yet, but wondering how I would go about getting output (or generating output).  Would it be via stdout?  A file?

I think you’d probably want it to be a file that would then have to be read in a downstream task if you know the location

Would you recommend something like the Job uploads results to s3 or something and the task reads from that?

I would yeah. That’s the easiest in my opinion

ah.  but nothing is built in to handle stdout or logs or whatever?

There actually <https://github.com/PrefectHQ/prefect/blob/1867f205405e1f68b828ce9dcf00bbd4969903bc/src/prefect/tasks/kubernetes/job.py#L744-L761|is> , but I’m not 100% sure this is working right now. The code has a section to read the pod logs.

Our task library doesn’t have strong guarantees in general because a lot is community contributed, but at the same time, if you had to tweak it for your use case, it might be a good starting point

\But I think the current code is just to stream the logs to the Prefect logs. You might need the ReadNamespaceLogs task

does that exist?  or would i have to write it?