Hi! I have a flow that executes a

RunNamespacedJob

task. Because of a known bug the way I'm obtaining the logs are via

ReadNamespacedPodLogs

. All that working well. Recently I changed the pod spec to run the application as an

initContainer

then some docker image to capture results as the actual

container

. So I have an image that generates content and logs in the

ìnitContainer

and a

container

that process the produced content (upload to gcs basically). Problem: the logs from the initContainer are not being captured by prefect, only the logs from the container. Any recommendation? 🙏

This is interesting, I've never seen this pattern before to use init containers to run an actual application, I thought they are meant more for setup scripts before some other app can start. Is the only reason to use an init container here because init containers run until completion regardless of failure? I wonder whether this is something you can instead tackle using prefect triggers/state handlers, or with https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/?

I decided that :D Idea is we have dynamic flows that triggers random docker images. All I have is a docker image (and some k8s settings). That docker image is 100% independent of prefect, but still I want to run it within prefect. And here is the magic. I want to have a way to produce artifacts (ex: input payload to a http request, response, CSV, etc) then store those on GCS. Something I can do is say to the image "put your artifacts in this folder" but that's it there is no way to connect with prefect artifact controls.

ahh I see, definitely a hard problem orchestrating processes that are completely independent and don't speak the same language but need to work in coordination and exchange data I asked our Kubernetes expert if he has some ideas, will let you know if he recommends sth

awesome, thank you!

I got a response and generally speaking we assume that jobs that we’re orchestrating only have one container, and it seems like a valid feature request to capture logs from all containers. One of the challenges is that each of the containers has its own log output and we’d need to aggregate them all, and some of them can run in parallel (e.g. multiple init containers or multiple containers). Due to the focus on Prefect 2.0, we don't have resources to make changes to this 1.0 task, but if you would want to contribute yourself or write custom logic for this in your flow, you could potentially add a flag to

ReadNamespacedPodLogs

to specify container names and for the future, I can imagine integration like this in prefect-kubernetes Collection in 2.0

I'll give it a try 👍

nice work!

this is working amazingly well, thanks for your push! 👌

Awesome work getting this done success kid