Lawrence Finn

    Lawrence Finn

    1 year ago
    Is there a way to get the output from a kubernetes
    RunNamespacedJob
    and use it as an input for another task?
    Kevin Kho

    Kevin Kho

    1 year ago
    @Lawrence Finn, what output are you looking for? Does your job produce some data that you want to read?
    Lawrence Finn

    Lawrence Finn

    1 year ago
    I haven’t built anything yet, but wondering how I would go about getting output (or generating output). Would it be via stdout? A file?
    Kevin Kho

    Kevin Kho

    1 year ago
    I think you’d probably want it to be a file that would then have to be read in a downstream task if you know the location
    Lawrence Finn

    Lawrence Finn

    1 year ago
    Would you recommend something like the Job uploads results to s3 or something and the task reads from that?
    Kevin Kho

    Kevin Kho

    1 year ago
    I would yeah. That’s the easiest in my opinion
    Lawrence Finn

    Lawrence Finn

    1 year ago
    ah. but nothing is built in to handle stdout or logs or whatever?
    Kevin Kho

    Kevin Kho

    1 year ago
    There actually is , but I’m not 100% sure this is working right now. The code has a section to read the pod logs.
    Our task library doesn’t have strong guarantees in general because a lot is community contributed, but at the same time, if you had to tweak it for your use case, it might be a good starting point
    \But I think the current code is just to stream the logs to the Prefect logs. You might need the ReadNamespaceLogs task
    Lawrence Finn

    Lawrence Finn

    1 year ago
    does that exist? or would i have to write it?
    oh i see it