j

    Jake

    4 months ago
    Hello! We have a flow that is highly resource intensive. We already specify custom resource requests but that alone is not enough. When the agent schedules it; we would like it to make use of a priority class to ensure that it has access to all the resources it needs. How can this be done? I see in the docs that there is a way to provide a custom job template:
    flow.run_config = KubernetesRun(job_template_path="<s3://bucket/path/to/spec.yaml>")
    But I’m not sure how to write that template / how this template is used (are there any docs on this?). Thanks!
    Kevin Kho

    Kevin Kho

    4 months ago
    So the template is loaded in and then used and merged with the other settings you provide. If you don’t provide a job template, there is actually a default one used here. I guess this is a starting template, the requirement is you need a container named
    flow
    The agent needs access to that s3 address during runtime
    j

    Jake

    4 months ago
    this is exactly what I was looking for thanks!
    If I provide the job_template; the do the other settings get merged in (labels, flow metadata, resource requests etc.) if they are not part of the template?
    Kevin Kho

    Kevin Kho

    4 months ago
    Yes they do. Priority given to stuff in KubernetesRun, they will override the template specifications
    j

    Jake

    4 months ago
    Amazing! thank you