z

    Zach Hodowanec

    1 year ago
    Hi Prefect Team! I am wondering if there is a way to set static
    run_config
    parameters on a Kubernetes Agent for subsequent flows to consume rather than duplicating similar
    run_configs
    across various flows. We currently make use of the
    PREFECT__CLOUD__AGENT__ENV_VARS
    to pass along
    storage
    configurations, but not having much success attempting to update the execution environment to use an internal custom image. I have tried passing the
    IMAGE
    and the
    PREFECT__CONTEXT__IMAGE
    environment variables to my job spec thus far to no avail.
    As a follow-up, when using the
    PREFECT__CONTEXT__IMAGE
    environment variable I am seeing the value set correctly in my configmap, however when the job kicks off the env variable is using the
    prefecthq/prefect:0.14.6
    as it's value. What's interesting about this use case is that if don't pass in a
    PREFECT__CONTEXT__IMAGE
    value at all the k8s job does not assign a value for it.
    Jenny

    Jenny

    1 year ago
    Hi @Zach Hodowanec - thanks for your message. I want to make sure I'm understanding your request and use case correctly. Am I right that what you'd like to do is re-use your run-configs for different flows? You might find this discussion on flow deployment useful but come back if you've got more questions.
    z

    Zach Hodowanec

    1 year ago
    Hi @Jenny - thanks for your response. Yes, instead of defining a
    flow.run_config = KubernetesRun(image=os.getenv("MY_CUSTOM_IMAGE")
    on every flow I was hoping to set something on the agent config itself to override the default execution environment image. Since we know each flow is going to execute on a Kubernetes agent it seems redundant to also define the
    flow.run_config
    as such. Potentially this issue is more in line with what I'm trying to accomplish?
    Jenny

    Jenny

    1 year ago
    Thanks Zach. I've checked with the team and I don't think that's possible right now. But I'm going to open an issue to request it. If you want to add any more information in that issue to help us understand your use case, please do! @Marvin open "Enable default run_config on an agent"
    Marvin

    Marvin

    1 year ago
    Jenny

    Jenny

    1 year ago
    One more suggestion is that you could you use job template: https://docs.prefect.io/orchestration/agents/kubernetes.html#custom-job-template
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Note that that won't work currently for
    image
    (fixing that now).
    z

    Zach Hodowanec

    1 year ago
    Ok, thank you both! @Jim Crist-Harif Is
    flow.run_config = KubernetesRun(image=os.getenv("MY_CUSTOM_IMAGE")
    set on each flow the best/recommended way to go about it for now?
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Setting on the run config is the best option for now, yeah. Whether using an environment variable is the best way of doing that is up to you.
    z

    Zach Hodowanec

    1 year ago
    Great, thanks! Any idea on timeline for making
    image
    more configurable?
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    Should be in next release, almost done with the PR
    z

    Zach Hodowanec

    1 year ago
    Thanks @Jim Crist-Harif! Just to clarify - this will still require use to use a custom job spec and does not expose the ability to override the
    image
    for Prefect's default job spec, correct?
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    This requires the agent to be deployed with a custom job spec, but requires no configuration on the flows.
    So there's no
    --image
    flag on the agent, you'd write your own job template and pass it to the agent as
    --job-template your-template.yaml
    .
    The default job template (https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/kubernetes/job_template.yaml) is completely empty (the bit that is there is only for user reference), a minimal one that does what you want would be:
    apiVersion: batch/v1
    kind: Job
    spec:
      template:
        spec:
          containers:
            - name: flow
              image: your-image-here
    z

    Zach Hodowanec

    1 year ago
    Great, thanks!