b

    Brian Mesick

    1 year ago
    I’m curious if other folks are running into this. Using a KubernetesRun run config and Docker storage for our flows, we almost always need to manually bump the Docker
    image_tag
    to a new version in order for K8s to pick up the new image. Does anyone have a more automated way of doing this? It’s not uncommon for someone to forget to make that change and leads to confusion about what’s running in prod.
    Michael Adkins

    Michael Adkins

    1 year ago
    Hi @Brian Mesick -- this is because the default image pull policy on the kubernetes jobs is
    IfNotPresent
    if you change this to
    Always
    it'll always pull the newest image. Overriding tags is generally not recommended by Docker though so 🤷
    b

    Brian Mesick

    1 year ago
    Thanks, I think I asked this before and got the same answer. I’m hoping this is a thing we can automate once we get CI running on our flows.
    Jim Crist-Harif

    Jim Crist-Harif

    1 year ago
    If you want you can set this globally on your agent by providing a default template with
    imagePullPolicy
    configured: https://docs.prefect.io/orchestration/agents/kubernetes.html#custom-job-template
    b

    Brian Mesick

    1 year ago
    Awesome, thanks @Jim Crist-Harif
    Digging further into this, with a K8s agent I’m not seeing where the
    prefect agent kubernetes start
    command line is actually defined. Any idea where one might override the job template the agent this way? Env var perhaps?
    Ah I think I see. We’re probably doing an
    install
    and so should use
    --image-pull-policy
    I’ll take a look at that angle.
    Kevin Kho

    Kevin Kho

    1 year ago
    @Marvin archive “Docker images not updating with Kubernetes RunConfig”
    Marvin

    Marvin

    1 year ago