Hi there, hope that someone could help me, I've got a Kubernetes Agent correctly installed, and I need to use Docker Storage for my flows, but i need to push the docker images on a private registry. My jobs fail with this error

Kubernetes Error: rpc error: code = Unknown desc = Error: image XXXXXX not found

So I've tried to tweak the job spec using

KubernetesJobEnvironment(_job_spec_file_="job_spec.yaml")

adding an

imagePullSecrets

to the Job template But the custom yaml that i defined seems not to be used when the job is spawned by the agent, what am I doing wrong? Here is how i register the flow:

flow.environment = KubernetesJobEnvironment(job_spec_file="job_spec.yaml")
      flow.storage = Docker(
        registry_url="REGISTRYURL",
        python_dependencies=['psycopg2-binary', 'pymssql'],
        )
      flow.register(args.register)

Hi @mogui mogui, the initial job started by the k8s agent isn't configured by the environment (confusing, I know). The k8s agent always starts an initial job, which if you're using

KubernetesJobEnvironment

then starts a second job. We're doing some work right now to simplify things (which has been merged, but is undocumented and is a "secret" feature still). You have two options here: • Configure image pull secrets on your k8s agent as well by setting

IMAGE_PULL_SECRETS

in your agent environment. This will add the image pull secrets to the initial job as well, and should fix things for you without other changes.

IMAGE_PULL_SECRETS=... prefect agent start kubernetes

• Use the new (secret)

KubernetesRun

style configuration. This lets you configure the entire k8s job as part of your registered flow, and skips the intermediate job started using the old environment-style configuration. Note that with the following configuration your job spec is stored in prefect cloud, so don't put anything private in there (there are ways to get around this, but as I said all of this is currently secret and undocumented). Some docs here: https://docs.prefect.io/api/latest/run_configs.html#kubernetesrun

from prefect.run_configs import KubernetesRun

# Configure a flow to run as a k8s job, using
# a job spec at the following path.
# 
# Note that `run_config` based config replaces `environment`
# based config, so you shouldn't configure `flow.environment`
flow.run_config = KubernetesRun(
    job_template_path="job_spec.yaml"
)

In my case I had to provide both the registry_url and image_name

Docker(registry_url="<https://FOO>", image_name="Bar")

Then the image to be pushed is

<https://FOO/BAR>

I dont see the image_name in your example. I am now curious what happens if I leave it off

Hi Asif, that's something separate (

Docker

storage builds an image, and optionally pushes it to a registry). To answer your question, if you leave the image name off the

Docker

storage config, a name will be generated for you based on the flow. The question above was about configuring a k8s job, and why some configuration (the image pull secrets) weren't being used in the started job.

Mb. Thanks Jim

@Jim Crist-Harif Many thanks I missed the IMAGE_PULL_SECRETS env var, it did the trick! I'm stil having problem with my repository but unrelated with prefect, I'v tryied with a private registry using github packages and everything works like a charm! Interesting the secret feature, yes I think documentation for the Kube agent could be more verbose Thanks again