Thread
#prefect-community
    Joseph Finlayson

    Joseph Finlayson

    1 year ago
    Hello. I hope this is the right place. I'm really having a nightmare with the agent creating jobs on my prefect deployment. the job itself has "ErrImgPull" as it somehow cannot get access to the image.
    │   ----     ------          ----              ----                  -------                                                                 │
    │   Normal   Scheduled       <unknown>         default-scheduler     Successfully assigned prefect/prefect-job-3afb5e32-qx8wd to first-3kwtz │
    │   Normal   Pulling         13s               kubelet, first-3kwtz  Pulling image "<http://registry.digitalocean.com/unmanaged-registry/prefect-job|registry.digitalocean.com/unmanaged-registry/prefect-job> │s:ac673b8facdc4d18a75da5e8fb98b63b"                                                                                                        │
    │   Warning  Failed          13s               kubelet, first-3kwtz  Failed to pull image "<http://registry.digitalocean.com/unmanaged-registry/pref|registry.digitalocean.com/unmanaged-registry/pref> │
    │ ect-jobs:ac673b8facdc4d18a75da5e8fb98b63b": rpc error: code = Unknown desc = Error response from daemon: Get <https://registry.digitalocean> │
    │ .com/v2/unmanaged-registry/prefect-jobs/manifests/ac673b8facdc4d18a75da5e8fb98b63b: unauthorized: authentication required                  │
    │   Warning  Failed          13s               kubelet, first-3kwtz  Error: ErrImagePull
    Agent YAML looks like this
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: prefect
      name: prefect-agent
      namespace: prefect
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: prefect
      template:
        metadata:
          labels:
            app: prefect
        spec:
          containers:
          - args:
            - prefect agent start kubernetes         command:
            - /bin/bash
            - -c
            env:
            - name: NAMESPACE
              value: prefect
            - name: PREFECT__CLOUD__AGENT__AUTH_TOKEN
              value: XXXXX
            - name: PREFECT__CLOUD__API
              value: <https://api.prefect.io>
            - name: IMAGE_PULL_SECRETS
              value: unmanaged-registry
    The generated job itself doesn't seem to have anything set regarding an imagepull secret
    m

    Marwan Sarieddine

    1 year ago
    @Joseph Finlayson in case you want to avoid having to set an image pull secret on the agent deployment - you can instead set the image pull secret directly on the service account
    Joseph Finlayson

    Joseph Finlayson

    1 year ago
    This is already done unfortunately
    m

    Marwan Sarieddine

    1 year ago
    weird - are you sure the image secret is correctly formed? i.e. did you try to deploy a pod (without using prefect) with an image from
    unmanaged-registry
    ?
    Joseph Finlayson

    Joseph Finlayson

    1 year ago
    Hey @Marwan Sarieddine I think that was the issue! Digital ocean has started auto-injecting secrets, but they don't seem to work!