I also tried inspecting the pod using

kubectl describe pods {pod_name}

Am using a private Docker registry, and have passed in

image_pull_secrets

to KubernetesRun (the secret was obtained by running

kubectl get secrets

).

RUN_CONFIG = KubernetesRun(
    image="<http://registry.digitalocean.com/registry-name/hello-world:latest|registry.digitalocean.com/registry-name/hello-world:latest>",
    labels=["label"],
    image_pull_secrets=["secret-token-value"]
)

Below is the full output of the pod inspection:

Name:         prefect-job-7b9f3bf6-snmn9
Namespace:    default
Priority:     0
Node:         flowstate-pool-8cacb/10.104.0.7
Start Time:   Sat, 06 Mar 2021 18:10:28 +0800
Labels:       controller-uid=c29400c3-72bc-4da3-8dd7-44863e3ecd20
              job-name=prefect-job-7b9f3bf6
              <http://prefect.io/flow_id=e0f1f3a1-1aea-42b1-8fee-d3915a41c07c|prefect.io/flow_id=e0f1f3a1-1aea-42b1-8fee-d3915a41c07c>
              <http://prefect.io/flow_run_id=98219371-92bf-4ea8-8e71-cf08002dde71|prefect.io/flow_run_id=98219371-92bf-4ea8-8e71-cf08002dde71>
              <http://prefect.io/identifier=7b9f3bf6|prefect.io/identifier=7b9f3bf6>
Annotations:  <none>
Status:       Failed
IP:           10.244.0.157
IPs:
  IP:           10.244.0.157
Controlled By:  Job/prefect-job-7b9f3bf6
Containers:
  flow:
    Container ID:  <containerd://61e0fa1c035643fc97ddb38fdaf082142888baf7084900d43cd844da269297a2>
    Image:         <http://registry.digitalocean.com/registry-name/hello-world:latest|registry.digitalocean.com/registry-name/hello-world:latest>
    Image ID:      <http://registry.digitalocean.com/registry-name/hello-world@sha256:30aa9b254d5f8c2c7bc04134e81faf2a99301896eafdcd7b0ff0c6f52f7d81a6|registry.digitalocean.com/registry-name/hello-world@sha256:30aa9b254d5f8c2c7bc04134e81faf2a99301896eafdcd7b0ff0c6f52f7d81a6>
    Port:          <none>
    Host Port:     <none>
    Args:
      prefect
      execute
      flow-run
    State:          Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Sat, 06 Mar 2021 18:10:31 +0800
      Finished:     Sat, 06 Mar 2021 18:10:31 +0800
    Ready:          False
    Restart Count:  0
    Environment:
      PREFECT__BACKEND:                             cloud
      PREFECT__CLOUD__AGENT__LABELS:                ['label']
      PREFECT__CLOUD__API:                          <https://api.prefect.io>
      PREFECT__CLOUD__AUTH_TOKEN:                   token_value
      PREFECT__CLOUD__USE_LOCAL_SECRETS:            false
      PREFECT__CONTEXT__FLOW_RUN_ID:                98219371-92bf-4ea8-8e71-cf08002dde71
      PREFECT__CONTEXT__FLOW_ID:                    e0f1f3a1-1aea-42b1-8fee-d3915a41c07c
      PREFECT__CONTEXT__IMAGE:                      <http://registry.digitalocean.com/registry-name/hello-world:latest|registry.digitalocean.com/registry-name/hello-world:latest>
      PREFECT__LOGGING__LOG_TO_CLOUD:               true
      PREFECT__ENGINE__FLOW_RUNNER__DEFAULT_CLASS:  prefect.engine.cloud.CloudFlowRunner
      PREFECT__ENGINE__TASK_RUNNER__DEFAULT_CLASS:  prefect.engine.cloud.CloudTaskRunner
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from secret-token-value (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  secret-token-value:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  secret-token-value
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     <http://node.kubernetes.io/not-ready:NoExecute|node.kubernetes.io/not-ready:NoExecute> op=Exists for 300s
                 <http://node.kubernetes.io/unreachable:NoExecute|node.kubernetes.io/unreachable:NoExecute> op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  16s   default-scheduler  Successfully assigned default/prefect-job-7b9f3bf6-snmn9 to flowstate-pool-8cacb
  Normal  Pulling    15s   kubelet            Pulling image "<http://registry.digitalocean.com/flowstate/hello-world:latest|registry.digitalocean.com/flowstate/hello-world:latest>"
  Normal  Pulled     14s   kubelet            Successfully pulled image "<http://registry.digitalocean.com/flowstate/hello-world:latest|registry.digitalocean.com/flowstate/hello-world:latest>" in 1.769408916s
  Normal  Created    14s   kubelet            Created container flow
  Normal  Started    13s   kubelet            Started container flow

Passing the image pull secret using the -i flag to

kubernetes install

doesn’t seem to work as well

In case this helps anyone, you can inspect the logs from your Kubernetes pod using

kubectl logs {pod_id}

. In my specific case, I had built the Docker image on a M1 Mac, and was running into the error:

exec user process caused exec format error

when inspecting the running pod. Rebuilding the Docker image on an Intel Mac worked, so the error was due to image built on arm64 instead of x86.