https://prefect.io logo
Title
p

Patrick Alves

11/16/2022, 8:15 PM
Hi there, I have a ETL flow running on Prefect 2.6.4 that runs in a Kubernetes Cluster. I am having issues with the KubernetJob retries. I’ve create a Job template where I set the backoffLimit as 0, so it should not retry any failed flow/Job. (I will add the full template as a comment. ) The point is that Prefect is not setting the backoffLimit in Job manifest when it creates the KubernetesJob. See the config of the Job created by Prefect (see the 2nd image). I really need to avoid these retries, any tips?
{
  "apiVersion": "batch/v1",
  "kind": "Job",
  "metadata": {
    "namespace": "prefect",
    "labels": {
      "purpose": "prefect"
    }
  },
  "spec": {
    "template": {
      "spec": {
        "backoffLimit": 0,
        "completions": 1,
        "ttlSecondsAfterFinished": 300,
        "imagePullSecrets": [
          {
            "name": "harbor"
          }
        ],
        "containers": [
          {
            "env": [],
            "name": "prefect-job",
            "image": "prefecthq/prefect:2.3.0-python3.9",
            "imagePullPolicy": "Always",
            "envFrom": [
              {
                "secretRef": {
                  "name": "job-credentials-envs"
                }
              }
            ],
            "resources": {
              "requests": {
                "memory": "256Mi",
                "cpu": "250m"
              },
              "limits": {
                "memory": "1000Mi",
                "cpu": "2000m"
              }
            }
          }
        ],
        "parallelism": 1,
        "restartPolicy": "Never"
      }
    }
  }
}
r

redsquare

11/16/2022, 8:45 PM
should it not be inside the tenplate not spec
p

Patrick Alves

11/17/2022, 12:08 PM
You’re right @redsquare Thanks, it worked now!
👍 1