Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

image.png

Hi there,

I have a ETL flow running on Prefect 2.6.4 that runs in a Kubernetes Cluster.
I am having issues with the KubernetJob retries.
I’ve create a Job template where I set the backoffLimit as 0, so it should not retry any failed flow/Job.
(I will add the full template as a comment. )

The point is that Prefect is not setting the backoffLimit in Job manifest when it creates the KubernetesJob.
See the config of the Job created by Prefect (see the 2nd image).

I really need to avoid these retries, any tips?

```{
  "apiVersion": "batch/v1",
  "kind": "Job",
  "metadata": {
    "namespace": "prefect",
    "labels": {
      "purpose": "prefect"
    }
  },
  "spec": {
    "template": {
      "spec": {
        "backoffLimit": 0,
        "completions": 1,
        "ttlSecondsAfterFinished": 300,
        "imagePullSecrets": [
          {
            "name": "harbor"
          }
        ],
        "containers": [
          {
            "env": [],
            "name": "prefect-job",
            "image": "prefecthq/prefect:2.3.0-python3.9",
            "imagePullPolicy": "Always",
            "envFrom": [
              {
                "secretRef": {
                  "name": "job-credentials-envs"
                }
              }
            ],
            "resources": {
              "requests": {
                "memory": "256Mi",
                "cpu": "250m"
              },
              "limits": {
                "memory": "1000Mi",
                "cpu": "2000m"
              }
            }
          }
        ],
        "parallelism": 1,
        "restartPolicy": "Never"
      }
    }
  }
}```

should it not be inside the tenplate not spec

<https://kubernetes.io/docs/concepts/workloads/controllers/job/#running-an-example-job|https://kubernetes.io/docs/concepts/workloads/controllers/job/#running-an-example-job>

You’re right <@U03LH98F8S2>
Thanks, it worked now!