@Marvin, I want to override specific job configurations (cpu and memory pod defs) for specific prefect deployments (running on Kubernetes) for example, my work pool is defined like:

...
  "job_configuration": {
...
    "job_manifest": {
      "kind": "Job",
      "spec": {
        "template": {
          "spec": {
            "containers": [
              {
                "env": "{{ env }}",
                "args": "{{ command }}",
                "name": "prefect-job",
                "image": "{{ image }}",
                "resources": {
                  "limits": {
                    "cpu": "1000m",
                    "memory": "6G",
                    "ephemeral-storage": "1Gi"
                  },
                  "requests": {
                    "cpu": "1000m",
                    "memory": "6G",
                    "ephemeral-storage": "1Gi"
                  }
...

I know that we can set "Job Variables" for deployment, but im not sure if thats the approach and how to set it for example to have "memory": 60G

To override specific job configurations like CPU and memory for Prefect deployments running on Kubernetes, you can use job variables. These allow you to customize deployment settings dynamically. Here's a concise guide to achieve this: 1. Job Variables: You can define job variables in your deployment to override the default settings in your job configuration. This can be done by passing a dictionary to the

job_variables

argument of the

.deploy()

method. 2. Prefect YAML Configuration: In your

prefect.yaml

file, you can specify job variables under the

work_pool

configuration. For example:

yaml
   work_pool:
     name: your-work-pool-name
     job_variables:
       resources:
         limits:
           memory: "60G"

3. Using the Prefect UI or CLI: You can also override job variables directly from the Prefect UI or CLI when initiating a flow run. This allows you to adjust configurations without altering the code. For more detailed instructions, you can refer to the following resources: - Kubernetes Infrastructure Examples - Customize Infrastructure Concepts These steps should help you customize your Kubernetes job configurations effectively. If you have further questions or need additional assistance, feel free to ask.

I have this in my deployment config - but it's affects nothing

{
  "resources": {
    "limits": {
      "cpu": "3000m",
      "memory": "50G",
      "ephemeral-storage": "1Gi"
    },
    "requests": {
      "cpu": "3000m",
      "memory": "50G",
      "ephemeral-storage": "1Gi"
    }
  }
}