Running prefect with workers deployed on an EKS Cluster. How do I specify the resources (CPU, memory) for the job pod that runs the flow?

Overriding Resource Limits at the Deployment Level To allow overriding resource limits for individual deployments: 1. Edit the Kubernetes work pool and go to the Advanced tab. 2. In the first element of the containers section of the spec, add:

"resources": {
  "requests": {
    "cpu": "{{ cpu_request }}",
    "memory": "{{ memory_request }}"
  },
  "limits": {
    "cpu": "{{ cpu_limit }}",
    "memory": "{{ memory_limit }}"
  }
}

• In the

properties

of the

variables

section, add:

"cpu_request": {
  "type": "string",
  "title": "CPU Request",
  "description": "The minimum CPU given to infrastructure created by a worker."
},
"cpu_limit": {
  "type": "string",
  "title": "CPU Limit",
  "description": "The maximum CPU allowed for infrastructure created by a worker."
},
"memory_request": {
  "type": "string",
  "title": "Memory Request",
  "description": "The minimum memory given to infrastructure created by a worker."
},
"memory_limit": {
  "type": "string",
  "title": "Memory Limit",
  "description": "The maximum memory allowed for infrastructure created by a worker."
}

Now you can specify

cpu_request

,

cpu_limit

,

memory_request

, and

memory_limit

in the job_variables for individual deployments. For more information on overriding job variables, refer to the Prefect documentation.

Thank you so much!

is there a simple way to start the worker for a fresh pool from cli with these additional arguments? When I tried starting a worker from CLI with a similar template (to replace the default template) with additional arguments I was getting a schema error (because the default worker has hard coded different set of arguments). So, every time I wanted to set up a new work pool I had to start the k8s worker to generate the default template and replace it with copy-paste. What worked for me was to implement a new worker class that has these arguments hard-coded, but that's cumbersome.