@Marvin I want to set limits / requests for EKS pods. I believe I set this within the advanced configuration of the Work Pool but can’t seem to find the docs that detail how to add requests and limits for all flows.

To set resource requests and limits for EKS pods for all flows in Prefect, you can modify the default base job template in the work pool configuration. Here's how you can do it: 1. In the Prefect UI, navigate to the Work Pools section. 2. Select the work pool you want to configure or create a new one if needed. 3. Click on the Advanced tab within the work pool configuration. 4. Edit the JSON representation of the base job template. For setting a CPU request, you can add the following section under the `variables`:

json
"cpu_request": {
  "title": "CPU Request",
  "description": "The CPU allocation to request for this pod.",
  "default": "default",
  "type": "string"
}

Next, you should add the

requests

and

limits

settings to the

containers

item under

job_configuration

, like so:

json
... 
"containers": [
  { 
    ..., 
    "resources": {
      "requests": {
        "cpu": "{{ cpu_request }}",
        "memory": "YOUR_MEMORY_REQUEST"
      },
      "limits": {
        "cpu": "YOUR_CPU_LIMIT",
        "memory": "YOUR_MEMORY_LIMIT"
      }
    } 
  }
],
...

By providing the relevant values for CPU and memory requests and limits, you ensure that the Kubernetes scheduler allocates the correct amount of resources for the pods running your flows. You can find more details on this process in the Prefect documentation regarding Kubernetes deployment and configuring work pools: - Kubernetes Deployment Guide - Work Pool Configuration Guide Please replace "YOUR_CPU_LIMIT", "YOUR_MEMORY_REQUEST", and "YOUR_MEMORY_LIMIT" with the appropriate values you want to use for CPU and memory requests and limits. Note that this configuration will apply to all flows running with the specified work pool.

@Marvin can you provide a specific documentation link where it details adding the memory and cpu requests

Certainly! You can find detailed documentation on adding memory and CPU requests to the base job template in the Work Pool's advanced configuration in the Prefect documentation here: - Kubernetes Deployment Guide To reiterate, you'll be editing the JSON in the Advanced tab of the Work Pool configuration to include the

resources

section with the

requests

and

limits

for CPU and memory.

@Marvin Can I add this to my kubernetes manifest instead?

Yes, you can configure memory and CPU requests and limits in the Kubernetes manifest for Prefect agents. The Prefect documentation details how to modify the default base job template to include resource requests and limits. If you prefer to handle this at the Kubernetes level, you can adjust the agent's deployment YAML manifest directly. To set resource requests and limits, add a

resources

block under the container spec in your Kubernetes manifest as follows:

yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prefect-agent
spec:
  ...
  template:
    ...
    spec:
      containers:
      - name: agent
        image: prefecthq/prefect:latest
        ...
        resources:
          requests:
            memory: "64Mi"
            cpu: "250m"
          limits:
            memory: "128Mi"
            cpu: "500m"

Ensure that you replace the memory and CPU request and limit values (

"64Mi"

,

"250m"

,

"128Mi"

,

"500m"

) with the appropriate values according to your requirements. This configuration will be applied to the Prefect agent pod to guarantee that it has the appropriate resources allocated for its operation. Keep in mind that this will only configure the resources for the agent itself, not the individual flow run jobs, which are managed separately. For configuring resources for individual flow run jobs, you should refer to the Kubernetes Deployment Guide where you can modify the base job template through the Advantaged tab in the Work Pool configuration or use a custom job template.

@Marvin I technically don’t have a manifest for agents , only a worker. Since this is the case I think I need to modify the advanced configuration of the EKS work pool i have at the kubernetes level. Is this possible?

Yes, if you have a Kubernetes worker running for Prefect and you don’t have a separate manifest for agents, you will need to modify the advanced configuration of the EKS work pool at the Kubernetes level through the Prefect UI. This is indeed possible. By editing the advanced configuration of the work pool, you can amend the default job template that the Prefect Kubernetes agent uses to launch flow run jobs. This configuration defines how flow run jobs are deployed onto the Kubernetes cluster, including their resource requests and limits. Here's the general procedure for modifying the work pool's advanced configuration: 1. Log into the Prefect UI. 2. Navigate to the Work Pools section from the sidebar. 3. Select the EKS work pool that you have set up for your worker. 4. Click on the Advanced tab within the work pool configuration. 5. Modify the JSON representation of the base job template. You will need to edit the job configuration to include the

resources

block with the appropriate

requests

and

limits

sub-blocks as explained in the previous message. After saving your changes, the modified settings will be applied to all new flow run jobs that are dispatched to that work pool. Existing running jobs will not be affected by the change. Please refer to the Kubernetes Deployment Guide for detailed instructions and examples on how to configure the Work Pool’s advanced settings, including the resource requests and limits.

@Marvin is there a way to add this to my kubernetes manifest instead of manually modifying at the UI level? The helm chart automatically created my work pool when I specified the name

The documentation indicates that modifying the default base job template to set resource requests and limits is done through the Prefect UI by editing the JSON representation of the base job template in the advanced configuration of a work pool. This method allows you to set values like

cpu_request

and apply them when deployments run with the relevant work pool. However, your use case involves setting up resource requests and limits using a Kubernetes manifest file during the initial setup of the Prefect work pool using Helm. The documentation provided does not include explicit guidance on this specific scenario, suggesting that the standard approach is through the UI. While Helm is mentioned in the context of deploying a Prefect worker, which sets up the Kubernetes infrastructure to run flows, the details you provided imply that the work pool was automatically created with a default job template without specifying resource requests and limits in the manifest. Therefore, if you need to automate the setting of resource requests and limits for your work pools when deploying via Helm, you might consider these options: 1. Explore the Helm chart values: Check if the Helm chart for the Prefect agent includes parameters for setting resource requests and limits for the work pool. It's common for Helm charts to support such configurations via values that can be customized in your

values.yaml

file or passed via command line when installing the chart. 2. Post-deployment script: If the Helm chart does not provide a way to automate this configuration, you could write a post-deployment script that interacts with the Prefect API or CLI to update the work pool's job template based on your specifications. 3. Custom resource definitions: Depending on how the Prefect work pool is represented within Kubernetes after deployment, it might be possible to apply changes directly to the Kubernetes resource (likely a ConfigMap or a custom resource). You would need to know the exact structure and fields applicable. To get the most accurate and up-to-date information on whether the Helm chart can be used to set up resource requests and limits for a Prefect work pool, I would recommend checking the Helm chart's documentation or the

values.yaml

file that comes with the Prefect Helm chart. Consulting the Prefect community or Prefect support may also provide additional insights into how to achieve this automation.

@Marvin I think I am wrong, the kubernetes deployment did not automatically create it oops. The prefect.yaml in my code base did. Here is an example:

definitions:
  tags:
  - eks
  work_pool:
    name: eks_worker_pool
    job_variables:
      image: blah_example_image

can I add the cpu/memory requests/limits here?

The documentation excerpts provided do not specifically mention the ability to add CPU and memory requests and limits directly in the

prefect.yaml

file for a Prefect work pool configuration. They do describe the use of images and code storage for managed execution but do not cover the resource specification in the

prefect.yaml

. However, based on common practices in Prefect 2.x, configurations like work pools and associated settings are usually managed either through the UI or via the Prefect API or CLI. The

prefect.yaml

file is generally used for defining deployment specifications, including tags, work pool names, and job variables like the image. The standard way to set CPU and memory requests and limits is by editing the JSON representation of the work pool template in the Prefect UI, as mentioned earlier. There is no clear indication from the provided documentation that you can define resource requests and limits within the

prefect.yaml

directly. To achieve this, you may need to use the Prefect UI, API, or CLI to modify the configured work pool settings after the initial creation via the

prefect.yaml

. If you require further clarification or if there have been updates or features added to Prefect after the knowledge cutoff, I recommend referring to the official Prefect documentation or reaching out to the Prefect community or support for the most accurate and up-to-date information.

I’ll leave it to be a manual add for now, I’ve confirmed it works at the pod level (flows running on pods in eks)

There is no clear indication from the provided documentation that you can define resource requests and limits within the

prefect.yaml

directly. To achieve this, you may need to use the Prefect UI, API, or CLI to modify the configured work pool settings after the initial creation via the

prefect.yaml

.

🙅