@Marvin how can i limit the amount of memory and cpu that is used by deployed flows on kubernetes? The helm chart for worker does contain resources section - does it refer only to resources allocated to worker process, or to all jobs spawned by it?

In Prefect 3.x, when deploying flows on Kubernetes using the Helm chart, the

resources

section in the Helm chart typically refers to the resources allocated to the worker process itself. However, you can also specify resource requests and limits for the jobs spawned by the worker, which run the actual flow tasks. To limit the memory and CPU for the jobs spawned by the worker, you can use Kubernetes job specifications. For Prefect, this can be achieved using

infra_overrides

in your deployment configuration. Here's an example of how you might specify these overrides:

python
infra_overrides = {
    "customizations": [
        {
            "op": "add",
            "path": "/spec/template/spec/containers/0/resources",
            "value": {
                "requests": {
                    "cpu": "2000m",
                    "memory": "4Gi"
                },
                "limits": {
                    "cpu": "4000m",
                    "memory": "8Gi"
                }
            }
        },
    ]
}

# Load an already created Kubernetes Block
k8sjob = k8s_job.load("your_k8s_block")

deployment = Deployment.build_from_flow(
    flow=my_flow,
    name="example-flow",
    infrastructure=k8sjob,
    infra_overrides=infra_overrides,
)

deployment.apply()

This configuration allows you to define the CPU and memory requests and limits for the Kubernetes jobs that your Prefect worker will spawn. You should adjust the values according to your specific requirements. For further details, you might find the following resources useful: - Prefect Kubernetes Deployment Example - GitHub Issue on Infra Overrides If you need more detailed guidance, consider checking the Prefect documentation or relevant GitHub discussions for updates and community insights.