https://prefect.io logo
Title
d

datamongus

04/11/2023, 7:45 PM
In a prefect deployment with an infrastructure type of kubernetes-job, is it possible to include a resource request limit.
k

Kevin Grismore

04/11/2023, 7:47 PM
it is! You can add it under Customizations like this:
[  
  {
    "op": "add",
    "path": "/spec/template/spec/containers/0/resources",
    "value": {
      "limits": {
        "memory": "4000Mi"
      }
    }
  },
  <other customizations>
]
:upvote: 1
d

datamongus

04/11/2023, 7:49 PM
Awesome, thank you. Is it also possible to do this from the deployment yaml file, lets say to always ensure a job requests the amount it needs rather than overwriting the gke auto-pilot defaults everytime.
###
### A complete description of a Prefect Deployment for flow 'Cat Facts'
###
name: catfact
description: null
version: c0fc95308d8137c50d2da51af138aa23
# The work queue that will handle this deployment's runs
work_queue_name: test
work_pool_name: null
tags: []
parameters: {}
schedule: null
infra_overrides: {}
infrastructure:
  type: process
  env: {}
  labels: {}
  name: null
  command:
  - python
  - -m
  - prefect.engine
  stream_output: true
###
### DO NOT EDIT BELOW THIS LINE
###
flow_name: Cat Facts
manifest_path: null
storage: null
path: /Users/terry/test/testflows/catfact
entrypoint: catfact.py:catfacts_flow
parameter_openapi_schema:
  title: Parameters
  type: object
  properties:
    url:
      title: url
  required:
  - url
  definitions: null
c

Chris White

04/11/2023, 7:50 PM
Also the new Kubernetes Worker allows you to expose custom schemas for your deployments to consume; I believe the default template exposes a resource limit. Relevant documentation: • https://docs.prefect.io/latest/concepts/work-pools/#work-pool-configurationhttps://prefecthq.github.io/prefect-kubernetes/worker/
👀 1
1
🙌 1
:upvote: 1
d

datamongus

04/11/2023, 7:50 PM
I would imagine I could just drop it in there, however I noticed the difference between image_pull_policy vs. the kubernetes manifest
imagePullPolicy
so wasn’t sure if there are differences intentionally in the spelling convention.