Hi guys. I managed to successfully deploy prefect ...
# prefect-kubernetest
Tom
03/15/2024, 11:57 AMHi guys. I managed to successfully deploy prefect server and work pool to Kubernetes but I'm facing an issue which is a show stopper. All our clusters use dedicated node pools for different workloads just to protect them from each other. I configured nodeSelector and tolerations at the worker level and it works fine, but I cannot find any docs on configuring the same for actual jobs - which is crucial to properly manage huge workloads. Am I missing something?
k
Kevin Grismore
03/15/2024, 1:15 PMhi tom, I've got you covered! I'll be online in a little bit to walk you through it
t
Tom
03/15/2024, 1:15 PMGreat, waiting!
k
Kevin Grismore
03/15/2024, 2:28 PMokay, I'm here!
Kevin Grismore
03/15/2024, 2:28 PMthe place to start is on the work pools page in the UI. for the kubernetes work pool whose jobs you want to customize, click the three dots -> edit
Kevin Grismore
03/15/2024, 2:29 PMon the edit page, you'll see there's an advanced tab. click on that and take a look through it. you'll see a list of job variables and their types, as well as a templated k8s job manifest where those variables slot in
Kevin Grismore
03/15/2024, 2:34 PMyou can edit this template, adding new variables and modifying the job manifest to place those variables where they'll need to be. once you save, the new variables you added will be available as fields on the defaults tab, and can now be overridden per deployment as
job_variablest
Tom
03/15/2024, 3:00 PMI don't see any variables for setting up nodeSelector and tolerations in base job template. Can I just add them there?
k
Kevin Grismore
03/15/2024, 3:01 PMyep, you can add anything you want, just be sure it's valid as far as the k8s job spec is concerned
t
Tom
03/15/2024, 3:10 PMOk, thank you, will try that in a minute
Tom
03/15/2024, 4:36 PM@Kevin Grismore thank you very much. That method worked like a charm!
🙌 1
k
Kevin Grismore
03/15/2024, 4:36 PMwoohoo!
t
Tom
03/15/2024, 4:36 PMok, now scaling to millions of runs
😅 1
6 Views