Anyone using multiple workers in single cluster?
I...
# prefect-kubernetesk
KG
06/04/2024, 2:33 PMAnyone using multiple workers in single cluster?
I inherited a cloud deployment with 5 agents. After upgrading to Worker model (5 workers), I am seeing each of them attempt to submit a flow, with a Job error. Does this smell like unnecessary duplicate infra?
k
Kevin Grismore
06/04/2024, 2:34 PMI'm not sure why you'd need more than one worker since its job is just to talk to the k8s api
upvote 1
k
KG
06/04/2024, 2:36 PMMy thoughts as well.
What if multiple jobs get kicked off at the same time? Do you know if the worker would sequentially kick off those k8s jobs and have delay
KG
06/04/2024, 2:36 PMprobably insignificant delay if any
k
Kevin Grismore
06/04/2024, 2:37 PMit creates new threads for job submission, so while code execution is technically still sequential, it's mostly i/o waiting so the delay between anything that happens on the worker is just the time it takes to move between making requests
Kevin Grismore
06/04/2024, 2:37 PMin other words it's pretty fast
Kevin Grismore
06/04/2024, 2:39 PMit's fine to have multiple workers in a single work pool if you wanted to separate deployments across multiple queues and each worker is responsible for its own set of queues. but otherwise I wouldn't have more than one
k
KG
06/04/2024, 3:35 PMgratitude thank you
KG
06/04/2024, 5:32 PMReduced to 1 and seeing some random improvements, less Job scheduling errors
KG
06/04/2024, 5:33 PMMaybe worth including in docs "Upgrading Agents to Workers" 🤷
15 Views