Hi, what is difference between work pool and work queue. All my deployments are on work queue, should I redeploy ?
✅ 1
c
Chris White
04/10/2023, 7:49 PM
Hey Patrick! A work pool is a collection of work queues that are associated with similar infrastructure; as an example, here is one way you could consider organizing your work:
• 1 Process Work Pool for local development and testing
◦ one work queue per developer
• 1 K8s Work Pool for your dev environment
◦ one work queue for ad-hoc experimental runs
◦ one work queue for ETL jobs
◦ one work queue for high priority ETL jobs
• 1 K8s Work Pool for your prod environment
◦ same work queue setup as dev
That's just an example, but think of work pools as being 1-1 with your infrastructure environments, and work queues within each work pool as representing similar types of work that has the same priority and concurrency needs
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.