Slackbot
06/27/2023, 6:43 AMMarvin
06/27/2023, 6:44 AMbash
$ prefect agent start -p "work-pool-1" -p "work-pool-2"
This will start an agent that polls for work from both "work-pool-1" and "work-pool-2" [^1^].
Now go forth and conquer the world of multitasking, my human friend.
[^1^]: Prefect Work Pools DocumentationArjun Krishnamurthy
06/27/2023, 6:49 AMMarvin
06/27/2023, 6:50 AM-p
flags. Instead, you can use the --match
flag to poll multiple work pools with a common prefix. The agent will poll every work pool with a name that starts with the given string. New work pools matching this prefix will be found by the agent without needing to restart it. Here's an example:
bash
$ prefect agent start --match "foo-"
This example will poll every work pool that starts with "foo-"[^1^].
I hope this clears up the confusion. Remember, with great power comes great responsibility... to manage work pools.
[^1^]: Prefect Work Pools DocumentationArjun Krishnamurthy
06/27/2023, 6:53 AMMarvin
06/27/2023, 7:00 AM--match
flag is indeed for work queues and not work pools. Unfortunately, Prefect agents do not support polling multiple work pools out of the box.
However, you can work around this limitation by running multiple agents, each configured to poll a specific work pool. For example:
bash
$ prefect agent start -p "work-pool-1" &
$ prefect agent start -p "work-pool-2" &
This will start two separate agents, each polling a different work pool. While it's not as elegant as having a single agent polling multiple work pools, it should achieve the desired outcome.
I hope this clears up the confusion. Remember, even robots make mistakes, but at least we don't get tired of fixing them.