Ryan Peden02/09/2023, 10:06 PM
. Creating a new work pool can be done via the Work Pools page in the UI or via the CLI. To create a new work pool named “my-pool” via the CLI:
Each work pool starts out with a default queue. New queues can be added to a work pool via the UI or the CLI. To create a new work queue in a work pool via the CLI:
prefect work-pool create "my-pool"
Deployments can now be assigned to a work queue in a specific work pool. Use the
prefect work-queue create "high-priority" --pool "my-pool"
flag to specify the work pool and the
flag to specify the work queue when building a deployment.
Once a deployment has been created and is scheduling flow runs on a work queue, you can start an agent to pick up those flow runs by starting an agent with the
prefect deployment build \ --pool my-pool \ --queue high-priority \ --name high-priority \ high_priority_flow.py:high_priority_flow
Starting an agent with the
prefect agent start --pool my-pool
command allows the agent to pick up flow runs for the entire pool even as new queues are added to the pool. If you want to start an agent that only picks up flow runs for a specific queue, you can use the
A few other important parts of this release: • The ability to create a flow run from the UI with parameters from a previous run — #8405 • A generic
prefect agent start --pool my-pool --queue high-priority
block — #8401 • Override customizations functionality for CLI deployments v— #8349 • It's now possible to reset concurrency limits in CLI to purge existing runs from taking concurrency slots — #8408 To learn more about work pools, and everything else we've added or updated, check out the docs or see the relevant pull requests in the release notes.