@Marvin I followed the documentation at https://docs.prefect.io/contribute/develop-a-new-worker-type, but cannot get the worker to start. Digging into the source it looks like the worker type must be in the canonical

aggregate-worker-metadata.json

collections file. Is there a way to create a work pool and run a custom worker type without changing that file? This request seems related to this open issue, https://github.com/PrefectHQ/prefect/issues/10964, from last year but the docs made it seem like it was already possible.

It seems that there might be a bit of confusion regarding how to use a custom worker type in Prefect without modifying the

aggregate-worker-metadata.json

file. From the Prefect documentation and your description, this isn't directly addressed, but we can try to infer how to proceed. ### Current State 1. Worker Types: Both Prefect 2.x and 3.x documentations mention that if a worker type that meets your needs doesn't exist, you can develop a new worker type. The documentation seems to imply that creating custom workers is indeed possible but doesn't explicitly mention how to avoid using the

aggregate-worker-metadata.json

specifically. Prefect Documentation links: - Prefect 2.x: Developing a New Worker Type - Prefect 3.x: Infrastructure Concepts - Workers 2. Automatic Work Queue Creation: Prefect's documentation mentions that when starting a worker, work pools and queues are created automatically if they do not exist. This suggests flexibility in how workers are instantiated and might mean custom setup can be inferred or configured through available CLI options. ### Speculative Solution Though it isn't directly documented, you could explore these general steps: - Create your custom worker class inheriting from an existing worker base class in Prefect. - Start the worker with a custom work pool and optionally, a specific work queue through the CLI or programmatically. - The worker CLI seems to have the capability to infer or create the necessary structures for execution. You might want to delve into Prefect's code related to worker instantiation if you're comfortable with that approach or reach out to the community through forums or GitHub issues like you've mentioned. Lastly, keep an eye on the linked GitHub issue for any updates or similar problems that might have been posted since it could affect or resolve your situation. If you're hitting obstacles or errors, consider sharing them with the Prefect community for additional guidance.

hi @Cary - you shouldn't your worker to be in that file can you share the error you got?

can you try

prefect worker start --type gce-vm -p test

?

prefect worker start --type gce-vm -p test
Could not find a package for worker type 'gce-vm'.
Unable to start worker. Please ensure you have the necessary 
dependencies installed to run your desired worker type.

This error doesn't surprise me. While I followed the docs here, to install my library with the custom worker type I don't see how prefect could know which of the available modules would have the correct type...

Ooooo. Thank you!

that's correct

Gotcha. Thank you both for the info!