Hello @Marvin. I have a docker worker running an application inside a docker container that requires the use of nvidia-runtime. How can I launch such a worker and ensure that my docker container will use the nvidia-runtime?

To launch a Docker worker that uses the NVIDIA runtime, you typically need to specify the runtime in your Docker command. While the Prefect documentation does not explicitly outline steps for configuring a worker with the NVIDIA runtime, you can configure your container to use the NVIDIA runtime by adding the

--runtime=nvidia

flag in your

docker run

command. Here is a generic example:

bash
docker run --runtime=nvidia -e PREFECT_API_URL=<http://host.docker.internal:4200/api> my-prefect-image

Ensure that the NVIDIA Container Toolkit is installed and configured on the host system. Additionally, you may have to customize the Docker command to fit the specifics of your Prefect setup and your flow configuration. You can refer to the following Prefect documentation pages for more information on workers and Docker: - Prefect Workers - Prefect Docker Guide For the NVIDIA-specific configuration, please consult the NVIDIA Container Toolkit documentation.

@Marvin, but I am using a docker worker that is started as follows:

prefect worker start -t docker --pool <pool-name>

and the worker itself pulls the image and runs it. Running the docker container isn't in my control, the worker does that!

It appears the Prefect documentation does not directly provide information about configuring a Docker worker to use the NVIDIA runtime when it is started with the

prefect worker start

command. Given this limitation, your scenario might require a more custom setup or there might not be a documented feature in Prefect 2.x that supports this configuration out of the box. However, as a suggested approach, you could look into developing a new custom worker type that integrates NVIDIA runtime support. This process would involve a deeper dive into Prefect's plugin system for developing custom execution environments. Refer to Developing a New Worker Type for guidance on extending Prefect with custom worker types. If you are looking to configure the NVIDIA runtime with Prefect at the Docker level and need information from the community or developers, consider opening an issue or a discussion on the Prefect GitHub repository. For specific guidance on interacting with NVIDIA runtime from Docker, please consult the NVIDIA Container Toolkit documentation or Docker documentation related to runtime configuration. For the time being, without a direct feature or documented method from Prefect docs, you may need to develop a custom solution or seek assistance from the Prefect community or support channels. If other users have encountered this scenario, they might have shared their solutions or workarounds, which you can then apply to your Docker worker setup.

if you're just using normal docker yourself, how do you do this?

ensure that my docker container will use the nvidia-runtime

--runtime=nvidia

to the

docker run

command?

yeah

i dont believe these is a way to do that today natively - sounds like this is what you're looking for?