Hi folks. Can we run Docker using GPUs on Prefect? Any help would be appreciated.

Hey, I don't think you need to post this question in every channel.. You can delete the other ones.

Any answers to the question? If any help, I appriciate it.

It doesn't look like the Docker worker currently supports setting docker-py's

device_requests

parameter, which would let you decide whether or not to use GPU(s) on a per-flow-deployment basis. However, if you're willing to run a worker dedicated to running GPU-enabled flows, you can set an environment variable before you start the worker:

NVIDIA_VISIBLE_DEVICES=all

That should give you the same functionality you get when you use the Docker CLI with the

--gpus

flag.

Tried that and does not work.

hi, I've also been looking for the same feature, but realized Prefect doesn't officially support it. Prefect offer a job template for Docker runners, which relies on the Docker Python SDK for backend operations, the job template lacks an option to pass through device settings. It seems that the only workaround might be to edit the '/etc/docker/daemon.json' file to set the Nvidia Docker runtime as the default. This appears to be the sole method for utilizing GPUs with Docker deployments in Prefect.

It looks like someone opened a PR that should address this, but abandoned it before finishing it: https://github.com/PrefectHQ/prefect-docker/pull/85 It looks like it'd be easy to finish it. I'd like to use this functionality too, so I'll try to set aside some time this week to make the required changes, add some tests, and make sure GPU containers work as expected.