https://prefect.io
Join Slack
Is there a way to configure my Docker Worker Pool so it creates new containers with the proper flags...
# prefect-getting-started
s
Is there a way to configure my Docker Worker Pool so it creates new containers with the proper flags to mount GPUs devices (something similar to 
docker run --gpus all myimage
) ?
k
the docker worker uses the docker python sdk and starts a container using 
client.containers.create
, which supports these kwargs. However, the kwargs available by default in the docker work pool are a subset of those
specifically these as retrieved from the docker work pool config:
Copy code
"job_configuration": {
    "command": "{{ command }}",
    "env": "{{ env }}",
    "labels": "{{ labels }}",
    "name": "{{ name }}",
    "image": "{{ image }}",
    "image_pull_policy": "{{ image_pull_policy }}",
    "networks": "{{ networks }}",
    "network_mode": "{{ network_mode }}",
    "auto_remove": "{{ auto_remove }}",
    "volumes": "{{ volumes }}",
    "stream_output": "{{ stream_output }}",
    "mem_limit": "{{ mem_limit }}",
    "memswap_limit": "{{ memswap_limit }}",
    "privileged": "{{ privileged }}"
s
@Kevin Grismore Thank you for the reply. From what I understand it's not doable without code changes within Prefect right now ?
7 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by