# Use GPUs in Docker

To use the GPUs within a Docker setup, the system needs to be prepared before.

## For more information visit the original guides

- [docker-compose](<https://docs.docker.com/compose/how-tos/gpu-support/>)
- [Docker](<https://docs.docker.com/engine/containers/resource_constraints/#gpu>)

## Install the Nvidia container toolkit

### 1\. Configure production repository

```text
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
```

### 2\. Update packages

```text
sudo apt-get update
```

### 3\. Install the NVIDIA Toolkit packages

```text
sudo apt-get install -y nvidia-container-toolkit
```

### 4\. Configure NVIDIA Toolkit for containerd

```text
sudo nvidia-ctk runtime configure --runtime=containerd --set-as-default
```

### 5\. Restart containerd

```text
sudo systemctl restart containerd
```

## Referencing the GPUs for Docker deployments

There are multiple options for referencing the available GPUs. As our flavors currently represent only one GPU per node, the selections can be left to `all`. However you can also target the particular GPU via its ID that you can find with `nvidia-smi`.

```text
# docker-compose
...
    deploy:
      resources:
        reservations:
          devices:
              - driver: nvidia
                count: all
                capabilities: [compute, utility]

# Docker
docker run -it --rm --gpus all ubuntu nvidia-smi
```
