Enable Nvidia Multi-Instance GPU (MIG)

Info

This feature is only available on our H100 GPUs.

Nvidia MIG (Multi-Instance GPU) leverages a virtualization layer atop the GPU architecture to deliver isolated, independently managed GPU instances.

Utilizing a combination of PCIe and NVLink interconnects, MIG partitions the GPU’s resources – including memory, compute units, and I/O – into discrete virtual instances, each operating with its own dedicated scheduling and resource allocation policies.

This allows multiple users or applications to concurrently utilize the same physical GPU, maximizing utilization and minimizing contention.

MIG supports diverse deployment models, including virtualized workstations and cloud-based inference, offering a configurable and scalable solution for demanding workloads requiring high GPU performance and isolation.

Find more information in the official guide.

Note: All commands in this guide are executed as the root user. If you are not logged in as root, you will need to prefix each command with sudo.

Prerequisites:

Access to a GPU-optimized flavor
Successful setup of the GPU instance
Access to the GPU instance via SSH

Available GPU MIG profiles

Info

Please refer to the Nvidia MIG concepts guide for an in depth profile explanation.

Profile Name	Fraction of Memory	Fraction of SMs	Hardware Units	L2 Cache Size	Copy Engines	Number of Instances Available
MIG 1g.12gb	⅛	1/7	1 NVDEC / 1 JPEG / 0 OFA	⅛	1	7
MIG 1g.12gb+me	⅛	1/7	1 NVDEC / 1 JPEG / 1 OFA	⅛	1	1 (Single profile can include ME)
MIG 1g.24gb	¼	1/7	1 NVDEC / 1 JPEG / 0 OFA	⅛	1	4
MIG 2g.24gb	2/8	2/7	2 NVDECs / 2 JPEG / 0 OFA	2/8	2	3
MIG 3g.48gb	4/8	3/7	3 NVDECs / 3 JPEG / 0 OFA	4/8	3	2
MIG 4g.48gb	4/8	4/7	4 NVDECs / 4 JPEG / 0 OFA	4/8	4	1
MIG 7g.96gb	Full	7/7	7 NVDECs / 7 JPEG / 1 OFA	Full	Full	8

Check GPU Info

CLI

Shows GPU device status, including MIG instances, memory usage, and active processes

nvidia-smi

Enable MIG

Warning

Enabling MIG is not persistent. It will be disabled after every reboot.

CLI

Enables MIG (Multi-Instance GPU) mode on the specified GPU_ID

nvidia-smi -i <GPU_ID> -mig 1

-i <GPU_ID>: Index of the physical GPU (e.g. 0)
-mig 1: Enables MIG mode on the GPU

List GPU instance profiles

CLI

Lists all supported GPU instance profiles

nvidia-smi mig -lgip

mig: Subcommand to manage Multi-Instance GPU (MIG) features
-lgip: Lists all available GPU Instance Profiles (GIP)

Example output:

+-------------------------------------------------------------------------------+
| GPU instance profiles:                                                        |
| GPU   Name               ID    Instances   Memory     P2P    SM    DEC   ENC  |
|                                Free/Total   GiB              CE    JPEG  OFA  |
|===============================================================================|
|   0  MIG 1g.12gb         19     0/7        10.75      No     16     1     0   |
|                                                               1     1     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 1g.12gb+me      20     0/1        10.75      No     16     1     0   |
|                                                               1     1     1   |
+-------------------------------------------------------------------------------+
|   0  MIG 1g.24gb         15     0/4        21.62      No     26     1     0   |
|                                                               1     1     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 2g.24gb         14     0/3        21.62      No     32     2     0   |
|                                                               2     2     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 3g.47gb          9     0/2        46.38      No     60     3     0   |
|                                                               3     3     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 4g.47gb          5     0/1        46.38      No     64     4     0   |
|                                                               4     4     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 7g.94gb          0     0/1        93.12      No     132    7     0   |
|                                                               8     7     1   |
+-------------------------------------------------------------------------------+

List the possible placement

CLI

Lists valid GPU instance placement combinations for current hardware.

nvidia-smi mig -lgipp

mig: Subcommand to manage Multi-Instance GPU (MIG) features
-lgipp: Lists all available GPU Instance Placement Policies (GIPP)

Example output:

GPU  0 Profile ID 19 Placements: {0,1,2,3,4,5,6}:1
GPU  0 Profile ID 20 Placements: {0,1,2,3,4,5,6}:1
GPU  0 Profile ID 15 Placements: {0,2,4,6}:2
GPU  0 Profile ID 14 Placements: {0,2,4}:2
GPU  0 Profile ID  9 Placements: {0,4}:4
GPU  0 Profile ID  5 Placement : {0}:4
GPU  0 Profile ID  0 Placement : {0}:8

Create GPU instances

Options to reference the instances:

Profile ID (e.g. 9, 14, 5)
Short name of the profile (such as 3g.20gb)
Full profile name of the instance (such as MIG 3g.20gb)

Info

Due to a known issue with the APIs, the profile ID 9 or 3g.20gb must be specified first in order.

CLI

Creates a GPU instance with the specified profile

nvidia-smi mig -cgi 9,3g.20gb -C

mig: Subcommand to manage Multi-Instance GPU (MIG) features
-cgi 9,3g.20gb: Creates a GPU instance (CGI = Create GPU Instance) with profile ID 9, corresponding to the 3g.20gb GPU instance profile.
-C: Commits the changes, applying the configuration to the GPU.

Example output:

Successfully created GPU instance ID  2 on GPU  0 using profile MIG 3g.20gb (ID  9)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  2 using profile MIG 3g.20gb (ID  2)
Successfully created GPU instance ID  1 on GPU  0 using profile MIG 3g.20gb (ID  9)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  1 using profile MIG 3g.20gb (ID  2)

List the created instances:

CLI

Lists created GPU instances

nvidia-smi mig -lgi

mig: Subcommand to manage Multi‑Instance GPU (MIG) features
-lgi: Lists GPU Instances (LGI = List GPU Instances),

Example output:

+----------------------------------------------------+
| GPU instances:                                     |
| GPU   Name          Profile  Instance   Placement  |
|                       ID       ID       Start:Size |
|====================================================|
|   0  MIG 3g.20gb       9        1          4:4     |
+----------------------------------------------------+
|   0  MIG 3g.20gb       9        2          0:4     |
+----------------------------------------------------+

nvidia-smi

+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+----------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |         Memory-Usage |        Vol|         Shared        |
|      ID  ID  Dev |                      | SM     Unc| CE  ENC  DEC  OFA  JPG|
|                  |                      |        ECC|                       |
|==================+======================+===========+=======================|
|  0    1   0   0  |     11MiB / 20224MiB | 42      0 |  3   0    2    0    0 |
+------------------+----------------------+-----------+-----------------------+
|  0    2   0   1  |     11MiB / 20096MiB | 42      0 |  3   0    2    0    0 |
+------------------+----------------------+-----------+-----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Removing GPU instances

Refer to the original guide for more information.

Remove all instances

CLI

Deletes all Compute Instances from all GPU Instances on MIG-enabled GPUs

nvidia-smi mig -dci

mig: Subcommand to manage Multi‑Instance GPU (MIG) features
-dci: Without arguments, defaults to deleting all compute instances created under MIG.

Example output:

Successfully destroyed compute instance ID  0 from GPU  0 GPU instance ID  1
Successfully destroyed compute instance ID  1 from GPU  0 GPU instance ID  1
Successfully destroyed compute instance ID  2 from GPU  0 GPU instance ID  1

Deletes all GPU Instances (GIs) from all physical GPUs on a MIG-enabled system

nvidia-smi mig -dgi

mig: Subcommand to manage Multi‑Instance GPU (MIG) features
-dgi: Deletes GPU Instance(s) previously created—when used without any IDs, it defaults to destroying all GPU Instances across all GPUs.

Example output:

Successfully destroyed GPU instance ID  1 from GPU  0
Successfully destroyed GPU instance ID  2 from GPU  0

Removing specific instances

CLI

nvidia-smi mig -dci -ci 0,1,2 -gi 1

Example output:

Successfully destroyed compute instance ID  0 from GPU  0 GPU instance ID  1
Successfully destroyed compute instance ID  1 from GPU  0 GPU instance ID  1
Successfully destroyed compute instance ID  2 from GPU  0 GPU instance ID  1

Verify the change:

CLI

nvidia-smi

Example output:

+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+----------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |         Memory-Usage |        Vol|         Shared        |
|      ID  ID  Dev |                      | SM     Unc| CE  ENC  DEC  OFA  JPG|
|                  |                      |        ECC|                       |
|==================+======================+===========+=======================|
|  No MIG devices found                                                       |
+-----------------------------------------------------------------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Now remove the GPU instances:

CLI

nvidia-smi mig -dgi

Example output:

Successfully destroyed GPU instance ID  1 from GPU  0
Successfully destroyed GPU instance ID  2 from GPU  0

Working with Docker

To utilize the MIG instances in Docker, the system needs to be prepared before.

Additionally this sections show how to utilize the individual MIG instances per Docker container.

Install the NVIDIA Container Toolkit

1. Configure production repository

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg &&\
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

2. Update packages

apt-get update

3. Install the NVIDIA Toolkit packages

apt-get install -y nvidia-container-toolkit

4. Configure NVIDIA Toolkit for containerd

nvidia-ctk runtime configure --runtime=containerd --set-as-default

5. Restart containerd

systemctl restart containerd

Reference the MIG instance for the container

To utilize a MIG (Multi-Instance GPU) instance, you must specify the MIG device ID so Docker can recognize and use it.

Docker Compose Example

Make sure to append the device configuration within the services section of your docker-compose.yml:

...
    deploy:
      resources:
        reservations:
          devices:
              - driver: nvidia
                device_ids: ["MIG-382940c-9804-5fb5-b9cd-1h0t351a167c0"]
                capabilities: [compute, utility]

Docker CLI Example

When using the Docker CLI directly, specify the MIG device like this:

docker run -it --rm --gpus device=MIG-382940c-9804-5fb5-b9cd-1h0t351a167c0 ubuntu nvidia-smi

For more information visit the original guides: