content_image/996761.jpg
MPS is recommended for jobs that do not generate enough work for GPUs. If you have multiple runs concurrently, MPS helps to balance the workload of the GPU and increases its utilization. 

Also, MPI programs that have different MPI-tasks but share a GPU can see performance improvements by using MPS to control access to the GPU between tasks. 

Note:  the Volta MPS server supports 48 client CUDA contexts per-device. 

A recommendation is to use MPS with the EXCLUSIVE_PROCESS  mode to be sure that only the MPS server runs on GPUs. 

The CUDA program works by using MPS if the MPS control daemon runs in the system. The program at startup attempts to connect to the MPS control daemon, which creates an MPS server or reuses a server if it has the same user ID as the user who started the job. Therefore, each user has its own MPS server. 

The MPS server creates the shared GPU context for the different jobs in the system, manages its clients, and calls to GPU on behalf of them. An overview of this process is shown in Figure 9-24. 

<img src='content_image/996761.jpg'>

To run MPS, you need to run the following commands as root: 

1. (This step is optional.) Set the CUDA_VISIBLE_DEVICES  environment variable to inform the system which GPUs will be used. To use all GPUs, start from step 2 on page 294 and issue the following command: 

export CUDA_VISIBLE_DEVICES=0,1 #Use only first and second GPUs 

