title: How to reduce CUDA context size (Multi-Process Service)
body: <p>I followed Robert Crovella's <a href="https://stackoverflow.com/a/34711344/1185254">example</a> on how to use Nvidia's <a href="https://docs.nvidia.com/deploy/mps/" rel="nofollow noreferrer">Multi-Process Service</a>. According to <a href="https://docs.nvidia.com/deploy/mps/#topic_3_1_2" rel="nofollow noreferrer">docs</a>:</p>
  
  <blockquote>
    <p>2.1.2. <strong>Reduced on-GPU context storage</strong></p>
    
    <p>Without MPS each CUDA processes using a GPU allocates separate storage
    and scheduling resources on the GPU. In contrast, the MPS server
    allocates one copy of GPU storage and scheduling resources shared by
    all its clients.</p>
  </blockquote>
  
  <p>which I understood as the reduction of each of the processes' context sizes, which is possible because they are shared. This would increase free GPU memory and thus enable running more processes in parallel.</p>
  
  <p>Now, back to the example. Without MPS:</p>
  
  <p><a href="https://i.stack.imgur.com/zXCQm.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/zXCQm.png" alt="MPS disabled"></a></p>
  
  <p>And with MPS:</p>
  
  <p><a href="https://i.stack.imgur.com/u1PKg.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/u1PKg.png" alt="MPS enabled"></a></p>
  
  <p>Unfortunately each process still takes virtually the same (~300MB) amount of memory. Isn't this in contradiction to the docs? Is there a way to decrease per process memory consumption?</p>