title: nvidia-smi Volatile GPU-Utilization explanation?
body: <p>I know that <code>nvidia-smi -l 1</code> will give the GPU usage every one second (similarly to the following). However, I would appreciate an explanation on what <code>Volatile GPU-Util</code> really means. Is that the number of used SMs over total SMs, or the occupancy, or something else?</p>
  
  <pre><code>+-----------------------------------------------------------------------------+
  | NVIDIA-SMI 367.48                 Driver Version: 367.48                    |
  |-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |===============================+======================+======================|
  |   0  Tesla K20c          Off  | 0000:03:00.0     Off |                    0 |
  | 30%   41C    P0    53W / 225W |      0MiB /  4742MiB |     96%      Default |
  +-------------------------------+----------------------+----------------------+
  |   1  Tesla K20c          Off  | 0000:43:00.0     Off |                    0 |
  | 36%   49C    P0    95W / 225W |   4516MiB /  4742MiB |     63%      Default |
  +-------------------------------+----------------------+----------------------+
  
  +-----------------------------------------------------------------------------+
  | Processes:                                                       GPU Memory |
  |  GPU       PID  Type  Process name                               Usage      |
  |=============================================================================|
  |    1      5193    C   python                                        4514MiB |
  +-----------------------------------------------------------------------------+
  </code></pre>