<p>L2 utilization and hit rate are orthogonal concepts.</p>

<p>L2 utilization % measures how many operations (reads/writes/atomics) the L2 cache performed, compared to its peak performance.  You can alternatively think of this as a proxy for "how much L2 bandwidth did I use" given there is a fixed bandwidth between L1 and L2 on a given GPU.  Note, this metric is not measuring the % of L2 capacity used.  (to simplify, in the diagram below, think of it as measuring the throughput of arrows next to the red dots)</p>

<p>L2 cache hit rate measures when an L1 miss occurs, how often was it found in L2.  (in the diagram, think of L2 cache tags at the green X)</p>

<p><em>Original diagram from <a href="https://arxiv.org/pdf/1903.07486.pdf" rel="nofollow noreferrer">Dissecting the NVidia Turing T4 GPU via Microbenchmarking</a></em>
<a href="https://i.stack.imgur.com/k3aG1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/k3aG1.png" alt="enter image description here"></a></p>

<p>Hypothetically:</p>

<ul>
<li>Some CUDA kernel could read a single L1 cacheline (128B) per SM once, incurring a single L2 read that always hits.  The L2 utilization would be ~0%, with L2 hit-rate of 100%.</li>
<li>A different CUDA kernel could achieve ~100% L2 utilization and 100% L2 hit-rate, by performing tons of loads that either miss in L1, or were <a href="https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#data-movement-and-conversion-instructions" rel="nofollow noreferrer">"cache global" loads</a>, where the set of accessed addresses fit within the size of the L2.</li>
<li>Yet another CUDA kernel could achieve high L2 utilization and low L2 hit-rate, by performing tons of loads that either miss in L1, or were <a href="https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#data-movement-and-conversion-instructions" rel="nofollow noreferrer">"cache global" loads</a> that are scattered throughout a Gigabyte sized buffer (i.e. that don't all fit simultaneously in L2).</li>
</ul>

<p>See also</p>

<ul>
<li>The <a href="https://docs.nvidia.com/cuda/profiler-users-guide/index.html#metrics-reference-7x" rel="nofollow noreferrer">tables of metrics in the CUDA Toolkit Profiler documentation</a>.</li>
<li><a href="https://arxiv.org/pdf/1903.07486.pdf" rel="nofollow noreferrer">Dissecting the NVidia Turing T4 GPU via Microbenchmarking</a></li>
<li><a href="https://arxiv.org/pdf/1804.06826.pdf" rel="nofollow noreferrer">Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking</a></li>
</ul>
why L2 cache utilization is low even though L2 cache hit rate is about 93% in cuda?
<p>I'm doing expriments by using cuda.</p>

<p>I thought that if L2 cache hit ratio is high, performance will increase.</p>

<p>However, from nvprof, L2 cache utilization is low even though L2 cache hit rate is about 93%.</p>

<p>Why this happens? Are there examples that make it happen?</p>