as the compute capability of the card ([@wilt2013cuda] for more details). While not a requirement for good performance [@volkov2010better], it is generally a good idea to aim for high occupancy.

Memory on the GPU is organized into a hierarchy of memory spaces as shown in Figure \[fig:memory\_hierarchy\]. At the bottom, we have global memory which is accessible by all threads and is the most plentiful but the slowest memory. The next space of interest is the shared memory which is accessible only by threads within the same block and is configurable with the L1 cache to be at most 48KB per thread block on current generation GPUs. Shared memory is very fast and acts as a programmer controllable cache. Finally, we have the registers which are local to the threads. Registers are the fastest of all memory, but the total number of registers usable by a thread without performance implications is limited. If a kernel needs more registers than the limit, then registers are spilled to “local" memory, which is in the slow but cached global memory. Making good use of the faster memories and avoiding excessive accesses to the slower ones is key to good performance on the GPU. 