 <p>Your access pattern for <code>M</code> is perfectly coalesced, so the global memory with the help of the cache hierarchy of compute capability devices > 2.x will give the best performance for that array. </p> <p>You have the same access pattern in K and J, so, the accesses are also coalesced. So, there is no reason for not use the global memory and get the best performance</p> <p>As you commented that <em>memory limits are obviously not a problem</em> I understand that the size of the arrays will be small, so you would expect a good behaviour of the cache hierarchy.</p> <p>If you want to go one step forward, you could copy the <code>K</code> array to 1D texture memory and compare the performance of both approach. </p>