title: What are the latencies of GPU?
body: <p>I can find the latencies in terms of either ns or CPU cylces between CPU core and its cache, main memory, etc.</p>
  
  <p>But it seems so hard to find similiar information about modern GPU.</p>
  
  <p>Does anyone know about the latencies of GPU,  esepecially the latencies between modern nvidia GPU (GF110 or later) and their memory, thanks.</p>
  
  <p>GPU memory do have a much larger bandwidth, but what about their latencies?</p>
  
  <p>I heard that the latencies for GPU are just as high as these for CPU, so basically make the larger bandwidth largely pointless for many general purpose computing tasks, I just need to confirm this.</p>