<ul>
<li><strong><code>LATENCY     -</code></strong> an amount of <strong>time</strong> to get the response <code>[us]</code></li>
<li><strong><code>BANDWIDTH   -</code></strong> an amount of data-flow volume <strong>per unit of time</strong> <code>[GB</code><strong>/<code>s</code></strong>]`</li>
</ul>

<h2>Marketing papers are fabulous in mystifications with <code>LATENCY</code> <em>figures</em></h2>

<p>A term latency could be confused, if not taking carefully this <strong>whole context of transaction life-cycle</strong>: participating line-segments { amplification | retiming | switching | MUX/MAP-ing | routing | EnDec-processing (not speaking about cryptography ) | statistical-(de)compressing }, data-flow duration and framing / line-code-protective add-ons / ( opt. procotol, if present, encapsulation and re-framing ) additional surplus overheads, <strong>that continually increase <kbd>latency</kbd> but <em>also</em> increase data-<code>VOLUME</code></strong>.</p>

<p><a href="https://i.stack.imgur.com/hY9hs.png" rel="noreferrer"><img src="https://i.stack.imgur.com/hY9hs.png" alt="enter image description here"></a>
Just as an example, <strong>take any GPU-engine marketing.</strong> The huge numbers that are presented about GigaBytes of <strong><code>DDR5</code></strong> and <strong><code>GHz</code></strong> timing thereof silently are communicated in bold, what they omit to tell you is, that with all that zillions of things, each of your <code>SIMT</code> many-cores, yes, all the cores, have to pay a cruel <kbd><strong>latency</strong></kbd>-<strong>penalty</strong> and <strong>wait</strong> for more than <strong><code>+400-800</code></strong> <code>[GPU-clk]</code>s just to receive the first byte from GPU-over-hyped-GigaHertz-Fast-DDRx-ECC-protected bank of memory.<br></p>

<p><strong>Yes, your Super-Engine's <code>GFLOPs/TFLOPs</code> <em>have</em> to wait!</strong> ... because of (hidden) <strong><code>LATENCY</code></strong></p>

<p>And you wait with all the full parallel-<strong>circus</strong> ... because of <strong><code>LATENCY</code></strong></p>

<p>( ... and any marketing bell or whistle cannot help, believe or not ( forget about cache promises too, these do not know, what the hell there would be in the far / late / distant memory cell, so cannot feed you a single bit copy of such latency-"far" enigma from their shallow local-pockets ) )</p>

<hr>

<h2><code>LATENCY</code> ( and taxes ) cannot be avoided</h2>

<p>Highly professional <strong><code>HPC</code></strong>-designs only <strong>help to pay less</strong> penalty, while <strong>still cannot avoid <code>LATENCY</code></strong> (as taxes) <strong>penalty</strong> beyond some smart re-arrangements principles.</p>

<pre><code> CUDA Device:0_ has &lt;_compute capability_&gt; == 2.0.
 CUDA Device:0_ has [    Tesla M2050] .name
 CUDA Device:0_ has [             14] .multiProcessorCount         [ Number of multiprocessors on device ]
 CUDA Device:0_ has [     2817982464] .totalGlobalMem              [ __global__   memory available on device in Bytes [B] ]
 CUDA Device:0_ has [          65536] .totalConstMem               [ __constant__ memory available on device in Bytes [B] ]
 CUDA Device:0_ has [        1147000] .clockRate                   [ GPU_CLK frequency in kilohertz [kHz] ]
 CUDA Device:0_ has [             32] .warpSize                    [ GPU WARP size in threads ]
 CUDA Device:0_ has [        1546000] .memoryClockRate             [ GPU_DDR Peak memory clock frequency in kilohertz [kHz] ]
 CUDA Device:0_ has [            384] .memoryBusWidth              [ GPU_DDR Global memory bus width in bits [b] ]
 CUDA Device:0_ has [           1024] .maxThreadsPerBlock          [ MAX Threads per Block ]
 CUDA Device:0_ has [          32768] .regsPerBlock                [ MAX number of 32-bit Registers available per Block ]
 CUDA Device:0_ has [           1536] .maxThreadsPerMultiProcessor [ MAX resident Threads per multiprocessor ]
 CUDA Device:0_ has [         786432] .l2CacheSize
 CUDA Device:0_ has [          49152] .sharedMemPerBlock           [ __shared__   memory available per Block in Bytes [B] ]
 CUDA Device:0_ has [              2] .asyncEngineCount            [ a number of asynchronous engines ]
</code></pre>

<h2>Yes, telephone!<br>Why not?<br><br> A cool point to remind<br>a 8kHz-8bit-sampling on a 64k circuit switching<br>used inside an E1/T1 TELCO hierarchy</h2>

<p>A <strong><code>POTS</code></strong> telephone service used to be based on a <strong>synchronous</strong> <strong>fix-<code>latency</code></strong> switching ( late 70-ies have merged global, otherwise in-synchronise-able Plesiochronous Digital Hierarchy networks between Japanese-<code>PDH</code>-standard, Continental-<code>PDH</code>-<strong><code>E3</code></strong> inter-carrier standards and US-<code>PDH</code>-<strong><code>T3</code></strong> carrier services, which finally avoided many headaches with international carrier service jitter / slippage / (re)-synchronisation storms and drop-outs )</p>

<p><strong><code>SDH</code></strong>/<code>SONET-STM1 / 4 / 16</code>, carried on 155 / 622 / 2488 <code>[Mb/s]</code> <strong><code>BANDWIDTH</code></strong> SyncMUX-circuits.</p>

<p>The cool idea on <code>SDH</code> was the globally enforced fix structure of time-aligned framing, which was both deterministic and stable.</p>

<p>This allowed to simply memory-map (cross-connect switch) lower-order container-datastream components to be copied from incoming STMx onto outgoing STMx/PDHy payloads on the SDH-cross-connects ( remember, that was as deep as in late 70-ies so the CPU performance and DRAMs were decades before handling <code>GHz</code> and sole <code>ns</code> ). Such a box-inside-a-box-inside-a-box payload mapping provided both low-switching overheads on the hardware and provided also some means for re-alignment in time-domain ( there were some bit-gaps between the box-in-box boundaries, so as to provide some elasticity, well under a standard given maximum skew in time )</p>

<p>While it may be hard to explain the beauty of this concept in a few words, AT&amp;T and other major global operators enjoyed a lot the SDH-synchronicity and the beauty of the globally-synchronous SDH network and local side Add-Drop-MUX mappings.</p>

<hr>

<p>Having said this,<br>
<strong>latency controlled design</strong><br>
takes care of:<br>
- <code>ACCESS-LATENCY      :</code>how long time does it take to <strong>arrive</strong> for the first ever bit <code>:     [s]</code><br>
- <code>TRANSPORT-BANDWIDTH :</code>how many bits it can transfer/<strong>deliver</strong> each next unit of time<code>:  [b/s]</code><br>
- <code>VOLUME OF DATA      :</code>how many bits of data are there in total to transport <code>:        [b]</code><br>
- <code>TRANSPORT DURATION  :</code>how many units of time does it take<br>
- <code>___________________ :</code>to move/<strong>deliver</strong> whole <code>VOLUME OF DATA</code>to who has asked<code>: [s]</code></p>

<hr>

<h2>Epilogue:</h2>

<blockquote>
  <p>A very nice illustration of the principal independence of a <strong>THROUGHPUT</strong> ( <strong>BANDWIDTH <code>[GB/s]</code></strong> ) on <strong>LATENCY <code>[ns]</code></strong> is in <strong>Fig.4</strong> in a lovely <a href="https://arxiv.org/pdf/1512.05578v1.pdf" rel="noreferrer">ArXiv paper on <strong>Improving Latency</strong></a> from Ericsson, testing how manycore RISC-procesor Epiphany-64 architecture from Adapteva may help in driving LATENCY down in signal processing.<br><br>Understanding the <strong>Fig.4</strong>, extended in core-dimension,<br>can also show the possible scenarios<br><br>- how to increase of <strong>BANDWIDTH <code>[GB/s]</code></strong> <br>by more-core(s) involved into accelerated / TDMux-ed <code>[Stage-C]</code>-processing ( interleaved in time )<br>and also<br>- that <strong>LATENCY <code>[ns]</code></strong><br>can never be shorter than a sum of principal <strong><code>SEQ</code>-process-durations</strong> <code>== [Stage-A]</code>+<code>[Stage-B]</code>+<code>[Stage-C]</code>, independently of the number of available ( single/many )-cores the architecture permits to use.<br><strong>Great thanks to Andreas Olofsson &amp; the Ericsson guys. KEEP WALKING, BRAVE MEN!</strong></p>
</blockquote>
What is low latency access of data?
<p>What do you mean by low latency access of data?<br></p>

<p>I am actually confused about the definition of the term <kbd>"<strong>LATENCY</strong>"</kbd>.<br></p>

<p>Can anyone please elaborate the term "Latency".</p>