title: What's missing/sub-optimal in this memcpy implementation?
body: <p>I've become interested in writing a <code>memcpy()</code> as an educational exercise. I won't write a whole treatise of what I did and didn't think about, but here's 
  <a href="http://pastebin.com/MwvgKjiw" rel="noreferrer">some guy's implementation</a>:</p>
  
  <pre><code>__forceinline   // Since Size is usually known,
                  // most useless code will be optimized out
                  // if the function is inlined.
  
  void* myMemcpy(char* Dst, const char* Src, size_t Size)
  {
          void* start = Dst;
          for ( ; Size &gt;= sizeof(__m256i); Size -= sizeof(__m256i) )
          {
                  __m256i ymm = _mm256_loadu_si256(((const __m256i* &amp;)Src)++);
                  _mm256_storeu_si256(((__m256i* &amp;)Dst)++, ymm);
          }
  
  #define CPY_1B *((uint8_t * &amp;)Dst)++ = *((const uint8_t * &amp;)Src)++
  #define CPY_2B *((uint16_t* &amp;)Dst)++ = *((const uint16_t* &amp;)Src)++
  #define CPY_4B *((uint32_t* &amp;)Dst)++ = *((const uint32_t* &amp;)Src)++
  #if defined _M_X64 || defined _M_IA64 || defined __amd64
  #define CPY_8B *((uint64_t* &amp;)Dst)++ = *((const uint64_t* &amp;)Src)++
  #else
  #define CPY_8B _mm_storel_epi64((__m128i *)Dst, _mm_loadu_si128((const __m128i *)Src)), ++(const uint64_t* &amp;)Src, ++(uint64_t* &amp;)Dst
  #endif
  #define CPY16B _mm_storeu_si128((__m128i *)Dst, _mm_loadu_si128((const __m128i *)Src)), ++(const __m128i* &amp;)Src, ++(__m128i* &amp;)Dst
  
      switch (Size) {
      case 0x00:                                                      break;
      case 0x01:      CPY_1B;                                         break;
      case 0x02:              CPY_2B;                                 break;
      case 0x03:      CPY_1B; CPY_2B;                                 break;
      case 0x04:                      CPY_4B;                         break;
      case 0x05:      CPY_1B;         CPY_4B;                         break;
      case 0x06:              CPY_2B; CPY_4B;                         break;
      case 0x07:      CPY_1B; CPY_2B; CPY_4B;                         break;
      case 0x08:                              CPY_8B;                 break;
      case 0x09:      CPY_1B;                 CPY_8B;                 break;
      case 0x0A:              CPY_2B;         CPY_8B;                 break;
      case 0x0B:      CPY_1B; CPY_2B;         CPY_8B;                 break;
      case 0x0C:                      CPY_4B; CPY_8B;                 break;
      case 0x0D:      CPY_1B;         CPY_4B; CPY_8B;                 break;
      case 0x0E:              CPY_2B; CPY_4B; CPY_8B;                 break;
      case 0x0F:      CPY_1B; CPY_2B; CPY_4B; CPY_8B;                 break;
      case 0x10:                                      CPY16B;         break;
      case 0x11:      CPY_1B;                         CPY16B;         break;
      case 0x12:              CPY_2B;                 CPY16B;         break;
      case 0x13:      CPY_1B; CPY_2B;                 CPY16B;         break;
      case 0x14:                      CPY_4B;         CPY16B;         break;
      case 0x15:      CPY_1B;         CPY_4B;         CPY16B;         break;
      case 0x16:              CPY_2B; CPY_4B;         CPY16B;         break;
      case 0x17:      CPY_1B; CPY_2B; CPY_4B;         CPY16B;         break;
      case 0x18:                              CPY_8B; CPY16B;         break;
      case 0x19:      CPY_1B;                 CPY_8B; CPY16B;         break;
      case 0x1A:              CPY_2B;         CPY_8B; CPY16B;         break;
      case 0x1B:      CPY_1B; CPY_2B;         CPY_8B; CPY16B;         break;
      case 0x1C:                      CPY_4B; CPY_8B; CPY16B;         break;
      case 0x1D:      CPY_1B;         CPY_4B; CPY_8B; CPY16B;         break;
      case 0x1E:              CPY_2B; CPY_4B; CPY_8B; CPY16B;         break;
      case 0x1F:      CPY_1B; CPY_2B; CPY_4B; CPY_8B; CPY16B;         break;
      }
  #undef CPY_1B
  #undef CPY_2B
  #undef CPY_4B
  #undef CPY_8B
  #undef CPY16B
          return start;
  }
  </code></pre>
  
  <p>The comment translates as "Size is usually known as the compiler can optimize the code inline out most useless".</p>
  
  <p>I would like to improve, if possible, on this implementation - but maybe there isn't much to improve. I see it uses SSE/AVX for the larger chunks of memory, then instead of a loop over the last &lt; 32 bytes does the equivalent of manual unrolling, with some tweaking. So, here are my questions:</p>
  
  <ul>
  <li>Why unroll the loop for the last several bytes, but not partially unroll the first (and now single) loop?</li>
  <li>What about alignment issues? Aren't they important? Should I handle the first several bytes up to some alignment quantum differently, then perform the 256-bit ops on aligned sequences of bytes? And if so, how do I determine the appropriate alignment quantum?</li>
  <li>What's the most important missing feature in this implementation (if any)?</li>
  </ul>
  
  <hr>
  
  <h2>Features/Principles mentioned in the answers so far</h2>
  
  <ul>
  <li>You should <code>__restrict__</code> your parameters. (@chux)</li>
  <li>The memory bandwidth is a limiting factor; measure your implementation against it.(@Zboson)</li>
  <li>For small arrays, you can expect to approach the memory bandwidth; for larger arrays - not as much. (@Zboson)</li>
  <li>Multiple threads (may be | are) necessary to saturate the memory bandwidth. (@Zboson)</li>
  <li>It is probably wise to optimize differently for large and small copy sizes. (@Zboson)</li>
  <li>(Alignment <em>is</em> important? Not explicitly addressed!)</li>
  <li>The compiler should be made more explicitly aware of "obvious facts" it can use for optimization (such as the fact that Size &lt; 32 after the first loop). (@chux)</li>
  <li>There are arguments for unrolling your SSE/AVX calls (@BenJackson, <a href="https://stackoverflow.com/a/18320068/1593077">here</a>), and arguments against doing so (@PaulR)</li>
  <li><a href="https://stackoverflow.com/q/37070/1593077">non-temporal transfers</a> (with which you tell the CPU you don't need it to cache the target location) should be useful for copying larger buffers. (@Zboson)</li>
  </ul>