title: x86 mfence and C++ memory barrier
body: <p>I'm checking how the compiler emits instructions for multi-core memory barriers on x86_64. The below code is the one I'm testing using <code>gcc_x86_64_8.3</code>.</p>
  
  <pre><code>std::atomic&lt;bool&gt; flag {false};
  int any_value {0};
  
  void set()
  {
    any_value = 10;
    flag.store(true, std::memory_order_release);
  }
  
  void get()
  {
    while (!flag.load(std::memory_order_acquire));
    assert(any_value == 10);
  }
  
  int main()
  {
    std::thread a {set};
    get();
    a.join();
  }
  </code></pre>
  
  <p>When I use <code>std::memory_order_seq_cst</code>, I can see the <code>MFENCE</code> instruction is used with any optimization <code>-O1, -O2, -O3</code>. This instruction makes sure the store buffers are flushed, therefore updating their data in L1D cache (and using MESI protocol to make sure other threads can see effect). </p>
  
  <p>However when I use <code>std::memory_order_release/acquire</code> with no optimizations <code>MFENCE</code> instruction is also used, but the instruction is omitted using <code>-O1, -O2, -O3</code> optimizations, and not seeing other instructions that flush the buffers.</p>
  
  <p>In the case where <code>MFENCE</code> is not used, what makes sure the store buffer data is committed to cache memory to ensure the memory order semantics?</p>
  
  <p>Below is the assembly code for the get/set functions with <code>-O3</code>, like what we get <a href="https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(j:1,lang:c%2B%2B,source:&#39;%23include+%3Catomic%3E%0A%23include+%3Cthread%3E%0A%23include+%3Ccassert%3E%0A%0Astd::atomic%3Cbool%3E+flag+%7Bfalse%7D%3B%0Aint+any_value+%7B0%7D%3B%0A%0Avoid+set()%0A%7B%0A++any_value+%3D+10%3B%0A++flag.store(true,+std::memory_order_release)%3B%0A%7D%0A%0Avoid+get()%0A%7B%0A++while+(!!flag.load(std::memory_order_acquire))%3B%0A++assert(any_value+%3D%3D+10)%3B%0A%7D%0A%0A/*%0Aint+main()%0A%7B%0A++std::thread+a+%7Bset%7D%3B%0A++get()%3B%0A++a.join()%3B%0A%7D%0A*/&#39;),l:&#39;5&#39;,n:&#39;0&#39;,o:&#39;C%2B%2B+source+%231&#39;,t:&#39;0&#39;)),k:50,l:&#39;4&#39;,n:&#39;0&#39;,o:&#39;&#39;,s:0,t:&#39;0&#39;),(g:!((h:compiler,i:(compiler:g83,filters:(b:&#39;0&#39;,binary:&#39;1&#39;,commentOnly:&#39;0&#39;,demangle:&#39;0&#39;,directives:&#39;0&#39;,execute:&#39;1&#39;,intel:&#39;0&#39;,libraryCode:&#39;1&#39;,trim:&#39;1&#39;),lang:c%2B%2B,libs:!(),options:&#39;-O3+-fno-pie+-no-pie&#39;,source:1),l:&#39;5&#39;,n:&#39;0&#39;,o:&#39;x86-64+gcc+8.3+(Editor+%231,+Compiler+%231)+C%2B%2B&#39;,t:&#39;0&#39;)),k:50,l:&#39;4&#39;,n:&#39;0&#39;,o:&#39;&#39;,s:0,t:&#39;0&#39;)),l:&#39;2&#39;,n:&#39;0&#39;,o:&#39;&#39;,t:&#39;0&#39;)),version:4" rel="nofollow noreferrer">on the Godbolt compiler explorer</a>:</p>
  
  <pre><code>set():
          mov     DWORD PTR any_value[rip], 10
          mov     BYTE PTR flag[rip], 1
          ret
  
  
  .LC0:
          .string "/tmp/compiler-explorer-compiler119218-62-hw8j86.n2ft/example.cpp"
  .LC1:
          .string "any_value == 10"
  
  get():
  .L8:
          movzx   eax, BYTE PTR flag[rip]
          test    al, al
          je      .L8
          cmp     DWORD PTR any_value[rip], 10
          jne     .L15
          ret
  .L15:
          push    rax
          mov     ecx, OFFSET FLAT:get()::__PRETTY_FUNCTION__
          mov     edx, 17
          mov     esi, OFFSET FLAT:.LC0
          mov     edi, OFFSET FLAT:.LC1
          call    __assert_fail
  </code></pre>