title: How do "wait" and "notify" work for std::atomic in C++20?
body: <p>C++20 added <code>std::atomic</code>-based thread synchronization methods using wait (<code>std::atomic::wait</code>) and notify (<a href="https://en.cppreference.com/w/cpp/atomic/atomic/notify_one" rel="nofollow noreferrer"><code>std::atomic::notify_one</code></a> or <a href="https://en.cppreference.com/w/cpp/atomic/atomic/notify_all" rel="nofollow noreferrer"><code>std::atomic::notify_all</code></a>) functions for them. The link <a href="https://www.modernescpp.com/index.php/performancecomparison-of-condition-variables-and-atomics-in-c-20" rel="nofollow noreferrer">https://www.modernescpp.com/index.php/performancecomparison-of-condition-variables-and-atomics-in-c-20</a> provides examples of using two variables of type <a href="https://en.cppreference.com/w/cpp/atomic/atomic_flag" rel="nofollow noreferrer"><code>std::atomic_flag</code></a> to transfer control between two threads (&quot;ping-pong game&quot;). Here is the original code for this example (variant &quot;Two Atomic Flags&quot;):</p>
  <pre><code>// pingPongAtomicFlags.cpp
  
  #include &lt;iostream&gt;
  #include &lt;atomic&gt;
  #include &lt;thread&gt;
  
  std::atomic_flag condAtomicFlag1{};
  std::atomic_flag condAtomicFlag2{};
  
  std::atomic&lt;int&gt; counter{};
  constexpr int countlimit = 1'000'000;
  
  void ping() {
      while(counter &lt;= countlimit) {
          condAtomicFlag1.wait(false);               // (1)
          condAtomicFlag1.clear();                   // (2)
  
          ++counter;
          
          condAtomicFlag2.test_and_set();           // (4)
          condAtomicFlag2.notify_one();             // (3)
      }
  }
  
  void pong() {
      while(counter &lt; countlimit) {
          condAtomicFlag2.wait(false);
          condAtomicFlag2.clear();
          
          condAtomicFlag1.test_and_set();
          condAtomicFlag1.notify_one();
      }
  }
  
  int main() {
       auto start = std::chrono::system_clock::now();  
  
      condAtomicFlag1.test_and_set();                    // (5)
      std::thread t1(ping);
      std::thread t2(pong);
  
      t1.join();
      t2.join();
  
      std::chrono::duration&lt;double&gt; dur = std::chrono::system_clock::now() - start;
      std::cout &lt;&lt; &quot;Duration: &quot; &lt;&lt; dur.count() &lt;&lt; &quot; seconds&quot; &lt;&lt; std::endl;
  }
  </code></pre>
  <p>I changed this code a bit:</p>
  <pre><code>#include &lt;iostream&gt;
  #include &lt;atomic&gt;
  #include &lt;thread&gt;
  #include &lt;chrono&gt;
  
  std::atomic_flag condAtomicFlag1{};
  std::atomic_flag condAtomicFlag2{};
  
  std::atomic&lt;int&gt; counter{};
  constexpr int countlimit = 2;
  
  auto time() {
    static auto start = std::chrono::system_clock::now();
    return std::chrono::duration_cast&lt;std::chrono::seconds&gt;(std::chrono::system_clock::now() - start).count();
  }
  
  void ping() {
      while(counter &lt;= countlimit) {
          condAtomicFlag1.wait(false);               // (1)
          condAtomicFlag1.clear();                   // (2)
          std::cout &lt;&lt; time() &lt;&lt; &quot; sec: ping() flag1&quot; &lt;&lt; std::endl;
          
          ++counter;
  
          condAtomicFlag2.test_and_set();           // (4)
          condAtomicFlag2.notify_one();             // (3)
      }
  }
  
  void pong() {
      while(counter &lt;= countlimit) {
          condAtomicFlag2.wait(false);
          condAtomicFlag2.clear();
          std::cout &lt;&lt; time() &lt;&lt; &quot; sec: pong() flag2&quot; &lt;&lt; std::endl;
          
          condAtomicFlag1.test_and_set();
          condAtomicFlag1.notify_one();
      }
  }
  
  int main(int argc, char **argv)
  {
    std::cout &lt;&lt; time() &lt;&lt; &quot; sec: before pong() flag2&quot; &lt;&lt; std::endl;
    std::thread t2(pong);
    std::cout &lt;&lt; time() &lt;&lt; &quot; sec: after pong() flag2&quot; &lt;&lt; std::endl;
    std::this_thread::sleep_for(std::chrono::seconds(3));
    condAtomicFlag1.test_and_set();                    // (5)
    std::cout &lt;&lt; time() &lt;&lt; &quot; sec: flag1 test and set&quot; &lt;&lt; std::endl;
    //condAtomicFlag1.notify_one();                      // (6)
    //std::cout &lt;&lt; time() &lt;&lt; &quot; sec: flag1 notify_one&quot; &lt;&lt; std::endl;
    std::this_thread::sleep_for(std::chrono::seconds(3));
    std::cout &lt;&lt; time() &lt;&lt; &quot; sec: before ping() flag1&quot; &lt;&lt; std::endl;
    std::thread t1(ping);
    std::cout &lt;&lt; time() &lt;&lt; &quot; sec: after ping() flag1&quot; &lt;&lt; std::endl;
  
    t1.join();
    t2.join();
  
      return 0;
  }
  </code></pre>
  <p>Now, at first, only the <code>t2</code> (pong) thread is started, in which there is a waiting of a change in the state of the atomic flag <code>condAtomicFlag2</code>. Then the main thread goes to sleep for 3 seconds. After that, the state of the atomic flag <code>condAtomicFlag1</code> (<code>test_and_set</code>) changes, although the thread waiting for it has not been started yet. Then the main thread goes to sleep again for 3 seconds. Finally, thread <code>t1</code> (ping) is started, in which there is a waiting of a change in the state of the atomic flag <code>condAtomicFlag1</code>. Since earlier <code>condAtomicFlag1</code> was already changed to <code>true</code> when calling <code>condAtomicFlag1.test_and_set()</code>, the <code>t1</code> thread immediately starts working and the transfer of control from the <code>t1</code> thread to the <code>t2</code> thread and back starts a number of times (<code>countlimit + 1</code>). The application output looks pretty predictable to me:</p>
  <pre><code>0 sec: before pong() flag2
  0 sec: after pong() flag2
  3 sec: flag1 test and set
  6 sec: before ping() flag1
  6 sec: after ping() flag1
  6 sec: ping() flag1
  6 sec: pong() flag2
  6 sec: ping() flag1
  6 sec: pong() flag2
  6 sec: ping() flag1
  6 sec: pong() flag2
  </code></pre>
  <p>But now I will uncomment two lines in the code of the main function:</p>
  <pre><code>condAtomicFlag1.notify_one();                      // (6)
    std::cout &lt;&lt; time() &lt;&lt; &quot; sec: flag1 notify_one&quot; &lt;&lt; std::endl;
  </code></pre>
  <p>I figured this was just a notification to the waiting thread that the state of the <code>condAtomicFlag1</code> atomic flag has changed. And since we have not yet started a thread that will wait for this particular flag, nothing interesting will happen. But I got this application output:</p>
  <pre><code>0 sec: before pong() flag2
  0 sec: after pong() flag2
  3 sec: flag1 test and set
  3 sec: flag1 notify_one
  3 sec: pong() flag2
  6 sec: before ping() flag1
  6 sec: after ping() flag1
  6 sec: ping() flag1
  6 sec: pong() flag2
  6 sec: ping() flag1
  6 sec: pong() flag2
  6 sec: ping() flag1
  6 sec: pong() flag2
  </code></pre>
  <p>Here you can see that on the 3rd second after calling <code>condAtomicFlag1.notify_one()</code>,</p>
  <blockquote>
  <p>3 sec: flag1 notify_one</p>
  </blockquote>
  <p>for some reason, thread <code>t2</code> exited waiting in the <code>condAtomicFlag2.wait(false)</code> function,</p>
  <blockquote>
  <p>3 sec: pong() flag2</p>
  </blockquote>
  <p>that is, in fact, it waited until the state of the atomic flag <code>condAtomicFlag2</code> changed. But how could <code>condAtomicFlag1.notify_one()</code> affect the wait for a completely different flag in <code>condAtomicFlag2.wait(false)</code>?</p>
  <p>About <a href="https://en.cppreference.com/w/cpp/atomic/atomic/wait" rel="nofollow noreferrer"><code>std::atomic::wait</code></a> it is written that <em>&quot;These functions are guaranteed to return only if value has changed, even if underlying implementation unblocks spuriously&quot;</em>. That is, the return from the function is carried out only when the value changes, even in the case of a false unlock. But I do not see any changes to <code>condAtomicFlag2</code> in my case, and the return from the function is still carried out?</p>
  <p>It looks like I do not understand something in the work of the wait-notify bundle? Explain to me what is happening here and why the notification of one flag affects the state of the other?</p>
  <p>P.S. For the project I used the gcc compiler version 11.1.0 (Ubuntu 11.1.0-1ubuntu1 ~ 20.04)</p>