title: MPI_Lock_win / passive synchronization usage confusion
body: <p>I'm trying to convert an application from using standard point-to-point MPI calls (e.g., <code>MPI_Isend</code>, <code>MPI_Irecv</code>) to using MPI-3's one-sided calls. My goal is to improve performance on my hardware, which is a system that has Infiniband hardware support and an MPI implementation that's optimized for RDMA calls. I've been told that the hardware performs particularly well with passive synchronization mode, as opposed to active synchronization (i.e., <a href="https://cvw.cac.cornell.edu/MPIoneSided/pscw" rel="nofollow noreferrer">Post-Start-Complete-Wait</a>).</p>
  
  <p>However, even after reading through the <a href="https://www.mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf" rel="nofollow noreferrer">MPI standard documentation and examples</a>, I'm confused on how to actually use the calls. For context, my program has a setup phase where I will know the communication pattern and even the buffers of the send data and ultimate buffer of the receiver. So, it's straightforward to set up a window and use it. </p>
  
  <p>Specifically, with passive synchronization, I'm confused about when the "receiver" knows the data in the window has been written by the sender. What I want to do is have the sender produce the message data, then call <code>MPI_Win_lock</code> on the window and then do an <code>MPI_Put</code> and then wait for completion with a <code>MPI_Win_Unlock</code>. But, what is an efficient / recommended way for the "receiver" (window target) of the data to know when the message data has been written? Similarly, given that the communication pattern is iterated and the same receive buffer (the target's buffer) is used multiple times, how do I know that the receiver is done consuming the buffer and it can be reused?</p>
  
  <p>I can envision a couple of approaches:</p>
  
  <ol>
  <li><p>I can use an MPI_Barrier after the MPI_Win_unlock and before the receiver accesses the data. (This seems that it would work but I'm skeptical that this would yield better performance than active synchronization.)</p></li>
  <li><p>I can possibly use MPI_Lock and MPI_Unlock on the receiver (target), locking the window when the target is actually using the data so the access epoch can't start on the origin (but, is that the way it works? I've read that lock and unlock don't create critical sections in the traditional sense). </p></li>
  <li><p>Some sort of home-grown approach where the receiver polls for some sort of a nonce to be written, knowing the data is available when that happens.</p></li>
  </ol>
  
  <p>Docs for <code>MPI_Win_lock</code>: <a href="https://www.open-mpi.org/doc/v3.0/man3/MPI_Win_lock.3.php" rel="nofollow noreferrer">https://www.open-mpi.org/doc/v3.0/man3/MPI_Win_lock.3.php</a></p>
  
  <p>In general, how does a programmer synchronize with <code>MPI_Lock</code> and 'MPI_Unlock` in a way that's any more efficient than the active synchronization approach? It does feel like I need to just use post-start-complete-wait, but I'm hoping you can help me find a way to try passive synchronization as well. </p>