DAT_LMR_SYNC_RDMA_WRITE(3DAT) Direct Access Transport Library Functions


dat_lmr_sync_rdma_write - synchronize local memory with RDMA write on
non-coherent memory


cc [ flag... ] file... -ldat [ library... ]
#include <dat/udat.h>

dat_lmr_sync_rdma_write (
IN DAT_IA_HANDLE ia_handle,
IN const DAT_LMR_TRIPLET *local_segments,
IN DAT_VLEN num_segments


A handle for an open instance of the IA.

An array of buffer segments.

The number of segments in the local_segments argument.


The dat_lmr_sync_rdma_write() function makes effects of an incoming RDMA
Write operation visible to the Consumer. This operation guarantees
consistency by locally invalidating the non-coherent cache whose buffer
has been populated by remote peer RDMA write operations.

The dat_lmr_sync_rdma_write() function is needed if and only if the
Provider attribute specifies that this operation is needed after an
incoming RDMA Write operation. The Consumer must call
dat_lmr_sync_rdma_write() before reading data from a memory range in this
region that was the target of an incoming RDMA Write operation. The
dat_lmr_sync_rdma_write() function must be called after the RDMA Write
operation completes, and the memory range that was modified by the RDMA
Write must be supplied by the caller in the local_segments array. After
this call returns, the Consumer may safely see the modified contents of
the memory range. It is permissible to batch synchronizations of multiple
RDMA Write operations in a single call by passing a local_segments array
that includes all modified memory ranges. The local_segments entries need
not contain the same LMR and need not be in the same Protection Zone.

The Consumer must also use dat_lmr_sync_rdma_write() when performing
local writes to a memory range that was or will be the target of incoming
RDMA writes. After performing the local write, the Consumer must call
dat_lmr_sync_rdma_write() before the RDMA Write is initiated.
Conversely, after an RDMA Write completes, the Consumer must call
dat_lmr_sync_rdma_write() before performing a local write to the same

If the Provider attribute specifies that this operation is needed and the
Consumer attempts to read from a memory range in an LMR without properly
synchronizing using dat_lmr_sync_rdma_write(), the returned contents are
undefined. If the Consumer attempts to write to a memory range without
properly synchronizing, the contents of the memory range become


The operation was successful.

The DAT handle is invalid.

One of the parameters is invalid. For example,
the address range for a local segment fell
outside the boundaries of the corresponding
Local Memory Region or the LMR handle was


Determining when an RDMA Write completes and determining which memory
range was modified is the Consumer's responsibility. One possibility is
for the RDMA Write initiator to post a Send DTO message after each RDMA
Write that identifies the range in the body of the Send. The Consumer at
the target of the RDMA Write can receive the message and know when and
how to call dat_lmr_sync_rdma_write().

This call ensures that the Provider receives a coherent view of the
buffer contents after a subsequent remote RDMA Write operation. After the
call completes, the Consumer can be assured that all platform-specific
buffer and cache updates have been performed, and that the LMR range has
consistency with the Provider hardware. Any subsequent read by the
Consumer can void this consistency. The Provider is not required to
detect such access.

The action performed on the cache before the RDMA Write depends on the
cache type:

o I/O noncoherent cache will be flushed.

o CPU noncoherent cache will be invalidated.


See attributes(5) for descriptions of the following attributes:

|Interface Stability | Standard: uDAPL, 1.2 |
|MT-Level | Unsafe |


dat_lmr_sync_rdma_read(3DAT), libdat(3LIB), attributes(5)

September 22, 2020 DAT_LMR_SYNC_RDMA_WRITE(3DAT)