Take the 2-minute tour ×

In the parallel MPI program on for example 100 processors:

In case of having a global counting number which should be known by all MPI processes and each one of them can add to this number and the others should see the change instantly and add to the changed value.

Synchronization is not possible and would have lots of latency issue.

Would it be OK to open a shared memory among all the processes and use this memory for accessing this number also changing that?

Would it be OK to use MPI_WIN_ALLOCATE_SHARED or something like that or is this not a good solution?

share|improve this question

1 Answer 1

Your question suggests to me that you want to have your cake and eat it too. This will end in tears.

I write you want to have your cake and eat it too because you state that you want to synchronise the activities of 100 processes without synchronisation. You want to have 100 processes incrementing a shared counter, (presumably) to have all the updates applied correctly and consistently, and to have increments propagated to all processes instantly. No matter how you tackle this problem it is one of synchronisation; either you write synchronised code or you offload the task to a library or run-time which does it for you.

Is it reasonable to expect MPI RMA to provide automatic synchronisation for you ? No, not really. Note first that mpi_win_allocate_shared is only valid if all the processes in the communicator which make the call are in shared memory. Given that you have the hardware to support 100 processes in the same, shared, memory, you still have to write code to ensure synchronisation, MPI won't do it for you. If you do have 100 processes, any or all of which may increment the shared counter, there is nothing in the MPI standard, or any implementations that I am familiar with, which will prevent a data race on that counter.

Even shared-memory parallel programs (as opposed to MPI providing shared-memory-like parallel programs) have to take measures to avoid data races and other similar issues.

You could certainly write an MPI program to synchronise accesses to the shared counter but a better approach would be to rethink your program's structure to avoid too-tight synchronisation between processes.

share|improve this answer
thanks a lot for your answer. But would it be possible to do st like transactional memory in this problem and lock the variable or role back the operation if the variable has been accessed by one processor. –  zeve Oct 28 '13 at 17:27
MPI gives you the routines to write your code to do what you suggest, but it is a toolset not an off-the-shelf solution. Bear in mind that sometimes if what you are trying to do with MPI is difficult it is because either (a) it is difficult or (b) it is not an application for which MPI is a good toolset. –  High Performance Mark Oct 28 '13 at 19:24
MPI_ACCUMULATE allows for atomic increments of values in shared windows but still requires that memory access is synchronised either by locks (MPI_LOCK/MPI_UNLOCK) or with fences (MPI_WIN_FENCE). Also note that MPI_WIN_ALLOCATE_SHARED is an MPI-3.0 call and as such is still not widely implemented. –  Hristo Iliev Oct 29 '13 at 9:01

Your Answer


