Implementation details of MPI collective operations on a multi core machine <p>In MPI, each rank has a unique address space and communication between them happens via message passing. I want to know how MPI works in a multicore machine which has a shared memory. If the ranks are on two different machines with no shared memory, then MPI has to use messages for communication. But if ranks are on the same physical machine (but still each rank has a different address space), will the MPI calls take advantage of the shared memory. Say for example I'm issuing an ALLREDUCE call. I have two machines M1 &amp; M2 each with 2 cores. Rank R1, R2 are on core1 &amp; core2 of machine M1 and R3&amp;R4 are on C1&amp;C2 of machine M2. How would the ALLREDUCE happen. Will there be more than 1 message transmitted? Ideally I would expect R1&amp;R2 to do a reduce using the shared memory available to them (similarly R3&amp;R4) followed by message exchange between M1 &amp; M2. Is there any documentation where I can read about the implementation details of the collective operations in MPI?</p>