Take the 2-minute tour ×

I'm fairly new to programming with memory barriers/fences, and I was wondering how we can guarantee that setup writes are visible in worker functions subsequently run on other CPUs. For example, consider the following:

int setup, sheep;

void SetupSheep():    // Run once
    CPU 1: setup = 0;
    ... much later
    CPU 1: sheep = 9;
    CPU 1: std::atomic_thread_fence(std::memory_order_release);
    CPU 1: setup = 1;

Run afterwards (not concurrently), many, many times:

void ManipulateSheep():
    CPU 2: int mySetup = setup;
    CPU 2: std::atomic_thread_fence(std::memory_order_acquire);
    CPU 2: // Use sheep...

On CPU 2, if mySetup is 1, sheep is then guaranteed to be 9 -- but how can we guarantee that mySetup is not 0?

So far, all I can think of is to spin-wait on CPU 2 until setup is 1. But this seems quite ugly given that the spin-wait would only have to wait the first time ManipulateSheep() was called. Surely there must be a better way?

Note there's also a symmetrical problem with uninitialization code: Say you're writing a lock-free data structure which allocates memory during its lifetime. In the destructor (assuming all threads have finished calling methods), you want to deallocate all the memory, which means that you need the CPU that's running the destructor to have the latest variable values. It's not even possible to spin-wait in that scenario since the destructor would have no way of knowing what the "latest" state was in order to check for it.

Edit: I guess what I'm asking is: Is there a way to say "Wait for all my stores to propagate to other CPUs" (for initialization) and "Wait for all stores to propagate to my CPU" (for uninitialization)?

share|improve this question
It occurred to me that this must be possible in some form (perhaps architecture-dependent), since otherwise operating systems would have no way of safely moving a thread from one CPU to another while it's running. –  Cameron Nov 4 '12 at 3:44
Late comment, but when you think about it: the true way of running ManipulateSheep() on another CPU/core AFTER (even not-concurrently executed) your SetupSheep() should be "signaled" via some OS primitive or atomic-loop which guarantee acquire (aka sync) semantics.. So while the question is valid, I believe it's a little bit artificial (how you guarantee CPUs ordering?) –  Red XIII Oct 16 '13 at 13:55
add comment

2 Answers

Indeed, memory barriers don't give you any way to wait for a condition to become true. You almost certainly want to use functionality provided by your operating system to do that, such as a pthread condition variable, or lower-level primitives such as Linux's futex calls.

However, the barriers you've shown in your example are at least enough to ensure that ManipulateSheep can tell whether the sheep are ready yet.


share|improve this answer
add comment
up vote 0 down vote accepted

It turns out that #StoreLoad is exactly the right barrier for this situation. As explained simply by Jeff Preshing:


In C++11, std::atomic_thread_fence(std::memory_order_seq_cst) apparently acts as a #StoreLoad barrier (as well as the other three: #StoreStore, #LoadLoad, and #LoadStore). See this C++11 draft paper.

Side note: On x86, the mfence instruction acts as a #StoreLoad; this can generally be emitted with the _mm_fence() compiler intrinsic if need be.

So a pattern for lock-free code might be:

    CPU 1: setupStuff();
    CPU 1: std::atomic_thread_fence(std::memory_order_seq_cst);

Run parallel stuff

    CPU 2: std::atomic_thread_fence(std::memory_order_seq_cst);
    CPU 2: teardownStuff();
share|improve this answer
add comment

Your Answer


