Take the 2-minute tour ×

I've been using gcc's Intel-compatible builtins (like __sync_fetch_and_add) for quite some time, using my own atomic template. The "__sync" functions are now officially considered "legacy".

C++11 supports std::atomic<> and its descendants, so it seems reasonable to use that instead, since it makes my code standard compliant, and the compiler will produce the best code either way, in a platform independent manner, that is almost too good to be true.
Incidentially, I'd only have to text-replace atomic with std::atomic, too. There's a lot in std::atomic (re: memory models) that I don't really need, but default parameters take care of that.

Now for the bad news. As it turns out, the generated code is, from what I can tell, ... utter crap, and not even atomic at all. Even a minimum example that increments a single atomic variable and outputs it has no fewer than 5 non-inlined function calls to ___atomic_flag_for_address, ___atomic_flag_wait_explicit, and __atomic_flag_clear_explicit (fully optimized), and on the other hand, there is not a single atomic instruction in the generated executable.

What gives? There is of course always the possibility of a compiler bug, but with the huge number of reviewers and users, such rather drastic things are generally unlikely to go unnoticed. Which means, this is probably not a bug, but intended behaviour.

What is the "rationale" behind so many function calls, and how is atomicity implemented without atomicity?


As-simple-as-it-can-get example:

#include <atomic>

int main()
    std::atomic_int a(5);
    __builtin_printf("%d", (int)a);
    return 0;

produces the following .s:

movl    $5, 28(%esp)     #, a._M_i
movl    %eax, (%esp)     # tmp64,
call    ___atomic_flag_for_address   #
movl    $5, 4(%esp)  #,
movl    %eax, %ebx   #, __g
movl    %eax, (%esp)     # __g,
call    ___atomic_flag_wait_explicit     #
movl    %ebx, (%esp)     # __g,
addl    $1, 28(%esp)     #, MEM[(__i_type *)&a]
movl    $5, 4(%esp)  #,
call    _atomic_flag_clear_explicit  #
movl    %ebx, (%esp)     # __g,
movl    $5, 4(%esp)  #,
call    ___atomic_flag_wait_explicit     #
movl    28(%esp), %esi   # MEM[(const __i_type *)&a], __r
movl    %ebx, (%esp)     # __g,
movl    $5, 4(%esp)  #,
call    _atomic_flag_clear_explicit  #
movl    $LC0, (%esp)     #,
movl    %esi, 4(%esp)    # __r,
call    _printf  #
.def    ___atomic_flag_for_address; .scl    2;  .type   32; .endef
.def    ___atomic_flag_wait_explicit;   .scl    2;  .type   32; .endef
.def    _atomic_flag_clear_explicit;    .scl    2;  .type   32; .endef

... and the mentioned functions look e.g. like this in objdump:

004013c4 <__atomic_flag_for_address>:
mov    0x4(%esp),%edx
mov    %edx,%ecx
shr    $0x2,%ecx
mov    %edx,%eax
shl    $0x4,%eax
add    %ecx,%eax
add    %edx,%eax
mov    %eax,%ecx
shr    $0x7,%ecx
mov    %eax,%edx
shl    $0x5,%edx
add    %ecx,%edx
add    %edx,%eax
mov    %eax,%edx
shr    $0x11,%edx
add    %edx,%eax
and    $0xf,%eax
add    $0x405020,%eax

The others are somewhat simpler, but I don't find a single instruction that would really be atomic (other than some spurious xchg which are atomic on X86, but these seem to be rather NOP/padding, since it's xchg %ax,%ax following ret).

I'm absolutely not sure what such a rather complicated function is needed for, and how it's meant to make anything atomic.

share|improve this question
What version of GCC are you using? Can you show a small program that results in such poor code? I'm running a 4.7 snapshot from last month and it seems to produce decent code, with lock instructions in it. –  R. Martinho Fernandes Nov 14 '11 at 12:12
The memory model that you "don't need" comes to mind as a possible culprit. What does your code look like? Also what do you mean with the last sentence: "How is atomicity implemented without atomicity"? –  jalf Nov 14 '11 at 12:13
By "memory models", do you mean "memory orderings"? –  R. Martinho Fernandes Nov 14 '11 at 12:19
@R.MartinhoFernandes: Using gcc 4.6.1, __sync_fetch_and_add produces LOCK XADD or LOCK INC if you don't consume the output (just as expected), whereas something like std::atomic_int a(5); ++a; produces said 5 function calls. I'll edit and provide .s and objdump output. –  Damon Nov 14 '11 at 13:00
@R.MartinhoFernandes: Re: "memory models", indeed, my bad... wrong word. I meant "memory ordering". I just need atomic increment and decrement of counters, none of the comlicated esoteric stuff. –  Damon Nov 14 '11 at 13:03
show 1 more comment

3 Answers

up vote 11 down vote accepted

It is an inadequate compiler build.

Check your c++config.h, it shoukld look like this, but it doesn't:

/* Define if builtin atomic operations for bool are supported on this host. */

/* Define if builtin atomic operations for short are supported on this host.

/* Define if builtin atomic operations for int are supported on this host. */

/* Define if builtin atomic operations for long long are supported on this
   host. */

These macros are defined or not depending on configure tests, which check host machine support for __sync_XXX functions. These tests are in libstdc++v3/acinclude.m4, AC_DEFUN([GLIBCXX_ENABLE_ATOMIC_BUILTINS] ....

On your installation, it's evident from the MEM[(__i_type *)&a] put in the assembly file by -fverbose-asm that the compiler uses macros from atomic_0.h, for example:

#define _ATOMIC_LOAD_(__a, __x)                        \
  ({typedef __typeof__(_ATOMIC_MEMBER_) __i_type;                          \
    __i_type* __p = &_ATOMIC_MEMBER_;                      \
    __atomic_flag_base* __g = __atomic_flag_for_address(__p);          \
    __atomic_flag_wait_explicit(__g, __x);                 \
    __i_type __r = *__p;                           \
    atomic_flag_clear_explicit(__g, __x);                      \
    __r; })

With a properly built compiler, with your example program, c++ -m32 -std=c++0x -S -O2 -march=core2 -fverbose-asm should produce something like this:

movl    $5, 28(%esp)    #, a.D.5442._M_i
lock addl   $1, 28(%esp)    #,
movl    28(%esp), %eax  # MEM[(const struct __atomic_base *)&a].D.5442._M_i, __ret
movl    $.LC0, (%esp)   #,
movl    %eax, 4(%esp)   # __ret,
call    printf  #
share|improve this answer
And guess what, editing c++config.h to contain those defines fixes the issue, giving me exactly the lock addl, mfence sequence that you posted above, which is what I wanted, too. (I'll forward the issue to my compiler builder). Thank you very much. –  Damon Nov 15 '11 at 9:36
add comment

There are two implementations. One that uses the __sync primitives and one that does not. Plus a mixture of the two that only uses some of those primitives. Which is selected depends on macros _GLIBCXX_ATOMIC_BUILTINS_1, _GLIBCXX_ATOMIC_BUILTINS_2, _GLIBCXX_ATOMIC_BUILTINS_4 and _GLIBCXX_ATOMIC_BUILTINS_8.

At least the first one is needed for the mixed implementation, all are needed for the fully atomic one. It seems that whether they are defined depends on target machine (they may not be defined for -mi386 and should be defined for -mi686).

share|improve this answer
None of these are defined here although atomic insns are certainly available (I'm compiling for -march=core2) and work without problem using the __sync functions. I've tried to define these macros before including <atomic> just to see if that makes a difference, doesn't though. So basically you're saying this is probably a kind of "poor man's fallback implementation"? In that case, how would I enable the real one (without compiling my own gcc)? –  Damon Nov 14 '11 at 13:33
add comment

Some x86 instructions are atomic in their normal implementation. For example, a word-sized integer read or write is guaranteed to be atomic when read with mov if aligned properly- you don't need an explicit atomic instruction.

And, of course, there might be atomic instructions hidden in those functions.

share|improve this answer
The default memory ordering, sequential consistency, requires more guarantees than that. –  R. Martinho Fernandes Nov 14 '11 at 12:18
@R.MartinhoFernandes: well, DeadMG is right word sized aligned read writes are atomic (but still did not answer the OPs question). So whats the meaning of "sequential consistency" for a single read operation? –  smerlin Nov 14 '11 at 13:26
@smerlin: Yes, they are atomic in the narrow sense (happen either wholy or not at all), but operations on C++ "atomic" types are required to by "synchronized" (=atomic and ordered, where ordered means if one thread writes X and Y and another thread reads Y and X, it must not see new value of Y and old value of X), but mov on x86 does not guarantee that. –  Jan Hudec Nov 14 '11 at 15:00
@Jan Hudec: Popular memory-consistency models include x86's process consistency, in which writes from a given CPU are seen in order by all CPUs... wikipedias sources seem to contradict you –  smerlin Nov 14 '11 at 15:50
@smerlin: Than why would the compilator emit those mfence instructions all over the place if they were not necessary? –  Jan Hudec Nov 15 '11 at 15:47
show 2 more comments

Your Answer


