Take the 2-minute tour ×

One my thread writes data to circular-buffer and another thread need to process this data ASAP. I was thinking to write such simple spin. Pseudo-code!

    while (true) {
        while (!a[i]) {
            /* do nothing - just keep checking over and over */
        // process b[i]
        if (i >= MAX_LENGTH) {
            i = 0;

Above I'm using a to indicate that data stored in b is available for processing. Probaly I should also set thread afinity for such "hot" process. Of course such spin is very expensive in terms of CPU but it's OK for me as my primary requirement is latency.

The question is - am I should really write something like that or boost or stl allows something that:

  1. Easier to use.
  2. Has roughly the same (or even better?) latency at the same time occupying less CPU resources?

I think that my pattern is so general that there should be some good implementation somewhere.

upd It seems my question is still too complicated. Let's just consider the case when i need to write some items to array in arbitrary order and another thread should read them in right order as items are available, how to do that?


I'm adding test program to demonstrate what and how I want to achive. At least on my machine it happens to work. I'm using rand to show you that I can not use general queue and I need to use array-based structure:

#include "stdafx.h"
#include <string>
#include <boost/thread.hpp>
#include "windows.h" // for Sleep

const int BUFFER_LENGTH = 10;
int buffer[BUFFER_LENGTH];
short flags[BUFFER_LENGTH];

void ProcessorThread() {
    for (int i = 0; i < BUFFER_LENGTH; i++) {
        while (flags[i] == 0);
        printf("item %i received, value = %i\n", i, buffer[i]);

int _tmain(int argc, _TCHAR* argv[])
    memset(flags, 0, sizeof(flags));
    boost::thread processor = boost::thread(&ProcessorThread);
    for (int i = 0; i < BUFFER_LENGTH * 10; i++) {
        int x = rand() % BUFFER_LENGTH;
        buffer[x] = x;
        flags[x] = 1;
    return 0;


item 0 received, value = 0
item 1 received, value = 1
item 2 received, value = 2
item 3 received, value = 3
item 4 received, value = 4
item 5 received, value = 5
item 6 received, value = 6
item 7 received, value = 7
item 8 received, value = 8
item 9 received, value = 9

Is my program guaranteed to work? How would you redesign it, probably using some of existent structures from boost/stl instead of array? Is it possible to get rid of "spin" without affecting latency?

share|improve this question
Why not just use a blocking bounded queue to exchange data between the threads? Every half-decent concurrency library should have one. (The corollary is: use a half-decent library of concurrency collections and utilities when writing multithreaded code.) –  millimoose Apr 25 '13 at 8:26
Are you sure you need threads if you're concerned about latency? –  Alexey Frunze Apr 25 '13 at 8:27
How sensitive is the latency - what is your upper limit where your code stops working? How long does it take to process, and how much time do you spend waiting (as a proportion and max/min of "per message")? If you say "don't know", then you need to do some research before even asking this question. –  Mats Petersson Apr 25 '13 at 8:27
@javapowered 1. I call bullshit. I doubt a good lock-free queue implementation will be any slower than a correct spinlock. 2. So your argument for reinventing the wheel is that you already reinvented the axle? Anyway. If you want maximum throughput and you're not IO-bounded, don't use threads. If you're doing any IO on your data, the synchronisation overhead will probably be negligible. –  millimoose Apr 25 '13 at 8:29
@javapowered "it's trading and [...] even 1 nanosecond is better than 2 nanoseconds" You do realize that to be of any use, your application needs to communicate with the real world and that any I/O will completely drown your (premature) optimization because of the tens or hundreds of milliseconds the I/O takes, right? –  syam Apr 25 '13 at 8:48
show 49 more comments

5 Answers

up vote 1 down vote accepted

If the consuming thread is put to sleep it takes a few microseconds for it to wake up. This is the process scheduler latency you cannot avoid unless the thread is busy-spinning as you do. The thread also needs to be real-time FIFO so that it is never put to sleep when it is ready to run but exhausted its time quantum.

So, there is no alternative that could match latency of busy spinning.

(Surprising you are using Windows, it is best avoided if you are serious about HFT).

share|improve this answer
i'm in the process of transferring to Windows. need to get rid of C# and then I ready. i'm still looking for something ready to use but it seems there are nothing except disruptor-cpp... –  javapowered Apr 25 '13 at 21:07
add comment

This is what Condition Variables were designed for. std::condition_variable is defined in the C++11 standard library.

What exactly is fastest for your purposes depends on your problem; You can attack it from several angles, but CVs (or derivative implementations) are a good starting point for understanding the subject better and approaching an implementation.

share|improve this answer
It is however not exactly a low latency operation, which means that this doesn't really answer the question –  Grizzly Apr 25 '13 at 16:35
Condition variables are, essentially, locks. The OP's problem only requires locking when the buffer is empty or full, it should be possible to use lockfree approaches the rest of the time. –  millimoose Apr 25 '13 at 16:39
@Grizzly if what you say is true… surely you can provide an answer that the OP can easily understand and implement, which also guarantees lower latency than CVs. so, how do you eliminate that spinning and achieve lower latency than CV while using two threads? –  justin Apr 25 '13 at 18:56
@millimoose and locks can be implemented using atomic ops on some platforms. locks are not always bad. if used correctly, it can be very very fast. sure, more specific implementations can be dreamed up and tested for a specific purpose, but that wouldn't make much sense to describe before the OP understands topics like CVs and sharing data using this approach. plus, there really isn't enough information in the OP to give a very specific solution. get it correct before trying to make it faster ;) –  justin Apr 25 '13 at 19:03
@justin Oh, I was coming from the correctness angle. You should (b)lock when you need to (b)lock, and going by the OP's description he certainly does when the buffer is full or exhausted. Except that's exactly what a bounded producer-consumer queue will do, correctly - it fits the desired semantics precisely, and is arguably the go-to solution to a producer-consumer scenario. The fact that lock-free collection implementations are widely available is, I hope, just a performance bonus, or at least no worse than locking around a primitive collection. –  millimoose Apr 25 '13 at 19:20
add comment

Consider using C++11 library if your compiler supports it. Or boost analog if not. And in your case especially std::future with std::promise.

There is a good book about threading and C++11 threading library:

Anthony Williams. C++ Concurrency in Action (2012)

Example from cppreference.com:

#include <iostream>
#include <future>
#include <thread>

int main()
// future from a packaged_task
std::packaged_task<int()> task([](){ return 7; }); // wrap the function
std::future<int> f1 = task.get_future();  // get a future
std::thread(std::move(task)).detach(); // launch on a thread

// future from an async()
std::future<int> f2 = std::async(std::launch::async, [](){ return 8; });

// future from a promise
std::promise<int> p;
std::future<int> f3 = p.get_future();
std::thread( [](std::promise<int>& p){ p.set_value(9); }, 
             std::ref(p) ).detach();

std::cout << "Waiting..." << std::flush;
std::cout << "Done!\nResults are: "
          << f1.get() << ' ' << f2.get() << ' ' << f3.get() << '\n';


share|improve this answer
I love futures to death but they're a high-level construct that's not even close to low-overhead. –  millimoose Apr 25 '13 at 12:00
and the relevance of this is? –  Nim Apr 25 '13 at 12:02
Also I doubt they're an appropriate mechanism here. Futures are for when the consumer pulls data from a long asynchronous producing task. Here the producer is continuously pushing data to a consumer thread. So, yeah, this answer is barely on point. –  millimoose Apr 25 '13 at 12:27
add comment

If you want a fast method then simply drop to making OS calls. Any C++ library wrapping them is going to be slower.

e.g. On Windows your consumer can call WaitForSingleObject(), and your data-producing thread can wake the consumer using SetEvent(). http://msdn.microsoft.com/en-us/library/windows/desktop/ms687032(v=vs.85).aspx

For Unix, here is a similar question with answers: Windows Event implementation in Linux using conditional variables?

share|improve this answer
a think any synchronization will cost me about 5-15 microseconds. intead I likely should spent one CPU core but have latency of less than 1 microsecond... –  javapowered Apr 25 '13 at 13:42
@javapowered: Where are you pulling those numbers from? Why 1 microsecond? Why such round even numbers? Sounds like you're just making stuff up. –  GManNickG Apr 25 '13 at 18:53
10-15 microseconds is what i generally spent when using BlockingCollection in C#. I guess that any other "fair lock" approach will cost something like that. –  javapowered Apr 25 '13 at 19:10
Consider that if you're hammering one CPU core, the CPU may throttle everything back to keep cool. That may hurt your overall timing. –  Graham Perks Apr 26 '13 at 1:24
add comment

Do you really need threading?

A single threaded app is trivially simple and eliminates all the issues with thread safety and the overhead of launching threads. I did a study of threaded vs non threaded code to append text to a log file. The non threaded code was better in every measure of performance.

share|improve this answer
by definition i have two threads already because I have to receive the same data from two UDP multicast sockets. –  javapowered Apr 25 '13 at 21:31
I guess I misunderstand this comment. Why does receiving from two sockets require two threads? –  Jay Apr 26 '13 at 4:50
good question. and the answer i think because you CAN. I think you have to do that in parallel. assume on one socket you receive packet 1 and on another socket you receive packet 2. you can now in parallel parse and decode both packets so overall latency is as low as possible. if you using just one thread packet 2 have to wait until packet 1 is processed. –  javapowered Apr 26 '13 at 6:08
add comment

Your Answer


