Take the 2-minute tour ×

My program processes database rows in different threads depending on one of the row's fields. The main thread spawns off the "workers", performs a query, and then, for each row, needs to wake up all of the workers for one of them to consume the row.

Now, using pthread_cond_broadcast() seems like the most logical choice. However, the workers in this case must all wait inside pthread_cond_wait() using the same mutex.

In my case this is suboptimal, because it means, the workers will be woken up one at a time (which I do not need) -- instead of all at once. Yes, I do want them all to wake up -- they would all read one field from the new DB-row, after which all but one will go back to waiting for the next row. I do not need to synchronize them.

I thought, I'd use a dummy thread-specific mutex with the pthread_cond_wait() in each thread, but that is not working (only one thread is woken up). The standard says, waiting for the same condition variable using different mutexes (as I do) is undefined.

So, is there a way to notify all threads at once? Thanks!

share|improve this question
Are you trying to have the workers figure out which one actually needs to wake up (based on the row data)? Why not let the main thread figure out the right worker and queue data to it? –  dan3 Dec 30 '12 at 5:51
Yes, the workers are to figure out, which of them will take the job. I'd rather not do it in the main thread, because then I'd need to loop the comparison, whereas the worker threads just need to perform one comparison each. –  Mikhail T. Dec 30 '12 at 6:17
Can't you use a hash table or a binary search (instead of a linear search) to figure out the recipient? Anyway, unless you have as many cores as workers, comparisons won't happen in parallel (and because of context switching, this could be even slower than a loop). –  dan3 Dec 30 '12 at 6:28
Yes, I can use a hash table or a tree -- but that's still slower, than simply letting each worker perform one comparison. My way the figuring out is done in time O(1). With a hash or a tree it is still O(log(n)). –  Mikhail T. Dec 30 '12 at 6:31
"big-O" is dodgy and misleading. For small values of n, O(log(n)) where an operation takes about 30 cycles is far better than O(1) where an operation takes a few thousand cycles. Also note that your O(1) is actually "O(1) on n different threads/CPUs", which sounds a lot like O(n) to me. –  Brendan Dec 30 '12 at 6:56

2 Answers 2

With condition variables, the assumption is that there is some associated "condition" (the data row in your case) that needs to be checked and updated exclusively (hence the mutex). Whichever other mechanism you choose, you will need to figure out how to ensure exclusive access to your "work queue" (whether it's a single slot or a real queue).

With a shared queue, you will always have 2 writers (main thread + intended worker) and N-1 readers to the data structure. You could use read-write locks (rwlock) to ensure integrity.

Alternatively you could have N separate queues (one per worker). You would push a copy of the data row to each worker.

As far as waking up several threads at once: you could have your workers "sleep" (e.g. with select()) and wake them up with pthread_signal() (in a loop).

You could also use pthread_barrier_wait().



  1. initialize the barrier with pthread_barrier_init() (with count = 1 + # of workers)
  2. in each worker, call pthread_barrier_wait() in a loop; when that returns, new data is ready
  3. in the main thread, call pthread_barrier_wait() to signal the workers

Unfortunately (as the OP notes), in the next iteration, no worker will be woken up until the previously activated worker finishes its job.

A simpler architecture would have the main thread dispatch events to the proper worker (instead of waking all workers up and having them figure out which one is the intended recipient). Unless you have as many cores as workers, the tests will not really happen in parallel anyway. Also, even if you have enough cores to let the workers run in parallel, N-1 of them will not learn that the "winner" has taken up the job before they finish testing, so the total amount of work across all cores is higher.

share|improve this answer
But then, wouldn't all threads have to wait while one of them is busy processing the previous row? I don't want the main thread to do the figuring out, which worker to wake up - that's because it would have to do comparisons sequentially (in a loop). My way -- if it were possible -- each worker would perform one comparison in parallel with everyone else. –  Mikhail T. Dec 30 '12 at 6:25
Yes, I do have 8 cores. The number of threads is currently 5 and may rise to 12 or 15 some day. I am puzzled, why such a simple need is not addressed by the pthread API. Simply making the mutex-argument of pthread_cond_wait() optional would've solved the problem... –  Mikhail T. Dec 30 '12 at 6:45
Sorry, in my case there are no "two writers" -- only one, the main thread. Think of an officer bringing mail to barracks: he announces (broadcasts) the name on each letter and one soldier claims it (after comparing the name announced with his own). This is much faster than for the officer to bring each letter to each person. In my case rows are "letters" and threads are "soldiers"... –  Mikhail T. Dec 30 '12 at 9:05
I gave you several examples of architectures that satisfy your requirements. In the rwlock example, the "recipient thread" would remove the "letter" from the work queue (this would be the second writer). There may be other designs, but in any case you will need to notify the main thread from the recipient thread. Think things through. –  dan3 Dec 30 '12 at 9:48

I think you need to describe more about the problem and why you're (attempting) to do it this way to begin with. I wouldn't be surprised if the best way is to do something completely different that doesn't involve waking up all threads at once without a mutex.

To me, your description sounds like:

  • Main thread spawns several threads (where spawning a thread is relatively expensive)
  • Main thread does a query, while the spawned threads start, do very little, then block (where starting/restarting and blocking are relatively expensive)
  • For each row the main thread wakes up every thread (relatively expensive restarting and blocking) and every thread except one of them goes back to waiting (very wasteful)

Without knowing why you're doing any of this, I'd assume that not using any threads at all would be faster (e.g. that the main thread would be able to process a row faster than the main thread can examine the row and tell one spawned thread to process it and hassle other threads for no reason).

If processing a row takes a long time, then I'd consider having worker threads waiting on a FIFO queue, such that the main thread pushes a "process this row" command onto the queue and the first thread that grabs it from the queue processes that row.

Of course I have no idea why you want to do what you want to do, and therefore any suggestion is just a guess.

TL;DR: I think your question is a bit like someone that wants to lose weight asking "what is the best way to chop off my own legs" (where the most practical answer has nothing to do with the question actually asked).

share|improve this answer
On startup each thread has to read a (potentially very large) file building a hash-table from its contents -- n files. Each row has to be checked to find out, whether it necessitates an update to the corresponding file -- a fairly expensive thing to do. At the end, the files, which have to be updated, need to be rewritten. The initial opening, the checking, and the final rewrite -- as well as complete independence of different workers -- justify parallelizing in my opinion. Maybe, waking them all up for each row is too expensive -- I do not know. I implemented lookup in the main thread... –  Mikhail T. Dec 30 '12 at 8:55
On startup each thread has to read a file building a hash-table (for an unstated reason) from its contents - n files (and n threads with one thread per file, or with n/m files per thread?). Each row (with unspecified contents) has to be checked (in some way) to find out whether it necessitates an update to the corresponding file (can this be done first so that files that don't need to be updated don't get read/processed at all? If not why not?) At the end, files which have to be updated (but haven't been modified at all yet?) need to be rewritten. –  Brendan Dec 30 '12 at 17:07
Something somewhere does sound like it should be done in parallel. The problem is determining what, then how. To me it sounds like maybe the main thread should do the query, then start a worker thread for each row of the query; where each worker thread does whatever has do be done to determine if the file needs to be updated and to update the file, without any synchronisation between threads at all (other than main thread waiting for all workers to finish). –  Brendan Dec 30 '12 at 17:11
Yes, Brendan, this is exactly how I've structured my program. The main thread kicks off the workers and, while they are initializing (which includes loading the existing local tables into memory), initiates a query. When the records start arriving from the DB, the main thread dispatches each to the appropriate worker. My original plan was for the main thread to simply update a global variable (with the new DB-row) and notify all threads of it. The one thread, to whom the row belonged, would then process it. I had to do it differently (with a per-thread mutex and cond-var)... –  Mikhail T. Jan 11 '13 at 0:17

Your Answer


