Take the 2-minute tour ×

How can I write only every third item in a char buffer to file quickly in C++?

I get a three-channel image from my camera, but each channel contains the same info (the image is grayscale). I'd like to write only one channel to disk to save space and make the writes faster, since this is part of a real-time, data collection system.

C++'s ofstream::write command seems to only write contiguous blocks of binary data, so my current code writes all three channels and runs too slowly:

char * data = getDataFromCamera();
int dataSize = imageWidth * imageHeight * imageChannels;
std::ofstream output;
output.open( fileName, std::ios::out | std::ios::binary );
output.write( data, dataSize );

I'd love to be able to replace the last line with a call like:

int skipSize = imageChannels;
output.write( data, dataSize, skipSize );

where skipSize would cause write to put only every third into the output file. However, I haven't been able to find any function that does this.

I'd love to hear any ideas for getting a single channel written to disk quickly. Thanks.

share|improve this question
Is the channel data interleaved or continuous? That is, is the pattern 123123123123 or 111222333111222? –  MikeD Jul 20 '10 at 15:23
I'm betting interleaved. –  Steven Sudit Jul 20 '10 at 15:36
Yeah, it's interleaved - 123123123 –  Ross Jul 20 '10 at 15:56
Add a local with a codecvt facet that only writes every third byte into the buffer. Once you have written the facet you can imbue any stream with the locale and it will only look at every third byte. See below for details. –  Loki Astari Jul 20 '10 at 17:07
add comment

6 Answers

up vote 2 down vote accepted

Let's say your buffer is 24-bit RGB, and you're using a 32-bit processor (so that operations on 32-bit entities are the most efficient).

For the most speed, let's work with a 12-byte chunk at a time. In twelve bytes, we'll have 4 pixels, like so:


Which is 3 32-bit values:


We want to turn this into ABCD (a single 32-bit value).

We can create ABCD by applying a mask to each input and ORing.

ABCD = A000 | 0BC0 | 000D

In C++, with a little-endian processor, I think it would be:

unsigned int turn12grayBytesInto4ColorBytes( unsigned int buf[3] )
   return (buf[0]&0x000000FF) // mask seems reversed because of little-endianness
        | (buf[1]&0x00FFFF00)
        | (buf[2]&0xFF000000);

It's probably fastest to do this another conversion to another buffer and THEN dump to disk, instead of going directly to disk.

share|improve this answer
This is a nice little trick that did result in an appreciable speed up! If I have to copy data to a new buffer before writing, at least I can do it in an interesting and efficient way. –  Ross Jul 21 '10 at 14:59
add comment

You'll probably have to copy every third element into a buffer, then write that buffer out to disk.

share|improve this answer
+1 Buffering the data should give you an appreciable speedup over writing it out one byte at a time. You could even sub-class std::ofstream and add a write_pattern(char* ptr, int dataSize, int write_bytes, int skip_bytes) member function that would do it all for you. –  bta Jul 20 '10 at 15:20
It does look like copying the necessary data into its own buffer and then writing that to disk is the best option. As detailed in Tom Sirgedas's answer below, though, it turns out that I can gain a small speed up by realizing that what I really need is one channel from each pixel, but not necessarily the same channel from each pixel. By treating the input buffer as uint32's and masking intelligently, I can do better than naively copying every third element to a new buffer. –  Ross Jul 21 '10 at 15:04
Yup -- Tom's answer definitely got an up-vote from me; it's very nice. Actually, looking at it, I guess I'm the only one who's up-voted it. Oh well... –  Jerry Coffin Jul 21 '10 at 15:34
add comment

You can use a codecvt facet on a local to filter out part of the output.
Once created you can imbue any stream with the appropraite local and it will only see every third character on the input.

#include <locale>
#include <fstream>
#include <iostream>

class Filter: public std::codecvt<char,char,mbstate_t>
   typedef std::codecvt<char,char,mbstate_t> MyType;
   typedef MyType::state_type          state_type;
   typedef MyType::result              result;

    // This indicates that we are converting the input.
    // Thus forcing a call to do_out()
    virtual bool do_always_noconv() const throw()   {return false;}

    // Reads   from -> from_end
    // Writes  to   -> to_end
    virtual result do_out(state_type &state,
             const char *from, const char *from_end, const char* &from_next,
             char       *to,   char       *to_limit, char*       &to_next) const
       // Notice the increment of from
       for(;(from < from_end) && (to < to_limit);from += 3,to += 1)
            (*to) = (*from);
       from_next   = from;
       to_next     = to;

       return((to > to_limit)?partial:ok);

Once you have the facet all you need is to know how to use it:

int main(int argc,char* argv[])
   // construct a custom filter locale and add it to a local.
   const std::locale filterLocale(std::cout.getloc(), new Filter());

   // Create a stream and imbue it with the locale
   std::ofstream   saveFile;

   // Now the stream is imbued we can open it.
   // NB If you open the file stream first. 
   // Any attempt to imbue it with a local will silently fail.
   saveFile << "123123123123123123123123123123123123123123123123123123";

   std::vector<char>   data[1000];
   saveFile.write( &data[0], data.length() /* The filter implements the skipSize */ );
                                           // With a tinay amount of extra work
                                           // You can make filter take a filter size
                                           // parameter.

share|improve this answer
add comment

There is no such a functionality in the standardlibrary afaik. Jerry Coffin's solution will work best. I wrote a simple snippet which should do the trick:

const char * data = getDataFromCamera();
const int channelNum = 0;
const int channelSize = imageWidth * imageHeight;
const int dataSize    = channelSize * imageChannels;
char * singleChannelData = new char[channelSize];
for(int i=0; i<channelSize ++i)
    singleChannelData[i] = data[i*imageChannels];
try {
    std::ofstream output;
    output.write( singleChannelData, channelSize );
catch(const std::ios_base::failure& output_error) {
    delete [] channelSize;
delete [] singleChannelData;

EDIT: i added try..catch. Of course you could aswell use a std::vector for nicer code, but it might be a tiny bit slower.

share|improve this answer
@Justin: you would need a division aswell (singleChannelData[i/imageChannels]=data[i];), and multiplications are normally faster than divisions. And afaik there are special cüu instructions for ++ operations, which might be faster than +=3 aswell –  smerlin Jul 20 '10 at 15:50
Woops, I wasn't thinking. You're correct (and yes, increment by 1 is likely faster than increment by 3). –  Justin Ardini Jul 20 '10 at 15:54
add comment

First, I'd mention that to maximize writing speed, you should write buffers that are multiples of the sector size (eg. 64KB or 256KB)

To answer your question, you're going to have to copy every 3rd element from your source data into another buffer, and then write that to the stream.

If I recall correctly Intel Performance Primitives has functions for copying buffers, skipping a certain number of elements. Using IPP will probably have faster results than your own copy routine.

share|improve this answer
add comment

I'm tempted to say that you should read your data into a struct and then overload the insertion operator.

ostream& operator<< (ostream& out, struct data * s) {
share|improve this answer
add comment

Your Answer


