title: Retrieving File Data Stored in Buffer
body: <p>I'm new to the forum, but not to this website. I've been searching for weeks on how to process a large data file quickly using C++ 11. I'm trying to have a function with a member that will capture the trace file name, open and process the data. The trace file contains 2 million lines of data, and each line is structured with a read/write operation and a hex address:</p>
  
  <pre><code>r abcdef123456
  </code></pre>
  
  <p>However, with a file having that much data, I need to read in and parse those 2 values quickly. My first attempt to read the file was the following:</p>
  
  <p><div class="snippet" data-lang="js" data-hide="true">
  <div class="snippet-code snippet-currently-hidden">
  <pre class="snippet-code-js lang-js prettyprint-override"><code>void getTraceData(string filename)
  {
    ifstream inputfile;
    string file_str;
    vector&lt;string&gt; op, addr;
  
    // Open input file
    inputfile.open(filename.c_str());
    cout &lt;&lt; "Opening file for reading: " &lt;&lt; filename &lt;&lt; endl;
  
    // Determine if file opened successfully
    if(inputfile.fail())
    {
      cout &lt;&lt; "Text file failed to open." &lt;&lt; endl;
      cout &lt;&lt; "Please check file name and path." &lt;&lt; endl;
      exit(1);
    }
  
    // Retrieve and store address values and operations
    if(inputfile.is_open())
    {
      cout &lt;&lt; "Text file opened successfully." &lt;&lt; endl;
  
      while(inputfile &gt;&gt; file_str)
      {
        if((file_str == "r") || (file_str == "w"))
        {
          op.push_back(file_str);
        }
        else
        {
          addr.push_back(file_str);
        }
      }
    }
    inputfile.close();
    cout &lt;&lt; "File closed." &lt;&lt; endl;
   }</code></pre>
  </div>
  </div>
  </p>
  
  <p>It worked, it ran, and read in the file. Unfortunately, it took the program 8 minutes to run and read the file. I modified the first program to the second program, to try and read the file in faster. It did, reading the file into a buffer in a fraction of a second versus 8 mins. using ifstream:</p>
  
  <p><div class="snippet" data-lang="js" data-hide="false">
  <div class="snippet-code">
  <pre class="snippet-code-js lang-js prettyprint-override"><code>void getTraceData()
  {
     // Setup variables
   char* fbuffer;
   ifstream ifs("text.txt");
   long int length;
   clock_t start, end;
  
   // Start timer + get file length
   start = clock();
   ifs.seekg(0, ifs.end);
   length = ifs.tellg();
   ifs.seekg(0, ifs.beg);
  
   // Setup buffer to read &amp; store file data
   fbuffer = new char[length];
   ifs.read(fbuffer, length);
   ifs.close();
   end = clock();
  
   float diff((float)end - (float)start);
   float seconds = diff / CLOCKS_PER_SEC;
  
   cout &lt;&lt; "Run time: " &lt;&lt; seconds &lt;&lt; " seconds" &lt;&lt; endl;
  
   delete[] fbuffer;
  }</code></pre>
  </div>
  </div>
  </p>
  
  <p>But when I added the parsing portion of the code, to get each line, and parsing the buffer contents line-by-line to store the two values in two separate variables, the program silently exits at the while-loop containing getline from the buffer:</p>
  
  <p><div class="snippet" data-lang="js" data-hide="false">
  <div class="snippet-code">
  <pre class="snippet-code-js lang-js prettyprint-override"><code>void getTraceData(string filename)
  {
   // Setup variables
   char* fbuffer;
   ifstream ifs("text.txt");
   long int length;
   string op, addr, line;
   clock_t start, end;
  
   // Start timer + get file length
   start = clock();
   ifs.seekg(0, ifs.end);
   length = ifs.tellg();
   ifs.seekg(0, ifs.beg);
  
   // Setup buffer to read &amp; store file data
   fbuffer = new char[length];
   ifs.read(fbuffer, length);
   ifs.close();
  
   // Setup stream buffer
   const int maxline = 20;
   char* lbuffer;
   stringstream ss;
  
   // Parse buffer data line-by-line
   while(ss.getline(lbuffer, length))
   {
    while(getline(ss, line))
    {
     ss &gt;&gt; op &gt;&gt; addr;
    }
    ss.ignore( strlen(lbuffer));
   }
   end = clock();
  
   float diff((float)end - (float)start);
   float seconds = diff / CLOCKS_PER_SEC;
  
   cout &lt;&lt; "Run time: " &lt;&lt; seconds &lt;&lt; " seconds" &lt;&lt; endl;
  
   delete[] fbuffer;
   delete[] lbuffer;  
  }</code></pre>
  </div>
  </div>
  </p>
  
  <p>I was wondering, once my file is read into a buffer, how do I retrieve it and store it into variables? For added value, my benchmark time is under 2 mins. to read and process the data file. But right now, I'm just focused on the input file, and not the rest of my program or the machine it runs on (the code is portable to other machines). The language is C++ 11 and the OS is a Linux computer. Sorry for the long posting.</p>