title: C++ Read file into Array / List / Vector
body: <p>I am currently working on a small program to join two text files (similar to a database join). One file might look like:</p>
  
  <pre><code>
      269ED3
      86356D
      818858
      5C8ABB
      531810
      38066C
      7485C5
      948FD4
  </code></pre>
  
  <p>The second one is similar:</p>
  
  <pre><code>
      hsdf87347
      7485C5
      rhdff
      23487
      948FD4
  </code></pre>
  
  <p>Both files have over 1.000.000 lines and are not limited to a specific number of characters. What I would like to do is find all matching lines in both files.</p>
  
  <p>I have tried a few things, Arrays, Vectors, Lists - but I am currently struggling with deciding what the best (fastest and memory easy) way.</p>
  
  <p>My code currently looks like:</p>
  
  <pre><code>
  
      #include iostream>
      #include fstream>
      #include string>
      #include ctime>
      #include list>
      #include algorithm>
      #include iterator>
      using namespace std;
  
  
      int main()
      {
  
          string line;
  
          clock_t startTime = clock();
  
          list data;
          //read first file
          ifstream myfile ("test.txt");
          if (myfile.is_open())
          {
              for(line; getline(myfile, line);/**/){
                  data.push_back(line);
              }
  
              myfile.close();
          }
  
          list data2;
          //read second file
          ifstream myfile2 ("test2.txt");
          if (myfile2.is_open())
          {
              for(line; getline(myfile2, line);/**/){
                  data2.push_back(line);
              }
  
              myfile2.close();
          }
          else cout  data2[k], k++
          //if data[j] > a;
  
          return 0;
  
  
      }
  
  </code></pre>
  
  <p>My thinking is: With a vector, random access on elements is very difficult and jumping to the next element is not optimal (not in the code, but I hope you get the point). It also takes a long time to read the file into a vector by using push_back and adding the lines one by one. With arrays the random access is easier, but reading >1.000.000 records into an array will be very memory intense and takes a long time as well. Lists can read the files faster, random access is expensive again.</p>
  
  <p>Eventually I will not only look for exact matches, but also for the first 4 characters of each line.</p>
  
  <p>Can you please help me deciding, what the most efficient way is? I have tried arrays, vectors and lists, but am not satisfied with the speed so far. Is there any other way to find matches, that I have not considered? I am very happy to change the code completely, looking forward to any suggestion!</p>
  
  <p>Thanks a lot!</p>
  
  <p>EDIT: The output should list the matching values / lines. In this example the output is supposed to look like:</p>
  
  <pre><code>
      7485C5
      948FD4
  </code></pre>