title: How can I compare two files in Python in a more efficiently way?
body: <p>I have to compare two large files, but I'm having some problems related to performance.</p>
  
  <p>So, let's consider two files <strong><code>X</code></strong> and <strong><code>Y</code></strong>.</p>
  
  <p><code>X</code> has 42000 records. One word per line.</p>
  
  <p><code>Y</code> has 881000. Three words per line, i.e. three columns.</p>
  
  <p>I want to compare the words of the <code>X</code> file with the first word of the <code>Y</code> file.</p>
  
  <p>If I find the <strong><code>X_word</code></strong> in the <strong><code>Y_first_column_word</code></strong>, then I write the the word of the second column of the <code>Y</code> file to a <code>file(Y_second_column_word)</code>.</p>
  
  <p>See the code:</p>
  
  <pre><code>to_file = open( output_file, 'w' )                # opening the file to write
  f1      = open( input_file1, "rU" ).readlines()   # reading 1st file  42000 records
  f2      = open( input_file2, "rU" ).readlines()   # reading 2nd file 881000 records
  
  for i, w1 in enumerate( f1 ):
      for j, line in enumerate( f2 ):
          w2 = line.split(',')                      # splitting words from  2nd file
          if w1.strip() == w2[0].strip():           # removing trails
              if w2[1].strip() == '':               # when it is blank, get 1st column word 
                  w2[1] = w2[0]
              print&gt;&gt;to_file, w2[1]
  
  to_file.close()                                   # closing the file
  </code></pre>
  
  <p>I've carried out tests run with test data, and it does what I want. But when I run it with the real data it becomes unresponsive. My last try spent 18 hours.</p>
  
  <p><strong>Is there any way to improve this code to get it running in a more efficient way?</strong></p>