title: Find Common Region in two CSV File in PYTHON
body: <p>I have two CSV files with 10 columns each where the first column is called the "Primary Key". </p>
  
  <p>I need to use Python to find the common region between the two CSV files. For example, I should be able to detect that rows 27-45 in CSV1 is equal to rows 125-145 in CSV2 and so on. </p>
  
  <p>I am only comparing the Primary Key (Column One). The rest of the data is not considered for comparison. I need to extract these common regions in two separate CSV files (one for CSV1 and one for CSV2).</p>
  
  <p>I have already parsed and stored the rows of the two CSV files in two 'list of lists',  <code>lstCAN_LOG_TABLE</code> and <code>lstSHADOW_LOG_TABLE</code>, so the problem reduces down to comparing these two list of lists.</p>
  
  <p>I am currently assuming is that if there are 10 subsequent matches (<code>MAX_COMMON_THRESHOLD</code>), I have reached the beginning of a common region. I must not log single rows (comparing to true) because there would be regions equal (As per primary key) and those regions I need to identify.  </p>
  
  <pre><code>for index in range(len(lstCAN_LOG_TABLE)):
      for l_index in range(len(lstSHADOW_LOG_TABLE)):
          if(lstSHADOW_LOG_TABLE[l_index][1] == lstCAN_LOG_TABLE[index][1]):  #Consider for comparison only CAN IDs
              index_can_log = index                                           #Position where CAN Log is to be compared
              index_shadow_log = l_index                                      #Position from where CAN Shadow Log is to be considered
              start = index_shadow_log
              if((index_shadow_log + MAX_COMMON_THRESHOLD) &lt;= (input_file_two_row_count-1)):
                  end = index_shadow_log + MAX_COMMON_THRESHOLD
              else:
                  end = (index_shadow_log) + ((input_file_two_row_count-1) - (index_shadow_log))
              can_index = index
              bPreScreened = 1
              for num in range(start,end):
                  if(lstSHADOW_LOG_TABLE[num][1] == lstCAN_LOG_TABLE[can_index][1]):
                      if((can_index + 1) &lt; (input_file_one_row_count-1)):
                          can_index = can_index + 1                           
                      else:
                          break   
                  else:
                      bPreScreened = 0
                      print("No Match")
                      break
              #we might have found start of common region         
              if(bPreScreened == 1):      
                  print("Start={0} End={1} can_index={2}".format(start,end,can_index))
                  for number in range(start,end):
                      if(lstSHADOW_LOG_TABLE[number][1] == lstCAN_LOG_TABLE[index][1]):                           
                          writer_two.writerow(lstSHADOW_LOG_TABLE[number][0])
                          writer_one.writerow(lstCAN_LOG_TABLE[index][0])
                          if((index + 1) &lt; (input_file_one_row_count-1)):
                              index = index + 1                           
                          else:
                              dump_file.close()   
                              print("\nCommon Region in Two CSVs identifed and recorded\n")                           
                              return
  dump_file.close()   
  print("\nCommon Region in Two CSVs identifed and recorded\n")
  </code></pre>
  
  <p>I am getting strange output. Even the first CSV file has only 1880 Rows but in the common region CSV for the first CSV I am getting many more entries. I am not getting desired output.</p>
  
  <p><strong>EDITED FROM HERE</strong></p>
  
  <p><strong>CSV1:</strong></p>
  
  <pre><code>216 0.000238225 F4  41  C0  FB  28  0   0   0   MS CAN
  109 0.0002256   15  8B  31  0   8   43  58  0   HS CAN
  216 0.000238025 FB  47  C6  1   28  0   0   0   MS CAN
  340 0.000240175 0A  18  0   C2  0   0   6F  FF  MS CAN
  216 0.000240225 24  70  EF  28  28  0   0   0   MS CAN
  216 0.000236225 2B  77  F7  2F  28  0   0   0   MS CAN
  216 0.0002278   31  7D  FD  35  28  0   0   0   MS CAN
  </code></pre>
  
  <p><strong>CSV2:</strong></p>
  
  <pre><code>216 0.0002361   0F  5C  DB  14  28  0   0   0   MS CAN
  216 0.000236225 16  63  E2  1B  28  0   0   0   MS CAN
  109 0.0001412   16  A3  31  0   8   63  58  0   HS CAN
  216 0.000234075 1C  6A  E9  22  28  0   0   0   MS CAN
  40A 0.000259925 C1  1   46  54  30  44  47  36  HS CAN
  4A  0.000565975 2   0   0   0   0   0   0   C0  MS CAN
  340 0.000240175 0A  18  0   C2  0   0   6F  FF  MS CAN
  216 0.000240225 24  70  EF  28  28  0   0   0   MS CAN
  216 0.000236225 2B  77  F7  2F  28  0   0   0   MS CAN
  216 0.0002278   31  7D  FD  35  28  0   0   0   MS CAN
  </code></pre>
  
  <p><strong>EXPECTED OUTPUT CSV1:</strong></p>
  
  <pre><code>340 0.000240175 0A  18  0   C2  0   0   6F  FF  MS CAN
  216 0.000240225 24  70  EF  28  28  0   0   0   MS CAN
  216 0.000236225 2B  77  F7  2F  28  0   0   0   MS CAN
  216 0.0002278   31  7D  FD  35  28  0   0   0   MS CAN
  </code></pre>
  
  <p><strong>EXPECTED OUTPUT CSV2:</strong></p>
  
  <pre><code>340 0.000240175 0A  18  0   C2  0   0   6F  FF  MS CAN
  216 0.000240225 24  70  EF  28  28  0   0   0   MS CAN
  216 0.000236225 2B  77  F7  2F  28  0   0   0   MS CAN
  216 0.0002278   31  7D  FD  35  28  0   0   0   MS CAN
  </code></pre>
  
  <p><strong>OBSERVED OUTPUT CSV1</strong></p>
  
  <pre><code>340 0.000240175 0A  18  0   C2  0   0   6F  FF  MS CAN
  216 0.000240225 24  70  EF  28  28  0   0   0   MS CAN
  216 0.000236225 2B  77  F7  2F  28  0   0   0   MS CAN
  216 0.0002278   31  7D  FD  35  28  0   0   0   MS CAN
  </code></pre>
  
  <p>And many thousands of redundant row data </p>
  
  <p><strong>EDITED - SOLVED AS PER ADVICE (CHANGED FOR TO WHILE):</strong></p>
  
  <p><strong>LEARNING:</strong> <em>In Python FOR Loop Index cannot be changed at RunTime</em></p>
  
  <pre><code>dump_file=open("MATCH_PATTERN.txt",'w+')
  print("Number of Entries CAN LOG={0}".format(len(lstCAN_LOG_TABLE)))
  print("Number of Entries SHADOW LOG={0}".format(len(lstSHADOW_LOG_TABLE)))  
  index = 0   
  while(index &lt; (input_file_one_row_count - 1)):
      l_index = 0
      while(l_index &lt; (input_file_two_row_count - 1)):
          if(lstSHADOW_LOG_TABLE[l_index][1] == lstCAN_LOG_TABLE[index][1]):  #Consider for comparison only CAN IDs
              index_can_log = index                                           #Position where CAN Log is to be compared
              index_shadow_log = l_index                                      #Position from where CAN Shadow Log is to be considered
              start = index_shadow_log
              can_index = index
              if((index_shadow_log + MAX_COMMON_THRESHOLD) &lt;= (input_file_two_row_count-1)):
                  end = index_shadow_log + MAX_COMMON_THRESHOLD
              else:
                  end = (index_shadow_log) + ((input_file_two_row_count-1) - (index_shadow_log))              
              bPreScreened = 1
              for num in range(start,end):
                  if(lstSHADOW_LOG_TABLE[num][1] == lstCAN_LOG_TABLE[can_index][1]):                      
                      if((can_index + 1) &lt; (input_file_one_row_count-1)):
                          can_index = can_index + 1                           
                      else:
                          break   
                  else:
                      bPreScreened = 0
                      break
              #we might have found start of common region         
              if(bPreScreened == 1):      
                  print("Shadow Start={0} Shadow End={1} CAN INDEX={2}".format(start,end,index))
                  for number in range(start,end):
                      if(lstSHADOW_LOG_TABLE[number][1] == lstCAN_LOG_TABLE[index][1]):                           
                          writer_two.writerow(lstSHADOW_LOG_TABLE[number][0])
                          writer_one.writerow(lstCAN_LOG_TABLE[index][0])
                          if((index + 1) &lt; (input_file_one_row_count-1)):
                              index = index + 1
                          if((l_index + 1) &lt; (input_file_two_row_count-1)):
                              l_index = l_index + 1                               
                          else:
                              dump_file.close()   
                              print("\nCommon Region in Two CSVs identifed and recorded\n")                           
                              return
              else:
                  l_index = l_index + 1
          else:
              l_index = l_index + 1
      index = index + 1   
  dump_file.close()   
  print("\nCommon Region in Two CSVs identifed and recorded\n")
  </code></pre>