title: How can I read and search multiple textfiles so that I can store a list of files that match my search?
body: <p>I hope you can help out a new learner of Python. I could not find my problem in other questions, but if so: apologies. What I basically want to do is this:</p>
  <ol>
  <li>Read a large number of text files and search each for a number of string terms.</li>
  <li>If the search terms are matched, store the corresponding file name to a new file called &quot;filelist&quot;, so that I can tell the good files from the bad files.</li>
  <li>Export &quot;filelist&quot; to Excel or CSV.</li>
  </ol>
  <p>Here is the code that I have so far:</p>
  <pre><code>#textfiles all contain only simple text e.g. &quot;6 Apples&quot;
  filelist=[]
  for file in os.listdir('C:/mydirectory/'):
      with open('C:/mydirectory/' + file, encoding=&quot;Latin1&quot;) as f:
          fine=f.read()
          
          if re.search('APPLES',fine) or re.search('ORANGE',fine) or re.search('BANANA',fine):    
            filelist.append(file)
  
  listoffiles = pd.DataFrame(filelist)
  writer = pd.ExcelWriter('ListofFiles.xlsx', engine='xlsxwriter')
  listoffiles.to_excel(writer,sheet_name='welcome',index=False)
  writer.save()
  print(filelist)
  
  </code></pre>
  <p>Questions:</p>
  <ol>
  <li>Surely, there is a more elegant or time-efficient way? I need to do this for a large amount of files :D</li>
  <li>Related to the former, is there a way to solve the reading-in of files using pandas? Or is it less time efficient? For me as a STATA user, having a dataframe feels a bit more like home....</li>
  <li>I added the &quot;Latin1&quot; option, as some characters in the raw data create conflict in encoding. Is there a way to understand which characters are causing the problem? Can I get rid of this easily, e.g. by cutting of the first line beforehand (skiprow maybe)?</li>
  </ol>