title: Repeat BeautifulSoup scrape for all files in a local folder
body: <p>I have thousands of html files in a few folders that I want to pull the data from the comments and put into a csv file.  This will allow me to format and clean it for a project.  For example, I have 640 html files in this folder:</p>
  
  <blockquote>
    <p>D:\My Web Sites\baseball 2\www.baseball-reference.com\boxes\ANA</p>
  </blockquote>
  
  <p>Here's my code pulling comments from a single file and putting into a CSV:</p>
  
  <pre><code># import libraries and files
  from bs4 import BeautifulSoup, Comment
  import re
  import csv
  
  # Get Page, Make Soup
  soup = BeautifulSoup(open("D:/My Web Sites/baseball 2/www.baseball-reference.com/boxes/ANA/ANA201806180.html"), 'lxml')
  
  # Get Description
  game_description = soup.findAll("div", {"scorebox_meta"})
  print (game_description)
  
  # Get Comment Data
  Player_Data = soup.find_all(string=lambda text:isinstance(text,Comment))
  for c in Player_Data:
      print c
      print "==========="
  
  # Results to CSV
  csvfile = "C:/Users/Benny/Desktop/anatest.csv"
  
  with open(csvfile, "w") as output:
      writer = csv.writer(output, lineterminator='\n')
      writer.writerows(Player_Data)
  </code></pre>
  
  <p>I need to pull the game data from each html file (all of it nested into comments in the html code), put the individual scrape results for each game file into a single CSV.  Any help with the code is super appreciated.</p>
  
  <p>Thanks,
  Benny</p>