title: Web Scraping with Google Compute Engine / App Engine
body: <p>I've written a python script that uses Selenium to scrape information from a website and stores it in a csv file. It works well on my local machine when I manually execute it but I now want to run the script automatically once per hour for several weeks and safe the data in a database. It may take about 5-10 minutes to run the script.</p>
  
  <p>I've just started off with Google Cloud and it looks like there are several ways of implementing it with either Compute Engine or App Engine. So far, I get stuck at a certain point with all three ways that I found so far (e.g. getting the scheduled task call a URL of my backend instance and getting that instance to kick off the script). I've tried to:</p>
  
  <ul>
  <li>Execute the script via Compute Engine and use datastore or cloud sql. Unclear if crontab can easily be set up.</li>
  <li>Use Task Queues and Scheduled Tasks on App Engine.</li>
  <li>Use backend instance and Scheduled Tasks on App Engine.</li>
  </ul>
  
  <p>I'd be curious to hear from others what they would recommend as the easiest and most appropriate way given that this is truly a backend script that does not need a user front end.</p>