title: Block Website Scraping by Google Docs
body: <p>I run a website that provides various pieces of data in chart/tabular format for people to read. Recently I've noticed an increase in the requests to the website that originate from Google Docs. Looking at the IPs and User Agent, it does appear to be originating from Google servers - <a href="http://whatismyipaddress.com/ip/66.249.89.221" rel="noreferrer">example IP lookup here</a>.</p>
  
  <p>The number of hits is in the region of 2,500 to 10,000 requests per day.</p>
  
  <p>I assume that someone has created one or more Google Sheets that scrape data from my website (possibly using the <a href="https://support.google.com/docs/answer/3093339?hl=en" rel="noreferrer">IMPORTHTML</a> function or similar). I would prefer that this did not happen (as I cannot know if the data is being attributed properly).</p>
  
  <p><strong>Is there a preferred way to block this traffic that Google supports/approves?</strong> </p>
  
  <p>I would rather not block based on IP addresses, as blocking Google servers feels wrong and may lead to future problems or IPs could change. At the moment I am blocking (returning 403 status) based on User Agent containing <code>GoogleDocs</code> or <code>docs.google.com</code>.</p>
  
  <p>Traffic is mostly coming from 66.249.89.221 and 66.249.89.223 at present, always with the user agent <code>Mozilla/5.0 (compatible; GoogleDocs; apps-spreadsheets; http://docs.google.com)</code></p>
  
  <p>As a secondary question: <strong>Is there a way to trace the document or its account owner?</strong> I have access to the URLs that they are accessing, but little else to go on as the requests appear to proxy through the Google Docs servers (no Referer, Cookies or other such data in the HTTP logs).</p>
  
  <p>Thank you.</p>