title: disallow certain url in robots.txt
body: <p>We implemented a rating system on a site a while back that involves a link to a script. However, with the vast majority of ratings on the site at 3/5 and the ratings very even across 1-5 we're beginning to suspect that search engine crawlers etc. are getting through. The urls used look like this:</p>
  
  <pre><code>http://www.thesite.com/path/to/the/page/rate?uid=abcdefghijk&amp;value=3
  </code></pre>
  
  <p>When we started we add the following to our robots.txt:</p>
  
  <pre><code>User-agent: *
  Disallow: /rate
  </code></pre>
  
  <p>Is this incorrect or are googlebot and others simply ignoring our robots.txt?</p>