title: Allow search bots to crawl your sites without session IDs
body: <p>Google's <a href="http://www.google.com/support/webmasters/bin/answer.py?hl=en&amp;answer=35769" rel="nofollow noreferrer">Webmaster guidelines</a> state </p>
  
  <blockquote>
    <p>Allow search bots to crawl your sites without session IDs or arguments that track their path through the site. These techniques are useful for tracking individual user behavior, but the access pattern of bots is entirely different. Using these techniques may result in incomplete indexing of your site, as bots may not be able to eliminate URLs that look different but actually point to the same page.</p>
  </blockquote>
  
  <p>My ASP.NET 1.1 site uses custom authentication/authorization and relies pretty heavily on session guids (similar to <a href="http://www.15seconds.com/Issue/021119.htm" rel="nofollow noreferrer">this approach</a>). I'm worried that allowing non-session tracked traffic will either break my existing code or introduce security vulnerabilities.</p>
  
  <p>What best practices are there for allowing non-session tracked bots to crawl a normally session tracked site? And are there any ways of detecting search bots other than inspecting the user agent (i don't want people to spoof themselves as googlebot to get around my session tracking)?</p>