title: How can I iterate through the pages of a website using Python?
body: <p>I'm new to software development, and I'm not sure how to go about this. I want to visit every page of a website and grab a specific bit of data from each one. My problem is, I don't know how to iterate through all of the existing pages without knowing the individual urls ahead of time. For example, I want to visit every page whose url starts with </p>
  
  <blockquote>
    <p>"http://stackoverflow.com/questions/"</p>
  </blockquote>
  
  <p>Is there a way to compile a list and then iterate through that, or is it possible to do this without creating a giant list of urls?</p>