title: JSOUP is only returning some of the div tags
body: <p>I am trying to use JSOUP to get all the div tags with a class from this URL. I want to make a list of all the names and prices for each product. Specifically I am looking for the div tags that have class="item-name" and class="item-price".</p>
  
  <pre><code>public static void main(String[] args) throws IOException {
  
          Document doc = Jsoup.connect("http://www.games-workshop.com/gws/catalog/listProducts.jsp?catId=cat440176a").get();
  
          Elements content = doc.select("div[class]");
  
          for(Element src : content) {
  
                  System.out.println(src.text());
  
          }
  
  }
  </code></pre>
  
  <p>When I run this code it doesn't return all of the div tags with a class attribute it only returns some of the ones from the top and some from the bottom of the page. I am assuming this has something to do with the way they handle their webpages since I am getting some of the div tags. I was hoping some one may be able to shed some light on why I am only getting some of the div tags. </p>
  
  <hr>
  
  <p><strong>EDIT</strong></p>
  
  <p>Ok I decided to try a new approach after looking at some other articles on other people having inconsistent html pages. It seems to me this has something to do with the scripts generating the html I am looking for. Currently I am trying to get htmlUnit to generate the page then I want to use JSOUP to parse it.</p>
  
  <pre><code>public static void main(String[] args) throws IOException {
                      WebClient webClient = new WebClient();
          webClient.setCssEnabled(true);
          webClient.setAppletEnabled(true);
          webClient.setJavaScriptEnabled(true);
          webClient.setTimeout(7000);
          WebRequest request = new WebRequest(new URL("http://www.games-workshop.com/gws/catalog/listProducts.jsp?catId=cat440176a"));
          Page page = webClient.getPage(request);
                      String webpage = page.getWebResponse().getContentAsString();
  
                      System.out.println(webpage);
  }
  
  }
  </code></pre>
  
  <p>When I run this code I get a lot of red errors. Here are the errors <a href="http://pastebin.com/LHr7R7U1" rel="nofollow">http://pastebin.com/LHr7R7U1</a>. I hope some one can help me with a work around.</p>