title: Java - How do I Retrieve Text from Website (Not Source)
body: <p>I want to use java to retrieve text from a website. I can easily get the source by doing: (Thank you random internet person who posted this somewhere else)</p>
  
  <pre><code>    import java.io.BufferedReader;
      import java.io.InputStreamReader;
      import java.net.URL;
      import java.net.URLConnection;
  
      public class WebCrawler{
          public static void main(String[] args) {
              try {
                  URL google = new URL("http://stackoverflow.com");
                  URLConnection yc = google.openConnection();
                  BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
                  String inputLine;
                  while ((inputLine = in.readLine()) != null) {
                      System.out.println(inputLine);
  
                  }
                  in.close();
              } catch (Exception e) {
                  e.printStackTrace();
              }
          }
      }
  </code></pre>
  
  <p>However this leaves me with the problem of some sites returning 403s. Is there a way of getting around this? </p>
  
  <p>Very simply I was hoping to use java to create a simple bot that would scan a forum thread and automatically respond based of user queries. Am I able to do this in java? or do I need to look at it from the perspective of another language/ data retrieval method?</p>
  
  <p>Thank you for your time.</p>