title: Incorrect Java HttpClient's response stream
body: <p>In my application I need to parse a website and save some data from ir to the database. I am using HttpClient to get the page content. My code looks like this:</p>
  
  <pre><code>        HttpClient client = new DefaultHttpClient();
          System.out.println(doc.getUrl());
          HttpGet contentGet= new HttpGet(siteUrl + personUrl);
          HttpResponse response = client.execute(contentGet);
  
          String html =  convertStreamToString(response.getEntity().getContent());
  
         /*
            parse the page
         */
  
      /***********************************************************************/
  
      public static String convertStreamToString(InputStream is) throws Exception {
      BufferedReader reader = new BufferedReader(new InputStreamReader(is));
      StringBuilder sb = new StringBuilder();
      String line = null;
      while ((line = reader.readLine()) != null) {
        sb.append(line + "\n");
      }
      is.close();
      return sb.toString();
  }
  </code></pre>
  
  <p>I am doing this in a loop - I try to get content of some pages (their structure is the same). Sometimes it works fine, but unfortunately, my response in many cases is a sequence of similar trash liek this:</p>
  
  <pre><code>�=�v7���9�Hǳ$�d7/�$�st��؎I��X^�$A6t_D���!gr�����C^��k@��MQ�2�d�8�]
  </code></pre>
  
  <p>I
  I don't know where is the problem, please help me.</p>
  
  <hr>
  
  <p>I have displayed headers of all responses that I got. For correct ones, there are:</p>
  
  <pre><code>Server : nginx/1.0.13
  Date : Sat, 23 Mar 2013 21:50:31 GMT
  Content-Type : text/html; charset=utf-8
  Transfer-Encoding : chunked
  Connection : close
  Vary : Accept-Encoding
  Expires : Thu, 19 Nov 1981 08:52:00 GMT
  Cache-Control : no-store, no-cache, must-revalidate, post-check=0, pre-check=0
  Pragma : no-cache
  Set-Cookie : pfSC=1; path=/; domain=.profeo.pl
  Set-Cookie : pfSCvp=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; domain=.profeo.pl
  </code></pre>
  
  <p>For incorrect ones:</p>
  
  <pre><code>Server : nginx/1.2.4
  Date : Sat, 23 Mar 2013 21:50:33 GMT
  Content-Type : text/html
  Transfer-Encoding : chunked
  Connection : close
  Set-Cookie : pfSCvp=3cff2422fd8f9b6e57e858d3883f4eaf; path=/; domain=.profeo.pl
  Content-Encoding : gzip
  </code></pre>
  
  <p>Any other suggestions? My guess is that this gzip encoding is a problem here, but what can I do about it?</p>