title: How to handle IncompleteRead: in python
body: <p>I am trying to fetch some data from a website. However it returns me <code>incomplete read</code>. The data I am trying to get is a huge set of nested links. I did some research online and found that this might be due to a server error (A chunked transfer encoding finishing before
  reaching the expected size). I also found a workaround for above on this <a href="http://bobrochel.blogspot.in/2010/11/bad-servers-chunked-encoding-and.html?showComment=1358777800048">link</a></p>
  
  <p>However, I am not sure as to how to use this for my case. Following is the code I am working on</p>
  
  <pre><code>br = mechanize.Browser()
  br.addheaders = [('User-agent', 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1;Trident/5.0)')]
  urls = "http://shop.o2.co.uk/mobile_phones/Pay_Monthly/smartphone/all_brands"
  page = urllib2.urlopen(urls).read()
  soup = BeautifulSoup(page)
  links = soup.findAll('img',url=True)
  
  for tag in links:
      name = tag['alt']
      tag['url'] = urlparse.urljoin(urls, tag['url'])
      r = br.open(tag['url'])
      page_child = br.response().read()
      soup_child = BeautifulSoup(page_child)
      contracts = [tag_c['value']for tag_c in soup_child.findAll('input', {"name": "tariff-duration"})]
      data_usage = [tag_c['value']for tag_c in soup_child.findAll('input', {"name": "allowance"})]
      print contracts
      print data_usage
  </code></pre>
  
  <p>Please help me with this.Thanks</p>