BeautifoulSoup not working on malformed utf-8 HTML <p>I'm starting to play with <code>BeautifulSoup</code> but its not working. Just tried to obtain all links with <code>find_all('a')</code> and the reponse is always <code>[]</code> or <code>null</code>. Issues could be caused by iso/utf-8 encoding or malformed html, right?</p> <p>I've found that if I only take code between <code>&lt;body&gt;&lt;/body&gt;</code> tags it will work ok, so we can prob discard encoding. </p> <p>So what to do? Is there a soup built-in function to fix malformed html? Maybe use RE to just take <code>&lt;body&gt;</code> contents? Any clues? Its probably a common issue...</p> <p>By the way, i'm dealing with Portuguese (pt_BR) language, Win64, Python27 and example not-working page is <a href="http://www.tudogostoso.com.br/" rel="nofollow">http://www.tudogostoso.com.br/</a></p> <p>EDIT: What i've done so far</p> <pre><code>#im using mechanize br = mechanize.Browser() site = 'http://www.tudogostoso.com.br/' r = br.open(site) #returned html IS OK. outputed and tested a lot html = r.read() soup = BeautifulSoup(html) for a in soup.find_all('a', href=True): print "Found the URL:", a['href'] #nothing happens #but if html = &lt;body&gt;...&lt;/body&gt; (cropped manually) its works and prints all the links </code></pre>