Take the 2-minute tour ×

This is an extension of this question. I'm trying to parse HTML snippets embedded in an XML backup of a Blogger blog and retag them with InDesign tags.

Blogger doesn't standardize the HTML for any of its posts, and the posts can be written in Word, Windows Live Writer, the native Blogger interface, or text editors, resulting in tons of different forms of HTML. Some posts don't mark paragraphs and only use double <br>s in between paragraphs—others use actual <p> tags.

What's the best way to parse this unstandard conglomeration of tags?

Additionally, each post is not a complete HTML file--just a snippet that gets inserted into a template—which means that there is no overall HTML structure to parse (<html><body></body></html>, etc.) Does that have any effect on XML/HTML parsing?

Here's some potential examples, mostly standard HTML, missing paragraphs:

This is a section of a blog post. It has <a href="#">links</a> and lists and stuff. Weee....
Here's a list
<br />
And another paragraph here...

The Word HTML looks like this - http://www.timeatlas.com/mos/images/stories/word_html_tags.png

share|improve this question
Can you post a reasonably sized snippet of the worst case input? –  Sinan Ünür Jun 11 '09 at 16:43
HTML parsing is easy with the right module. Are you really asking about how you would normalize it? –  brian d foy Jun 12 '09 at 21:23
Yeah. At this point there are 4-5 different styles of HTML that can come through, so I'm wondering if I need to standardize/normalize/tidy the html first before traversing the DOM... –  Andrew Jun 13 '09 at 11:15

4 Answers 4

up vote 2 down vote accepted

The HTML generated by Word is relatively easier to deal with. I would just get rid of all the tag attributes (unless you care about styles). That would live you with fairly plain HTML which you can then style.

HTML::TokeParser::Simple can help make that relatively painless.

As for the other stuff, that will take some trial and error. I am going to think more about that and post later if I can think of something clever.

Later Update:

Well, here is something that makes me cringe a little but it seems to work:


use strict;
use warnings;

use File::Slurp;
use Text::Markdown qw( markdown );

my $html = read_file \*DATA;

$html =~ s{(?:<br(:? ?/)*>)}{\n\n}g;

print markdown( $html );

Here's a list
<br />
And another paragraph here...


stuff. Weee....</p>

<p>Here's a list</p>


<p>And another paragraph here...</p>
share|improve this answer


share|improve this answer

As I said in the other question, I like XML::Twig. It can handle both XML and HTML.

share|improve this answer
Can it take care of the untagged text and convert it to a real <p>? That's my main concern... –  Andrew Jun 11 '09 at 20:44
No, for that you need to use the same algorithm Blogger is using. –  Chas. Owens Jun 11 '09 at 23:08
This is why I would use XML::LibXML, which has explicit support for repairing HTML documents. –  jrockway Jun 12 '09 at 4:54

FWIW, I tend to use XML::LibXML for all my XML and HTML needs. Here is a one-liner that will convert a line of "bad" HTML into a well-formed XHTML document:

perl -MXML::LibXML -ne 'my $p = XML::LibXML->new->parse_html_string($_); print $p->toString'

In your case, you probably want to use the DOM to emit a new document that has the correct tags. This is straightforward; XML::LibXML uses the same W3C DOM that JavaScript does.

As an example, this input:


Gets translated into:


This is probably what you want, and remember, use the DOM to translate... don't worry about this printed representation.

share|improve this answer
Cool. Is there any way to not get a full blown XML/HTML file and just get the <p>Foo</p><p>Bar<br/>Baz!</p>, or would I need to traverse the DOM to get just that out? –  Andrew Jun 12 '09 at 5:35
You need to traverse the DOM. –  jrockway Jun 12 '09 at 12:23

Your Answer


