Slashdot Mirror
Convert from HTML to XML With HTML Tidy
An anonymous reader writes "HTML Tidy, a powerful tool to help convert old HTML pages to newer standards, such as XML. This tip demonstrates how to convert HTML documents to XML (or more specifically, XHTML) with a simple, open source tool. This conversion is useful for webmasters who are migrating to XML. It can also help XML converts who have to interface with legacy HTML tools."
I've always been interested in X(HT)ML, but I've never wanted to sit down and convert every single page by hand. This tool might be just what I need.
You're right, I wouldn't steal a car. But if it were possible, I sure as hell would download one!
A few days ago I had to convert HTML pages into XHTML, stripping out a few extra elements and attributes. I used xsltproc, from libxslt , which uses the parser from libxml2 , and this has the option of parsing strict HTML into an XML DOM.
XMLTidy can be useful when you have a not-so-strict HTML, but for most quick conversions I've found libxml2 &co to be quite light and easy.
dakkar - mobilis in mobile
HTML Tidy has been our for years.
Check out the Tidy Homepage or the project on SourceForge.
Popisms.com - Connecting pop culture
more like tips for newbies
>
but yeah this is a great tip.. especially if you are writing web-scrapers to extra data from web pages and/or convert them to RSS. Just use Tidy to tidy it, and then your favorite XML parser can slurp it right up and you can use XPath to pull out what you need.
Look out though, there are some cases that Tidy chokes on. One that I keep running into is shit like this:
<table>
<form>
<tr><td>...</td></tr>
</table
</form>
Basically mixing stuff in between table rows. Something like that anyway. Just be ready to handle a fatal error from Tidy, it surprised me at first because I thought it could eat anything.
Again, it's converted to XHTML for a few years. I only posted the original message because I was quite surprised to see it on Slashdot. It's not uncommon to see a story that is a month or two old on the homepage, but several years old is crazy.
Popisms.com - Connecting pop culture
Ian Hickson makes a good case here that using XHTML may not be the right direction to go -- at least at this point.
- say that I use XHTML
- make it easier to parse my pages
HTML 4.01 doesn't make you expressly close your tags, which causes XML processors to choke and die. I'd rather write it in a usable format once than have to Tidy-parse every time I want to update my search engine. Plus XSLT really is cool. I've got (somewhere) a stylesheet I wrote that will validate form data for me and then I can apply other xslt stylesheets to make the output, further seperating the output from the script that does the magic. Great way to update the look of a page without messing up (accidentally, of course) the code I wrote months ago.If you are running MacOS with BBEdit, you can use the BBTidy plugin to get HTML Tidy integration in BBEdit.
JP
The date for the referenced article is 18 Sep 2003, less than two weeks ago.
Larry
4 years. I remember it being implemented before the XHTML recommendation was final. I remember it particularly because I've been using XHTML on my website since I converted it, then, in 1999, with HTML-Tidy.
Yes, HTMLTidy can "convert" an HTML page to XHTML. It basically adds CDATA marks, closes tags and create CSS classes instead of attributes like "background".
:
:
:
But correct XHTML is more than that. The goal is to actually give the right context to every element of the text.
When you have an horror like
My company
to display a title, how do you want an automatic tool like Tidy to convert it to
My company
?
It just can't. It will see a table with no caption, no column headers and three elements : two images and a text that is not supposed to be a title at all.
Converting an HTML web site with no semantic to XHTML using Tidy is useless. The result will still be unparsable (it will, but elements will have no meaning), the site will still be unaccessible to alternative browsers, it will still be a hell to maintain, etc. Of course easy navigation with the keyboard shortcuts using Mozilla is out of question.
And the code will even be larger because of the indentation, closing and styles created by Tidy.
All benefits of XHTML/CSS are totally lost.
Look at an horror like
http://www.skyrock.com/
Try to access it with Lynx or the built-in browser of a phone or PDA with no support for styles (ex: Sony/Ericsson P800).
You don't see anything but the names of three files supposed to be images. And this is all you can see on the web site. You don't see any link nor any text.
Convert this to XHTML using Tidy.
The site still doesn't look like anything but three useless filenames. It's just twice longer to load because the code is larger.
Correct XHTML sites have to be designed the right way from the ground up. There's no magic to convert an horror to something clean. And even manually, the best way to do so is almost always to restart from scratch.
{{.sig}}
Argl, I forgot to enable "Extrans" before submitting the previous post :(
:
:
:
Let's try again, sorry for the noise, I believed
"plain old text" would escape HTML tags.
---
Yes, HTMLTidy can "convert" an HTML page to XHTML. It basically adds CDATA marks, closes tags and create CSS classes instead of attributes like "background".
But correct XHTML is more than that. The goal is to actually give the right context to every element of the text.
When you have an horror like
<table><tr><td width="100%" align="center"><img src="transparentpix.gif" width="20"><font size="9"><b>My company</b></font><img src="transparentpix.gif" width="20"></td></tr></table>
to display a title, how do you want an automatic tool like Tidy to convert it to
<h1>My company</h1>
?
It just can't. It will see a table with no caption, no column headers and three elements : two images and a text that is not supposed to be a title at all.
Converting an HTML web site with no semantic to XHTML using Tidy is useless. The result will still be unparsable (it will, but elements will have no meaning), the site will still be unaccessible to alternative browsers, it will still be a hell to maintain, etc. Of course easy navigation with the keyboard shortcuts using Mozilla is out of question.
And the code will even be larger because of the indentation, closing and styles created by Tidy.
All benefits of XHTML/CSS are totally lost.
Look at an horror like
http://www.skyrock.com/
Try to access it with Lynx or the built-in browser of a phone or PDA with no support for styles (ex: Sony/Ericsson P800).
You don't see anything but the names of three files supposed to be images. And this is all you can see on the web site. You don't see any link nor any text.
Convert this to XHTML using Tidy.
The site still doesn't look like anything but three useless filenames. It's just twice longer to load because the code is larger.
Correct XHTML sites have to be designed the right way from the ground up. There's no magic to convert an horror to something clean. And even manually, the best way to do so is almost always to restart from scratch.
{{.sig}}