The Cafes

Here’s part 14 of the ongoing serialization of Refactoring HTML, also available from Amazon and Safari.

XSLT (Extensible Stylesheet Language Transformations) is one of many XML tools that work well on HTML documents once they have first been converted into well-formed XHTML. In fact, it is one of my favorite such tools, and the first thing I turn to for many tasks. For instance, I use it to automatically generate a lot of content, such as RSS and Atom feeds, by screen-scraping my HTML pages. Indeed, the possibility of using XSLT on my documents is one of my main reasons for refactoring documents into well-formed XHTML. XSLT can query documents for things you need to fix and automate some of the fixes.
Read the rest of this entry »

Here’s part 13 of the ongoing serialization of Refactoring HTML, also available from Amazon and Safari.

John Cowan’s TagSoup (http://home.ccil.org/~cowan/XML/tagsoup/) is an open source HTML parser written in Java that implements the Simple API for XML, or SAX. Cowan describes TagSoup as “a SAX-compliant parser written in Java that, instead of parsing well-formed or valid XML, parses HTML as it is found in the wild: poor, nasty and brutish, though quite often far from short. TagSoup is designed for people who have to process this stuff using some semblance of a rational application design. By providing a SAX interface, it allows standard XML tools to be applied to even the worst HTML.”
Read the rest of this entry »

Here’s part 13 of the ongoing serialization of Refactoring HTML, also available from Amazon and Safari.

Regular expressions are well and good for individual, custom changes, but they can be tedious and difficult to use for large quantities of changes. In particular, they are designed more to work with plain text than with semistructured HTML text. For batch changes and automated corrections of common mistakes, it helps to have tools that take advantage of the markup in HTML. The first such tool is Dave Raggett’s Tidy (www.w3.org/People/Raggett/tidy/), the original HTML fixer-upper. It’s a simple, multiplatform command-line program that can correct most HTML mistakes.
Read the rest of this entry »

Here’s part 12 of the ongoing serialization of Refactoring HTML, also available from Amazon and Safari.

Manually inspecting and changing each file on even a small site is tedious and often cost-prohibitive. It is much more effective to let the computer do the work by searching for mistakes and, when possible, automatically fixing them. A number of tools support this, including command-line tools such as grep, egrep, and sed; text editors such as jEdit, BBEdit, TextPad, and PSPad; and programming languages such as Java, Perl, and PHP. All these tools provide a specialized search syntax known as regular expressions. Although there are small differences from one tool to the next, the basic regular expression syntax is much the same.

For purposes of illustration, I’m going to use the jEdit text editor as my search and replace tool in this section. I chose it because it provides pretty much all the features you need, it has a reasonable GUI, it’s open source, and it’s written in Java, so it runs on essentially any platform you’re likely to want. You can download a copy from http://jedit.org/.

However, the techniques I’m showing here are by no means limited to that one editor. In my work, I normally use BBEdit instead because it has a slightly nicer interface. However, it’s payware and only runs on the Mac. There are numerous other choices. If you prefer a different program, by all means use it. What you’ll need are:

• Full regular expression search and replace
• The ability to recursively search a directory
• The ability to filter the files you search
• A tool that shows you what it has changed, but does not require you to manually approve each change
• Automatic recognition of different character encodings and line-ending conventions

Any tool that meets these criteria should be sufficient.

Read the rest of this entry »

Stupid user stories are a tradition in I.T., and there’s a whole subgenre of clueless manager/boss/executive stories. However for once this is a story where the manager was absolutely right, and the I.T. staff (or at least the incompetent boobs who built this system, if not for the poor schmucks who had to maintain it) were colossally wrong, with devastating consequences. And to make matters worse they still don’t realize what they did wrong or how to fix it.

Here’s the story from Andrew Brandt at InfoWorld:

Being part of an online community can reap rich rewards. Allowing the tools that fuel those communities to wreak havoc on your company Web site — well, that’s probably not what you had in mind.

Of course, when it’s your boss who is insisting on tapping those tools, sometimes you have to buck hierarchy and sneak behind his back to help him toe the prudent IT line, as the administrator of a business-to-business Web site quickly found out.

The tool in question was a toolbar called Alexa, which tracks the surfing habits of its users and spiders Web sites to build a ranking system for comparing the popularity of Web sites. The admin debated the value of the toolbar with his boss often, though perhaps “debate” is too delicate a term.

“I told him time and again to uninstall it, and even did so myself a number of times, but he’d put it back every time,” the admin says.

“Then, one day, all dynamic content on the main page [of the b-to-b’s Web site] just vanished. I brought it back from backup and chalked it up to a bug. Then it happened again a little while later. I started snooping around our logs,” he says.

As it turns out, Alexa’s spiders had been ignoring the robots.txt file — and were instead capturing usernames and passwords.

“It logged into the administrative area and followed the ‘delete’ link for every entry,” the admin says. “My dumb-ass boss still didn’t want to uninstall Alexa — could have strangled the man.”

Fallout: The data was restored, with some difficulty, and Alexa’s spider was prevented, through other means, from accessing the administrative side of the Web site.

Moral: When confronted with the classic pointy-haired boss, Machiavellian subterfuge sometimes becomes necessary. Try using the Image File Execution Options registry key to prevent Alexa — or whatever undesirable, dangerous, or obnoxious program he or she keeps using to make your life miserable — from running.

Unfortunately Brandt draws the wrong moral from this story, or at least not the most important one.
Read the rest of this entry »