In Praise of Draconian Error Handling, Part 2

The fundamental reason to prefer draconian error handling is because it helps find bugs. I was recently reminded of this when Peter Murray-Rust thought he had found a bug in XOM. In brief, it was refusing to parse some files other tools let slip right through. In fact, XOM’s strict namespace handling had uncovered a cascading series of bugs that had been missed by various other parsers including Xerces-2j and libxml.

But before I describe what happened, let’s see if you can eyeball this bug. I’ll make it easier by cutting out the irrelevant parts so you know you’re looking right at the bug. Here’s the instance document we start with:

<!DOCTYPE svg SYSTEM 
"http://www.w3.org/TR/2000/03/WD-SVG-20000303/DTD/svg-20000303-stylable.dtd">
<svg/>

And the referenced DTD is:

<!ENTITY % StylableSVG "INCLUDE" >
<!ENTITY % ExchangeSVG "IGNORE" >
<!ENTITY % SVGNamespace "http://www.w3.org/2000/svg-20000303-stylable" >
<!ENTITY % Shared PUBLIC "-//W3C//DTD SVG 20000303 Shared//EN" "svg-20000303-shared.dtd" >
%Shared;

Then in svg-20000303-shared.dtd we find this:

<!ATTLIST svg
  xmlns CDATA #FIXED "%SVGNamespace;"
  %stdAttrs; >

Not obvious, is it? In fact, I looked at this one for quite a while, and consulted several spec documents before Tatu Saloranta figured out what was actually wrong here. If it helps the relevant part of the XML specification is Section 4.4, XML Processor Treatment of Entities and References.

Give up? OK. Here’s what’s happening:

Although the parameter entity reference %SVGNamespace; appears in a DTD. it appears inside a default attribute value. The parameter entity reference is therefore not resolved until the attribute actually appears in the document. However, parameter entity references are not resolved in document content (including attribute values). There it’s just literal text. In essence this document is:

<svg xmlns="%SVGNamespace;"/>

Although this bug was nigh-on impossible to find by eye, XOM picked it right up. It noticed that Xerces was setting a namespace URI to %SVGNamespace;. XOM didn’t know what the namespace was supposed to be, but it knew that %SVGNamespace; was not a legal absolute URL, and it refused to allow that. Hence the bug was flagged, and we eventually figured out where and how in the DTD the bug really was. And then we figured out that this was an old DTD from a working dfraft spec, and the tool that was having trouble should be upgraded to the new DTD anyway.

However no other XML library I’m aware of would have caught this because none of them make the check that a namespace URL should be absolute. Technically that’s only good practice, not a strict requirement of the spec. However it is a strict requirement for Infoset based specs such as XInclude and XProc. A document that uses non-absolute URLs does not have an Infoset, and therefore one cannot use XInclude or XProc on them, at least not with any confidence that the results will be consistent from tool to tool because the spec simply does not define how to handle this case.

In practice, I have yet to encounter an XML document that did use a relative namespace URI for any good reason. I have had people request an option not to reject documents that use non-absolute URIs for namespaces, and merely allow any string to be passed in. However, if I accepted that request, then XOM would no longer catch real bugs like this. Furthermore, document producers would have less incentive to generate correct documents. The more error correction tools perform, the more error correction they have to perform. Documents gradually deviate further and further from the spec, and tools try to keep up. However different tools implement different forms and amounts of error correction, until eventually we might as well not have a spec at all.

Instead of participating in this race to the bottom, XOM refuses to play. It will parse namespace well-formed XML documents only. If you want to parse something else, you can’t. Fix the documents or fix the process generating the malformed documents. XOM isn’t changing.

This entry was posted on Friday, June 5th, 2009 at 9:21 am and is filed under XML. You can follow any responses to this entry through the Atom feed. Both comments and pings are currently closed.