The Cafes » Testing

Optimization is a subject fraught with witchcraft, voodoo, and superstition. Far too often it is done badly and pseudoscientifically. The first rule of optimization is that you can’t do it without measuring . Almost any guess you make about performance is at best irrelevant and often actively harmful. A common technique is to write a benchmark that runs the piece of code you’re trying to optimize several thousand times, and prints out the time it takes to run. Then tune until the time comes down.

They’re at least ten reasons why this technique fails more often than it succeeds (forgetting to account for HotSpot initialization, optimizing a piece of code in which the program spends a trivial amount of time, etc.) but recently I encountered an 11th: the benchmark may not be exercising the code you think it is. This turns out to be surprisingly common, and the more complex the code is, the more likely this is to occur.
(more…)

Friday I installed one of my Christmas presents, a complete set of 1:100000 scale topo maps of the U.S. for my Garmin eTrex Vista GPS receiver. I’d rather use the more detailed 1:25000 maps I bought from James Associates; and I’d rather use my Mac to load them onto the GPS unit; but Garmin won’t document the protocol for uploading maps; and that protocol doesn’t seem to have been effectively reverse engineered yet; so I had to boot Windows and load them from the PC using Garmin’s own MapSource.
(more…)

One advantage of test-driven development I’ve rarely seen emphasized is that it enables experimental programming. This was recently brought home to me while I was working on XOM. Steve Loughran had requested the ability to use a NodeFactory when converting from a DOM document. It was a reasonable request so I set off to implement it.

My first iteration satisfied Steve’s use-case. It was actually a fairly major rework of the code, but the test suite assured me that it all worked and I hadn’t broken anything that used to work. However, Wolfgang Hoschek quickly found a real bug in the new functionality I had added; and things got a little sticky here.

To satisfy John Cowan, the DOMConverter uses some fancy non-recursive algorithms. This enables it to process arbitrarily deep documents without stack overflows. However the algorithm is far from obvious; and even though the code is well-written it’s hard to come back to it a couple of years later and figure out just how it works.

But this is where test-driven development really shines: I don’t have to understand the code to fix it. (more…)