Comments on: Test Everything, No Matter How Simple

By: Elliotte Rusty Harold

Elliotte Rusty Harold — Thu, 28 Sep 2006 11:19:00 +0000

No. NullPointerException is a special case. See Effective Java, Item 23 and Item 42. From p. 176, “If a caller passes null in some parameter for which null values are prohibited, convention dictates that NullPointerException be thrown rather than illegalArgumentException.”

By: Heine Pedersen

Heine Pedersen — Thu, 28 Sep 2006 07:55:10 +0000

Not to be nitpicking - but shouldn’t you throw an IllegalArgumentException instead of a NullPointerException?

From the J2SE API on IllegalArgumentException
“Thrown to indicate that a method has been passed an illegal or inappropriate argument.”

By: J. B. Rainsberger

J. B. Rainsberger — Sun, 10 Sep 2006 05:31:11 +0000

I assure you I definitely understand the issue. I simply don’t understand your reasoning. Let me try to be very clear. As I read it, you state the following:

1. I found a bug in some code.
2. When I went to fix the bug, I first did it before writing a test, then I stopped myself and decided to write the test.
3. It was in writing the test that I realized I tried to fix the bug in the wrong place.
Therefore, test all code, no matter how simple.

It’s that last step that doesn’t sit right with me. It seems like a sideways jump, rather than a step.

I agree that when we find a bug, we should write a test to expose the bug before fixing it. Without that, how do we know we fixed it? Even so, I think you conflate two lines of reasoning and use a conclusion from one line to justify a statement in the other.

One line of reasoning regards not fixing bugs until there is a test that fails because of the bug.

Another line of reasoning regards the ROI on testing code that I feel is Too Simple to Break. You think the R is worth the I.

The way I read it, you have concluded that it is worth testing Too Simple to Break code, because a test helped you find and fix a bug in code that wasn’t Too Simple to Break: namely setType(). If setType() is supposed to do more than just this.type = type, then it is Complex Enough to Break, and so should absolutely be tested.

So you seem to have used an example of finding a bug in code that is Complex Enough to Break to argue that one should test absolutely everything, including code that is Too Simple to Break. I just don’t understand how that reasoning works. I do certainly agree with the notion that by writing a test, you slowed down, thought about what you were doing, then realized you were about to fix the wrong code, but that’s generally a good idea anyway.

By: Elliotte Rusty Harold

Elliotte Rusty Harold — Wed, 23 Aug 2006 12:04:24 +0000

You’re not quite understanding the problem. Indeed _setType() should not be tested directly, because it is not public API. The test should go through the setType() method instead. That was never the issue.

My point is that my initial “fix” of changing the _setType() method was incorrect, and writing the test method first helped me see that. When I developed the fix just by looking at the code, I didn’t notice that. Viewing the code from the perspective of a client class, rather than the model class, helped me see things differently.

By: J. B. Rainsberger

J. B. Rainsberger — Tue, 22 Aug 2006 19:00:03 +0000

It seems to me the bug is in setType() and not in _setType(), so I would continue to test setType(), as you have done here, and not to test _setType(), as you have done here. In t his case, _setType() remains Too Simple to Break, and indeed, is not broken.