Eliminating Final
All the hoohaw over finality, its goodness or badness, and whether or not it should be the default, suggests it’s worth exploring the background. Why do I feel so strongly that final should be the default (at least for methods) and what changes could be made to modify this belief?
First a couple of ground rules:
- I am only interested in code other people will use. Closed, one-off code; or code that is only modified by the members of one small team, is relatively easy and thus uninteresting.
-
I am going to talk here about potential modifications to the Java (or other) programming languages that would enable the relaxation of my rule that all methods should be final until proven otherwise. Absent such modifications, I remain convinced that
finalis necessary and a good idea.
Design by Contract and its Implications
I’m a firm believer in Design By Contract. Every public method should specify what its legal inputs are and what its expected output is. Every class should document its class invariants. Furthermore, it’s the class’s responsibility to enforce all this. Incorrect input (i.e. input outside the range of the preconditions) should have defined behavior (usually an exception, particularly IllegalArgumentException) and not cause a violation of the post-condition or class invariant. This has been described in numerous texts, most compellingly in Bertrand Meyer’s Object Oriented Software Construction.
It is a general principle of object oriented programming that an instance of the subclass can be used anywhere an instance of the superclass is expected. This is the very definition of polymorphism. This means that the subclasses must maintain the class invariants and postconditions of their superclasses because client code might be depending on those invariants and postconditions. Failure to do so is a contract violation. For example, if a Clock class promises that the getHours() method always returns a value between 1 and 12, then this must still be true for a MilitaryClock subclass’s getHours() method. The MilitaryClock subclass can add a different getMilitaryHours() method that returns a value between 0 and 23. However it must not violate the rule that getHours() always returns a number between 1 and 12.
Subclasses can relax but not restrict the preconditions. For example, it is OK for the MilitaryClock‘s setHours() method to accept any time between 0 and 23. However, it would not be OK for it to require the input to be between 1 and 10. The superclass accepts 11 and 12 as valid input, so the subclass must as well.
Given these restrictions, it follows that most methods should be final. There’s no other way to guarantee their behavior in subclasses. Failing to mark them final risks postcondition, precondition, and class invariant violation. In extreme cases, it can cause security breaches.
Allowing subclassing but avoiding such violations isn’t too hard. Just mark all methods as final, and leave the class non-final. Allowing methods to be overridden is much trickier. Ideally only methods that have no preconditions, postconditions, or effect on the class’s invariants should be allowed to be overridden. In practice, you can’t always achieve that so you document the class up the wazoo, put lots of warnings in the code and the JavaDoc, and hope the subclassers read it. It’s not an ideal solution, though, so you really don’t want to do this unless you know you need to.
The Fix
The fix is simple. Make preconditions, postconditions, and class invariants language level constructs. Then make them automatically inherited by all subclasses. This would allow subclasses to override methods from the superclass, but would not allow them to change the superclass’s defined behavior. They could only change things the superclass did not explicitly promise.
They’re a lot of other benefits to this scheme as well. First of all it would mean more people would use design by contract, and thus write better code. For those of us who already use design by contract, it would make our code much simpler. But what interests me for the moment is that it would dramatically reduce the need for final. final is really overkill for 90% of the uses to which it’s put.
(There’s still one place you’d need final in this scheme. A method that has side effects not visible in its input and output might still want to declare itself final.)
The Workaround
On occasion when I’ve really needed overridden methods, but couldn’t accept the loss of conditions and invariants, I’ve done something like this:
public final void setHours(int hours) {
try {
_setHours(hours);
}
catch (IllegalArgumentException ex) {
if (hours >= 1 && hours <= 12) {
throw new RuntimeException("Subclass tightened precondition");
}
}
if (hours < 1 || hours > 12) {
throw new RuntimeException("Hours out of range: " + hours);
}
}
protected void _setHours(int hours) {
if (hours < 1 || hours > 12) {
throw new IllegalArgumentException("Hours out of range: " + hours);
}
this.hours = hours;
}
This allows the subclass to plug in to the superclass without being able to change the preconditions, postconditions, and invariants. They’re a lot of variations to this approach depending on exactly where and what you need to check. This is a complex and ugly kludge and I don’t like it, but sometimes it’s necessary for trustworthy code.
Any takers?
The language that comes closest to what is needed is Eiffel. Indeed Eiffel was designed with many of these principles in mind. However it doesn’t quite get there. Eiffel lets condition checking be turned off at runtime, and that’s unsuitable for public APIs. Turning off checks at runtime is like including air bags in a new car model during design and street testing, then removing them just as you start selling the cars to consumers. No matter how rigorously you test, the clients of your code will encounter situations and uncover bugs you did not find in testing.
One final note
These same concerns also illustrate why interfaces are vastly overused in Java today. They’re a lot of high-powered theorists who just love interfaces and abstract factories; and it’s not a coincidence that these are the same people who hate final. They’re wrong about this too, and for the same reason. Java interfaces have no ability to enforce preconditions, postconditions, and invariants. This means they’re appropriate only when a type really doesn’t have any of those, which is rare.
There are a few cases where you just need interfaces, and you have to let go of preconditions, postconditions, and invariants. In that case, you document everything six ways to Sunday and pray the people who implement your interfaces read and follow the documentation. But make no mistake: this is an extreme measure suitable for use only in unusual circumstances. This is not the default. Most designs in Java should use concrete classes, not interfaces.
Of course this would change if interfaces could declare preconditions, postconditions, and invariants as part of the methods’ signatures. If that were so, it would then become rational to use interfaces more routinely. However that is not the case with Java today.
Preconditions, postconditions, and invariants are at the core of data encapsulation. They are the first and most important pillar of object oriented programming. Inheritance and polymorphism are second and third respectively, and are less important. Not unimportant, mind you; just less important. Breaking data encapsulation to support polymorphism is like choosing a restaurant because they have wonderful desserts even when you know the entree and appetizer are going to suck. Maybe you do it if you’re the pastry chef at Chez Panisse checking out your competition, but you don’t eat there every night. Interfaces and non-final methods are the same: a tool for very special purposes, but never the first thing you should pull out of your toolbox.
May 30th, 2006 at 8:57 am
Whoa, typo dude!
You seem to have pasted a link to my weblog when you meant to link to someone who is any one or more of the following : (a) high powered, (b) a theorist, (c) someone who loves abstract factories, or (d) someone who hates final.
Glossing over (a), (b), and (c), let me say that in that particular article I defended the use of final for implementation inheritence.
Now let me respond to your specific suggestion, namely adding pre- and post-conditions to the language, a’la Eiffel. This is a suggestion with great merit in the context of ever-more-restrictive static analysis of programs. Is it better than my suggestion of separating interface from implementaion inheritence? I think they are orthogonal, as Design-By-Contract is clearly a specification of the interface and not the implementation, as you point out.
p.s.: A small nit: For the example you give, conditions on the use of hours, isn’t this pointing out a deeper flaw in most programming languages, namely overloading ‘base’ types like int, float, and string? Should we say that setHours(…) takes an int with preconditions? or should we have a new type, like an enum, that only has values in the range 0..11 (or 1..12)?
May 30th, 2006 at 10:23 am
I’m not a high-powered theorist, but I love interfaces, abstract factories, and final. OK, I don’t really love abstract factories all that much.
May 30th, 2006 at 11:10 am
i agree mostly with elharo’s points. any non-trivial program has a vast number of degrees-of-freedom. Most of them are errors, some are don’t care and very few are the good ones. The easier it is for the programmer to restrict those unwanted degrees-of-freedom, the easier it is to get right programs. The class-designer should define setHours( hourType hour ) with hourType ( 0..23 ). That obsoletes a handwritten testcase or runtime-check. Something like setHours( -5 ) shouldnt compile! Of course, this makes design more difficult – you must think to the end and come to a decision! This puts more responsibility to the class-designer. And it may make more work for the class-user.
It’s all about restriction: is a product-code a string? is a length a double ? rarely – ‘hello world’ or the text of the mao-bible is not a valid product code (at least no in my apps); for most real-world applications any length larger than earth-circumfence is invalid data. A processor should throw an exception when you try to store -275 into temperatureType. So strings and doubles are hidden away behind restrictive classes.
When we discuss ‘final is restrictive’ – why not discuss ‘private is restrictive’? I can imagine programmers feeling annoyed by inaccessible members and methods…
Interfaces have another important use: they are a good way to prevent class-cycles. Even if they are not necessary for the object-model and are implemented by only one single class – if they serve no other purpose than to prevent class-cycles, they are a good thing to have!
May 31st, 2006 at 1:15 am
Wow, talk about a bad attitude. Why are you so concerned that people might use your library in a way that you didn’t intend? Is it so wrong when someone creates a subclass that you disapprove of? Isn’t that their business?
This sort of paranoia is appropriate when designing an API that’s made available to untrusted code where security is an issue, but not for friendly collaboration between developers.
May 31st, 2006 at 10:24 am
It has nothing to do with what the client uses the library for. That is their business. It has everything to do with making sure the library works as advertised. That is my business.
Sometimes one method in the library depends on another. If that method A is changed, method B may no longer function. Sometimes there’s more than two people involved. Developer B can change a method’s behavior in a subclass and thus break the code for developer C.
Even if security is not an issue, “Friendly collaboration” implies teams small enough for everyone to know each other and yell at each other if they break each other’s code. As I said at the beginning, “code that is only modified by the members of one small team, is relatively easy and thus uninteresting.”
Bad practices you can get away with on small teams and simple projects cause problems for large teams and complex projects.
May 31st, 2006 at 12:07 pm
Hi Elliotte, just a note that we published a news announcement to this excellent blog entry on InfoQ. I hope it brings it more visibility:
http://www.infoq.com/news/Final-should-be-default
May 31st, 2006 at 1:13 pm
“It has everything to do with making sure the library works as advertised. That is my business.”
Nothing works as advertised… except in books.
“Sometimes one method in the library depends on another. If that method A is changed, method B may no longer function. Sometimes there’s more than two people involved. Developer B can change a method’s behavior in a subclass and thus break the code for developer C.”
Good automated tests (i.e. Unit Tests) should prevent this. Tests are designed from the “User of the interface” perspective. So changes in the interface should be caught very early.
“Bad practices you can get away with on small teams and simple projects cause problems for large teams and complex projects.”
Rigid code is a bad practice. Everything should be changeable in the fast-moving world of software.
June 1st, 2006 at 2:17 am
It seems to me that the real issue here is fault isolation for a library’s public API. It’s well-known that a public API should always check its arguments (throwing IllegalArgumentException). On the other hand, a public API’s postconditions are usually not tested in the production code. Typically they’re checked by asserts in unit tests, and we assume that the library conforms to its postconditions if it passes its test suite.
However, whenever a library allows the client code to install a callback method, it’s the opposite. Then we can use a unit test to verify that the library calls the hook with the correct arguments, but for proper fault isolation, the library needs to check the hook’s postconditions when it returns. Returning from a callback method is an entry point into the library, just like a method call.
Any overridable method is potentially a callback, so this implies that the library should always check the return values of overridable methods for reasonableness. That’s enough of a burden that overridable methods probably shouldn’t be the default in a public API. Typically, the public methods should be final and the overridable methods should be protected and have default implementations that are either empty or entirely optional.
But I’m still not sure how important this is. It seems to me that there are all sorts of ways you can misuse a library that aren’t checked, such as calling a non-threadsafe class from multiple threads, or synchronizing on the wrong thing, or just hanging in the middle of a callback. Typically libraries try to check for the most common programming errors, but they don’t check for every possible programming error.
June 1st, 2006 at 7:33 am
“It seems to me that the real issue here is fault isolation for a library’s public API. ”
It seems that the real issue here is trust. It sounds like what you want is a library that will work properly no matter what a fellow programmer does. And if she ‘misuse’s the public API, or makes a mistake, you want to take 100% control of error reporting. You do not trust her to do things well. You do not trust her to debug her own problems.
If she writes her own callbacks, by passing in objects conforming to an interface or by overriding something in the library, you don’t even trust her to write her own unit tests to make sure that her own application is working properly. You want unit tests for the library that will anticipate her every move! Of course this can only be done by restricting her freedom, deprecating her from a programmer into a user.
Here’s a practical example: I am working on a Ruby project. You get truly private variables, no negotiation. But you do not get final methods. It’s better than that: you can change methods at runtime, and in fact that’s how a lot of programming works: classes from the Rails framework are actually patched in the application.
That’s what makes it possible to add version tracking to model data with a one-liner. And in my own case, I need to go one further: when one model has a reference to another model, I need to track when the reference gets out of date. I need to patch the way references work so that I can refer to a model’s ID and also its version number.
Needless to say, if the author of ActiveRecord locked his framework down, there wouldn’t be an acts_as_versioned plugin until he felt that his framework should include it. And if the author of acts_as_versioned locked his plugin down, I wouldn’t be able to created versioned_references.
June 1st, 2006 at 9:47 am
A minor disagreement:
“Bad practices you can get away with on small teams and simple projects cause problems for large teams and complex projects. ”
I would restate this as: Practices that are appropriate and cost effective on smaller teams may be inappropriate or even disastrous in larger, disconnected projects.
June 1st, 2006 at 12:22 pm
Actually, if the ActiveRecord framework were locked down, you wouldn’t be stuck, but you’d have to copy the source code and patch it to do what you want. Then it would be a very good idea to send your patches upstream, so they’re in the next release. By being able to override anything without changing the ActiveRectord source code, you avoid this beneficial communication with the original developers. They have no idea what you’re doing and won’t be able to avoid breaking your code in their next release.
June 1st, 2006 at 1:52 pm
Off Topic… sort of…
“By being able to override anything without changing the ActiveRectord source code, you avoid this beneficial communication with the original developers.”
I think there’s a slightly different culture around this. The process at the moment (Rails is still very young) is to write a plugin that “monkey patches” Rails to do what you want if it is something you want to reuse. You can share the plugin with the world. This is how “acts_as_versioned”, “acts_as_paranoid,” and “acts_as_threaded” work at the moment.
The Rails team can incorporate the ideas into Rails or leave them as plugins if it makes more sense to let developers pick and choose. DHH says that every 37signals project uses 5-6 plugins, so he clearly likes the plugin philosophy.
I think plugin is to framework as mixin is to class. With all of the same connotations and implications raised by this interesting debate about flexibility, power, safety, ease of use, &tc.
July 10th, 2006 at 7:36 am
Design by Contract, like provably correct programs, is a good concept.
However, besides the fact that perhaps half the Java developers out there aren’t interested and capable of using it correctly, DbC has a fatal flaw. That flaw is that pre-conditions, post-conditions and invariants do not have to be correct and *complete* for the program to work correctly. So, human nature being what it is, they invariably aren’t correct. I would like to see some real DbC code and see if I couldn’t find a heap of missing conditions. These holes imply that the benefits of the approach are less than perfect.
I have seen it attempted in requirements Use Cases, by good BAs, and the results aren’t encouraging. Because it’s not the primary driver, the proper attention is not given to it being correct and complete.
So, give me the flexibility of non-final methods, and as a developer I’ll take the consequences. That is, unless you can give me a water-tight DbC scheme. If you are going to suggest unit tests as a solution, then I will use the same reason for not making methods final.
I know it’s not quite the same thing, but I’ve had to create an effective subclass of java.util.Date, for practical reasons, using delegation, and it was not pretty. A non-final Date class would have been valuable there. I suspect non-final methods would provide similar benefits.
The library working as advertised is an internal concept and entirely within the capability of the library provider to guarantee with the use of DbC. There are thousands of libraries out there that “work as advertised†without using it.
I believe that the practical benefits of being able to inject, at runtime if desired, concrete classes that satisfy an interface brings far greater benefits than anything you are suggesting.
July 12th, 2006 at 6:38 pm
C++ requires you to specify a method as “virtual” if you want to allow subclasses to be able to override them.
Isn’t that pretty close to what you want?
Basically the “virtual” keyword means in the contract “be careful, derived classes might change this behaviour”.
(apart from that, the compiler can optimize better when he knows he doesn’t need to look up the actual method to be called. In fact, it can eventually inline the method.)
C++ is still a powerful language, not really obsoleted by Java or C#.
The only big drawback of C++ is it’s syntax (which in turn is inherited from C), especially the use of preprocessor macros make it a PITA to process, which makes refactoring and other source code processing “potentially impossible” (read: no tool will be totally reliable).
Oh, and of course that many people never bothered to actually learn proper C++… or where to find C++ libraries (e.g. boost) that can somewhat compete with the huge class libraries of Java and C#. It’s all there, just not in one big help file as with java and C#.
In fact, Java has been hitting some of the “library problems” that have been scaring away people from C/C++. Todays java apps often come with a collection of .jar files. Replace one, and you’re likely to break something. Many people end up having a dozen copies (or different versions) of xerces on their computer, what a waste… My system has over 100 java “library” jars installed (including xerces, xalan, lucene, junit, ant, openoffice and commons) – fortunately my linux distribution handles this dependency hell just as it does for all the other languages.
All this was just “delayed” by java having a larger standard library than C++ has. And C# in turn seems to have an even larger lib. But – if you’ve ever come to develop Java for a mobile device, you’ll be aware that not all of these libraries are available on every system…
July 12th, 2006 at 7:22 pm
Many of the objections in my previous post would be overcome it the contract formed the basis for unit testing the classes. If there were a tool in the style of Agitator that used heuristics for testing based on the Contract then that would elevate the importance of the contract, make it more complete and kill two birds with one stone.
Then it would remain to achieve the same flexibility of using interfaces with Contracts. Is it not possible to place contracts on interface definitions? Do Contracts, in practice, reply on implementation details for their operation? I suspect they must and this would be a problem.
June 20th, 2007 at 4:34 pm
All this problems arise because nowadays interfaces don’t define complete contracts.
I’m waiting for the day in which we can define post and pre-conditions inside interfaces.
June 18th, 2011 at 6:28 am
Your blog is so informative … keep up the good work!!!!
October 12th, 2015 at 7:00 pm
Language need unit testing sopuprt, yes yes yes!And if we are going to make unit testing a first-class citizen of language design, there is no reason why tests must be structured in classes and methods. IMHO tests have a fundamentally different structure than classes and objects. See the Noop link by Sakuraba above, or also how tests are structures in RSpec. Also tools such as Lint should treat test code different! What is a code smell in production might be great test code and vice versa. See the work of Stefan Reichhart from our lab ( Rule-based Assessment of Test Quality , TOOLS 2007) for a test-lint for Smalltalk.