Several things concern me though. One is syntax. I really, really want the syntax to be simple and have no surprises or significant learning curve. I want existing Java programmers to be able to read Java code that uses closures (or first class functions) and immediately be able to see what the code is doing without any prior training in the new syntax. I want the syntax to be that clear. If we can’t have that, I don’t want to do it. I want Java to still be a lingua franca.

Secondly I want closures to be really, really easy to write. No gotchas. A little training may be allowed here, but there shouldn’t be any special cases or surprises. Programmers’ first instincts of how to write something should be correct. In particular, I do not want to see any issues like “the closure works if the external variables are declared final, but fails if they’re not.” Or, “the closure works if the external variables are not modified, but stops compiling if a line is added that modifies one.” That is, implicit final is not a workaround. The only things that break the closure should be:

1. Syntax error inside the closure itself
2. Breakage in the closure syntax (i.e. leaving off a trailing curly brace or some such in the closure definition)
3. Removing or renaming an external variable the closure depends on.

In other words, the only things that should cause a closure to fail to compile or run in the expected fashion should be exactly those things that cause today’s code to fail to compile or run. If you rename a local variable from x to y and forget to rewrite the statement foo(x);, then the compiler complains. This is a common bug we all know how to handle when it happens to us. I want to make sure that adding closures does not add any new failure types or patterns.

Those are my requirements for any proposal to introduce closures or first class functions into the Java language. I am prepared to reject any proposal that does not meet these two criteria. If no proposal can satisfy these two criteria, then we shouldn’t add closures or first class functions.

So what are the proposals on the table, and how well do they meet these criteria?

CICE

CICE has a number of extra rules:

“The construct is legal only if ClassOrInterfaceType represents a class or interface type with a single abstract method (a SAM type).” In other words, add a new method to an interface and the closure breaks. That’s not a problem because it’s already the case that adding a method to an interface breaks existing implementations. However, it’s a problem that we can only use closures with single-method interfaces. That’s exactly the sort of gotcha I want to avoid that will trip up people writing closures.

“Any visible local variable that is initialized or assigned exactly once in the enclosing scope, as well as any visible parameter to an enclosing method that is never otherwise assigned, is accessible but not assignable within the body of the CICE, whether or not it is explicitly qualified as final.” In other words, adding a line that reassigns a variable, either in the closure or outside of it, can unexpectedly break things. This is a big problem. This is exactly the sort of gotcha I want to avoid.

“Any local variable that is explicitly qualified as public is accessible, and also assignable (unless also qualified as final), within the body of the CICE. Formal parameters and for-loop variables may not be qualified as public.” OK. We have to explicitly mark the variables we want to use in the closure. This seems like a slightly weaker version of only being able to use final variables, though why everything isn’t public by default I’m not sure. Furthermore the prohibition on public formal parameters (I think this means method arguments) and for loop indices seems unnecessarily confusing and gotcha prone.

There’s a second gotcha with this. The word public is being overridden to means something quite different from its usual meaning. These variables are not visible from outside the class. I’m sure this is being done to avoid introducing a new keyword, but it’s still added complexity for the language.

“Access to any other local variable or parameter from an enclosing scope is illegal within the body of a CICE.” It’s just not going to be obvious to people why they can use some variables in closures and not others. I think I’m coming to the conclusion that implicit final is a very bad thing, and a strong negative for any closure proposal. (Actually I’m coming around to the idea that implicit anything is a very bad idea, or at the least a code smell that warrants a second look.)

“It is, in many cases, technically feasible to infer types from the formal parameter list and method body of a CICE. It is worth exploring the pros and cons of doing so.” I’m not sure exactly what’s being proposed here, but it sounds like more dangerous implicit code that will surprise and confuse. Java’s a strongly, statically typed language. Let’s keep it that way. Making one small part of it implicitly typed is just inconsistent and confusing.

BGGA

How does the BGGA proposal satisfy or fail these criteria? First I think it fails the syntax test. It is uglier than the CICE proposal, and I have the same reaction to it I first had to generics and Ruby: I don’t understand it. Despite ten+ years of Java experience I cannot just look at the code and figure out what it is doing. I’m sure I’d get used to it over time but what about people who are not full-time Java programmers? It is on the one hand too concise, preferring large amounts of punctuation over keywords. On the other hand it in effect introduces an infinite number of new keywords to the language. Reading code that uses this seems likely to be too complex.

What about gotchas? Here I think BGGA comes out on top. Once you decipher the syntax, there appear to be fewer special cases to worry about than with CICE. It is a more complete, cleaner proposal, though likely many gotchas won’t become apparent until I actually start to try writing code with this. A few things do bother me though.

First of all, I’m not sure I like that return returns from the enclosing scope rather than from the closure itself. That’s a surprise.

I also don’t like the implicit return at the end of the closure to get the value out, or that there’s no way to yield that value early. That’s a surprise too. It also makes the code harder to read. Either return should return from the closure, not the enclosing scope, or there needs to be a new yield keyword that returns from the closure.

“At runtime, if a break statement is executed that would transfer control out of a statement that is no longer executing, or is executing in another thread, the VM throws a new unchecked exception, UnmatchedNonlocalTransfer.” This is unnecessary. I think this is more like returning a value from a method when that value is ignored and not stored. This should not cause an exception. The same is true for continue and return. The closure should silently exit when it complete and the enclosing scope no longer exists. Anything else violates the principle of least surprise, and exceptions are always surprising. It’s not as if the programmer actually needs to be told that the enclosing scope no longer exists, any more than they need to be told that the variable they’re returning from a method will be ignored. That’s an issue for the calling code, not the called code.

Like CICE, BGGA closure conversions only work with single method interfaces. That’s a gotcha.

Bottom Line

Both major current proposals flunk my criteria. If I’m forced to choose between them today, I choose neither: do not add closures to the Java language.

However, some of the problems may be fixable. In particular removing implicit final from CICE eliminates a lot of my objections there. Removing public local variables eliminates a few more. Is what’s left still useful and worth doing? I’m not sure.

The BGGA proposal is trickier to judge. I’d really like to see a fully written out version of Neal Gafter’s talks on the subject. I’ve watched his video twice now, but it’s just not as easy to dig into video as text. Rather than writing the spec first, I think you need to write some code that uses the proposal. That’s the only way to see if the syntax makes sense.

Listing 1. The simplest possible closure

3.times {puts "Inside the times method."} Results: Inside the times method. Inside the times method. Inside the times method.

times is a method on the 3 object. It executes the code in the closure three times. {puts "Inside the times method."} is the closure. It’s an anonymous function that’s passed into the times method and prints a static sentence. This code is tighter and simpler than the alternative with a for loop, shown in Listing 2:

Listing 2: Looping without closures

for i in 1..3 puts "Inside the times method." end

Personally I find the latter example simpler, clearer, and easier to understand. For one thing, when I see the word times suffixed to a number like 3 I expect multiplication, not action. But even if times were changed to a more reasonable method name such as do or act, I’d still prefer the for loop. (Perhaps what you really want here is “do 3 times”. That might really be clearer. )

I don’t know what it is some people have against for loops that they’re so eager to get rid of them. This isn’t the first or even the second time CS theorists have revolted against for loops (or equivalent). One advantage cited for RATFOR over traditional Fortran-77 was the ability to use index-less while loops instead of DO loops (Fortran’s equivalent of for). The Java Collections API brought us a confusing mass of iterators with weird modification behavior to avoid having to do something as simple and obvious as indexing our walks through a list. Then in Java 5 this wasn’t good enough. Some people were still ignoring iterators and stubbornly persisting in indexing their list traversals, so we got a whole new indexless for ( String arg : args ) syntax.

What about a simple indexed loop offends people so much that they invent massive, complex syntax just to avoid it? Personally I find the indexed for loop syntax to be familiar and comforting. That’s why I deliberately designed XOM to support indexed access to the various components of the document tree. No iterators. No fancy loops. Just plain, old

for (int i = 0; i < element.getChildCount() ; i++) {
  Node child = element.getChild(i);
  //...
}

The times syntax does avoid the explicit declaration and creation of a loop variable, though I’m sure that’s still happening behind the scenes and there’s no performance difference. Still, it is nice that you don’t have the variable getting in your way if you don’t want it. Certainly in C and even in Java programmers are always enbugging their code with wrongly scoped loop indices or fencepost errors. This makes indexless loops a nice feature for a language that has it from the get-go like Ruby. However I don’t think this is a big enough advantage to justify changing Java.

A much more serious concern is that indexless loops don’t have to be serial. That is, there’s nothing in a statement like 3.times {doSomething()} that promises any particular order of execution. In fact, just maybe we can do all three actions at the same time. This enables parallel processing, and is going to be very important as multicore processors and multi-CPU systems become even more common. For example, consider the code to sum an array:

double sum = 0;
for (int i = 0; i < array.length; i++) {
  sum += array[i];
}

The programmer probably doesn't care (and certainly shouldn't care) that we add array[0] to the sum before array[5]. If the array is large, we could actually execute this on eight separate CPUs, each summing an eighth of the array, and then add the subtotals at the end. A smart compiler could figure this out, but it's easier to do that if there's nothing in the code that refers to the loop index.

The for syntax implies serialization where you may not need it. The closure syntax doesn't necessarily guarantee the order of execution of the various statements. However, sometimes you actually do need a particular order of execution, or you need to refer to the loop index from within or outside of the code. For example, consider this simple loop that concatenates an array of strings named args:

String s = "";
for (int i = 0; i < args.length; i++) {
  s += args[i];
}

String concatenation is not commutative, and it's really important that we add the strings in the proper order. A really smart compiler might still break this up into multiple threads. but it would have to be a lot more careful that the intended order was preserved.

Thus the closure syntax and the for syntax really aren't equivalent and closures can't replace for loops. They might supplement them, but this is only relevant if they really can be run on multiple processors simultaneously. In functional systems, this works because there are no side-effects. Thread safety is almost free. However, this isn't true in Java. In Java you have to think very carefully about thread safety, and typical closure code doesn't do that. Unless we have true functional programming, I'm not sure I see the point.

The current proposals for closures in Java all seem to still have sequential execution of code. For instance, the BGGA proposal makes a big point out of allowing break and continue inside closures, but what does that mean if the different iterations of the loop are in fact running on different processors at the same time? If the code is going to be sequential anyway, I prefer the style that makes that more obvious. The traditional indexed loop does that. A closure doesn't.

When I need to teach cross-platform subjects like DOM or data structures to mixed language audiences, I can easily use Java to show examples and everyone can follow them, even if they’re not Java programmers. Everyone reads at least pidgin Java. Only C++ programmers can read C++. Only Ruby programmers can read Ruby. (Still better than Perl though. No one can read Perl, Perl programmers included.)

Of course, if you’re traveling in one area, it’s better to speak the native language if you can. French gets you farther in Lyons than English, but if you’re at an international conference English is your only hope. If you’re at RubyConf, Ruby will get you far. However if you’re at OOPSLA or Software Development, Java is the way to go.

Java is the programmer’s lingua franca.

P.S. The Call for Papers for Software Development Best Practices 2007 has just been posted. Aside from a token C++ track, it’s a pretty language neutral conference; and we don’t normally bother to check presenters’ preferred programming language when deciding which presentations to accept. However, if you are showing code in a talk, I do recommend that you show it in Java.

For example, here is the Java 5 code to start a thread whose run method
invokes a method named foo:

    new Thread(new Runnable() {
        public void run() {
            foo();
        }
    }).start();

If we adopt this proposal, the following code would be equivalent:

    new Thread(Runnable(){ foo(); }).start();

Here is the Java 5 code to sort a list of strings by length (from shortest to
longest):

    List<String> ls = ... ;
   Collections.sort(ls, new Comparator<String>() {
        public int compare(String s1, String s2) {<
            return s1.length() - s2.length();
        }
    });

Here is the same code rewritten using the proposed syntax:

    List<String> ls = ... ;
    Collections.sort(ls,
        Comparator<String>(String s1, String s2){ return s1.length() - s2.length(); });

Which is more legible to you? To me, the answer’s obvious: the anonymous inner class version that’s nicely indented. It’s not an ideal solution, but it’s better than the CICE alternative. The CICE alternative is a confusing mishmash of brackets, braces, and parentheses. Frankly, it’s starting to look like Perl. It’s shorter than the traditional approach, but that’s not a good thing. The redundancy and white space of the current syntax helps us read it, understand what’s going on, and makes it more legible. The compiler doesn’t need it, but we humans do. The CICE proposal lets the compiler fill in too much. The more that is implied by the code, the more the compiler fills in, the harder the code becomes to understand. Indeed the only way to explain or teach this stuff is to rewrite the example using traditional syntax, and tell students what’s the compiler is really doing is that.

We’ve already seen the confusion caused by the compiler writing code for us. Since Java 1.0, the compiler has helpfully filled in default constructors and default calls to super(), and this is a much simpler case. That’s probably a net benefit, but barely at best. Problems with default constructors trip up almost every student learning Java eventually; often more than once. Explicit code is good. Implicit code is bad. Explicit code you can read. Implicit code you have to remember. Reading is easy. Remembering is hard.

The second thing I don’t like about CICE is that it proposes a lot of weird rules about defaults. For instance, sometimes local variables will be final by default and sometimes they won’t. They’ll be final if they can be; that is if nothing assigns to them more than once. Then they can be used inside this new CICE expression. Otherwise they won’t be. However that’s going to cause massive debugging pain. Everything will be working fine, and then you add a simple statement like x = x*2; and suddenly the compiler starts spewing weird error messages that have nothing obvious to do with what you changed. I’m sure Bloch and Lea can figure this out in two minutes when it happens to them. I’m equally sure that my students and other not-as-expert programmers are going to waste many hours on this problem.

Why I Don’t Like Anonymous Inner Classes

It’s worth noting that my objections are grounded a little further back. I don’t really like anonymous inner classes all that much, and frankly I’m not that fond of regular inner classes either. They’re a constant source of annoyance because they never work quite like I expect due to all those silly rules about final local variables, this inside inner classes, and other arcana I can never quite remember. At this point I’ve given up on getting it right the first time. I just write the code I want to write, then debug the inevitable compiler error messages. I’ve gotten pretty good at that, but it’s still an annoyance.

When I teach Java, I tell my students not use inner classes of any kind; and indeed I forbid them from doing so on some of their homeworks. I find that if I let them use inner classes they just become a crutch to replace global variables. The students never learn how to use methods to pass information and commands back and forth between objects. They just touch all the private parts directly, and they don’t know or care that behind the scenes the compiler is generating getters and setters. I do teach them what inner classes and anonymous inner classes look like, because they’ll have to read code other people have written using inner classes. However I recommend that they don’t use them themselves.

CICE is going the wrong way. Rather than removing the things that cause the confusion (inner classes) it’s adding more of them, and worse yet hiding them under the carpet where only the compiler sees them. I want code that’s more explicit and obvious, not less.

CICE is not syntax sugar. CICE is syntax cayenne pepper. It takes one of the ugly, confusing (and unnecessary) parts of the Java language and makes it even uglier and more confusing. Possibly real closures might be better, and might indeed remove a lot of the ugliness that is inner classes. However CICE doesn’t do that. CICE makes matters worse, not better.