I Can Outrun a 767
Over long distances, airplanes are the fastest way to travel (at least until we invent teleportation) but over short distances that isn’t true. If you want to get from La Guardia Airport to O’Hare, take a plane; but if you want to go from La Guardia to JFK, take a cab. Over shorter distances still, a bicycle may beat a car; and over the shortest distances, the fastest way to get somewhere is usually walking. Terry Pratchett explained it well in Interesting Times:
Of the three things that most people know about the horse, the third is that over a short distance, it can’t run as fast as a man. As Rincewind had learned to his advantage, it has more legs to sort out.
You naturally choose your mode of transport according to the distance you plan to travel. A few hundred meters or less, walk. A kilometer or two, take a horse (or more likely these days, a bicycle). 10K to a few hundred kilometers, take a car. And beyond a few hundred kilometers, airplane is the fastest choice; and the longer the journey the better a choice it becomes.
Programming is much the same.Short, simple projects should walk. That is, they should be done in the shortest, simplest fashion imaginable. This will get the answer the soonest. The programming equivalent of walking is writing a quick throwaway program that produces one answer, and which is then never run again. Probably this works up to about 100 lines of code. (This does tend to favor languages like Python or Ruby that get more done with fewer lines of code.)
However, most journeys aren’t short enough to walk; and most programs aren’t simple enough to write like that. Longer programs–in particular programs too long to keep on one screen or keep in your head in their entirety at one time–need more powerful tools and techniques. Which tools and techniques they need depends on their size and their complexity. Up to about 1,000 lines of code, the only tool you need may be structured programming and adherence to naming conventions to make the code look good.
As programs head past 1,000 lines of code, that’s no longer enough to achieve quality software. This is where you need to begin organizing programs into larger modules: classes and perhaps packages in most languages. It also becomes important here to document your code well, to spell words properly, and to take greater care with your typing.
Jump another order of magnitude to 10,000 lines of code or more, and that’s no longer enough. Just keeping the classes straight in your head can be a problem. The next thing you need is unit testing and test first development. On large enough, complex enough projects, test driven development will reach the goal faster despite having to write additional code to support the tests, just as traveling by car enables you to visit your aunt on the other side of town sooner, despite hunting for your keys and stopping for gas on the way. The larger and more complex the project, the more important this is.
Past 10,000 lines of code other tools and techniques are necessary too. Source code control of some kind becomes a necessity. It will take significant time to set up the repository and some time to manage it. Think of this annoying process as arriving at the airport two hours early to go through the airport security line. Despite this delay, traveling by air still enables you to reach far away cities sooner. Again, the larger and more complex the project, the more important source code control is. However for short projects, it may not be needed.
As projects grow, other tools and techniques gradually shift from hindrances to neutral to necessities. Design patterns, continuous integration, automated bug tracking, design documentation, and more are more necessary the larger and more complex a project is. However they aren’t necessary for all projects. On smaller projects, they may actually slow developers down. I’ve seen more than one simple project bogged down by an insistence on using design patterns and techniques really only suitable for much large projects. Abstract Factory and Factory Method are especially commonly overused techniques. (Hint: if there aren’t at least two independent implementations of each interface, then you don’t need a factory. It’s like driving the family SUV to your neighbor’s house across the street.)
The question then becomes what’s small and what’s large? How many projects are of which sizes? Where are the inflection points where each tool or technique begins to pay back its initial cost?
I don’t know the exact answers, but I suspect it’s not a linear distribution. There are relatively few destinations. Past 100,000 lines of code, you really need to use every tool in your toolbox to have any hope of producing quality software. Anything you omit will hurt you.
Problems arise, though, when we attempt to apply lessons learned at the wrong scale. For instance, many computer science students work primarily or only on small, simple textbook examples and problems where well commented code, naming conventions, unit testing and so forth merely get in their way. They’re just check marks from the grader, not something they need to make the code work. Consequently they learn the wrong lesson; and too many get out into the world and never learn the right one.
It’s like noticing that you walked to the corner store faster than you could drive, and therefore deciding to walk to Tierra del Fuego. Sure you could do it, and you could develop a million-lines-of-code enterprise system without source code control or unit tests–but it’s going to take you hundreds of times longer, and require a lot more pain and effort.
October 31st, 2006 at 2:49 pm
Over long distances, a horse can’t run as fast as a man either. Humans are cursorial hunters; a man in top condition can catch up to a horse in top condition by running after it until it drops dead of heat exhaustion. This is probably why we are able to even think about writing 10,000-line programs.
October 31st, 2006 at 3:46 pm
I can believe that of a thoroughbred overbred for short races, but is that true for a more normal working horse like cowboys ride?
October 31st, 2006 at 6:53 pm
Humans really get big advantages over quadrupedal mammals (including horses) when we walk – because of our somewhat unique gait (straightish legs and pendula leg motion): so yes a human, even one that is not so fit can walk further than a horse can either run or walk – we can keep going after the horse drops from exhaustion. Neill Alexander (Leeds University, UK) has published a lot work on biomechanics if you’re interested.
November 1st, 2006 at 7:34 am
This is a common discussion topic for small planes as well. I own a very slow (~120 knots), single-engine Piper Warrior II. If I fly from Ottawa to Montreal, my total flight time (including taxiing) is the same as a 767, and my door-to-door time is probably an hour more than driving either way. If I fly from Ottawa to New York City, my total flight time is about 45 minutes more than the airlines, but my door-to-door time is still at least an hour less, because I don’t have to arrive couple of hours early to have my shoes x-rayed. I can also beat the airlines on longer trips when the trip is not hub-to-hub, because I don’t get the three-hour layovers in Pittsburgh, and in cities like Toronto, I can choose a small airport close to downtown instead of landing far, far out in the suburbs.
November 1st, 2006 at 9:06 am
Excellent article, Elliote. I usually am not a fan of stretched analogies, but in this case it is spot-on. Your approach is much more reasonable than the “if you don’t develop using concept XYZ, then you are doing the project wrong.” Now if only I could get those pointless factory classes out of our project without offending the pattern-addicts!
November 1st, 2006 at 12:53 pm
I generally agree with your message, but I wish it were expressed in terms a customer could relate to, rather than a developer. How will I know how many lines of code it will be before I commit to a development approach? (Admittedly, powers of ten are pretty large buckets.) Where I work, we tried to use Function Point Analysis to determine the size of a program based on the business requirements (number of tables in DB, number of fields in records, etc.) but the discipline to follow through wasn’t there.
November 2nd, 2006 at 10:55 am
[…] Elliotte Rusty Harold seems to be on a roll these past few days: I Can Outrun a 767 […]
November 2nd, 2006 at 2:46 pm
Learning advanced techniques for large programs while writing small programs makes sense in school as long as it is pointed out that the student is learning a technique and that it will not make sense to apply it in the workplace for small programs. School IS where you are introduced to advanced techniques, though the need to cover many of them in a short time doesn’t allow their application where they would be the most useful.
November 3rd, 2006 at 3:53 am
I liked the analogy, too. E.g., I usually take my bike to work, it takes 4 minutes down and 6 up, while it takes at least 10 to drive and 17 to walk. However, there’s more to a travel than distance. A couple of days ago, we experienced the season’s first snow, and before I have changed the tyres (on both bike and car) it’s better to walk. When there’s a lot of snow and I have to wear a thick jacket, going uphill home is too exhausting, so I prefer walking. On longer travels in the terrain, skis are fastest (and often the only possibility). The fastest way of travelling by “foot” is using skates, but of course that requires a frozen and smooth river.
OK, so what about the analogy, in other words, what is weather, terrain and surface in the software development world? Perhaps surface could translate to technology, e.g. mud is legacy code, a frozen river is a homogenous environment for which one very suitable technology exists, like Fortran for numeric code, and the outbacks represent a mixed environment? Perhaps the weather could translate to organisational/human aspects, with heavy snow meaning a lot of resistance and little possibility to see far ahead.
The point isn’t to make a perfect fit, but to see that distance isn’t the only important characteristic of a travel. The distance aspect leads to heuristics like “if you don’t know how long the journey will be, you should prepare for a longer one than what you expect”. However, including weather, surface and terrain in the analogy leads to a richer set of problems. If it’s realistic that it will both be rainy, snowy, windy, freezy etc. in the same project, how should you plan? E.g. we always pack extra food and bring a showel on trips in the snowy, Norwegian mountains, because a heavy snowfall could prevent us from going straight home. How far you can see from the start, how well you know the conditions (I’ve seen women wearing high-healed shoes in the Grand Canyon), how much change you may expect during the travel, may be just as important as the travel distance.
November 8th, 2006 at 6:33 am
As soon as unit testing enters a project there’s (or at least should be) two independent implementations of each interface …
November 27th, 2006 at 5:19 pm
Actually the distribution of projects by size is in my experience very far away from being linear. The vast majority of projects are small. The Microsoft empire is built on this.
I disagree that the usefulness of source control is related to project size. It is more related to project duration (that bug has been in for half a year and nobody noticed).
January 30th, 2007 at 9:39 am
Sweet 🙂
April 9th, 2007 at 4:46 am
“It’s like noticing that you walked to the corner store faster than you could drive, and therefore deciding to walk to Tierra del Fuego. Sure you could do it, and you could develop a million-lines-of-code enterprise system without source code control or unit tests–but it’s going to take you hundreds of times longer, and require a lot more pain and effort.”
Excellent!
April 26th, 2007 at 4:48 pm
I can believe that of a thoroughbred overbred for short races, but is that true for a more normal working horse like cowboys ride?
August 23rd, 2007 at 7:10 am
I have an interesting anecdote from a late friend of mine. Apparently, during the war they were loading bombs onto a plane. Alas one of them dropped onto the tarmac. Everybody fled the scene post haste. My friend was flat out in a jeep when someone overtook him on a bicycle!
Regards David
March 5th, 2008 at 6:53 pm
OK, so what about the analogy, in other words, what is weather, terrain and surface in the software development world? Perhaps surface could translate to technology, e.g. mud is legacy code, a frozen river is a homogenous environment for which one very suitable technology exists, like Fortran for numeric code, and the outbacks represent a mixed environment? Perhaps the weather could translate to organisational/human aspects, with heavy snow meaning a lot of resistance and little possibility to see far ahead.
December 20th, 2009 at 2:45 pm
Excellent work. You have gained a new subscriber. I hope you keep up the good work and I eagerly await more of the absorbing posts.
February 7th, 2012 at 4:21 am
[…] This one is great!! Yes you can ‘outrun-a-767′ You can’t get a picture, when you’re inside the frame. […]