Over long distances, airplanes are the fastest way to travel (at least until we invent teleportation) but over short distances that isn’t true. If you want to get from La Guardia Airport to O’Hare, take a plane; but if you want to go from La Guardia to JFK, take a cab. Over shorter distances still, a bicycle may beat a car; and over the shortest distances, the fastest way to get somewhere is usually walking. Terry Pratchett explained it well in Interesting Times:
Of the three things that most people know about the horse, the third is that over a short distance, it can’t run as fast as a man. As Rincewind had learned to his advantage, it has more legs to sort out.
You naturally choose your mode of transport according to the distance you plan to travel. A few hundred meters or less, walk. A kilometer or two, take a horse (or more likely these days, a bicycle). 10K to a few hundred kilometers, take a car. And beyond a few hundred kilometers, airplane is the fastest choice; and the longer the journey the better a choice it becomes.
Programming is much the same.Short, simple projects should walk. That is, they should be done in the shortest, simplest fashion imaginable. This will get the answer the soonest. The programming equivalent of walking is writing a quick throwaway program that produces one answer, and which is then never run again. Probably this works up to about 100 lines of code. (This does tend to favor languages like Python or Ruby that get more done with fewer lines of code.)
However, most journeys aren’t short enough to walk; and most programs aren’t simple enough to write like that. Longer programs–in particular programs too long to keep on one screen or keep in your head in their entirety at one time–need more powerful tools and techniques. Which tools and techniques they need depends on their size and their complexity. Up to about 1,000 lines of code, the only tool you need may be structured programming and adherence to naming conventions to make the code look good.
As programs head past 1,000 lines of code, that’s no longer enough to achieve quality software. This is where you need to begin organizing programs into larger modules: classes and perhaps packages in most languages. It also becomes important here to document your code well, to spell words properly, and to take greater care with your typing.
Jump another order of magnitude to 10,000 lines of code or more, and that’s no longer enough. Just keeping the classes straight in your head can be a problem. The next thing you need is unit testing and test first development. On large enough, complex enough projects, test driven development will reach the goal faster despite having to write additional code to support the tests, just as traveling by car enables you to visit your aunt on the other side of town sooner, despite hunting for your keys and stopping for gas on the way. The larger and more complex the project, the more important this is.
Past 10,000 lines of code other tools and techniques are necessary too. Source code control of some kind becomes a necessity. It will take significant time to set up the repository and some time to manage it. Think of this annoying process as arriving at the airport two hours early to go through the airport security line. Despite this delay, traveling by air still enables you to reach far away cities sooner. Again, the larger and more complex the project, the more important source code control is. However for short projects, it may not be needed.
As projects grow, other tools and techniques gradually shift from hindrances to neutral to necessities. Design patterns, continuous integration, automated bug tracking, design documentation, and more are more necessary the larger and more complex a project is. However they aren’t necessary for all projects. On smaller projects, they may actually slow developers down. I’ve seen more than one simple project bogged down by an insistence on using design patterns and techniques really only suitable for much large projects. Abstract Factory and Factory Method are especially commonly overused techniques. (Hint: if there aren’t at least two independent implementations of each interface, then you don’t need a factory. It’s like driving the family SUV to your neighbor’s house across the street.)
The question then becomes what’s small and what’s large? How many projects are of which sizes? Where are the inflection points where each tool or technique begins to pay back its initial cost?
I don’t know the exact answers, but I suspect it’s not a linear distribution. There are relatively few destinations. Past 100,000 lines of code, you really need to use every tool in your toolbox to have any hope of producing quality software. Anything you omit will hurt you.
Problems arise, though, when we attempt to apply lessons learned at the wrong scale. For instance, many computer science students work primarily or only on small, simple textbook examples and problems where well commented code, naming conventions, unit testing and so forth merely get in their way. They’re just check marks from the grader, not something they need to make the code work. Consequently they learn the wrong lesson; and too many get out into the world and never learn the right one.
It’s like noticing that you walked to the corner store faster than you could drive, and therefore deciding to walk to Tierra del Fuego. Sure you could do it, and you could develop a million-lines-of-code enterprise system without source code control or unit tests–but it’s going to take you hundreds of times longer, and require a lot more pain and effort.