The Nastiest Bug
There’s one trap I fall into repeatedly while doing software development of any kind: testing, debugging, coding, documenting, anything. And it happens in every language I’ve ever worked in: Java, C++, Perl, CSS, HTML, XML, etc. The only difference is how much time I waste tracking down the bug.
I have this on my mind now because I just lost at least half an hour to this while working on the CSS stylesheet for this very web site. I have had many students show up during my office hours for help with debugging this problem. I have had at least one company pay me lots of money to fix this problem for them (though they didn’t know this was their bug or they wouldn’t have needed to call me in the first place). I can virtually guarantee you’ve made this mistake too. What is the mistake?
Editing the Wrong File
The problem usually occurs when there are two files named Navigator.java, or wp-admin.css or prices.pl, or whatever. More often than not, they are different versions of the same file. Usually they’re in different directories. Maybe you’ve opened one from a backup directory, or you’ve accidentally saved the file onto your desktop instead of your src directory. Sometimes there are entirely separate copies of the source tree. (That’s what happened to me today. I was actually editing the CSS file for The Cafes, but loading the one from Mokka mit Schlag in my browser.)
This mistake is so obvious we rarely talk about it. Once you realize that’s what you’re doing, the fix is totally obvious. You feel like a bonehead, fix the problem, and move on; at least until the next time it happens. Certainly no one wants to stand up and admit they’ve done this. It is an incredibly stupid mistake, one only a bonehead could make more than once. At least it feels like that to me every time I make it, which is at least once a month.
Until you realize what you’re doing, this is the most frustrating bug imaginable. It usually happens to me during debugging or iterative development. I intend to make a small change and verify that the feature is added or the bug is fixed. I make the change and run the test. (Whether the test is manual or automated doesn’t matter here.) Nothing has changed.
“That’s funny,” I think. I look at the code again and twiddle it a little bit. Still nothing.
I try something else. Nothing.
I back out the edit and start over. Nothing.
I start digging deeper, and trying to figure out how the really obvious piece of code could possibly not be doing what it is obviously doing. Maybe I set a breakpoint; then run the code in the debugger. If I’m lucky the breakpoint isn’t reached. You think that would tell me what the problem is, but it rarely does. My usual reaction is, “Hmm, I must have accidentally hit Run instead of Debug. let’s try that again.” So I run the code in the debugger a second time. Still nothing.
At this point, I’m getting very frustrated. Sometimes I’ll drop out of my IDE to the command line and try it there. Sometimes that will actually work if the command line is picking up different files than the IDE. Sometimes it won’t.
Sooner or later (but later more than sooner) I finally realize what the problem is. I slap myself on the head and move on. The problem is fixed for the moment, but this keeps happening!
Is there any way to avoid this? I am convinced that the programming community is losing person-years of productivity to this mistake. Pair programming doesn’t help. I’ve absolutely been half of pairs where both pairs of eyes were totally focused on looking for bugs in the wrong file. IDEs don’t help. The fact that they often need their own copy of the source tree makes the problem worse rather than better. Ditto for source code control systems. The problem is at its worst in server side environments because the need to deploy the compiled archive introduces another step where the wrong file can replace the right one. Worse yet, the symptoms of this bug in a server side environment look very much like a failure to reload a compiled class or restart the server, even if that’s not what’s happening at all.
This is such an incredibly dumb bug there must be a way to prevent it, but for the life of me I can’t think of one. Ideas?
March 2nd, 2006 at 7:55 pm
Yeah, we all do this. There’s a few things I do to try to keep this from happening.
First, try not to name two files the same thing if you can help it. Sometimes you can’t, but if you can, do it. This isn’t just good for preventing this particular mistake, but it’s also easier on your brain, because it makes switching between mental contexts quicker and cleaner. You don’t have to go through that same few seconds of confusion when you switch between files.
Also, I try not to keep backup copies around. If I do, they are zipped up, simple so I can’t accidentally navigate into them. Again, this isn’t always avoidable. Other benefits are that you’ll have less clutter, which again, is easier on your brain.
March 3rd, 2006 at 9:10 am
Some of this could be prevented by better integration between IDEs, project management, and source control tools. I’ve been bitten by this because of the need to develop different versions of the same codebase at the same time. Better integration between those tools would be a good thing anyway.
March 3rd, 2006 at 10:10 am
The answer is simple: the first step in debugging should always be to verify that you are editing the proper file.
Add a line that writes to the console/screen/whatever.
If it doesn’t work, consider where the correct source file may actually live.
If it does work, move on and work on the bug.
March 3rd, 2006 at 12:26 pm
Yes, but once you have a suspicion you are editing the wrong file it would be nice if the output from the execution could tell you where the real file is. Like each piece of the output had a super-tooltip that could tell you the provenance of the data and the instructions that produced it. Awesome!
March 3rd, 2006 at 7:28 pm
The main reason I see for this is editors insist on only showing the short form of the filename, not the full path. The bug is (more) obvious if you can see the full file name with path.
I’ve had a bit better luck with this bug in Eclipse when “sync editor with navigator” is on. When you’re editing the version from your build directory (doh!), it doesn’t have the right icon in the navigator. It’s subtle, though. It’d be nice if there was a line right below the editor tabs that showed the full path of the file you’re editing, instead of having to hover over the tab, or a warning “you’re editing a file that isn’t on your source path… Are you sure?”
March 3rd, 2006 at 9:40 pm
gnu emacs (and other emacsen?) offers some help in preventing this problem: files of the same name are suffixed with according to the order they were opened. This is almost an effective solution for me … except the original file name is not updated to include a suffix. So when editing the original file there’s no indication its one of a bunch.
I wish new IDE’s would adopt this convention including a suffix for the original file when needed.
March 3rd, 2006 at 9:45 pm
sorry for the dupe – I didn’t escape the <pointy brackets> in the first post.
gnu emacs (and other emacsen?) offers some help in preventing this problem: files of the same name are suffixed with <n> according to the order they were opened. This is almost an effective solution for me … except the original file name is not updated to include a <1> suffix. So when editing the original file there’s no indication its one of a bunch.
I wish new IDE’s would adopt this convention including a <1> suffix for the original file when needed.
March 4th, 2006 at 4:29 am
You make a good point. I think this falls into two categories:
Different versions of the same file
Different files with the same name
The first is the case where you have a backup file somewhere (and that’s the one you’re editing). The solution, as you rightly point out, is to only have one copy of a file. Fortunately, that’s pretty easy to manage; don’t have backup files. Use a version control system (e.g. subversion) instead. That way, your files are always the version control system and there’s only one file available. When you’ve finished with updates (and of course, checked it in!) then delete the file from disk. It’s not necessary to keep it around, and you can get it out again afterwards.
More importantly, don’t let live systems read from files on the file system; get them out of version control. This might be accomplished by having a version checked out on a regular basis (with a CVS tag so you don’t have to use HEAD; I use STABLE, for example). Alternatively, if you’re using subversion, you can even mount the version control system via a WebDAV share (simple enough on most Unix operating systems, and to a lesser extent, even ‘doze boxes). Of course, you’d want the version control system to be local to the server to make it fast, and this presupposes that you’re sending source directly to the server and not needing an intermediary compile step.
Secondly, files with the same name but different projects. This often crops up with generic files (like ‘style.css’) and you’re not sure what you’re editing. There seems to be two ways of handling this; either give each project’s file its own name (e.g. styleCafeAuLait.org), or have a ‘common’ set of files that are checked into source control that you pull out into your project. I guess if you’re editing in an IDE, one other approach is to close all projects that you aren’t working on, which has the effect of not allowing you to edit those files (multiple projects open with ‘.classpath’ will show up in Open Resource in Eclipse, for example). I don’t believe the project set filters do that.
If you’re doing a lot of development work in Eclipse, you can have separate workspace. I’d recommend that you have a different workspace for each different major project/system that you’re working on. That way, when it starts up, you know you’re in ~/CafeAuLaitWorkspace instead of ~/CafeConLecheWorkspace and you’ll only see projects related to that that you’ve already checked out.
In a nutshell; version control is probably the way to keep them separate, and use the partitioning mechanisms in your IDE/editor to keep unrelated projects apart (or better still, delete after synchronizing them or close them when you’re not working on them). Neither of these will help if you have multiple copies of ‘style.css’ in the same project though.
March 4th, 2006 at 10:22 am
“Like each piece of the output had a super-tooltip that could tell you the provenance of the data and the instructions that produced it. Awesome!”
If you would actually be in awe of a tool-tip, well, you need to get out more.
March 4th, 2006 at 4:54 pm
I find that particular problematic with resource files which are either in the current working directory, in a special resource directory, or in the classpath (using Java). Often there are different versions, and you are guaranteed to edit the wrong one. The problem then is that it’s a data file which cannot print off it’s location, as it is read in by some class which you might not even have direct access to…
Or compiling a file and having the class-file in the wrong place so that you edit the right file, but execute the wrong one.
Perhaps you could have a FileReader class debug version which prints every filename it opens, which in the production version is disabled. Doesn’t help with classpath-searches though 🙁
March 7th, 2006 at 11:43 am
I often have this problem in Eclipse, not when dealing with two files of the same name, but when copying one file to another with a different name. I typically then blindly go to edit the copied file, and only realize a bit later that Eclipse didn’t automatically open the copied file, but rather left me at the editor for the old file. Thus, I’ve been editing the file I’m copying from, not the one I want to edit.
I couldn’t count how many times I’ve done this, but I’m *starting* to get smart to it and check which file I’m on before I start.
March 7th, 2006 at 3:57 pm
[…] to httpd.conf. Again the installer should have done this, but it didn’t. Believe it or not, it took me longer than it should have to figure out the problem because of yet another variation of the nastiest bug. This time, I had httpd.conf files from two different web servers open in several different terminal windows, and I checked for the AddType line in what I thought was the new server but was in fact the old server. […]
March 9th, 2006 at 12:59 pm
Stephen was onto the answer: Have the editing window itself show more of the pathname.
I disagree that full pathname is necessary, or even useful. Big long strings of similar-looking noise all look alike. What you want to see is the DIFFERENCES WITH A DISTINCTION.
On Mac OS X, one example of this is Property List Editor. A significant fraction of the files you edit with it will be named “Info.plist”, located in a “Contents” directory, which would make a recent-files list extraordinarily useless.
So what PLE does is show the “Info.plist” name followed by ONLY the unambiguous part of its actual location, which is usually the enclosing “Xyzzy.app” app-bundle folder, or whatever the nearest actual enclosing folder is.
For example, I have 9 “Info.plist” items, 4 of them from a .app context, 3 from a .SpeechVoice bundle, and 2 from a .SpeechSynthesizer bundle.
So I say the solution is to show JUST ENOUGH context, with JUST ENOUGH difference, that a glance alone will tell the user what’s going on.
Sadly, not even Property List Editor does this in the editing-window’s title-bar, where it would be even more useful. However, all its windows support the cmd-click to reveal a popupmenu of the entire path to the file. Still, this is not good enough, because it’s not clear enough AT A GLANCE. If I always had the discipline or presence of mind to cmd-click before editing, then there isn’t even a problem. It’s precisely because I’m NOT disciplined and error-free that I need the context at a glance.
March 29th, 2006 at 2:45 pm
This is a problem. IntelliJ IDEA helps a little by using different icons and colors on a file’s editor tab. E.g., if you open the copy of messages.properties deployed to your web app, its name is brown instead of black because it isn’t under version control. Nevertheless, it’s still too easy to open the wrong one.
Cheers,
11011011