Welcome to The Cafes
Welcome to The Cafes, a new site from Elliotte Rusty Harold for content that’s longer than a blog posting and shorter than a book.
This site is currently alpha at best. While I encourage you to bang on it, please do not expect it to be stable, at least not yet. As The Mythical Man-Month taught us, “Plan to throw one away. You will anyway.” This is the one I plan to throw away. If you post really long, well thought-out comments, please, please, please save your work on your local system first. I make no promises that early comments will be preserved here or even accepted into the database in the first place. The comments system is held together by duct tape and spackle (and PHP, and MySQL, and a few other tools). Even if it actually works for long enough for a comment to be posted, the likelihood I’ll be able to preserve all those comments over a period of years or even weeks is not good. I expect I will need to do major reengineering on the back end sooner rather than later that may involve throwing it out and starting over from scratch.
Given these admitted problems, why roll my own system from scratch rather than use any of the much better engineered open source systems? The answer will be found not in the back end but the front end, which in at least one crucial way is going to be a huge leap forward from any commenting system I’ve ever seen. Of course, after I reveal it to the world, I’m sure somebody’s going to point out that www.somewhere.I’ve.never.seen.org has been doing exactly this for the last six years; but none of the content management systems I explored seemed capable of handling it without major transplant surgery.
Given my known predilection for Java, you may be asking why I implemented this site on top of PHP instead of JSPs and servlets. The short answer is that Tomcat, and everything that sits on top of it, is an almost undocumented, uninstallable, unmaintainable mess. Note that I am not criticizing the servlet APIs or JSPs here. If I were actually confident that I could install Tomcat, integrate it with Apache, keep it running, and successfully deploy servlets and JSPs on top of it, I would be happy to write Java code instead of PHP. However, the fact is I’ve not been able to keep even the simplest, trivial little servlets running for the last two years, and I think Tomcat is largely responsible for that. Programmers and system administrators are users too. Sadly no one pays much attention to the proper design of user interfaces for this class of users, and Tomcat’s a classic example of that. I attribute this about half to poor design of the servlet deployment system and half to inadequate documentation. PHP is just so much simpler to deploy than JSP that I’d rather learn a new language than struggle with Tomcat one more time. In fact, now that I’ve got PHP deployed on this server I think I’ll rip out the few servlets I do have “running” here (actually, they’re down again at the moment and I haven’t yet figured out what went wrong this time) and replace them with PHP when I get a minute.
Not that PHP is without its own problems, of course. I did have to manually edit the autoconf file to get the Tidy extension installing correctly, and the config options should be a lot clearer about which directories one should actually point to for extension libraries, and whether one needs to point to them at all. Nonetheless. even with these problems, installing PHP was an order of magnitude easier than installing Tomcat and at least two orders of magnitude better documented.
Now that I think about it the difficulty of installing Tomcat (and other server side software) is another symptom of the same attitude that led me to write my own comment system. The problem is that the user interface is far too tightly coupled to the back end. Most programmers start by designing the back end. Then the front end has to fit within the back end’s limitations. The best case scenario here is that the front end is designed by a competent user interface designer who does the best job they can within those limitations. The worst case scenario is that the same programmer who does the back end also writes the front end. In that case, the front end matches the back end way too closely, and it exposes details that the end user should never see or care about it. For instance, consider the classic Unix install from source:
$ ./configure $ make $ sudo make install
Why are these three separate steps? Why not just “install”? I understand why these are three separate steps for developers, but end users should not need to go through this rigamarole. Sometimes there’s a Debian package or an rpm that makes this a little easier, but all too often it has unresolvable dependencies or is based on an out of date version, or the wrong architecture, and you have to go back to source.
Many packages then add additional steps, for instance modifying Apache’s httpd.conf file, before anything will work. This should be done automatically. PHP almost figured this out. It added the necessary “LoadModule php5_module” command to my httpd.conf, but I still had to use vi to add “AddType application/x-httpd-php .php .phtml” to httpd.conf before it would work. Why couldn’t it have done this automatically?
These installation procedures are designed around how the product developers think about installation rather than how the end users think about it. Breaking out configure, compile, and install as three separate steps makes sense for the programmer hacking on the product’s code. It does not make sense to the sysadmin installing a finished product. That user just wants to say “Install this”, and maybe be asked a couple of questions about where to put the product, which options to include, and whether or not to restart the server when the installation is through. If the installation system had been designed up front, by simply sitting down and saying “What do we want the users experience of installing our software?”, without any consideration of what was expected from the back end, then this would have been obvious. Instead installation was left to be hacked together at the last minute on top of a build system the programmers already had lying around, full of extra options no end user should ever actually touch. This is insane troll logic! A build system is not an installer. Build systems, make files, Ant and the like, are all fine tools for programmers; but they are not appropriate tools for end users or system administrators, even if those users and sysadmins are programmers in another life. Installing software requires an installer, not a build tool.
And this is just one small example. There are many other ways in which the user interface gets tied into the developer’s model of how the system works, rather than the end user’s model. If we begin by designing the user interface that allows the user to complete the tasks they need to complete, and only then design the back end to support this, we will at least develop a backend that can support the front end. All the content management systems I looked at were so tied to the what back end expected that I couldn’t even implement my front end without hacking code in the back end. It was impossible to sit my front end requirements on top of their back end.
By the way, this is all very much a Unix problem. The Mac doesn’t have it. One of the products I evaluated and gave up on on Linux (Plone) installed like a charm when I first tested it on my Mac. No muss, no fuss. In fact, I’m seriously considering whether any future servers I buy should be Macs, just for the ease of deployment.
But Mac programmers have always been a weird bunch of ducks, like a few wood ducks mixed in with a flock of mallards. More often than not, Mac programmers do design (and sometimes even code) the user interface first, before they write the back end to support it. In too many other development groups, the user interface is the last thing done rather than the first.
November 17th, 2004 at 8:00 pm
Egads — Tomcat still funky
This echoes my own experience, but of years ago. I deployed a Java app and we had crashing and memory suckage. Finally took it off and haven’t been running Java on the server since.
But I assumed everyone else was having better experiences, since then, and that Tomcat had been fixed.
Last week we used Tomcat in class and it worked very nicely, serving itself for test programs. But it’s too bad to hear things are still messy, doesn’t make me want to try it again. Did you try JBoss? Apparently that is easier to install and has its own server etc. Might be what real folks are using.
As far as installs go — my experience too. They make it weird and obscure and say “it’s easy!” I’ve paid someone to figure this stuff out, but that’s expensive. I’d like to do it myself. I wonder if Windows is easier to administer as a server? Perhaps the latest version doesn’t need rebooting every two weeks.
November 17th, 2004 at 8:37 pm
What, no TagSoup
You’re using Tidy instead of TagSoup? Huh.
November 17th, 2004 at 9:47 pm
What, no TagSoup
Actually at the moment I’m not using either. I do need to add some code to tidy up the comments HTML before it goes into the database though. The site’s running on top of PHP, not Java, so I’m not sure I could easily fit TagSoup into that system.
November 18th, 2004 at 4:35 am
Comments…
Well, you say that “no one’s done comments like this before” – what’s different about the comments? If you mean the code, well, that’s cool. But I don’t see anything here that looks substantially different from comments on other sites.
As for more general comments, I like the clean layout, but I don’t think you need to explain that the content is “longer than a typical blog post, and but much shorter than a full book”. This doesn’t matter to most readers. (Of course, it may just be temporary.
November 18th, 2004 at 4:36 am
Re comments…
I think you should have a link for Comments, rather than Read Comments – as it is, it looks as though I’ll be able to read, but not post, comments.
November 18th, 2004 at 1:35 pm
What’s different is not the appearance (which actually is sort of ugly; I need to work on that), but rather the workflow. It allows authenticated comments with no registration step, and an absolute minimum of information required. You get to pick your password the first time you login. Or you can skip passwords completely and use e-mail confirmation.
November 19th, 2004 at 3:26 pm
Re comments…
That makes sense. I’ve made that change. One open question is whether and how much the comments should be threaded. The database schema has a place for the ID of the post being replied to, but so far there’s no way in the user interface to indicate that. Should I include a “Reply to this” link below each comment? If I did that would suggest some form of threaded display. I may eventually add something like that, but probably only if the number of comments on some articles routinely grows so large that it’s hard to follow to the bottom. Personally I’m not so fond of threaded displays unless the number of comments is so large it really doesn’t fit on one page. Then you need to think about how to select which posts to show and which not to. I can only hope the site gets so popular that I have such problems. 🙂
November 20th, 2004 at 9:58 am
Tomcat issues
Much of the difficulty with Tomcat seems to revolve arounds its integration with Apache. If you used Tomcat exclusively, without using Apache to serve static pages, you could probably avoid a lot of the issues I’ve encountered over the last couple of years. That might work in a classroom setting, but I don’t think it could scale to where I hope this site will go.
November 30th, 2004 at 12:52 pm
Threaded comments
Threaded comments are on my TODO list. I’ve actually got a database field ready for that, but the front end would look a little trickier.
December 3rd, 2004 at 1:33 am
Tomcat documentation
You’re right that the documentation concerning the use of Tomcat and the Apache webserver together is not very good. German speakers have an advantage when it comes to Tomcat documentation. Peter Rossbach, who used to teach at the University in Frankfurt am Main, writes a column devoted to Tomcat in the monthly magazine “Javamagazin” published by Software & Support Verlag. Peter Rossbach is a committer to the Apache Foundation, co-author of the book titled “Tomcat 4.X”, as well as consulting on J2EE projects. Recently exactly this topic was covered in the Tomcat column in Javamagazin. The tips in that article described the problems and pitfalls and how they can be solved. Posted by JOrenchak@aol.com on Friday, December 3rd, 2004 at 1:33 AM
November 29th, 2004 at 6:10 pm
Re comments…
One disadvantage here to the lack of threaded comments is that it leaves open too much room for human error in picking something for the Subject line. Because although one can self-thread visually with a good subject line, most of us are bound to mess up and leave it blank, or type something incorrect there. Just a thought. I hope the site *does* get popular enough that you do have such problems.
November 29th, 2004 at 6:07 pm
XML and the Whedonverse
A typo you made in “Buffy in Cyberspace” seems almost intentional, “demonstarted”, at least if you hyphenate it as “demon-started” (“Because a recent show really impressed me with a level of cluefulness I hadn’t previously seen demonstarted in Hollywood.”). Bwahahaha! Hmmm, now what does that mean, exactly? The other day I was thinking about Giles’ aborted attempt to have Willow “scan” (no doubt in a rather useless PDF-y form) his entire library of demonology, and about the later appearance of the (apparently) incomplete online searchable database, “Demons, Demons, Demons” (which appeared on Angel the Series even more prominently). And someone said something in her Livejournal that got me to really thinking about it. Of course, I then wrote a long, ranty reply about how the new post-season seven Watcher’s Council should hire “us” on, to mark everything up in XML and throw it all into Philologic or some other search engine to make it all truly useful, quickly searchable, sortable, and grokable. With the advances in OCR — the burned single-edition stuff from “Gingerbread” and the blowing up of the Council in season seven notwithstanding — it would be much more useful and complete than anything currently existing in the ‘verse. Of course, Andrew has probably already undertaken just this task, being the nerd that he is. Although when I start thinking like this, putting myself in the ‘verse as a programmer to help them, I know for sure I am _becoming_ Andrew.
(Of course, my real point was that the old Quentin Travers paternalistic mentality would have precluded any such digital archive, because keeping the knowledge in paper form only [untranslated to boot] keeps that knowledge in the hands of those who understand it, the Watchers, thereby giving them a purpose and maintaining their power. Making the data accessible to the Slayer and the lay-person [Xander] means that the old power structure crumbles, but since it has crumbled anyhow, now our time is at hand! XML will prevent the next apocalypse by saving all that “Leafing Randomly Through Various Tomes Over Donuts and Coffee” time [leaving out that scary amount of time the markup might take to actually DO, haha]. Or there’s the scary Wolfram & Hart data access solution, ick.)
November 29th, 2004 at 6:05 pm
Post updated Java 3.0 article
Maybe you can post an updated “Why we need Java 3.0” article?
November 29th, 2004 at 6:03 pm
Article Sites and Content Flow
Several websites have been getting authors and “luminaries” to write articles or blog for them in an aggregated location in order to keep the content flowing. Just curious why you decided not to join the bandwagon and post your articles elsewhere. But, on the other hand, since you’ve come up with your own articles site, how about getting a few pals to write articles for The Cafes as well? Or simply open up submissions to all, and you can pick/choose what to post?
November 29th, 2004 at 5:36 pm
Tomcat and the Whedonverse
Java, XML, and the Whedonverse intersected for me some years ago. See Buffy in Cyberspace from way back in Episode 8. 🙂
November 29th, 2004 at 4:29 pm
Tomcat and Unicode Our main problem with Tomcat was Unicode. For the longest time, only 4.0.6 did unicode correctly, specifically with difficult stuff like Devanagari. We couldn’t upgrade to a less memory-sucky, crashy, newer version because they broke some Unicode thingamadoo in the newer versions. Around 5.? we just gave up and went back to using PHP with Sablotron (now with libxml2 built right in, instead), and the occasional system call the Xmlstarlet. Much, much, much, much faster, easy Apache integration, etc., plus non-developers can ramp up quite quickly and our SysAdmins no longer loathe us for our hungry Java Solaris-box-killing ways of yore.
P.S. — I once imagined that my love of the Whedonverse and my love of programming, XML, etc. would never cross paths, but now that you’ve quoted ‘insane troll logic’, well, clearly my delusion of separation was also insane troll logic. Because, geeks. 😉
November 29th, 2004 at 9:06 am
Older Tomcat
I had no trouble installing servlets on the 4.x series, but once Tomcat began to include its graphical frontend, I did run into trouble. For some crazy reason I could not get my servlets loaded, and as a result abandoned upgrading. And you’re right, the documentation is lousy.
As to integrating with Apache, I managed to do it using the older method. (Sorry, but I can’t remember the names of the older and newer modules. It’s been a while.) That documentation was awful! I’m certain that it was written by someone for whom English is not his native language. Don’t get me wrong, my father speaks English as a third language, but he’s not writing documentation for anything. I had hoped the most recent Tomcat would be better, but your experience isn’t encouraging. I’ll have to see for myself. By the way, nice job on the site. I’m all for longer articles. I’m still meaning to get up to the Albany XML Developers meeting the next time you’re speaking.
November 29th, 2004 at 12:36 pm
Tomcat and Apache
You’re trying Tomcat with Apache, eh? That explains much.
I’ve found Tomcat to be relatively easy to install and configure for years. Then again, I’ve never needed to integrate with Apache. Personally, I can’t imagine running a site where Tomcat was the weakest link instead of bandwidth. I think I can explain Tomcat’s popularity, despite any configuration or documentation issues it has: It’s free and open source. If you’re developing for the Internet, you will be constrained by bandwidth costs before you tax Tomcat’s ability to serve more pages. In the worst case, you can always get a bigger box. If you’re developing for an intranet you probably aren’t serving much static content. So, Apache integration is poor, because it isn’t really very important.
November 29th, 2004 at 1:36 pm
Jetty is a good alternative
We’ve been running our production apps with Jetty since we dumped JBoss about a year and a half ago (or more… I forget). Had a few wrinkles on the way, but nothing jumps immediately to mind as a hair pulling, kick the monitor off the desk type episode. Up until recently we were serving in excess of 2 million page views per day with nary a hiccup.
PS. If you enter your nickname incorrectly, it asks you to correct, but there’s no input box to do so (obviously you can hit the back button to fix the problem though)
May 26th, 2008 at 10:34 am
[…] a year ago I launched a The Cafes with some fanfare to host shorter writings on a variety of subjects that didn’t already fit into Cafe au Lait […]