REST is like Quantum Mechanics
A resource is identified by URI and may emit representations. There’s no way to tell from the representations what the resource “is”; I tend to believe a resource is what its publisher says it is as a good rule of thumb. But it doesn’t affect the software very much.
–Tim Bray on the xml-dev mailing list, Wednesday, 23 Jul 2003
REST is like Quantum Mechanics; or, more specifically, resources are like atoms (and I do mean atoms, not ATOMs). In quantum mechanics you cannot actually say what an atom is, where it is, or how fast it is moving. You can only predict how it will respond to certain experiments, and then only in a probablistic fashion. As soon as you try to figure out what the wave function actually represents, well then you fall down a sink hole of bad physics and worse philosophy.
Resources are the same. When designing RESTful systems, you never see the resources. All you see is the URL and the representation of the resource the URL provides. What resource does the URL identify? Who knows? The only way to reason about it is through the Copenhagen interpration.
On the server side things are a little different. You may be able to look behind the curtain and in fact know quite a bit more about the real resource than the client can. (This is simply not possible in quantum mechanics.) However it pays off not to do this. The more indirection you have between the resource and the actual physical representation the more flexible your design will be. If you tie your URLs to particular file system layouts or database queries, then you’re going to have a really hard time porting that system to a new operating system or database.
It’s best to design your URLs without respect to how they will actually be implemented. Design them so they make sense to human clients and search engines. Then worry later about how you’ll actually implement the backend that serves representations of those URLs.
Sadly server support for such schemes is very lacking in 2006. Doing it on top of Apache requires serious mod_rewrite voodoo. (That’s how WordPress creates the very nice URLs you see on this site.) PHP doesn’t help out here since it’s still very tied to a one-URL, one .phtml file model.
Rails improves matters a little bit, but doesn’t really take the plunge of completely decoupling the file system from the URL and database structure. <troll>In fact, its whole convention-over-configuration attitude is exactly contrary to what’s needed. You can configure Rails to do the right thing, but by default it does the wrong thing. This makes it faster and easier to build small simple systems; but quite a bit harder to build more complex, more flexible, more scalable systems. i.e. Rails makes the easy things easy, but the hard things harder. </troll> On the flip side, the easy things are easy enough that even with the more than linear growth in complexity as you move to a more decoupled URL structure, it’s still possible in Rails. It’s just not as easy as it would be if flexibility were as big a design goal in Rails as a cool 15-minute demo.
I have not yet seen a web server system that implements the full decoupling of URL space from implementation details I desire. I have seen sites do this. Yahoo and Amazon come to mind. However, they either use custom servers or a lot of mod_rewrite scripts sitting on top of Apache. I’m still imagining what I want, but I think it would look something like this:
A centralized configuration database accessed through the web server itself that allows:
- An individual URL to be mapped to an arbitrary file or script
- A URL subtree to be mapped to a filesystem subtree or script
Of course, the devil’s in the details. If you aren’t careful, you’ll just end up with something as ugly and complex as mod_rewrite. To that end, I’d prefer not to have full regular expressions in the language, but perhaps they’re necessary.
Doing this right means we need full and complete support for HTTP 1.1, not just the popular subset of POST and GET you see in a lot of products today. This includes:
- Each script has access to the full and complete URL and the full HTTP request by which it was invoked, including headers and body.
- Scripts support all HTTP methods including POST, PUT, and DELETE; not just GET.
- All four operations can (potentially) be applied to the same URL. You don’t necessarily have separate URLs for POST and GET.
- HTTP authentication, both DIGEST and BASIC. Different authentication credentials can be applied to different operations. For instance, authentication might be necessary for PUT or DELETE but not GET. URLs can be added to realms based on criteria more complex than just parent directory. For instance, all URLs on the site containg the string “edit” might require authorization.
- Pluggable authentication to allow experiments with other, hopefully better authentication systems going forward.
- No dependence on cookies.
- No dependence on JavaScript. Individual applications might use JavaScript, but the framework itself shouldn’t require it of clients.
- Content and language negotiation. It should be possible, for instance, to send one client HTML and another XML+XSLT; or one client French and another English.
- Full support for the various HTTP response codes.
I don’t know any content management systems that meet this requirement. Do you? Is there a system out there than can do this today, or is it a subject for current research? I suspect one could built this on top of the Java Servlet API or Apache. Any takers?
April 23rd, 2006 at 10:30 am
As one of the many who has struggled with the nightmare of Spring and Hibernate configuration with its huge and repetitive XML documents (and that’s supposed to be lightweight  Java development!), I for one welcome our new Rails masters for picking a straightforward convention and sticking to it. Project/controller/action actually makes a great deal of sense.
As for the supposed need to be flexible about file systems and databases, I have a two-pronged response: (a) There is only one file system interface today, as long as you avoid corner cases like \ or + in filenames. (b) How often does one really need to change databases in a project? The notion that database independence is important is a chimera.
April 24th, 2006 at 9:12 am
I agree with John that database, and to some degree OS, independence is typically a red herring. Might be a good thing for a software vendor who needs to support multiple such, but most clients/users never change databases.
In quantum mechanics, it is smaller particles than atoms that the Heisenberg Uncertainty Principle (you can’t know both the location and the speed at the same time) applies. I think you mean “electrons” and not atoms in your analogy.
April 24th, 2006 at 12:24 pm
Again, in response to John’s comments:
(a) Well, there are really two file systems in common use: Unix style and Windows style (where the reverse-solidus, a.k.a. \, is the directory separator and so is pretty common!).
(b) It depends on the project. It’s not too uncommon to start of with a proof of concept using a small/cheap DBMS such as MySQL and then move to a commercially supported system (e.g. Oracle) as volume/usage grows.
Of course, as always, YMMV! If you think it hughly unlikely that you are going to change DBMS and/or FS then pick a system that is simpler but less flexible: i.e. YAGNI.
April 24th, 2006 at 12:28 pm
That should, of course, read “highly unlikely”, apologies to Hugh if he took offense at that!
April 24th, 2006 at 1:54 pm
Struts provides some of the indirection you speak of in that it allows and even encourages indirection between URL’s and resources implementations. However it is too deep in the stack to implement all the HTTP features you wish for.
April 24th, 2006 at 10:51 am
The Heisenberg Uncertainty Principle applies to objects at all scales: not just electrons but atoms, molecules, crystals, bacteria, dogs, people, planets, stars, and galaxies. At the larger levels, roughly bacteria and up, the effects are so small as to be effectively unobservable. However, in the realm of atoms quantum effects cannot be ignored. For instance, physicists have done double-slit experiments with atoms and molecules, not just electrons and photons.
They’re a lot of other quantum effects that come into play at the level of atoms and molecules. For instance, there’s a whole subfield of chemistry called quantum chemistry that essentially consists of approximating solutions to the Schrodinger equation for multiatom molecules.
April 30th, 2006 at 9:43 am
[…] Interesting musing about the need for better URL configuration by Elliotte Rusty Harold complete with a little Rails bashing. Fun. […]
May 1st, 2006 at 7:58 am
“As soon as you try to figure out what the wave function actually represents, well then you fall down a sink hole of bad physics and worse philosophy.”
Actually, that whole statement is just the sort of bad philosophy universally adopted by physicists who think philosophy doesn’t matter. 🙂
May 1st, 2006 at 12:17 pm
I believe there is a spin-out of HP that created a framework that mapped URLs to “anything”, i.e., Java methods, files, etc. Sorry I can’t remember the name. But it tackled many of the resource issues you describe.
May 1st, 2006 at 12:22 pm
Unless I’m mistaken, common web browsers require JavaScript for REST. That is, I haven’t had success with form actions other that GET and POST. Or in other words, I haven’t been able to do PUT or DELETE without XMLHttpRequest.
Anyone else with other experience?
May 2nd, 2006 at 12:40 am
[…] On the real web, data isn’t just about documents. It isn’t just about indivisible resources. We might like to think there are just atoms, but there really are subatomic particles, and ERH, GData, the HTTP spec, and everyone else admits as much. Look for the word query, and you’ll find it’s no secret. […]
May 4th, 2006 at 8:43 pm
Browsers aren’t required for REST. HTML forms don’t support PUT or DELETE, but those aren’t the right verbs for most of the things HTML forms are used for anyway. XForms will support DELETE, though not until the next version. Some web editors such as Netscape Composer do support PUT. They just don’t support it through an HTML form.
May 5th, 2006 at 11:20 am
Hey Elliotte,
I wrote the below short article this morning, and then a colleague pointed out your article. Interesting how we are thinking along much the same lines. /Roger
Recall your school days when you learned that atoms, electrons, photons can exhibit both wave behavior as well as particle behavior.
Analogously ….
Consider CNN: it serves up news stories in a variety of formats – as HTML, audio, video, RSS. Thus, news story information is an abstract thing that can take on many different formats.
Quantum particles exhibit a wave-particle duality. Information exhibits a HTML-XML-MP3-MPEG-JPEG-… plurality.
Physicists can’t really explain quantum particles; they can only tell you how they will behave. Similarly, we can’t really say what information is; we can only give it various representations (HTML, XML, MP3, …)
May 15th, 2006 at 7:17 am
[…] REST is like Quantum Mechanics (tags: REST HTTP design to_read elliotte_rusty_harold) […]
December 27th, 2006 at 5:40 am
Hi Elliotte,
I’ve discovered this post a bit late but I think that our Restlet project nicely supports your requirements. I’ve entered a RFE to support pluggable authentication mechanisms in our reference implementation, but the Restlet API itself is allowing several schemes (BASIC and Amazon for now).
The project has just reached 1.0 RC1:
http://www.restlet.org
I would be interested to hear your comments.
Best regards,
Jerome
December 29th, 2006 at 3:42 pm
The PUT request isn’t really a problem, since updating a object requires a form.
Putting :method => :put in the options parameter of the form generator will make a hidden field which emulates the PUT behaviour. Since deleting is usually with links, this is a problem, since i don’t think it’s possible with a link.
January 5th, 2007 at 8:36 am
Forms cannot emulate PUT because you have to POST to a different URL than the one you’re creating.