REST is like Quantum Mechanics

A resource is identified by URI and may emit representations. There’s no way to tell from the representations what the resource “is”; I tend to believe a resource is what its publisher says it is as a good rule of thumb. But it doesn’t affect the software very much.

–Tim Bray on the xml-dev mailing list, Wednesday, 23 Jul 2003

REST is like Quantum Mechanics; or, more specifically, resources are like atoms (and I do mean atoms, not ATOMs). In quantum mechanics you cannot actually say what an atom is, where it is, or how fast it is moving. You can only predict how it will respond to certain experiments, and then only in a probablistic fashion. As soon as you try to figure out what the wave function actually represents, well then you fall down a sink hole of bad physics and worse philosophy.

Resources are the same. When designing RESTful systems, you never see the resources. All you see is the URL and the representation of the resource the URL provides. What resource does the URL identify? Who knows? The only way to reason about it is through the Copenhagen interpration.

On the server side things are a little different. You may be able to look behind the curtain and in fact know quite a bit more about the real resource than the client can. (This is simply not possible in quantum mechanics.) However it pays off not to do this. The more indirection you have between the resource and the actual physical representation the more flexible your design will be. If you tie your URLs to particular file system layouts or database queries, then you’re going to have a really hard time porting that system to a new operating system or database.

It’s best to design your URLs without respect to how they will actually be implemented. Design them so they make sense to human clients and search engines. Then worry later about how you’ll actually implement the backend that serves representations of those URLs.

Sadly server support for such schemes is very lacking in 2006. Doing it on top of Apache requires serious mod_rewrite voodoo. (That’s how WordPress creates the very nice URLs you see on this site.) PHP doesn’t help out here since it’s still very tied to a one-URL, one .phtml file model.

Rails improves matters a little bit, but doesn’t really take the plunge of completely decoupling the file system from the URL and database structure. <troll>In fact, its whole convention-over-configuration attitude is exactly contrary to what’s needed. You can configure Rails to do the right thing, but by default it does the wrong thing. This makes it faster and easier to build small simple systems; but quite a bit harder to build more complex, more flexible, more scalable systems. i.e. Rails makes the easy things easy, but the hard things harder. </troll> On the flip side, the easy things are easy enough that even with the more than linear growth in complexity as you move to a more decoupled URL structure, it’s still possible in Rails. It’s just not as easy as it would be if flexibility were as big a design goal in Rails as a cool 15-minute demo.

I have not yet seen a web server system that implements the full decoupling of URL space from implementation details I desire. I have seen sites do this. Yahoo and Amazon come to mind. However, they either use custom servers or a lot of mod_rewrite scripts sitting on top of Apache. I’m still imagining what I want, but I think it would look something like this:

A centralized configuration database accessed through the web server itself that allows:

  1. An individual URL to be mapped to an arbitrary file or script
  2. A URL subtree to be mapped to a filesystem subtree or script

Of course, the devil’s in the details. If you aren’t careful, you’ll just end up with something as ugly and complex as mod_rewrite. To that end, I’d prefer not to have full regular expressions in the language, but perhaps they’re necessary.

Doing this right means we need full and complete support for HTTP 1.1, not just the popular subset of POST and GET you see in a lot of products today. This includes:

  • Each script has access to the full and complete URL and the full HTTP request by which it was invoked, including headers and body.
  • Scripts support all HTTP methods including POST, PUT, and DELETE; not just GET.
  • All four operations can (potentially) be applied to the same URL. You don’t necessarily have separate URLs for POST and GET.
  • HTTP authentication, both DIGEST and BASIC. Different authentication credentials can be applied to different operations. For instance, authentication might be necessary for PUT or DELETE but not GET. URLs can be added to realms based on criteria more complex than just parent directory. For instance, all URLs on the site containg the string “edit” might require authorization.
  • Pluggable authentication to allow experiments with other, hopefully better authentication systems going forward.
  • No dependence on cookies.
  • No dependence on JavaScript. Individual applications might use JavaScript, but the framework itself shouldn’t require it of clients.
  • Content and language negotiation. It should be possible, for instance, to send one client HTML and another XML+XSLT; or one client French and another English.
  • Full support for the various HTTP response codes.

I don’t know any content management systems that meet this requirement. Do you? Is there a system out there than can do this today, or is it a subject for current research? I suspect one could built this on top of the Java Servlet API or Apache. Any takers?

17 Responses to “REST is like Quantum Mechanics”

  1. John Cowan Says:

    As one of the many who has struggled with the nightmare of Spring and Hibernate configuration with its huge and repetitive XML documents (and that’s supposed to be lightweight  Java development!), I for one welcome our new Rails masters for picking a straightforward convention and sticking to it. Project/controller/action actually makes a great deal of sense.

    As for the supposed need to be flexible about file systems and databases, I have a two-pronged response: (a) There is only one file system interface today, as long as you avoid corner cases like \ or + in filenames. (b) How often does one really need to change databases in a project? The notion that database independence is important is a chimera.

  2. Andrzej Taramina Says:

    I agree with John that database, and to some degree OS, independence is typically a red herring. Might be a good thing for a software vendor who needs to support multiple such, but most clients/users never change databases.

    In quantum mechanics, it is smaller particles than atoms that the Heisenberg Uncertainty Principle (you can’t know both the location and the speed at the same time) applies. I think you mean “electrons” and not atoms in your analogy.

  3. Ian Phillips Says:

    Again, in response to John’s comments:

    (a) Well, there are really two file systems in common use: Unix style and Windows style (where the reverse-solidus, a.k.a. \, is the directory separator and so is pretty common!).

    (b) It depends on the project. It’s not too uncommon to start of with a proof of concept using a small/cheap DBMS such as MySQL and then move to a commercially supported system (e.g. Oracle) as volume/usage grows.

    Of course, as always, YMMV! If you think it hughly unlikely that you are going to change DBMS and/or FS then pick a system that is simpler but less flexible: i.e. YAGNI.

  4. Ian Phillips Says:

    That should, of course, read “highly unlikely”, apologies to Hugh if he took offense at that!

  5. Bill Goggin Says:

    Struts provides some of the indirection you speak of in that it allows and even encourages indirection between URL’s and resources implementations. However it is too deep in the stack to implement all the HTTP features you wish for.

  6. Elliotte Rusty Harold Says:

    The Heisenberg Uncertainty Principle applies to objects at all scales: not just electrons but atoms, molecules, crystals, bacteria, dogs, people, planets, stars, and galaxies. At the larger levels, roughly bacteria and up, the effects are so small as to be effectively unobservable. However, in the realm of atoms quantum effects cannot be ignored. For instance, physicists have done double-slit experiments with atoms and molecules, not just electrons and photons.

    They’re a lot of other quantum effects that come into play at the level of atoms and molecules. For instance, there’s a whole subfield of chemistry called quantum chemistry that essentially consists of approximating solutions to the Schrodinger equation for multiatom molecules.

  7. Todd Ditchendorf’s Blog » Blog Archive » REST is like Quantum Mechanics Says:

    […] Interesting musing about the need for better URL configuration by Elliotte Rusty Harold complete with a little Rails bashing. Fun. […]

  8. Al Lang Says:

    “As soon as you try to figure out what the wave function actually represents, well then you fall down a sink hole of bad physics and worse philosophy.”

    Actually, that whole statement is just the sort of bad philosophy universally adopted by physicists who think philosophy doesn’t matter. 🙂

  9. Adam Rosien Says:

    I believe there is a spin-out of HP that created a framework that mapped URLs to “anything”, i.e., Java methods, files, etc. Sorry I can’t remember the name. But it tackled many of the resource issues you describe.

  10. Tom Says:

    Unless I’m mistaken, common web browsers require JavaScript for REST. That is, I haven’t had success with form actions other that GET and POST. Or in other words, I haven’t been able to do PUT or DELETE without XMLHttpRequest.

    Anyone else with other experience?

  11. Bag O’ Tricks » Blog Archive » REST Query Language Says:

    […] On the real web, data isn’t just about documents. It isn’t just about indivisible resources. We might like to think there are just atoms, but there really are subatomic particles, and ERH, GData, the HTTP spec, and everyone else admits as much. Look for the word query, and you’ll find it’s no secret. […]

  12. Elliotte Rusty Harold Says:

    Browsers aren’t required for REST. HTML forms don’t support PUT or DELETE, but those aren’t the right verbs for most of the things HTML forms are used for anyway. XForms will support DELETE, though not until the next version. Some web editors such as Netscape Composer do support PUT. They just don’t support it through an HTML form.

  13. Roger L. Costello Says:

    Hey Elliotte,

    I wrote the below short article this morning, and then a colleague pointed out your article. Interesting how we are thinking along much the same lines. /Roger

    Recall your school days when you learned that atoms, electrons, photons can exhibit both wave behavior as well as particle behavior.

    Analogously ….

    Consider CNN: it serves up news stories in a variety of formats – as HTML, audio, video, RSS. Thus, news story information is an abstract thing that can take on many different formats.

    Quantum particles exhibit a wave-particle duality. Information exhibits a HTML-XML-MP3-MPEG-JPEG-… plurality.

    Physicists can’t really explain quantum particles; they can only tell you how they will behave. Similarly, we can’t really say what information is; we can only give it various representations (HTML, XML, MP3, …)

  14. links for 2006-05-15 at protocol7 Says:

    […] REST is like Quantum Mechanics (tags: REST HTTP design to_read elliotte_rusty_harold) […]

  15. Jerome Louvel Says:

    Hi Elliotte,

    I’ve discovered this post a bit late but I think that our Restlet project nicely supports your requirements. I’ve entered a RFE to support pluggable authentication mechanisms in our reference implementation, but the Restlet API itself is allowing several schemes (BASIC and Amazon for now).

    The project has just reached 1.0 RC1:

    I would be interested to hear your comments.

    Best regards,

  16. EgoH Says:

    The PUT request isn’t really a problem, since updating a object requires a form.
    Putting :method => :put in the options parameter of the form generator will make a hidden field which emulates the PUT behaviour. Since deleting is usually with links, this is a problem, since i don’t think it’s possible with a link.

  17. Elliotte Rusty Harold Says:

    Forms cannot emulate PUT because you have to POST to a different URL than the one you’re creating.