Comments on: The State of Native XML Databases

By: Malcolm Davis

Malcolm Davis — Wed, 20 Oct 2010 17:28:52 +0000

We have eXist running dozens of simultaneous database with some database as large as 1 GB, and a single xml file as large as 500 MB. (Release 1.4.0-rev10440, on Linux)

eXist seems to perform and scale well (this might be due configuration of Java and the OS)

eXist has some configuration and tuning items that need to be tweaked for scalability/performance. (Some require changes to a build file, and rebuild with Ant)

By: Peter Gibbon

Peter Gibbon — Tue, 03 Nov 2009 20:23:39 +0000

That’s incredible Gene, We’ve been working on a problem recently where we couldn’t scale eXist past 300 megabytes of data on a state of the art Sun Blade Server, let alone 2 gigabytes!?! How on earth did you do that?

Solutions to problems with XML Databases and XQuery generally either store thousands of smaller “record” like XML documents, ranging between 1 to 20k in size, or a few (like 3 or 4) very large XML documents (like 500 meg to 2 gigabytes in size).

11,000 xml documents averaging 200K in size sounds very much like a number made up our of thin air to me, especially given our experience with eXist.

When we tried MarkLogic and Sedna, both could handle the data fine. We are still in the decision making process but eXist is kind of an office joke here.

By: Gene Thomas

Gene Thomas — Thu, 22 Oct 2009 00:04:43 +0000

All of this talk about Oracle (not BDB XML), IBM DB2 9 (including express-C), and SQL Server (2005 & 2008) supporting XML is bogus since none of them support the full set of XPath Axes and do not fully support xQuery.

None of them support following or preceding sibling axes. For our project we considered MarkLogic, X-Hive (now XDb), eXist, and Sedna. We wound up choosing eXist and currently have almost 11,000 xml documents averaging 200K in size.

By: John

John — Wed, 12 Aug 2009 02:42:40 +0000

Currently at this time the answer is simple.

If you’ve got money, MarkLogic.
If you haven’t got money, Sedna.

Job done.

By: Rob Tweed

Rob Tweed — Fri, 19 Jun 2009 11:27:14 +0000

The next step, by the way, for M/DB:X is to add an optional JSON interface as an alternative. This makes it pretty interesting (and puts it up against CouchDB): persistent XML DOM-based storage of Javascript Objects, and an ability to cross-convert between XML or JSON in and XML or JSON out. And of course an ability to modify/construct and query the XML DOM realisation of a corresponding Javascript/JSON object.

By: Rob Tweed

Rob Tweed — Fri, 19 Jun 2009 07:08:40 +0000

Well I suppose if it really was a serious problem then the many high-volume users of SimpleDB would be complaining strongly on the Amazon AWS forums but not a dicky-bird so far to my knowledge. Indeed I suspect that much of Amazon’s logic for sticking to the common GET and POST of the broader web for their “REST interfaces” is that they’re more likely to be handled by intermediate proxies, firewalls, gateways etc in a standard way: if it works for the broader web, it will probably work for them.

By: Elliotte Rusty Harold

Elliotte Rusty Harold — Thu, 18 Jun 2009 12:52:21 +0000

They’re a lot of reasons to care about the difference between GET and POST (and PUT and DELETE) including the proper behavior of proxy and cache servers, security, repeated client requests, and more. It doesn’t look like it matters until it does, and then it matters a great deal.

By: Rob Tweed

Rob Tweed — Thu, 18 Jun 2009 09:16:16 +0000

I have to say that as someone who builds “back-end” stuff, the “Is it REST or isn’t it REST” arguments seem pretty academic and pedantic. Most web server gateways are entirely dumb and simply pass the HTTP verb through along with the name/value pairs that were delivered to the web server in the HTTP request: there’s no particularly magic meaning applied to the HTTP verb. It therefore really doesn’t matter what verb you use in most circumstances. In the case of M/DB:X and SimpleDB, delivering the name/value pairs to the back end is the important thing and the interpretation of what the request means in terms of a transaction is dealt with by analysing the name/value pairs, not the HTTP verb.

So the fact that you might be using a GET or POST to perform an edit or delete at the back end is pretty irrelevant except from a purist, religious view of REST. It’s a bit like saying you must only drink wine from cut glass goblets, not plastic cups. If all you’re interested in doing is swigging some wine, who really cares what you drink it from? I really think there are more important things to worry about than “does he use DELETE for a delete or PUT for an edit?” “Does it work?” and “Is it simple to use?” are more important in my book.

I really don’t want to get in a flame war about this, but for the life of me I can’t see what the fuss is about. OK for the sake of agreement here let’s call it “HTTP-interfaced” rather than REST, but personally I’m going to keep swigging that wine from the same plastic cup as the 800lb Amazon gorilla

By: Elliotte Rusty Harold

Elliotte Rusty Harold — Wed, 17 Jun 2009 13:06:03 +0000

Amazon is wrong and for the same reasons. Learn from other people's mistakes, but don't copy them. HTTP != REST.

By: Rob Tweed

Rob Tweed — Wed, 17 Jun 2009 07:06:17 +0000

As you’ll have realised from our documentation, we modelled the HTTP interface on Amazon’s SimpleDB and followed its principles but we applied that style of interface to an XML database, not the spreadsheet-like database of SimpleDB.

Amazon refer to their interface as REST (http://docs.amazonwebservices.com/AmazonSimpleDB/2009-04-15/DeveloperGuide/index.html?REST_RESTAuth.html), so who am I to argue?

Whatever you call it, it’s a simple and effective way to deliver an XML database as a set of web services in the cloud.