3251 Times Faster than MySQL, 9983 Times Faster than ORACLE

Posted 14 May 2002 at 04:18 UTC by KlausWuestefeld Share This

Performance scalability was the main concern people had when I first described Prevayler, an object persistence layer for Java, in an Advogato article last Christmas.

People have run some tests since then. Guess what the results were.


What were the results?

Prevayler runs queries 3251 times faster than using MySQL/JDBC.
Prevayler runs queries 9983 times faster than using ORACLE/JDBC.


Of course! What did you expect? Prevayler runs all queries in RAM while the database has to read from disk.

No. In fact, the tests ran with the whole databases buffered in RAM too. Therefore, these results aren't even reflecting the RAM vs Disk difference. They are reflecting only the difference between the Database/JDBC architecture and the prevalent architecture.


Why is Prevayler so much faster?

All objects are already in their native form. No object marshalling or inter-process communication is required. You are no longer restricted to the queries your database provides. You can finally use any algorithm and data-structure you want.


What were those queries like?

Each query would return 100 objects, filtering one million by attribute equality ("select * from QUERY_TEST where NAME=?"). I consider this a very simple task for a relational database. We are not even talking about complex object navigations.


Was there an index on the NAME field?

Yes.


Who implemented the JDBC version of the test?

It is the same test, actually, that runs for Prevayler and JDBC. It runs agains a TestSubject interface with two implementations. I wrote the Prevayler implementation and the people at the MyWMS project, sponsored by the german government, wrote the JDBC implementation.


And what about transaction processing?

Prevayler 1.03, which uses multiple concurrent transaction log files, was 3.8 times faster than ORACLE and only 1.8 times slower than MySQL. Besides, most systems' load (Web apps in particular) is more than 90% queries and less than 10% transaction processing.


I shall only believe it when I see it for myself.

You are free to download the latest Prevayler alpha version, inspect and run the tests for yourself. Send us your results and we shall publish them.

See: Prevayler Scalability Test Results.

a testing database instance, posted 14 May 2002 at 15:08 UTC by sye » (Journeyer)

I'd like to suggest that you set up a testing database instance for advogato users. I shall call it the DeliTalk Global Food Services database. All advogato users can enter their local grocery shopping prices, their favorite local restaurant menus, their own home cooking recipes and ingredients etc ...

A Testing Prevayler Instance, posted 14 May 2002 at 17:09 UTC by KlausWuestefeld » (Master)

Yes. That is a good idea.

Another option is to write a JSP/Servlets face for the Prevayler Bank Demo.

We might consider something with a much greater volume of data to illustrate Prevayler's robustness and scalability. Any ideas?

If anyone writes any of the above, I shall put it up on the prevayler.org server.

greater volume data, posted 14 May 2002 at 18:38 UTC by sye » (Journeyer)

Here's a few more ideas:

  • run DNS and reverse IP -domain search through your database?
  • run USPTO through it?
  • run EDGAR through it?

    A good database design is to attract more people willing to go through raw data so they are able to form true knowledge of their own from those raw data and to make a better living for themselves.

  • database schema - advogato library, posted 14 May 2002 at 19:00 UTC by sye » (Journeyer)

    fields: owner, media_type:media_code:title, owner_note ....
    (media_type: book, CD, DVD, newspaper clip, internet ...
    media_code: ISBN, ?, ? , ?, URL, ... )

    I have one record for input today:
    owner="sye";
    media_type:media_code:title="book:0-486-26375-4:the works of mencius";

    owner_note=
    This edition of Mencius' works contains the complete texts of his writings in Chinese, printed in large, clear characters. It also contains the revised version of the English translation prepared by the great Sinologist James Legge. Published in Canada by General Publishing Company ltd. This Dover edition, first published in 1970, is an unabridged and unaltered republication of the second, revised edition as published by the Clarendon Press, Oxford, in 1895 as Volume II in "The Chinese Classics Series.

    mencius and a salad, posted 14 May 2002 at 20:00 UTC by whytheluckystiff » (Master)

    When cross-pollinating databases, we find what eaters read. I know certain publishers in my neighborhood who would want to see results based on ISBN and DeliTalk Global ID. A grid display would work fine, printable and crease lines for placing in envelopes. Incorporate with ideas from sye, her word is absolute currency.

    examples, posted 15 May 2002 at 14:47 UTC by sye » (Journeyer)

    The marketing department of prevayler has to offer a lot of literature targeting at current SQL programmers. Something like "how Java class object can be seen through the eyes of SQL statements" and how to interpret the coding restrictions that my business classes have to be serializable and deterministic. Can you give us some real life examples?

    Serializable and Deterministic, posted 16 May 2002 at 20:35 UTC by KlausWuestefeld » (Master)

    Serialization is a fundamental service in Java and other languages. To be serializable, a Java class needs only to indicate that it wants to be serializable (for security reasons) and must use only other serializable classes. All collections included in the Java API are serializable.

    99.9% of all source code is deterministic. There are a few ugly things, though, that you have to avoid.

    Why web apps are more reliable than Desktop apps..., posted 17 May 2002 at 16:47 UTC by tapir » (Journeyer)

    People forget that there's a reason why web applications are faster to write and more reliable than desktop applications. It's the fact that requests are typically handled with nonpersistant processes so that the total state of the system is in the filesystem, a relational db or some other kind of specialized store.

    Desktop applications have a way of going bad the same way cheese goes bad. Every other line of code adds state to the system, and eventually the system is going to make a transition into a degraded or nonfunctioning state... This includes memory leaks.

    Web apps based on a relational database have an engineered state space... Abstractions in OO programming often hide the fact that programming decisions have consequences -- once you start throwing arbitrary objects into a persistant store, sooner or later somebody is going to make a mistake which gradually corrupts the store.

    Re: Why web apps are more reliable than Desktop apps..., posted 18 May 2002 at 00:30 UTC by tk » (Observer)

    I have heard of relational databases go bad due to bad DDL, so I don't think this is an issue.

    Re: Why web apps are more reliable than Desktop apps..., posted 18 May 2002 at 00:33 UTC by tk » (Observer)

    I have heard of relational databases go bad due to bad DDL and DML programming, so I don't think this is an issue.

    New Advogato Features

    New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

    Keep up with the latest Advogato features by reading the Advogato status blog.

    If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

    X
    Share this page