Slashdot Mirror
Prevayler Quietly Reaches 2.0 Alpha, Bye RDBMS?
"We've used relational databases for years despite incompatibilities in SQL implementation. Accessing them from an OOP paradigm has been so tedious, that Object-Relational mapping technologies have sprouted all over the Open Source landscape. Some competing examples and models are Hibernate, OJB, TJDO, XORM, and Castor; which in turn have supporting frameworks such as Spring and SQLExecutor. Because SQL is the dominant form of interfacing with the data in an RDBMS, there's now a specification to offer it a friendlier OO face.
Most of the above, including the SQL-variants, arguably appear to add yet another layer of complexity (even if only at the integration level) where they should be taking complexity away. These solutions are put together by some very smart people, but it's inescapable to get that feeling someone is missing the forest (simple answer) because all the trees (incompatible models) are in the way. If there are so many after-the-fact solutions attempting to simplify relational database access and manipulation from OO, isn't it reasonable to think that there is something generally wrong with trying to cobble-together two disparate concepts with what are essentially high-caliber hacks? Is Prevayler a better way?"
Is that really possible? How do you even benchmark that?
I'm all for trying to use Slashdot to promote your pet project, but don't couch your story in questions about people's use of your admittedly relatively unknown software.
taken! (by Davidleeroth) Thanks Bingo Foo!
This would be great for projects where interoperability isn't an issue or only occurs via edge connections like SOAP. However, I generally would be wary of a "database" which is only accessible in Java, via unique interface. What do you do with your Crystal Reports users? How do I get this into data cubes for analysis?
Frankly, this is simply a persistance layer with some nice properties. It *isn't* a database. A database stands at the center of your applications and makes itself available to as wide of an audiance as possible. It shouldn't limit your choice of tools in such an absurd manner.
Sig under construction since 1998.
Breakfast served all day!
A direct quote from your Wike:
Prevalence requires us to have enough RAM on our servers to contain all our business objects.
What do you do if you have a nice big data set that won't fit in memory? Businesses my company works with have millions of customers. Do they have to have all those gigabytes of data in memory to do anything?
Don't come at me with yet another memory resident thing that's supposed to be the greatest ever, when it doesn't come close to addressing the real needs of a database user.
I'm glad I'm not a subscriber right now cause if you payed to have the advertisements removed I don't think it would have caught this one.
--
WHO ATE MY BREAKFAST PANTS?
If you could cache an entire MySQL database in RAM, I'm sure your MySQL performance would improve dramatically.
If you could then optimize MySQL's search routines for working on memory instead of disk blocks, your MySQL performance would improve even more dramatically. As it is now, MySQL must go through all sorts of contortions, probably using B-Tree-like structures for indexing, and other fanciful datatypes I can't even conceive of without a PhD. The reason for all of this silliness is the fact that MySQL's backing store is disk, not memory.
In a prevalence system like Prevayler, one of the fundamental tenents of the system is that ALL of your objects are ALWAYS in memory, and are only serialized to disk when they change, for persistence.
So...yes: as always, a memory-based system will be three orders of magnitude (or more!) faster than a disk-based system. Prevayler vs. DBMS is no exception to this rule.
But when your website has grown popular, your prevalance database has swelled to 30 gigs and you find yourself hosting it on massive systems with 12 gigs of core memory and another 30 gigs of swap space -- when your memory access times are starting to look like disk access times because of swapping -- well, don't come crying to mwe.
Prevalence is a brilliant solution, for small projects. But they only scale to the size of your physical memory, or slightly (50-100%) larger. You can't expect them to scale beyond that.
Correct link for the previous Slashdot article -- important, since it provides the lucid, straightforward explanation of what the hell this thing actually is that today's pointless-link-filled, cutesy bit of Astroturfing neglects to offer.
What I'm listening to now on Pandora...
I'm,
Not sure what the proper term for referring to the articles that appear on Slashdot should be but I can say that this article is about the worst I've ever seen in terms of promoting a pet project.
Slashdot needs to go one step further with it's moderating functionality.
Slashdot needs a way that crappy articles like this one can be moderated into the bit bucket and I don't have to see it anymore!
Caution: Contents under pressure
I'm just going to assume their site uses Prevayler to store the page counter and revision information at the bottom of the linked page. Here's what I saw (times are Pacific):
[16:47] : This page has 10151 hits and 60 revisions
[16:50] : This page has 10156 hits and 60 revisions
[16:52]*: This page has 10170 hits and 60 revisions
[16:53] : This page has 10194 hits and 60 revisions
[16:54] : This page has 10220 hits and 60 revisions
[16:55] : This page has 10261 hits and 60 revisions
[16:56] : This page has 10311 hits and 60 revisions
[16:57] : This page has 10353 hits and 60 revisions
[16:58] : This page has 10413 hits and 60 revisions
[16:59] : This page has 10454 hits and 60 revisions
[17:00] : This page has 10503 hits and 60 revisions
[17:01] : This page has 10539 hits and 60 revisions
[17:02] : This page has 10578 hits and 60 revisions
[17:03] : This page has 10623 hits and 60 revisions
[17:04] : This page has 10666 hits and 60 revisions
[17:05] : This page has 10713 hits and 60 revisions
[17:06] : This page has 10748 hits and 60 revisions
[17:07] : This page has 10792 hits and 60 revisions
* - The Mysterious Future
Oh well. It did seemed to peek around 16:58, but I guess Slashdot users really didn't click on that link all that much. Too bad, that would have been fun ad-hoc to test.
A programmer is a machine for converting coffee into code.
Please don't go around saying, "Could this be the end of RDBMSes?" That is just a crock of shit, and it really bugs the hell out of me. How stupid do you think we are to have a tagline like that?
Please watch "Bowling for Columbine" and it says that this type of exaggeration by the media, and most notably US media, is driving the Americans crazy paranoid with fear. It seems like the editors have fallen for the same thing, and it should stop now!
Let's have a modicum of dignity and avoid all the hyperbole, please. We are all somewhat tech-savvy, please treat us with the respect that we deserve!
I'm sure the technology is good, but for crying out loud, mainframes are still around 40 years later. RDBMSes are going nowhere for the next 20+years until true AI comes around.
Although MySQL is about 2.8 times faster for transaction processing, Prevayler retrieves 100 objects among one million 3251 TIMES FASTER than MySQL via JDBC, even when MySQL has all data cached in RAM!
Now many different people will interpret this statement differently. But, my interpretation is that under very specific circumstances it is 3000 times faster than MySQL. However, for transaction processing, which is the primary and most common role of a RDBMS, MySQL is 2.8 times faster than Prevayler.
Bottom line... For normal circumstances, MySQL is faster and many commercial SQL database servers are faster still.
Hell no! Are you stupid?
I might be.. we use an in-memory Java objectbase solution for Ganymede.. when coupled with an on-disk transaction log, we get extremely high performance, transactional integrity, and the ability to run wherever there's a Java VM.
While I wouldn't claim that this kind of solution will scale indefinitely, it works very well indeed for our application and our dataset size.
It's not Prevayler, though, so I can't comment on his API or code quality.
- jon
Ganymede, a GPL'ed metadirectory for UNIX
If they just benchmarked reads ... then the results don't tell much.
The Raven
This sounds a lot like the Zope ZODB. For those who are still stuck in the stone age, Zope is a python based application server that essentially uses object serialisation to store its data.
Imagine if every page in your website was an object- with methods, properties, Access Control, etc. You have different classes for different types of documents, and each document internally knows how to render itself. The ZODB is essentially one big persistant object orientated namespace- You dont have to parse your data into SQL and back again, it's always just there, elimenating a huge amount of work (and bugs!). Having worked with it for a year, I can certainly testify that it is leaps and bounds over relational databases for most things.
I'd say RTFA, but the article is just a very plain website with very little information. However, if you look into it, all the transactions are actually SAVED so you don't have to worry about losing data in the middle of the day (i.e. a journaled file system). The daily serialization is just a snapshot of the system so you don't have to start from the beginning every time you restore the system.
At all times, they serialize commands to log files. Those commands are sent to every machine in your cluster. (A clock tick is one of those commands.) When you backup at night, one of those machines serializes it's objects to disk, and racks up commands to complete. Once it's done serializing to disk, it runs through the commands and catches up.
If all your servers go up in flames, you take your object serialization, and reload it. Then you run through the command logs, and you're right back where you were when the machines died. (Assuming you've got your command logs)
So no, they don't have their heads up their asses in that respect.
There are no trails. There are no trees out here.
I'm not about to give up PostgreSQL.
An RDBMS:
* Allows a wide variety of applications to operate on the data, in a wide variety of languages, at different times or simultaneously.
* Allows you to manipulate the data inside the database before you get it.
* Allows for a lot more storage, which is sometimes important when you need the memory for some other task.
What I like about an RDBMS (like postgres) is that my requirements constantly change. I try something for a while, then we have better ideas, but we need to work with our existing data. An RDBMS allows me a huge amount of flexibility with my data (also the reason I don't use MySQL...) and I've been able to drastically change the way my application works while still making use of the data that I have.
Maybe this is an OK database for some applications where you have the entire thing laid out in a perfect spec with no chance of a change. However, when I need to get at my data with another language, or it takes up more space than I thought, or I figure out that my application needs to change the way it works, I have no clue where to begin with a huge collection of "objects" that now happen to be obsolete. If I have relations, I have a solid representation of my data that an RDBMS can manipulate efficiently according to a fairly mature mathematical model. I'll take that over a collection of persistant objects any day.
That said, I think this has application in areas where you just need some persistent objects, and they need to be fast, and they don't take up much room. I don't encounter that very often, and when I do I usually just use postgres because it seems fast enough when the tables are cached. I suppose if the objects are really intricate this would be a nice system, because you wouldn't have to spend so much code on a mapping. It just seems so much more narrow as far as usefulness.
Social scientists are inspired by theories; scientists are humbled by facts.
I tried Prevayler and even loaded in 50k records from an old product table and object-ized them.
Yah, it was fast. Searches were great. I even figured out how to do complex object joins.
But I started having trouble when I tried to figure out how all the transactions worked. This was complicated by the wiki they're using, which was quite useless to me. It lead to many dead ends.
However, the real reason I found that I could not (possibly ever) use Prevayler is becuase it seemed the approach was for one machine and one machine only. There were no distributed mechanisms at all. Or at least, not how I'm used to working.
All the systems I've worked on in the past five years have all been with clusters of app servers. If all the objects on one machine were all in memory, then I couldn't think of an easy way to get them into the memory of the other machines. There was some talk about using Java Spaces, but that's kinda where I dropped off.
And the other issue was getting to the data from non app server machines. Like stuff to do back end reporting and things like that. I bascially figured out that for n-machine access, I needed something that, well, acted like a database.
I thought the idea was very interesting and maybe these things have been addressed. But when I really sat down with it for a few weeks, it just didn't pan out for me.
The object-oriented ideology as instantiated in C++ and Java is founded upon breaking data into objects, bearers of identity, which belong to classes, bearers of structure and behavior. (C++ and Java make little account of metaclasses, which are used in more dynamic object systems such as Python's class system and Common Lisp's CLOS. Templates are not metaclasses.) Objects have identity, so they can be equated; they are the unique bearers of attributes about themselves; and each object's structure is dictated by the class to which it belongs.
When object-oriented partisans look at a database, they see its relvars (or table headers) as bearers of structure and think of classes, and its tuples (or rows) as bearers of identity and think of objects. They see a database as a place to store objects persistently.
But this is not what an RDBMS does. An RDBMS isn't an object store; its relvars are not classes and its tuples are not objects. So what is an RDBMS? What is "relational" anyhow? Relational databases are founded upon relational mathematics, which is what you get when you cross set theory with predicate calculus.
Set theory is the branch of math that deals with collections of elements which behave according to formal axioms. Set theory lets you say, for instance, that if you have a non-null set R and a non-null set S, that you can construct a set R*S of all the possible pairs of elements from R and S.
Predicate calculus is the branch of logic that deals with quantified statements about entities. It lets you formalize logical arguments such as the syllogism: All men are mortal; Socrates is a man; therefore, Socrates is mortal. Predicate calculus deals with generalizations and instantiations of those generalizations.
What do we get by combining set theory and predicate calculus? We get a system that allows us to operate upon sets of tuples of values satisfying predicates. A relation holds tuples of values which make some predicate true. For instance, consider the predicate "Person x owes me y dollars." Tuples which satisfy this predicate will be pairs (x,y) for which the sentence is true. For instance, if Fred owes me 40 dollars, (Fred, 40) satisfies the predicate. It could thus be a tuple in the relation described by the predicate -- the one relating people's names to how much they owe me.
With the relational algebra (or an RDBMS) we can do operations upon this relation and others. We could, for instance, select a result set of all those people who owe me more than 50 dollars -- or join this result set with those people's addresses. Whatever result set we ask for will be calculated from the facts in the database. We might get back this result set:
(Barney, 75, 40 Elm Road)
(Megan, 60, 9 High Street)
Now, are the elements of this result set objects in the object-oriented sense? They are not. They do not have identity. The tuple about Barney is not Barney himself, or even a machine representation of him. It doesn't uniquely store attributes of Barney -- after all, we created it by joining tables which also contain such attributes. It is not even, truly, a fact about Barney exclusively -- for it is also a fact about the number 75, and about the address 40 Elm Road. It isn't an object; it's a tuple value, and values do not have identity as objects do.
Moreover, note that by joining, we can construct new relations from old ones. Thus, not only are tuples not objects, but relvars are not classes. After all, in OO we do not create new classes by joining, but by inheritance or encapsulation of old ones -- and creating a new class does not cause it to be instantiated into known-correct objects.
So what does this matter to OO people faced with RDBMS as a
That being said, the other gentlemen in this thread are talking about Two-Phase Commit. Yes, I do understand that Prevayler doesn't even have one-phase-commit to provide things such as Isolation, but that's beside the topic discussed in this thread. So please, let's both stop referring to it. Even if one-phase-commit can be implemented over RMI. But if you want to really simplify this issue to the extreme, you need to tell us how Prevayler handles rollbacks. After that, you could maybe explain to us (and to the RMI engineers at Sun) how to implement the Two-Phase-Commit protocol over RMI with a data store like Prevaylers that is not XA-compliant.
Thank you for playing(*).
(*) I told you I was going to be pedantic.
These folks are either very naive, or very silly.
They claim there's no need for two-phase commit (2pc), as though the only systems they need to interact with are (or will be) prevaylor.
Umm, hello. How about that 50TB database with all our transaction history? You gonna put that in your RAM-based database? No? Well, what happens when you need to do an insert into it, but commit only if the insert and the local transaction succeeds?
Hell, forget the 50TB database, what about the little Oracle database the guys down the hall use? Or the asynchronous queue that you post into?
It's a much bigger world than just your little project, guys, and you have to fit into it. 2PC is not an option. It's a requirement.
The whole "let's keep it in RAM" is cute, and for a lot of projects is probably all you need, but for any kind of large data set you just can't buy enough RAM to hold it all. Once it goes to disk, there's a whole new set of problems.
Also, the fact that you're responsible for defining and managing your transaction boundaries is really lame. It's not that hard to build check-in/check-out logic that can be used.
Come back when you have a real system that can handle real load with real datasets. Until then, I'll keep my RDBMS. You may have performance beat on the tiny systems, but who cares? THEY'RE TINY SYSTEMS!
Atomic, The A in RDBMS ACID (Atomic, Consistent, Isolation, Durable) transactions.
Atomic transactions consist of grouping of changes to tables or rows such that all or none of the changes take place. A rollback operation can reverse all the actions of the atomic transaction.
From the Greek atomos for indivisible
I don't mean to say anything against their product, which as far as I know is the greatest object persistence scheme ever hatched.
But they clearly don't know what a database actually is; they're confusing the issue with services that an RDBMS happens to perform as part of its job. It has always been possible to write procedural code that is faster than database queries because underneath a query is turned into a sequene of operations. When building a system to answer a single question, the system will always be faster without the database layer. Building a hash table or B-Tree to do a simple lookup simply can't be beat.
People have lost sight of history. Years ago, we used to keep our data in indexed files, and guess what? They were faster than databases of the day at doing the tasks they were designed to do.
However while databases are slower, in many cases much slower, than procedural code, they have an important property: they can be used to answer unanticipated questions acceptably quickly. How quickly is acceptably quickly? Well, if the database can come back with an answer faster than it takes a skilled programmer to come up with a special purpose program to answer the question, it has done its job. Compare this to their answer to the querying problem: write a java program. And that's a fine answer if having to answer an unanticipated question is a relatively rare event, which no doubt it is for many applications.
This gets to what a database is: it is a collection of information that that is organized to make reuse simple and efficient. This is different from business object re-use, which is about re-using logic; this is about re-using facts. Relational databases are unequaled at this task because they are based on sets and mathematical operations that are closed on these sets. This allows both the user, but more importantly the system's optimizer, to create sequences of operations that meet the user's requests.
These people may have a wonderful system for a lot of purposes, but they're really talking about a particular set of applications, for which their system might be better than storing object data in a database. Probably is, as far as I know. But really. No need for rollback because "transactions are instantaneous"? Well, they would be right if transactions really were instantaneous. HOwever, while their test case may be so fast they appear instantaneous, that's a long way from actually being instantaneous. In the real world bound by the laws of physics, transactions take finite time. Given enough objects to update, or high enough system load, or both, you will have to either (1) accept a possibility, albeit small, of inconsistent transactions being processed or (2) lock all the objects that might affected by a transaction, with attendent possibilties of deadlocking.
In short, I wish them luck; it sounds like they are producing some interesting and useful stuff. However, it isn't a database or a replacement for a database.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
I really appreciate the idea of a RAM targeted object repository layer, but as often happens, the Java guys tend to present their solutions as panaceias to every single application domain that slightly ressembles the problems they are attacking. I am really curious about how Prevalence compares to more naive in-RAM data storage approaches. Despite raising attention to the large quantities of memory not used in high end servers (and to the project itself), I simply can't see the point to compare it with mysql.
Furthermore, I can't avoid noticing the curious culture that has emerged with Java that you always have a single application/application instance for each machine. The dedicated server concept came to give reliability and CPU power, but WTF, do I really need a dedicated server to every application I run?
The objects are in memory. You implement the locking mechanisms yourself in your object model.
The "Prevayler Team" has written a persistent HashMap with a redo log, using the command pattern. This is exceptionally trivial and is in no way comparable to a database. A database has things like: 4GL query language, referential integrity constraints, data integrity, queryable metadata, separation of logical and physical layers, data independence, declarative rather than imperative querying, dynamically assembled queries, and gazillions of other things. These are the real features that we mean when we say "database." These features are absolutely necessary. Prevayler includes none of them. It is an extremely trivial persistent HashMap, that's all.
Thus, when the prevayler team says "throw away your database," I must assume one of two things. 1) They're trolling for publicity by saying outrageous and purposefully stupid things. Or 2) They are shockingly, mind-numbingly naive, and they don't know what a database is or what it does.
The author of Prevayler wrote this about himself: "Carlos Eduardo Villela is a 19-year old Brazilian graduate in Information Systems... almost 8 years experience has made him a Java and Python enthusiast."
Thus, I have to assume that the authors are mind-numbingly naive. Don't get me wrong, I'm sure the authors are very bright, and I know that some good insights that went into the implementation of prevayler. But let's not throw away our databases quite yet.
The somewhat naive authors of prevayler confidently announce the following on their website:
"No one has yet found a bug in Prevayler in a Production release. Who will be the first? [bold in original text]."
I already found a serious bug in the current production release. From the prevayler source:
ObjectOutputStream oos = logStream();
try {
oos.writeObject(command);
oos.reset();
oos.flush();
} catch (IOException iox) {
ObjectOutputStream does not guarantee atomicity. If your command object is larger than the page size of your disk, the "transaction" will take at least two page writes. A software failure between those page writes will lead to "half a transaction" being committed and a subsequent corruption of data. Once data integrity is lost, it is often difficult or impossible to recover. Prevayler has nothing to handle this case. Thus, prevayler does not correctly implement ACID, because it doesn't guarantee atomicity (half a transaction can be committed), consistency (referential integrity would be destroyed in such a case), isolation (this failure wouldn't be isolated to a single transaction) or durability (the problem would only show up upon reloading).
Finding this bug took very little searching. I am apparently the first person ever to find a bug in prevayler. Do I get a prize?
I posted a news item some days ago about oddb.org (see also linuxmednews.org). The complete 'datastructure' of oddb.org is done with Madlaine - a prevayler solution as mnemonic. Yes it has sped up development, yes the search queries are delivered faster, yes we are very happy with this new technology. You can have a look at our source at download.ywesee.com/ruby/oddb.org. You can test online at http://www.oddb.org. We are using Madlaine also on other mission-critical projects and it also works great there.
If you're tempted by the RAM-based performance of this, but you don't want to give up on all the good things a RDBMS offers, check out MySQL HEAP tables - they are a polymorphous implementation of MySql's tables that are designed to be stored in memory.