Prevayler Quietly Reaches 2.0 Alpha, Bye RDBMS?

ninejaguar asks: "Slashdot did an article on an Open Source product called Prevayler, which could theoretically resolve all the problems associated with OO's rough courtship with Relational databases. Slashdot covered Prevayler when it was still 1.x. Despite fear, doubt, and memory concerns, it has reached 2.0 alpha. Is anyone currently using this non-database solution in production? If so, has it sped development because of the lack of OO-to-RDBMS complexity? Was there a significant learning curve to speak of? The LGPL'd product could be incorporated into proprietary commercial software, and few might know about it. Is anyone considering using it in a transactional environment where speed is the paramount need? And, are there any objections to using Prevayler that haven't been answered at the Prevayler wiki? Would those who use MySQL find Prevayler to be a better solution because it's tiny (less than 100kb), 3000 times faster and is inherently ACID compliant?" Update: 09/24 19:25 GMT by C :Quite a few broken links, now fixed.

"We've used relational databases for years despite incompatibilities in SQL implementation. Accessing them from an OOP paradigm has been so tedious, that Object-Relational mapping technologies have sprouted all over the Open Source landscape. Some competing examples and models are Hibernate, OJB, TJDO, XORM, and Castor; which in turn have supporting frameworks such as Spring and SQLExecutor. Because SQL is the dominant form of interfacing with the data in an RDBMS, there's now a specification to offer it a friendlier OO face.

Most of the above, including the SQL-variants, arguably appear to add yet another layer of complexity (even if only at the integration level) where they should be taking complexity away. These solutions are put together by some very smart people, but it's inescapable to get that feeling someone is missing the forest (simple answer) because all the trees (incompatible models) are in the way. If there are so many after-the-fact solutions attempting to simplify relational database access and manipulation from OO, isn't it reasonable to think that there is something generally wrong with trying to cobble-together two disparate concepts with what are essentially high-caliber hacks? Is Prevayler a better way?"

Comments

Re:3000 times faster than Mysql?

[Anonymous Coward]

here

Broken link

[Otter]

Correct link for the previous Slashdot article -- important, since it provides the lucid, straightforward explanation of what the hell this thing actually is that today's pointless-link-filled, cutesy bit of Astroturfing neglects to offer.

Zope

[SlightOverdose]

This sounds a lot like the Zope ZODB. For those who are still stuck in the stone age, Zope is a python based application server that essentially uses object serialisation to store its data.

Imagine if every page in your website was an object- with methods, properties, Access Control, etc. You have different classes for different types of documents, and each document internally knows how to render itself. The ZODB is essentially one big persistant object orientated namespace- You dont have to parse your data into SQL and back again, it's always just there, elimenating a huge amount of work (and bugs!). Having worked with it for a year, I can certainly testify that it is leaps and bounds over relational databases for most things.

Re:Zope

[Evan]

Like the ZODB, except that the ZODB has pluggable storage backends, so that your objects can live in a FileStorage, DirectoryStorage, OracleStorage, etc. Except that ZODB has true transactions, with rollback and two-stage commit (allowing transactions to span other systems, such as an RDBMS). Oh, and except that the ZODB doesn't force all of your objects to live in memory at once; you set the cache size and it dynamically loads and unloads objects as necessary. Oh yeah, it also has ZEO for client-server access to object databases, so that you can spread your object processing across more than one computer. Did I forget anything? Sorry, the overdone Prevayler hype got to me :-)

By the way, ZODB is completely independent of Zope, although Zope relies on ZODB.

Re:Zope

[SlightOverdose]

The first thing you need to remember about the Zodb is that it's NOT a synonym for a relational database. You can use it to store the same kind of data, but you have to approach the problem from a completely different point of view.

At first people try and grasp how they would simulate tables, SQL queryies, etc. The answer is, you don't. If I want to store a large amount of records, I create a python dictionary (Think Java Hashtable, only better). I've written a 'product' called Shelf, which provides some simple APIs to do this. I can put dictionarys within dictionaries within dictionarys ad infinum, as well as lists, tuples, string, ints, whatever.

The 'query' language in this case, would be Python itself. Using list comprehension, I can search a dictionary of several hundred thousand records in a few hundred milliseconds.

i.e. return [(person['username'],person['password']) for person in staff if person.department in ['Marketing','Legal']]

Seems complex, but once you get the hang of it its more powerfull than SQL. (I can also write a normal python script to parse the data if need be).

Python supports several ways of allowing runtime parsing of code (ad-hoc querys, in this case).

When you modify data in zope, it records all transactions. If something fails, it rolls back the whole operation. You can also manually roll back any operation at a later date if you want. It also doesn't store the entire database in memory- it only loads whatever you access, and caches it for a period of time. This can all be tuned.

Zope supports spanning the ZODB over multiple servers. I've heard (although I'm still trying to confirm) of a Zope database containing over 300 gigabytes of information with no problems. I'll try and find a url.

The Zodb can only be accessed by zope, but all the code to do so is in a seperate module which can easily be imported into a standalone python session. On several occasions I've manually accessed the data using a python shell. You can also access the Zodb via FTP and WebDAV.

You can print a dictionary out as a string, save to a file, etc, and import into another python program at a later date using str() and eval(). This is trivial. Porting to other languages would require a little parsing, but is not out of the realms of possibility.

Re:Serialize once a night?

[batkid]

I'd say RTFA, but the article is just a very plain website with very little information. However, if you look into it, all the transactions are actually SAVED so you don't have to worry about losing data in the middle of the day (i.e. a journaled file system). The daily serialization is just a snapshot of the system so you don't have to start from the beginning every time you restore the system.

Re:Please, enough of the hyperbole bullshit

[Anonymous Coward]

I agree that this was a pretty shameless "story", but your comparison to "Bowling For Columbine" is somewhat ironic, since Michael Moore is at least as manipulative as the mass media he decries.

Here is a list of gross inaccuracies in Bowling For Columbine

Re:Serialize once a night?

[Elwood P Dowd]

At all times, they serialize commands to log files. Those commands are sent to every machine in your cluster. (A clock tick is one of those commands.) When you backup at night, one of those machines serializes it's objects to disk, and racks up commands to complete. Once it's done serializing to disk, it runs through the commands and catches up.

If all your servers go up in flames, you take your object serialization, and reload it. Then you run through the command logs, and you're right back where you were when the machines died. (Assuming you've got your command logs)

So no, they don't have their heads up their asses in that respect.

Re:Modding of Articles

[batkiwi]

Not to start the crosssite flamewars, but you're looking for http://www.kuro5hin.org/

or another slash-esque site running scoop software http://scoop.kuro5hin.org/ if kuro5hin doesn't jive with you

Distributed environment

[Lysol]

I tried Prevayler and even loaded in 50k records from an old product table and object-ized them.

Yah, it was fast. Searches were great. I even figured out how to do complex object joins.
But I started having trouble when I tried to figure out how all the transactions worked. This was complicated by the wiki they're using, which was quite useless to me. It lead to many dead ends.

However, the real reason I found that I could not (possibly ever) use Prevayler is becuase it seemed the approach was for one machine and one machine only. There were no distributed mechanisms at all. Or at least, not how I'm used to working.

All the systems I've worked on in the past five years have all been with clusters of app servers. If all the objects on one machine were all in memory, then I couldn't think of an easy way to get them into the memory of the other machines. There was some talk about using Java Spaces, but that's kinda where I dropped off.

And the other issue was getting to the data from non app server machines. Like stuff to do back end reporting and things like that. I bascially figured out that for n-machine access, I needed something that, well, acted like a database.

I thought the idea was very interesting and maybe these things have been addressed. But when I really sat down with it for a few weeks, it just didn't pan out for me.

Re:Distributed environment

[Anonymous Coward]

For in-memory distributed database, please have a look at Mnesia (http://www.erlang.org and http://www.erlang.org/doc/r9c/lib/mnesia-4.1.4/doc /html/index.html)

--
Mickael Remond

Re:Distributed environment

[onash]

this combined with JavaSpaces is really intresting, like http://www.j-spaces.com/database1.htm . I don't understand why Prevayler has reached Slashdot two times now, when its nothing new..

also, this is more an DBMS alternative than a RDBMS as you have to make all the relations in you'r own code, but not in the "DBS".

Re:Serialize once a night?

[angel'o'sphere]

This post is not INSIGHTFULL this post is STUPID, the author did not read the web site of prevayler.

e.g. he claimes:

And if you have a system crash, you lose all your data for the whole day.

Thats wrong. You lose nothing. Hint: before images(not really but similar). Hint: Command Pattern.

ARG, need holy water, a design pattern, the pure EVIL of OO geeks .... yeah forget it ...

angel'o'sphere
P.S.
I think if you needed half a second fore some few 1000 entries to serialize, yopu should have tried how long it takes with a million? One second more I guess. If not, consider using a BufferedWriter ...

Re:Why Java?

[Fred Nerk]

E.g.: PHP and Perl have no build in object
serialization and deserialization.

Perl:
use Storable;
my $string = freeze $bighash;
my $anotherhash = thaw $string;

PHP:
$string = serialize($bighash);
$anotherhash = unserialize($string);

Both of these are built-in (or close to it).

Re:Please, enough of the hyperbole bullshit

[Delirium Tremens]

Please do not compare this to hsqldb. The hsqldb project at least offers some drivers for remote connections to the persistent store.

Re:Persistance does not make a DB

[Elwood P Dowd]

I'm not sure what the ACs' problems are. My post has plenty of grammatical errors as well.

But the point is, in business, not only do we have a SQL database, there are thousands of vendors with products that expect SQL/ODBC databases.

Sure, if I'm writing something that requires a database, but will never ever have to interoperate with anything else that I haven't predicted, then Prevaylent is in the running. Otherwise, not. Their website is gleefully ignorant of this fact.

They imply that anyone not using their product is idiotic. They do it many times, and here the poster acts like their relative unpopularity is an enigma. They sound like assholes. I'll stand by that.

RDBMS != Object Store

[Frater 219]

It's true that an RDBMS doesn't map well to the object-oriented ideology. That's because an RDBMS does not store objects, or anything like them.

The object-oriented ideology as instantiated in C++ and Java is founded upon breaking data into objects, bearers of identity, which belong to classes, bearers of structure and behavior. (C++ and Java make little account of metaclasses, which are used in more dynamic object systems such as Python's class system and Common Lisp's CLOS. Templates are not metaclasses.) Objects have identity, so they can be equated; they are the unique bearers of attributes about themselves; and each object's structure is dictated by the class to which it belongs.

When object-oriented partisans look at a database, they see its relvars (or table headers) as bearers of structure and think of classes, and its tuples (or rows) as bearers of identity and think of objects. They see a database as a place to store objects persistently.

But this is not what an RDBMS does. An RDBMS isn't an object store; its relvars are not classes and its tuples are not objects. So what is an RDBMS? What is "relational" anyhow? Relational databases are founded upon relational mathematics, which is what you get when you cross set theory with predicate calculus.

Set theory is the branch of math that deals with collections of elements which behave according to formal axioms. Set theory lets you say, for instance, that if you have a non-null set R and a non-null set S, that you can construct a set R*S of all the possible pairs of elements from R and S.

Predicate calculus is the branch of logic that deals with quantified statements about entities. It lets you formalize logical arguments such as the syllogism: All men are mortal; Socrates is a man; therefore, Socrates is mortal. Predicate calculus deals with generalizations and instantiations of those generalizations.

What do we get by combining set theory and predicate calculus? We get a system that allows us to operate upon sets of tuples of values satisfying predicates. A relation holds tuples of values which make some predicate true. For instance, consider the predicate "Person x owes me y dollars." Tuples which satisfy this predicate will be pairs (x,y) for which the sentence is true. For instance, if Fred owes me 40 dollars, (Fred, 40) satisfies the predicate. It could thus be a tuple in the relation described by the predicate -- the one relating people's names to how much they owe me.

With the relational algebra (or an RDBMS) we can do operations upon this relation and others. We could, for instance, select a result set of all those people who owe me more than 50 dollars -- or join this result set with those people's addresses. Whatever result set we ask for will be calculated from the facts in the database. We might get back this result set:

(Barney, 75, 40 Elm Road)
(Megan, 60, 9 High Street)

Now, are the elements of this result set objects in the object-oriented sense? They are not. They do not have identity. The tuple about Barney is not Barney himself, or even a machine representation of him. It doesn't uniquely store attributes of Barney -- after all, we created it by joining tables which also contain such attributes. It is not even, truly, a fact about Barney exclusively -- for it is also a fact about the number 75, and about the address 40 Elm Road. It isn't an object; it's a tuple value, and values do not have identity as objects do.

Moreover, note that by joining, we can construct new relations from old ones. Thus, not only are tuples not objects, but relvars are not classes. After all, in OO we do not create new classes by joining, but by inheritance or encapsulation of old ones -- and creating a new class does not cause it to be instantiated into known-correct objects.

So what does this matter to OO people faced with RDBMS as a

Generic Prevayler

[_wintermute]

A little while ago, inspired by helpful members of the Prevayler mailing list I created a generic Prevayler system for object persistence in one of my current projects.

Basic objects implement a 'Persistent' interface and can also be 'Containers' which are simply Persistent objects that contain other Persistent objects. Basic CRUD operations can then be performed with a couple of simple Prevayler transactions. The system maintains the object graph hierarchy, aloowing you to perform transactions and queries using a 'path' that maps through the various containers. The code below shows the creation of a new persistent object (myItem) and Adds this to an object at the path 'item/subitem' (so there is a Container object located at 'item' which Contains a another Container called 'subitem' - myItem will be placed in the 'subitem' Container. The code then executes a 'select' , which will return the 'subitem' Container, which we can then use to access our newly created object if we so desired. We could also execute select("item/subitem/myItem.id") to access directly the object we just created.

Persistent myItem= new Persistent( id,. .. );
AddPersistent addSub = new AddPersistent("item/subitem", myItem);
Container container = store.selectItem("item/subitem");

There is also a Prevayler Transaction to Update Persistent objects, using a similar path methodology. After we make changes to myItem, we can then use code like that below to execute the Update.

UpdatePersistent update = new UpdatePersistent("item/subitem/myItem.id", myItem);

Notes: the query language is obviously begging for XPath.

As the whole system is generic, it means that after a select you have to cast to the correct type, which can be a pain. For example, you may have a Customer object which implements Container. The Container returned from a select will have to be explicitly cast to Customer. I guess you could write a typed wrapper.

Generic typing also means that if you want a single Container to maintain more than one collection of Persistent or sub-Containers in a transactional manner, you may have to have the one actual Map behind the scenes, and have accessor methods that grab objects from this Map and cast to the correct type. I haven't quite worked this part out in much detail..

I think I will publish the source code after I have done some refactoring and come up with a robust Exception mechanism. Although I must admit that Exceptions often do my head in. It seems that a lot of the time if your system breaks, you are simply fucked and there is often not a lot that you can do other than get out as gracefully as you can.

http://www.info-architects.net/vapourized/

Re:Well, when you operate on the atomic level...

[pivo]

Atomic, The A in RDBMS ACID (Atomic, Consistent, Isolation, Durable) transactions.

Atomic transactions consist of grouping of changes to tables or rows such that all or none of the changes take place. A rollback operation can reverse all the actions of the atomic transaction.

From the Greek atomos for indivisible

Re:Dunno if this link has been brought up before..

[Kalzus]

Nutz. Slash ate the link because I am a moron. The link is: [http://www-106.ibm.com/developerworks/library/wa- objprev/]; referenced by the original Slashdot article.

Use Open for Business and forget OOP

[almax]

I have been doing OOP for over a decade, but then I stumbled on Open for Business (http://www.ofbiz.org) which has an entity engine that basically throws out the OO model all together. At first I was offended, but as I thought about it, it makes sense - a bean class for each table just bloats things. Instead, OFBiz's entity engine handles the persistence and the service and event modules handle the business logic.

Re:Please, enough of the hyperbole bullshit

[Anonymous Coward]

These guys need to read the LRVM paper - a poor attempt by the OS community: PDF (by Satya and Co. at CMU). Of course, as a database researcher I don't think much of Prevlayer .. mainly because the database community went through this in 1990. Great paper to read is Carey and Dewitt's Of Objects and Databases: A decade of turmoil - it won the "Test of Time" award in VLDB 1996.

yes, it has sped up development, yes it's great.

[Zeno Davatz]

I posted a news item some days ago about oddb.org (see also linuxmednews.org). The complete 'datastructure' of oddb.org is done with Madlaine - a prevayler solution as mnemonic. Yes it has sped up development, yes the search queries are delivered faster, yes we are very happy with this new technology. You can have a look at our source at download.ywesee.com/ruby/oddb.org. You can test online at http://www.oddb.org. We are using Madlaine also on other mission-critical projects and it also works great there.

Re:Why Java?

[LizardKing]

I thought I'd amuse myself by checking out your website, and came across this gem:

Funny Code

... which starts off by cataloging some Perl atrocities. Then under the C/C++ section you write:

haven't got any good examples of C/C++ funny code yet

... and you still felt the need to question my own dislike of Perl?

Chris

alternatively, use MySql HEAP tables

[tartley]

If you're tempted by the RAM-based performance of this, but you don't want to give up on all the good things a RDBMS offers, check out MySQL HEAP tables - they are a polymorphous implementation of MySql's tables that are designed to be stored in memory.

Re:From the faq

[William Tanksley]

One acronym: VM (memory not machine).

I'm pro-OP, like you, but I have to point out that this isn't a good stance. Virtual memory doesn't help object prevalence; memory pagers aren't smart enough to handle the demands, and your OP system will fall behind a proper RDBMS.

Now, an OP system doesn't have to keep all its objects in its working set (i.e. in memory) at all times; intelligent design of this will result in the ability to use OP with a much larger problem than you have memory for, without a significant speed loss. But don't expect to be faster than a quick'n'dirty pseudo-RDMBS anymore.

A farm of prevlayer boxes is precisely one of the applications that 2.0 is intended for.

-Billy