Oracle Unveils New Open Source BerkeleyDB Release

Mark Brunelli writes to tell us that Oracle has released the newest version of the open source Oracle BerkeleyDB Java Edition. From the article: "The new release of the Java embeddable database is the third to come out in three years and the first new version to come out of Sleepycat Software since Oracle purchased the open source stalwart back in February. Rex Wang, Oracle's vice president of embedded systems and a former vice president of marketing at Sleepycat, said the latest release lets Java developers take advantage of a new Persistence application programming interface (API) that provides greater flexibility and new performance optimizations that enable applications to run faster."

Comments

Rex Wang?

[Anonymous Coward]

Rex Wang? Wasn't she one of the supervillains in one of the Austin Powers movies?

Re:Rex Wang?

[Rosco P. Coltrane]

Well he is Oracle's vice president...

Re:Rex Wang?

[Bootvis]

He is King Cock.

Re:Rex Wang?

[P3NIS_CLEAVER]

Braces Rex Wang.

Many using SQLite instead.

[Anonymous Coward]

I know that many developers, especially on embedded or hand-held systems, are starting to use SQLite these days instead of BDB. SQLite is small enough, and efficient enough, to be used suitably in such environments. It offers SQL support, which BDB does not. And its library-based architecture is a real plus in such environments. The fact that it's public domain makes it even more appealing.

SQLite is even starting to take over from MySQL for many smaller websites. The overhead (both in terms of system resources and administration) can't be justified when using a server-based system, especially when SQLite does the job just fine.

What we may even see is MySQL squeezed out of the market. I know many database developers who are starting to use PostgreSQL for their higher-end databases, and SQLite for the lower-end. MySQL could be considered the best contender for mid-range DBs, but SQLite and PostgreSQL keep converging, thus pushing MySQL out to a point of irrelevance. Of course, there has been a decade and more of development using MySQL, so for legacy reasons alone it will continue to be used for ages.

Re:Many using SQLite instead.

[AKAImBatman]

I know that many developers, especially on embedded or hand-held systems, are starting to use SQLite these days instead of BDB. SQLite is small enough, and efficient enough, to be used suitably in such environments. It offers SQL support, which BDB does not.

You're missing the point. Putting aside for the moment that we're discussing the Java version of BDB (which would be useless on the types of embedded platforms you're talking about), BDB is a basic database engine. Which means it's the type of software you can use to build a database management system on top of rather than being a DBMS in its own right. SQLite and HSQL are complete DBMS packages that will only meet your needs if an SQL engine is what you want.

For example, if you're looking to build an Object Storage Database, SQLite and HSQL would both be terrible choices. But BDB would fit the bill perfectly.

Re:Many using SQLite instead.

[frodo from middle ea]

that's the exact reason why Oracle and IBM have open sourced BerkeleyDB and cloudscape(derby) respectively.

Many developers use some sort of embeded / open source databases for prototyping and for small scale projects these DBs often end up being the production databases.

The reasoning is we have written some triggers/stored procedures etc in PostgreSQL/MySQL etc, and now it's too much of a hasel to convert every thing to Oracle or DB2.

This leads to a lot of loss in revenue for the big guys, especially from the small businesses. by open sourcing BerkeleyDB and cloudscape, Oracle and IBM wish to divert these developers to their DBs. Cloudscape supports all the datatypes of DB2 and I am sure BerkeleyDB supports Oracle Datatypes and some extensions as well.

So prototype in these open source DBs and migrate easily to Oracle or DB2 in production without too much changes.

Re:Many using SQLite instead.

[laffer1]

It depends on the person I suppose. I personally like MySQL better than postgresql as I find it easier to administer. (less crap on the file system, etc) I will look again. Perhaps postgresql has better jdbc support now than the last time I looked.

I'm glad to see oracle releasing open source products.

Re:Many using SQLite instead.

[ralph alpha]

Berkeley DB and SQLite don't even come close to filling the same niches. Berkeley DB is a highly optimized, generic, low-level tree-based database that can be used for a variety of purposes -- including providing the backing store for an SQL system. SQLite is an embeddable SQL parser wand extremely simplistic backing store. Berkeley DB is like a body without a head. SQLite is like a head without a body (or at least not much of one). You might want to note that "database" does not imply "SQL."

Re:Many using SQLite instead.

[JebusIsLord]

Sort of... SQLite isn't daemonized, it is more of a db library and set of apps like berkeley is. But usage-wise, it is a full SQL DBMS and therefore competes with MySQL.

Re:Many using SQLite instead.

[Skapare]

If I was already using BDB or the like, what does SQLite really give me? I didn't need SQL if I was using BDB (and many things really don't need it at all).

Larger applications are still going to need the power of a server based DB. There are a lot of choices like MySQL, PostgreSQL, and even Oracle when you can finally afford it. But I just don't see SQLite ever breaking through the "embedded library DB" ceiling to bump out MySQL. If Larry wants to destroy MySQL, then he should release a free open source version of "OracleLite" making sure everything written to work on that works w/o change on a full Oracle database. OracleLite wouldn't even need to do all that MySQL does to really put the pressure on.

Re:Many using SQLite instead.

[ajs]

Well, technically BDB is a DBMS. It's not an RDBMS, because it lacks relational table management features, but it is most certainly a DBMS (and far more powerful than many of the most popular commercial DBMSes of just a few decades ago).

Re:Many using SQLite instead.

[AKAImBatman]

Just an FYI, daemonization (is that even a word?) has very little to do with Database Management Systems. Daemonizing is simply a way of providing remote database services via a client/server architecture. Prior to the rise of common networking, DBMSes were regularly compiled into the program itself. Some mainframes even went as far as to compile the byte-level database access routines into every program. Which meant that a change in database schema would require that all programs be recompiled.

<old-foggie>You kids these days, with your fancy-smancy "Database Servers" think you know everything there is to know about database management, don'chaya? Why, back in MY day, stored the data in a fixed width heirarchy of records, and we LIKED it that way!</old-foggie> ;-)

Re:Many using SQLite instead.

[cortana]

Neither are most 'R'DMBSes ;)

Re:Many using SQLite instead.

[mritunjai]

Sorry, but you're totally mistaken about BerkeleyDB. Its just a database engine. i.e. you give it a key and value pair and it stores them. You give it the key and it retrieves the value. There is support for foreign keys and its completely transactional, but there is no SQL interface and no 'datatypes' whatsoever.

It'd take some time to figure out what exactly is Oracle trying to do with it.

Re:Many using SQLite instead.

[hey!]

I think you hit the nail on the had here.

If you are a database person, as opposed to a programmer who needs some kind of persistent store, database implies the prsence of certain abstractions that BDB doesn't have. It's the abstractions that matter, because they implement a kind of data reusability. This is a big aspect of databases that programmers don't always appreciate, but if you remmber the early adoption days of the relational model (as I do), the idea of abstracting data from its application was a big thing. We had methods already after all for managing storage: hash tables, b trees, record pointers. We even had models (the hierarchical model). But relational databases allow you to create a model of your data that is abstracted from what you are doing with it.

I'd go so far as to say that absracting data for reusability is what distinguishes a true database from other kinds of collections of data. Of course people do use words in a vague way, but this property definitely separates out what is unquestionably a datbase from what is only arguably a database.

There is no question of BDB being a database management system. Collections of data in BDB format might arguably be databases depending on the effort the programmer went through, but usually they're just indexed files.

Things like SQLLite that implement a generaly query and data manipulation language (e.g. including but not limited to SQL) is clearly a database management system, although it may not be a very good one.

Re:Many using SQLite instead.

[C_Kode]

What we may even see is MySQL squeezed out of the market. I know many database developers who are starting to use PostgreSQL for their higher-end databases, and SQLite for the lower-end.

I can understand porting to PostgreSQL for higher end stuff since PostgreSQL is more enterprise ready. I can see small home user sites using SQLite for there database. Even small apps can store configs among other things. On the other hand, For simplistic data storage but large amounts of it. (storing blogs, BBS (in todays sence not the old sense) or other of similar type) I think MySQL is best suited. It has replication, is extremely fast and very easy to manage. You can cluster it (in a sense) that allows it to scale to large read access without all the management overhead. MyDNS break several DNS barriers and it does it because it uses a DB as it's data storage. It scales because of MySQL's ability to scale and it's easy to manage because MySQL is easy to manage. It's ability to scale all you have to do is look at Slashdot. If you ratio it's total management to it's abilities it's the best DB. It just doesn't fit all jobs.

The only question to ask is: What does your job require?

Just as a note, my DB experience include.

MySQL (Linux & Windows)
PostgreSQL (Linux)
Oracle (Solaris & Linux)
Progress (AIX)
BerkleyDB (Linux)
SQL Server
MS Access
SQLite
Sybase (Tru64)

While I don't currently run this wide range of DBs I found each of them fit the roll that they were used even if you could have mixed and match some of them for what uses they were used with the same result.

Re:Many using SQLite instead.

[Anonymous Coward]

Enlighten us, ralph alpha: What is so simplistic about SQLite's backing store? A storage engine that puts a complete database in a single, relatively small, byte-order independent disk file with ACID transactions seems pretty advanced to me. What other storage engine has these features? And what features does SQLite's storage engine lack that you think are important?

Re:Many using SQLite instead.

[big dumb dog]

It'd take some time to figure out what exactly is Oracle trying to do with it.

Who has that kind of time?

Re:Many using SQLite instead.

[kfg]

we're discussing the Java version of BDB (which would be useless on the types of embedded platforms you're talking about)

Why? This is exactly the sort of thing Java was invented for. Don't confuse whichever head of the Hydra you are familiar with with the body of the beast.

KFG

Re:Many using SQLite instead.

[AKAImBatman]

This is exactly the sort of thing Java was invented for.

Java wasn't invented to run on systems with no JVM and lack of dynamic library support. SQLite and BDB, however, will run on those systems.

Re:Many using SQLite instead.

[kfg]

Java wasn't invented to run on systems with no JVM and lack of dynamic library support.

Well, D'oh! Cars weren't invented to run without wheels either. So if you want a car, you put wheels on it.

SQLite and BDB, however, will run on those systems.

Why do you believe those are the systems OP was talking about? There's nothing in "handheld" that implies that.

Bias note: You'd probably have to shoot me before you could get me to use Java in an embedable system, or any other for that matter. I'm no Java fanboy defending his religion. I'm a major detractor. As far as I'm concerned the main purpose that Java serves in the universal grand scheme of things is to have something to point and giggle at.I honestly simply don't see your point is all.

KFG

Re:Many using SQLite instead.

[AKAImBatman]

1. He suggests SQLite instead of BDB. Which means that the system must have native APIs exposed.

2. He mentions that BDB's "library-based archtecture" is a problem for said platforms. This denotes that he statically compiles SQLite on systems lacking dynamic libarary support.

3. Nowhere does he mention Java capabilities, such as the type you'd use to power an engine like HSQL. Instead he goes off into discussions about larger native engines like MySQL.

Argue all you want, but he was referring to non-Java platforms. As such, this version of BDB wouldn't be very useful. Also, while I haven't checked out the code yet, it's even likely that it won't run on a J2ME device due to optimizations utilizing memory mapping and File Channels. This is a key optimization in the C version, so it's likely that the design has been moved over to take advantage of J5SE's NIO capabilities.

Re:Many using SQLite instead.

[kfg]

Argue all you want

Ok. :)

1. He suggests SQLite instead of BDB.

That's right, as an alternative, suggesting the use of BDB as an option in the systems he's talking about.

2. He mentions that BDB's "library-based archtecture" is a problem for said platforms.

No, he says that SQLite's library-based architecture is a plus in such systems.

KFG

Re:Many using SQLite instead.

[AKAImBatman]

No, he says that SQLite's library-based architecture is a plus in such systems.

Ah, sorry. I misread that part. My point still holds, however. He's talking about native code, not Java.

That's right, as an alternative, suggesting the use of BDB as an option in the systems he's talking about.

He's comparing the native version of BDB. SQLite doesn't come in a Java version. And for J2ME platforms, you don't get the option of deploying a native binary. You either deploy all-Java, or you don't deploy at all. Which means that he should have suggested HSQL instead of SQLite.

Re:Many using SQLite instead.

[kfg]

SQLite doesn't come in a Java version.

Another plus. Now if only something could be done about the SQL part.

KFG

Re:Many using SQLite instead.

[AKAImBatman]

Now if only something could be done about the SQL part.

BDB comes in a (generally better supported than the Java version) native version AND it doesn't support SQL. So I guess that's double-plus good in your book.

Re:Many using SQLite instead.

[cant_get_a_good_nick]

Collections of data in BDB format might arguably be databases depending on the effort the programmer went through, but usually they're just indexed files.
This is of course the way that MySQL started, a collection of BDB files binded together to appear on the outside as a collection of tables.

Re:Many using SQLite instead.

[david.given]

SQLite is even starting to take over from MySQL for many smaller websites. The overhead (both in terms of system resources and administration) can't be justified when using a server-based system, especially when SQLite does the job just fine.

SQlite is so incredibly cool (for a database, of course; databases are intrinsically limited in coolness). It's tiny, it's easy to set up, it's portable, it's faster than MySQL for most operations, and it's not just open source, it's actually in the public domain. It doesn't do well if there are multiple processes trying to access the same database, and there's no network layer at all, but given that it stores the entire database in one file, it's the ideal choice for any application that needs to use complex state. I use it for keeping track of email addresses in an SMTP proxy --- it was so easy to use I was amazed.

I'd strongly, strongly recommend having a look if you're writing something that might want a database. Hell, given the command-line front end, you can even use it from shell scripts...

Re:Many using SQLite instead.

[Anonymous Coward]

For example, if you're looking to build an Object Storage Database, SQLite and HSQL would both be terrible choices.

Axiom is a Python object database built on top of SQLite, and it is claimed to be pretty awesome.

Re:Many using SQLite instead.

[unPlugged-2.0]

Finally!!

A topic I actually know something about.

There are several things wrong with your statement regarding the comparison of the two database. The first is that you assume that a database has to be relational in order to promote reusability. There can be many ways of promoting reusability including OODB and even the old ISAM model. BDB offers a very unique model of reusability and there are no constructs or foreign key relationships that are not possible to implement in BDB. Case in point MySQL can use BDB as it's underlying data mechanism.

The real questions is what kind of developer are you. If you have been used to developing with SQL then you will definitely find BDB's implementation hard to use. However if you think in terms of developing programmatic structures then BDB is actually pretty intuitive and you can do quite powerful things with it. We have api's that we have developed using the binding mechanisms that can do anything a sql database can with much less overhead of the query and row mechanism. Also reusability and sharing of tables has not been compromised in any way. Simply put BDB maps much better into an Object Oriented way of developing data vs. the Relational way of developing data. I don't want to start a religious war and claim one is better but I obviously have shown my bias towards the OO model.

However, I am in no way questioning the usefulness of SQL. It is merely a standard which developer can adhere to and be able to use a tool quickly. However that in itself does not measure the usefulness of a DBMS.

The way I look at is this and it sums up what I have researched in the course of building our embedded RFID devices.

- If you are looking for an embedded solution you want two things - performance, footprint and memory mgmt. The leader in this at least from our benchmarks so far has been BDB hands down.
- If you want a non-embedded solution then you need to take into account all the other features like failover, transactions, warehousing yada yada and for this I would rather go with something full featured like Postgre, EDB or <gasp> oracle

- If you don't feel like developing and just want something that does the embedded portion of your requirements then the myriad of embedded sql db's should suffice.

But when performance is an issue I haven't found anything that you can get to run faster than BDB (taking into account what is needed for small memory/limited results type applications)

My findings only (your results may vary)

--

Re:Many using SQLite instead.

[lpontiac]

It'd take some time to figure out what exactly is Oracle trying to do with it.

They bought it so that MySQL couldn't.

Re:Many using SQLite instead.

[Ed Avis]

For a long time MySQL was notorious for letting you put February 30th into a datetime column. Does sqlite still happily permit "hello there" in an integer column? To me this negates the biggest reason to use an RDBMS - to get some checking of data integrity and foreign key constraints.

Re:Many using SQLite instead.

[shutdown -p now]

It also has an excellent Tcl interface.

Re:Many using SQLite instead.

[Anonymous Coward]

Just to nitpick. It's Postgres, or PostgreSQL, not Postgre.

Sweet!

[AKAImBatman]

I'll have to see if they've finally added support for partial record retrivals. It used to be that this feature was only in the native code edition of BDB. It ended up being a showstopper for the Java version last time I tried to use it.

Re:Sweet!

[AKAImBatman]

In case anyone else is interested in this, the JavaDocs appear to make no mention of the previous limitations BDB imposed on partial records. Can anyone else confirm that BDB no longer loads the entire record to read partial records? If so, I have a few multi-gigabyte uses that this would be perfect for. :-)

Re:Sweet!

[greybird]

Hello, this is Mark Hayes, I'm one of the BDB Java Edition developers.

There are no restrictions on accessing partial records in BDB Java Edition -- it is the same as the BDB C Edition in this respect. However, you may have a misunderstanding about the way partial records work in both products: neither product has a BLOB capability.

In both products, when you do a partial get/put of a record, the entire record must be loaded into the BDB cache. The only advantage of a partial get/put is that it reduces the amount of memory copied between the BDB cache and your buffer. If you need the performance of a true BLOB implementation for very large records, I'm afraid you won't get it by using either BDB product.

We do get requests for a true BLOB capability from time to time. We have not implemented it yet, because it will take significant effort to implement it in a way that performs well for very large records. And the number of requests have not yet been high enough to outweigh other requested features.

Please keep the requests for this capability coming in if you need it. We do factor in these requests when we do product planning.

Thanks,
Mark

Re:Sweet!

[AKAImBatman]

Hi Mark, thanks for taking the time to respond!

There are no restrictions on accessing partial records in BDB Java Edition -- it is the same as the BDB C Edition in this respect. However, you may have a misunderstanding about the way partial records work in both products: neither product has a BLOB capability.

Well that's not so good. I was under the impression that BDB was able to easily handle large binary chunks up to 4GBs in size. From this page:

Because both keys and values can be up to four gigabytes in length, a single record can store images, audio streams, or other large data values. Large values are not treated specially in Berkeley DB. They are simply broken into page-sized chunks, and reassembled on demand when the application needs them. Unlike some other database systems, Berkeley DB offers no special support for binary large objects (BLOBs).

So as long as the data stayed small enough for BDB to address (i.e. <4GB), I thought that BDB was supposed to only load into memory the requested chunk of data. Was this an incorrect assumption? If so, how are large data sets handled on systems with limited memory? On many of the systems that BDB has traditionally been deployed on, it wasn't unusual for several hundred megs to be too large for main memory AND swap space combined.

It seems to me that BDB should be using memory mapping to force the operating system to manage memory chunks for it. Which would allow BDB to handle data sizes far above the total available memory. I thought that BDB already did this?

Anyway, thanks for your help!

Re:Sweet!

[MarkCollette]

Please do a quick reply to me when you reply to the other guy's question, so I can read it too :)

Re:Sweet!

[owlstead]

Warning: the word showstopper has a double meaning, but the one from Meriam Webster says the following:

Main Entry: showstopper
Pronunciation: -"stä-p&r
Function: noun
1 : an act, song, or performer that wins applause so prolonged as to interrupt a performance
2 : something or someone exceptionally arresting or attractive

Now, in IT it is mostly used to mean exactly the opposite, but it seems to me that introducing that meaning as well might let things get a bit confusing...

That is: I suppose you weren't using it in the "gold crown" sense :)

Re:Sweet!

[greybird]

From your reply I'm not sure whether you're asking about a single record or the entire database.

Each record is loaded into cache separately, and records are evicted when the cache is full according to an LRU algorithm. The total size of a database can therefore exceed the size of main memory and swap.

But for a single record, in most cases it is completely loaded into cache, even when a partial get/put is performed. Even when using memory mapping with the BDB C Edition, in most cases the complete record is loaded. This is why we say that there is no special support for BLOBs in BDB.

If you have further questions, please post to the BDB newsgroup for questions on the C Edition, or the bdbje mailing list for questions on the Java Edition, or to support@sleepycat.com for either product. I'm afraid I will miss your reply if we continue to discuss this here. Here are links for posting questions:

http://dev.sleepycat.com/community/discussion.html

Mark

Re:Sweet!

[greybird]

I posted a reply.
Mark

Alternate link

[Nerdfest]

Since TFA seems to be slashdotted, here's an alternate. I'm assuming that version 3.0 (May 17th) is the new version.

http://www.oracle.com/corporate/press/2006_may/ber keleydb-je-ga.html/

Re:Alternate link

[goldaryn]

Main article link is working again.

By the way, parent's link has an rogue trailing slash at the end, you may have to remove it to see the page.

Or just click here (same link)

"high-performance" Java?

[NynexNinja]

I thought it was funny that the article mentions "high-performance" and "Java" in the same sentence. IMHO, it's inappropriate to be writing "high-performance" applications in an interpreted language like Java. For the same amount of keystrokes, I can do it right the first time and write it in C and not have to worry about performance issues.

Re:"high-performance" Java?

[JebusIsLord]

oh for chrissakes don't turn this into a "java is slow" argument. 1996 called, and they want their discussion back.

Re:"high-performance" Java?

[Nerdfest]

For many (most?) things, java seems to be very close to C in speed. GUI interfaces although better are still lacking, but can generally be made usable. This aside, it's quite easy to interface to a java DB engine from a java application ... you don't need to deal with JNI, as would be the other option for an 'embedded' engine.

Re:"high-performance" Java?

[Anonymous Coward]

Well sir you are wrong. Java is increasingly being used for high performance, real time applications. If you don't believe me look up Sun's HotSpot VM and the specs for Java Real time. Check out the robotic vehicle called tommy (http://java.sun.com/developer/technicalArticles/J 2SE/tommy/ ). It used standard JAVA Se AI and Java ME for feedback control.

Re:"high-performance" Java?

[$RANDOMLUSER]

You're missing the humor in his troll:

> For the same amount of keystrokes, I can ... write it in C...

Too funny.

Re:"high-performance" Java?

[JebusIsLord]

Ack! I missed that, too... good troll. Comment revoked!

Re:"high-performance" Java?

[Anonymous Coward]

I'll bite

Java performance is often better than C when allocating and deallocating memory

Re:"high-performance" Java?

[arigi]

i don't think that java is slow; i actually think that BDB JE is slow (at least the previous version). i had to use it to build a p2p search engine for a class and in most cases, it turned out to be our bottleneck. after large numbers of records i found that its whole commit-log thing was taking quite a long time; and it's nigh impossible to find others with the same problem and a solution (i found a guy with 8 million records that never got a response to his problem). personally, this was enough to make me dislike BDB in all cases.

Re:"high-performance" Java?

[AKAImBatman]

i don't think that java is slow; i actually think that BDB JE is slow (at least the previous version).

The previous version loaded the entire record prior to reading/writing. This was a huge performance bottleneck if you were working with records above a kilobyte or two in size. In addition, the old version didn't make use of memory mapping like the C version did.

I'm still trying to confirm the exact design of the new version, but I get the impression that they're using memory mapping in Java to give the same sort of performance and sub-record select capabilities as the C version. In addition, they've added the ability for BDB to act as a natural object store.

Re:"high-performance" Java?

[rubycodez]

funny, in those benchmarks showing the speed of java, all I see is procedural code with primative data types. Start building objects, garbage collecting, and casting and my, my what a pig.

Re:"high-performance" Java?

[geoff lane]

OK Java isn't slow. But some Java programmers are terrible and write bloated and slow programs.

Re:"high-performance" Java?

[csnydermvpsoft]

funny, in those benchmarks showing the speed of java, all I see is procedural code with primative data types. Start building objects, garbage collecting, and casting and my, my what a pig.

So when you stick to basically writing code like you would in C, it's the same speed, but when you add more complicated, time-saving, features, it slows down? Wow, that's surprising. (</sarcasm>)

Re:"high-performance" Java?

[Tim C]

Just as some C programmers are terrible and write bloated, exploit-ridden programs.

So what? It's possible to write shit code in any language.

Re:"high-performance" Java?

[rubycodez]

java becomes much slower than c++ or objective c, not to mention a resource pig.

The benefits of BDB, with the addition of SQL.

[Anonymous Coward]

Actually, a subset of Java is quite heavily used in many mobile devices. This isn't the 1980s any more. A typical cell phone today is just as powerful as a decent PC from 1999 or 2000. And just like many other applications, data storage is a very real requirement.

While you are correct regarding BDB being superior for storing certain types of data (eg. objects), in the past BDB has often been used in embedded applications to store relational data. It isn't well suited to such uses, without additional layers on top, as you mention.

However, SQLite offers the small footprint of BDB, along with the efficient single-file persistence, with the additional SQL support. Those who have been using BDB for a rather bastardized method of storing relational data can now instead use a fairly full-featured SQL implementation. Best of all, their overhead is barely increased, if not outright decreased.

BDB has its role, there's no doubt about that. But its necessity is being eaten away rapidly by competitors (eg. SQLite) and vastly improved embedded/mobile hardware.

Transfer from Berkeley to Postgres?

[Doc Ruby]

Is there a wrapper for Postgres that lets me install PG+wrapper instead of BerkeleyDB for supporting apps that depend on BDB? Asterisk uses BDB, OpenLDAP uses BDB, and more. I'd rather wrap my existing Postgres (on which other apps depend) than refactor and port all the other apps off BDB. Even if I later port them from the BDB wrapper, it will be easier and more productive along the way.

If there's no such wrapper, is there a way to use this BDB source to strip out everything below the BDB interfaces, install mapping to PG interfaces, then integrate with PG?

Re:Transfer from Berkeley to Postgres?

[Limburgher]

Shoot, that'd be great, or even for SQLite. My biggest problem with OpenLDAP is the BDB backend, which can be slow and get wedged. I've integrated a de-wedging script into my startup so that in the event of power failure the machine will still boot normally, but I think SQLite would fit nicely. I know OpenLDAP supports multiple backends, but I've not had much success in finding out how to migrate to another backend from BDB.

Re:Transfer from Berkeley to Postgres?

[Anonymous Coward]

OpenLDAP can be made to use a SQL database as its backend. Check the docs for info. However, it probably won't be as quick as BDB baring in mind that it can be tricky to map relation SQL queried data to hierachical LDAP queried data. And BDB can be made to go super fast if properly tweaked.

As for Asterisk et all, it is unlikly you could do this without much work. Why would you want to? The Asterisk developers picked BDB for a reason. I presume that they wanted a low level embedded database. What makes you think that the code that accesses these dbs uses SQL? It probably doesn't.

Re:Transfer from Berkeley to Postgres?

[Doc Ruby]

I asked Safehaus in several emails for details of their SW they claim replaces BDB with eg. Postgres, but I've never gotten a response.

Asterisk uses BDB for configs and MySQL for CDRs and other persistent app data, and I integrate it with apps already using Postgres. Especially when I want to join across those multiple databases, it's a real pain. None of these apps are using any DB features unique to any of those RDBMS'es, though the interface calls are often annoyingly idiomatic. DB migration tools should be a lot more available, especially since the DB market is so competitive, grabbing competitors' customers by any means necessary.

-1, Wrong

[Jerk City Troll]

Java classes get compiled to native code with machine-specific optimizations at runtime. If you had done any research on the subject, you would find that Java out-performs other languages at times. Java is fast. This argument is over, so please can it.

Re:-1, Wrong

[Espectr0]

Java classes get compiled to native code with machine-specific optimizations at runtime. If you had done any research on the subject, you would find that Java out-performs other languages at times. Java is fast. This argument is over, so please can it.

Java can be almost as fast a C code, and in some weird cases even more. The problem is that java requires a jvm, and loading the jvm plus having the garbage collector running means your program will start slower, and use more ram. Also the fact that it is not easy to write a good UI while keeping threading under consideration makes your program look even slower.

Java may be "fast enough" but your typical java app needs bigger system requirements than it would need in other languages.

So in theory, java is as fast as C, but in practice it is probably slower, especially for desktop applications.

Re:-1, Wrong

[g1zmo]

Not to troll, because I understand your point and I honestly want to believe what you say. But why is OpenOffice dog-ass slow? And Azureus? Why does my system slow to a crawl for more than a minute when I launch Yahoo Pool or Slime Volleyball? (Speaking of Slime Volleyball, why does that damn game play at different speeds on every computer?) I'm talking about a 3GHz/2GB machine here.

Wave your benchmarks in my face all you want. As far as I'm concerned, these are the only metrics that mean a damn thing.

Re:-1, Wrong

[AKAImBatman]

But why is OpenOffice dog-ass slow?

OpenOffice isn't written in Java. It optionally uses it for various scripting components. The speed of OOo (which is actually pretty good these days) has everything to do with the performance of C/C++ code.

And Azureus?

Works fine here. Maybe if you put more than 64MB in your system, you'd see better response out of programs that are memory demanding? And I can't think of anything that demands more memory than an app that caches hundreds of megs of P2P data in memory.

Why does my system slow to a crawl for more than a minute when I launch Yahoo Pool or Slime Volleyball?

For the exact same reason your system slows down when you launch that ActiveX game embedded in a webpage. (Hint: The game is using CPU.)

Speaking of Slime Volleyball, why does that damn game play at different speeds on every computer?

Because the developer programmed it that way? Seems obvious to me.

I'm talking about a 3GHz/2GB machine here.

Suuuuurrrreee you are. Which is to say that you have a 1GHz machine with 64MB of RAM, but you felt like bashing Java because Firefox is running slow on your system. (Hint: It needs at least 128MB to handle all the preloading it does.)

This is almost as good as the "I've been waiting for 20 minutes for a file to copy!" troll. Keep up the bad work and maybe you'll even start a new slashdot meme. (You wish.) </sarcasm>

* Yes, the 64MB remark is sarcastic hyperbole. I don't know how much RAM you have, but you're obviously doing something wrong.

Re:-1, Wrong

[Umbral Blot]

And Azureus?

Works fine here. Maybe if you put more than 64MB in your system, you'd see better response out of programs that are memory demanding? And I can't think of anything that demands more memory than an app that caches hundreds of megs of P2P data in memory.

I don't know about you but Azureus runs horribly on my machine and I have a 1GB of ram. It runs fine initially, but after a while it is hogging system resources to an unbelievable extent, making it impossible to leave running for a period of time if you want to get any work done. On the other hand uTorrent runs fine, even for days on end. Now I'm not saying that this is necessarily Java's fault, but something is wrong with Azureus, and I think Java may be contributing to the problem.

Re:-1, Wrong

[AKAImBatman]

Now I'm not saying that this is necessarily Java's fault, but something is wrong with Azureus, and I think Java may be contributing to the problem.

It has nothing to do with Java, and everything to do with Azureus's design. Azureus gobbles up system resources for performance. In addition, it's constantly computing pretty charts, graphs, and statistics which further its resource requirements. Not to mention all the plugins it loads to provide extra features.

People keep holding up Azureus as a "problem" even though its only "problem" is that it's a heavyweight application. If you don't want its heafty requirements, use something else. It's perfectly understandable if not everyone wants to have The Ultimate Control(TM) over their BitTorrent downloads.

Personally, I run Azureus for weeks at time while downloading large numbers of CD and DVD files. (Hello, distros.) It hogs some resources while it does its work, but not enough to bug me. Just make sure you limit your upload speed to a rate slightly below what your line can handle, or it will completely choke the connection. I learned this the hard way with the *original* Python BitTorrent client. I'd have it running for one file, then wonder why I couldn't surf anywhere. :-)

Re:-1, Wrong

[Anonymous Coward]

The most computationally demanding consumer applications today are 3D games. Please point me to a state-of-the-art 3D game written in Java, and then your argument will have some credibility. State-of-the-art means something competitive with Far Cry, Doom 3, or The Chronicles of Riddick, not Quake 1.

The truth is that despite the advanced dynamic compilation techniques that have been brought to bear on it over the past ten years, Java is still not competitive with C and C++.

Re:-1, Wrong

[asuffield]

Java classes get compiled to native code with machine-specific optimizations at runtime.

That is quite irrelevant. Java as a language is slower than C (and both are slower than Fortran) because the language imposes constraints upon expression. Java's object-heavy type system and required garbage collection impose performance penalties which are not present in C. Java's lack of native pointers imposes performance penalties in some specific circumstances - mostly those where C can use pointer arithmatic for manipulating data. A maximally optimal implementation of both Java and C will show C having a significant (but not dominating) performance advantage.

All of these things are well-known in the field of language design. High-level languages don't have to be slower than low-level ones... but they usually are. Not because of some details of the implementation, but because of the nature of the language and the things it can express. All of those things that make Java 'safe' also make it slower. That's one of the tradeoffs involved in designing languages. (Java also has a number of design flaws in its type system, making it slower than it needs to be, but that isn't relevant here).

The 'benchmarks' that you improperly cite are meaningless. You can't just pick two random pieces of code that do roughly the same thing and compare them. The details of the algorithms used are important. For any given piece of Java code that appears to be running faster than the C version, I can show you how to make the C version run at least as fast.

Re:-1, Wrong

[Anonymous Coward]

I'm talking about a 3GHz/2GB machine here.

Suuuuurrrreee you are. Which is to say that you have a 1GHz machine with 64MB of RAM

I'm willing to give him the benefit of the doubt and just say that his machine is probably full of spyware that got installed by exploiting some buffer overflow in those amazingly fast C programs he's using.

One thing people forget in the Java is slow debate is how much more secure Java apps are simply because they make certain classes of exploits impossible. If Java's detractors insist on holding up memory intensive apps like Azureus and Eclipse as examples of how Java is slow, Java's proponents should be allowed to trumpet insecure C programs and the resultant drop in performance due to spyware. That, or we can just compare the execution speed of any Java application to Norton AV and win hands down ;-P

Microsoft uses BDB (and therefore Oracle), tho

[Anonymous Coward]

Microsoft's new Groove product uses BDB. Pretty fun pointing out to them that MSFT depends on Oracle for scalable database work.

Greedy Corporation Struggles to Stay Relevant

[Anonymous Coward]

Booooorrringgg

BDB on flash drives

[tcopeland]

We've been using the C version of BDB (via Guy Decoux's Ruby extension) as a data store for indi; it's been working pretty well. It doesn't take up much space, either, so it's easy to fit it on a flash drive.

Of course, on the backend we use PostgreSQL. Rails + PostgreSQL is good stuff.

5, Insightful

[Jerk City Troll]

Your comments are all valid, except I disagree with the last one. I find myself pleasantly surprised by the performance of Eclipse and applications based on its platform (in terms such as interface responsiveness). The other point I would make in contrary to yours is that Java is, these days at least, primarily intended for server applications. Most code written on the Java platform is never touched by the end user, but rather agents acting on behalf of the user. Hence interface response is rarely a concern.

Apache Derby/Cloudscape also available

[possible]

I have been impressed with Apache Derby, the open source project that grew from IBM's donation of its Cloudscape database. An embeddable pure-Java implementation of a file-backed SQL database that supports many things you'd want in a DBMS, including transactions.

Its optimizer is not the smartest in the world, but we use Derby and if you get your indexes right, you can usually get very good performance on very large datasets.

Re:Apache Derby/Cloudscape also available

[Anonymous Coward]

define large dataset. I have a customer with 64 TB online in a single database at any given time.

Re:Apache Derby/Cloudscape also available

[possible]

The size in bytes is not really that important as regards database performance. The important question is: what is the row count?

Persistant + java?

[Anonymous Coward]

Ok, if you don't mind slow (and not free software) how about "totally not appropriate for long lived processes"?

Show me a scheduler built in java and it will have 'hiccups' occasional. And require restarts after a while becuase of mysterious GC issues.

"Java isn't Slow" argument

[Anonymous Coward]

The reason that you can get away with saying that "Java is usually as fast as C," or any other language compiled to native code, is that USUALLY, speed doesn't matter. Anything waiting on user input -- or that only does a few small calculations -- doesn't need a speed boost. The change isn't noticeable.

For a high-traffic database, speed is paramount. I wouldn't choose Java.

Money, they cost money, eughhh

[Anonymous Coward]

Any of the other good acid compliant DBMSes, say postgre, derby, firebird et al are distributed under a license which allows them to be redistributed for no fee in comercial products. This doesn't, MySQL doesnt so i won't use them. Eugh, Money.

You're trolling...but...

[Phil John]

...remember with Databases, unless it's all in memory, your big bottleneck is disk performance.

Re:You're trolling...but...

[Anonymous Coward]

Gee... I hate it when my disk gets stuck in a bottleneck.

Re:You're trolling...but...

[Anonymous Coward]

Oh, please. I have just finished benchmarking a number of databases, all on the same computer. The performance numbers are all over the place. Some databases are good, some suck. Good hardware will get you nowhere without good software.

Years behind db4o

[albert.meunier]

Interesting! The new BDB Java release does attempt to store objects directly. That's quite an advance in comparison to the awkward API of BDB 2.x.

However BerkeleyDB JE is still years behind db4o. The querying functionality of BDB is very rudimentary. It requires manual selection of indexes to be used, not really sophisiticated in comparison to db4o's Query-By-Example and Native Query concepts.

It would be interesting to see BerkeleyDB and db4o race head to head in the Poleposition benchmark.

Why OOo is slow.

[Bob9113]

But why is OpenOffice dog-ass slow?

I'm not sure about the rest, but I can address this one.

StarOffice 5.x was the series where the people at Sun started to think, "hmmm, this could actually be something someday" - and they started showing it to a very small group of people. Those people said, "Well, that's not bad, but it crashes all the time."

StarOffice 6.x was more stable, and is when Sun spawned OpenOffice.org. It reached a slightly larger audience who said, "Well, it's stable now, but it's missing a whole bunch of important features."

OpenOffice 1.0 and 1.1 started to address the feature issues, and reached a larger audience. Those people said, "Well, it's getting there, but it doesn't handle Microsoft formats perfectly, and the user interface doesn't feel like MS Office."

OpenOffice 2.0 fixed that, and is reaching a much larger audience. Now people are saying, "This is getting pretty good, but it's not fast enough - for example, it chews a lot of CPU doing large spreadsheets."

Care to guess what version 3.0 will be improving upon? I don't actually know - haven't followed their developer notes or anything, but I'd place a big fat wager on performance.

It's the natural evolution of software. Sometimes the evolution happens behind closed doors, as is most common with proprietary software, but it almost always follows that evolutionary process. Core Stability, Major Features, Minor Features and User Friendliness, Performance. You can almost set your watch by it.

Disk bottleneck is just an excuse

[alxtoth]

Indexing/partitioning/remodelling the brain-dead table design can lead to significant performance gains

BDB has best performance of all choices

[unPlugged-2.0]

My company has been using Berkeley DB for quite some time and I must say that is indeed quite a good product. Sleepycat has great support and the tool itself is very stable. The java version is fairly new and doesn't have the features of the C version but works great as object storage.

However it is not very user friendly and also not very programmer friendly. It doesn't have any single protocol like JDBC for access because you can literally program it to do anything. Also the documentation is very sparse and there is only one GUI admin tool as opposed to about a jillion ones for SQL DB's. However BDBXMLAdmin from the java.net project is quite good and though its not quite a phpmysql competitor it is adequate https://bdbxmladmin.dev.java.net/. What it makes up in difficulty it absolutely makes up for in terms of raw speed and portability it just can't be beat and we have benchmarked it vs. HSQL and MySQL running in ISAM mode.

It looks like the oracle acquisition may actually be paying off as they were generally very slow in new releases and I don't know about the size of their development team but I would have thought they would have produced more for the time and the apparent caliber of the management team (bunch of Phd's that worked on the initial product). I hope that oracle can bring some muscle and polish to their product and maybe this will be one that they don't kill off with their questionable ideals.

Here's to hoping that BDB gets the recognition it deserves and I predict that with the right involvement it will be a strong player in the embedded applicaitons space, something oracle could never hope to compete in without a tool like this.

We are currently building a product for embedded devices in the RFID space and some of the things that need to be done can't be done with any other embedded db period. It really is so great to work with and handles Java and XML storage quite well.

------- Long time Addict, First Time Poster

Re:BDB has best performance of all choices

[shutdown -p now]

What it makes up in difficulty it absolutely makes up for in terms of raw speed and portability it just can't be beat and we have benchmarked it vs. HSQL and MySQL running in ISAM mode.

Out of curiosity, have you compared it to SQlite, perfomance-wise? The latter claims to be faster than MySQL in many cases, and is closer to BDB in terms of what it is intended to be used for, so it would seem an interesting comparison to make.

Re:BDB has best performance of all choices

[unPlugged-2.0]

I would assume that SQLLite and HSQL would compare similarly. HSQL was faster than MySQL as well in the pole position benchmarks and in ours too but not entirely fair since it runs in memory.

If you know how SQLLite compares to HSQL then let me know and you will have your answer to a certain extent.

Our evaluation phase is over so I can't justify doing any tests on SQL Lite but I may try it if I ever get some spare time as I am sure I could just use the HSQL tests against SQLLite as well

Rex Wang, what an insanely cool name!

[Cannelloni]

Rex Wang, that's the name I would pick for my own personal super hero!

Depends.

[jd]

If I compile the Java code with GCJ, then I shouldn't need a JVM. If someone would get around to building a processor-in-memory hardware Java implementation, you'd have a Java Physical Machine and again would not require a virtual machine. "Need", then, is a little strong. It would be more correct to say that - as of right now - most Java applications will indeed be run in a virtual machine, and require the components within that machine, but they are only run virtually because that is how it is traditionally done.

[GN]dbm any one?

[bobs666]

Now we can buy ndbm from Oracle.

That is mostly what BerkeleyDB is.

As for handheld scaners, I put tiny tcl http://tinytcl.sourceforge.net/ on mine. I am sure calling ndbm would be easy. execpt the handheld was a DOS_5. I don't have the time to go find all the junk like 'C' compilers for that. The next version of the scanned will run Linux and all will be well.

Will it be backward- and forward-compatible?

[mi]

The past BDB releases suffered from being unable to read older/newer .db files sometimes. This is why various software packages (like Evolution, for example) bundle their own versions.

If Oracle solves this problem, I'll be a very happy camper...