Oracle Unveils New Open Source BerkeleyDB Release

Mark Brunelli writes to tell us that Oracle has released the newest version of the open source Oracle BerkeleyDB Java Edition. From the article: "The new release of the Java embeddable database is the third to come out in three years and the first new version to come out of Sleepycat Software since Oracle purchased the open source stalwart back in February. Rex Wang, Oracle's vice president of embedded systems and a former vice president of marketing at Sleepycat, said the latest release lets Java developers take advantage of a new Persistence application programming interface (API) that provides greater flexibility and new performance optimizations that enable applications to run faster."

Comments

Many using SQLite instead.

[Anonymous Coward]

I know that many developers, especially on embedded or hand-held systems, are starting to use SQLite these days instead of BDB. SQLite is small enough, and efficient enough, to be used suitably in such environments. It offers SQL support, which BDB does not. And its library-based architecture is a real plus in such environments. The fact that it's public domain makes it even more appealing.

SQLite is even starting to take over from MySQL for many smaller websites. The overhead (both in terms of system resources and administration) can't be justified when using a server-based system, especially when SQLite does the job just fine.

What we may even see is MySQL squeezed out of the market. I know many database developers who are starting to use PostgreSQL for their higher-end databases, and SQLite for the lower-end. MySQL could be considered the best contender for mid-range DBs, but SQLite and PostgreSQL keep converging, thus pushing MySQL out to a point of irrelevance. Of course, there has been a decade and more of development using MySQL, so for legacy reasons alone it will continue to be used for ages.

Re:Many using SQLite instead.

[AKAImBatman]

I know that many developers, especially on embedded or hand-held systems, are starting to use SQLite these days instead of BDB. SQLite is small enough, and efficient enough, to be used suitably in such environments. It offers SQL support, which BDB does not.

You're missing the point. Putting aside for the moment that we're discussing the Java version of BDB (which would be useless on the types of embedded platforms you're talking about), BDB is a basic database engine. Which means it's the type of software you can use to build a database management system on top of rather than being a DBMS in its own right. SQLite and HSQL are complete DBMS packages that will only meet your needs if an SQL engine is what you want.

For example, if you're looking to build an Object Storage Database, SQLite and HSQL would both be terrible choices. But BDB would fit the bill perfectly.

Re:Many using SQLite instead.

[ralph alpha]

Berkeley DB and SQLite don't even come close to filling the same niches. Berkeley DB is a highly optimized, generic, low-level tree-based database that can be used for a variety of purposes -- including providing the backing store for an SQL system. SQLite is an embeddable SQL parser wand extremely simplistic backing store. Berkeley DB is like a body without a head. SQLite is like a head without a body (or at least not much of one). You might want to note that "database" does not imply "SQL."

Re:Many using SQLite instead.

[mritunjai]

Sorry, but you're totally mistaken about BerkeleyDB. Its just a database engine. i.e. you give it a key and value pair and it stores them. You give it the key and it retrieves the value. There is support for foreign keys and its completely transactional, but there is no SQL interface and no 'datatypes' whatsoever.

It'd take some time to figure out what exactly is Oracle trying to do with it.

Re:Many using SQLite instead.

[hey!]

I think you hit the nail on the had here.

If you are a database person, as opposed to a programmer who needs some kind of persistent store, database implies the prsence of certain abstractions that BDB doesn't have. It's the abstractions that matter, because they implement a kind of data reusability. This is a big aspect of databases that programmers don't always appreciate, but if you remmber the early adoption days of the relational model (as I do), the idea of abstracting data from its application was a big thing. We had methods already after all for managing storage: hash tables, b trees, record pointers. We even had models (the hierarchical model). But relational databases allow you to create a model of your data that is abstracted from what you are doing with it.

I'd go so far as to say that absracting data for reusability is what distinguishes a true database from other kinds of collections of data. Of course people do use words in a vague way, but this property definitely separates out what is unquestionably a datbase from what is only arguably a database.

There is no question of BDB being a database management system. Collections of data in BDB format might arguably be databases depending on the effort the programmer went through, but usually they're just indexed files.

Things like SQLLite that implement a generaly query and data manipulation language (e.g. including but not limited to SQL) is clearly a database management system, although it may not be a very good one.

Re:Many using SQLite instead.

[david.given]

SQLite is even starting to take over from MySQL for many smaller websites. The overhead (both in terms of system resources and administration) can't be justified when using a server-based system, especially when SQLite does the job just fine.

SQlite is so incredibly cool (for a database, of course; databases are intrinsically limited in coolness). It's tiny, it's easy to set up, it's portable, it's faster than MySQL for most operations, and it's not just open source, it's actually in the public domain. It doesn't do well if there are multiple processes trying to access the same database, and there's no network layer at all, but given that it stores the entire database in one file, it's the ideal choice for any application that needs to use complex state. I use it for keeping track of email addresses in an SMTP proxy --- it was so easy to use I was amazed.

I'd strongly, strongly recommend having a look if you're writing something that might want a database. Hell, given the command-line front end, you can even use it from shell scripts...

Re:Rex Wang?

[Bootvis]

He is King Cock.

Sweet!

[AKAImBatman]

I'll have to see if they've finally added support for partial record retrivals. It used to be that this feature was only in the native code edition of BDB. It ended up being a showstopper for the Java version last time I tried to use it.

Re:Sweet!

[greybird]

Hello, this is Mark Hayes, I'm one of the BDB Java Edition developers.

There are no restrictions on accessing partial records in BDB Java Edition -- it is the same as the BDB C Edition in this respect. However, you may have a misunderstanding about the way partial records work in both products: neither product has a BLOB capability.

In both products, when you do a partial get/put of a record, the entire record must be loaded into the BDB cache. The only advantage of a partial get/put is that it reduces the amount of memory copied between the BDB cache and your buffer. If you need the performance of a true BLOB implementation for very large records, I'm afraid you won't get it by using either BDB product.

We do get requests for a true BLOB capability from time to time. We have not implemented it yet, because it will take significant effort to implement it in a way that performs well for very large records. And the number of requests have not yet been high enough to outweigh other requested features.

Please keep the requests for this capability coming in if you need it. We do factor in these requests when we do product planning.

Thanks,
Mark

Alternate link

[Nerdfest]

Since TFA seems to be slashdotted, here's an alternate. I'm assuming that version 3.0 (May 17th) is the new version.

http://www.oracle.com/corporate/press/2006_may/ber keleydb-je-ga.html/

Re:"high-performance" Java?

[JebusIsLord]

oh for chrissakes don't turn this into a "java is slow" argument. 1996 called, and they want their discussion back.

Re:"high-performance" Java?

[$RANDOMLUSER]

You're missing the humor in his troll:

> For the same amount of keystrokes, I can ... write it in C...

Too funny.

The benefits of BDB, with the addition of SQL.

[Anonymous Coward]

Actually, a subset of Java is quite heavily used in many mobile devices. This isn't the 1980s any more. A typical cell phone today is just as powerful as a decent PC from 1999 or 2000. And just like many other applications, data storage is a very real requirement.

While you are correct regarding BDB being superior for storing certain types of data (eg. objects), in the past BDB has often been used in embedded applications to store relational data. It isn't well suited to such uses, without additional layers on top, as you mention.

However, SQLite offers the small footprint of BDB, along with the efficient single-file persistence, with the additional SQL support. Those who have been using BDB for a rather bastardized method of storing relational data can now instead use a fairly full-featured SQL implementation. Best of all, their overhead is barely increased, if not outright decreased.

BDB has its role, there's no doubt about that. But its necessity is being eaten away rapidly by competitors (eg. SQLite) and vastly improved embedded/mobile hardware.

Transfer from Berkeley to Postgres?

[Doc Ruby]

Is there a wrapper for Postgres that lets me install PG+wrapper instead of BerkeleyDB for supporting apps that depend on BDB? Asterisk uses BDB, OpenLDAP uses BDB, and more. I'd rather wrap my existing Postgres (on which other apps depend) than refactor and port all the other apps off BDB. Even if I later port them from the BDB wrapper, it will be easier and more productive along the way.

If there's no such wrapper, is there a way to use this BDB source to strip out everything below the BDB interfaces, install mapping to PG interfaces, then integrate with PG?

Re:Transfer from Berkeley to Postgres?

[Limburgher]

Shoot, that'd be great, or even for SQLite. My biggest problem with OpenLDAP is the BDB backend, which can be slow and get wedged. I've integrated a de-wedging script into my startup so that in the event of power failure the machine will still boot normally, but I think SQLite would fit nicely. I know OpenLDAP supports multiple backends, but I've not had much success in finding out how to migrate to another backend from BDB.

-1, Wrong

[Jerk City Troll]

Java classes get compiled to native code with machine-specific optimizations at runtime. If you had done any research on the subject, you would find that Java out-performs other languages at times. Java is fast. This argument is over, so please can it.

Re:-1, Wrong

[Espectr0]

Java classes get compiled to native code with machine-specific optimizations at runtime. If you had done any research on the subject, you would find that Java out-performs other languages at times. Java is fast. This argument is over, so please can it.

Java can be almost as fast a C code, and in some weird cases even more. The problem is that java requires a jvm, and loading the jvm plus having the garbage collector running means your program will start slower, and use more ram. Also the fact that it is not easy to write a good UI while keeping threading under consideration makes your program look even slower.

Java may be "fast enough" but your typical java app needs bigger system requirements than it would need in other languages.

So in theory, java is as fast as C, but in practice it is probably slower, especially for desktop applications.

BDB on flash drives

[tcopeland]

We've been using the C version of BDB (via Guy Decoux's Ruby extension) as a data store for indi; it's been working pretty well. It doesn't take up much space, either, so it's easy to fit it on a flash drive.

Of course, on the backend we use PostgreSQL. Rails + PostgreSQL is good stuff.

Years behind db4o

[albert.meunier]

Interesting! The new BDB Java release does attempt to store objects directly. That's quite an advance in comparison to the awkward API of BDB 2.x.

However BerkeleyDB JE is still years behind db4o. The querying functionality of BDB is very rudimentary. It requires manual selection of indexes to be used, not really sophisiticated in comparison to db4o's Query-By-Example and Native Query concepts.

It would be interesting to see BerkeleyDB and db4o race head to head in the Poleposition benchmark.

Re:"high-performance" Java?

[geoff lane]

OK Java isn't slow. But some Java programmers are terrible and write bloated and slow programs.

Re:"high-performance" Java?

[csnydermvpsoft]

funny, in those benchmarks showing the speed of java, all I see is procedural code with primative data types. Start building objects, garbage collecting, and casting and my, my what a pig.

So when you stick to basically writing code like you would in C, it's the same speed, but when you add more complicated, time-saving, features, it slows down? Wow, that's surprising. (</sarcasm>)

BDB has best performance of all choices

[unPlugged-2.0]

My company has been using Berkeley DB for quite some time and I must say that is indeed quite a good product. Sleepycat has great support and the tool itself is very stable. The java version is fairly new and doesn't have the features of the C version but works great as object storage.

However it is not very user friendly and also not very programmer friendly. It doesn't have any single protocol like JDBC for access because you can literally program it to do anything. Also the documentation is very sparse and there is only one GUI admin tool as opposed to about a jillion ones for SQL DB's. However BDBXMLAdmin from the java.net project is quite good and though its not quite a phpmysql competitor it is adequate https://bdbxmladmin.dev.java.net/. What it makes up in difficulty it absolutely makes up for in terms of raw speed and portability it just can't be beat and we have benchmarked it vs. HSQL and MySQL running in ISAM mode.

It looks like the oracle acquisition may actually be paying off as they were generally very slow in new releases and I don't know about the size of their development team but I would have thought they would have produced more for the time and the apparent caliber of the management team (bunch of Phd's that worked on the initial product). I hope that oracle can bring some muscle and polish to their product and maybe this will be one that they don't kill off with their questionable ideals.

Here's to hoping that BDB gets the recognition it deserves and I predict that with the right involvement it will be a strong player in the embedded applicaitons space, something oracle could never hope to compete in without a tool like this.

We are currently building a product for embedded devices in the RFID space and some of the things that need to be done can't be done with any other embedded db period. It really is so great to work with and handles Java and XML storage quite well.

------- Long time Addict, First Time Poster