Slashdot Mirror
MySQL Creator Contemplates RAM-only Databases
Aavidwriter writes "Peter Wayner asks Michael 'Monty' Widenius of MySQL, 'When will RAM prices make disk drives obsolete for database developers?' From Monty's answers, it sounds like hard drives may be nothing but backups before long." From experience, I'd wager that RAM failure rates are less than hard drive failure rates, so it might also mean more stability from that perspective.
But also a very strong Memory Manager. We've all seen a poorly written program corrupt memory.
I remember reading somewhere that, due to things like thermal radiation and cosmic rays, every so often a bit in RAM is changed by 'accident'... isn't the ECC RAM (which, IIRC, negates the effects of such interaction) horrendously expensive though, more so than the 'normal' SDRAM variants we have these days?
but doesn't RAM need power running through it to hold its data? If this is true and we do switch to RAM for our SQL servers, all it would take is one fool to trip over a power chord (or just a power-outtage) to lose one heck of a lot of data.
With our Exchange server, we use a Platypus Qik Drive to send our retrieval times through the basement. We put the database on Qik Drives (but mirror it hourly on to HDDs)...it makes our effective Exchange bandwidth limited to the gigabit ethernet port on the server.
Q: "Why do sound techs say 'check 1, 2'?"
A: "Cause if they could count any higher they'd be lighting techs."
Surely a well tuned database server stores uses quite a lot of RAM for buffering?
Nobody in their right mind would have a busy database server which accesses the hard disk like crazy. A few years back I saw Oracle servers running NT with 4GB of RAM, so I guess they're using even more now.
If you have a database that is stored in RAM and periodically written out to the hard disk (for backup reasons) then you get better performance than if you have a database that is reading and writing most of the time.
UPS would prevent the data loss, the database could be written to disk when the power fails.
Considering the non-existent ACID support in MySQL it sounds like a good idea, it's not like MySQL will get any more errorprone than it is now...
When our site was slashdotted last year, we were able to cope with the load after putting our database into RAM. It's probably not the best solution, since the RAM would get deleted if the system crashes (or the power goes out, etc.), but it's a good temporary measure.
OLPC Australia
I guess the issue with databases is not only speed and reliability, but a totally different ballgame called 'user-perception'. Even now, tape drives are used to archive databases; despite the fact that less than 1 in 1000 of the tape media get used for actually retrieving the data during a crash. NAS devices and the like have changed this, but the temptation remains to use tape.
I guess the RAM vs disk debate is on similar lines - but there are some vital differences:
1. Disks (esp. IDE) have become a commodity item and can be accessed by different system architectures easily.
2. IDE and SCSI standards have stood the test of time - 13 and 20 years respectively, unlike RAM wihch has evolved from Parity, non-EDO, EDO, DRAM, SDRAM, DDR-RAM, RAMBUS RAM etc., and suffers several patent and copyright encumberances.
3. Although RAM prices are driving down, the h/w to interface speciality RAM banks is proprietary and hence cost-prohibitive, and comes with attendant long-term supportability risks - think Palladium, or even Server mobos over the last 10 years. TCO for RAM based systems could thus be much higher than disk-based systems.
Overall, except for apps that need super-high speeds, and users that can risk proprietary stuff, disk-based databases shall remain.
My 0.02
If you keep throwing chairs, one day you'll break windows....
And large image-bases are too much for RAM today. Remeber the 640K limit? I'd guess that more than 50% current corporate databases would fit in a single gigabyte, including indexes.
The real speed improvements, according to the guys working on projects like Bamboo on sourceforge come not from the fact that it's in RAM, they test against SQL in RAM and show that most of the performance improvements come from keeping the data in the same process space as the application operating on the data. If they're right, putting MySQL in RAM or Oracle or Microsoft SQL Server is a small improvement.
The large speed improvements come from lowering the process boundary hopping overhead--marshaling and all that good stuff.
RAM-resident Database? Yes, that would be Google -- a massive, massive cluster of x86 boxen with a couple gigs of RAM apiece. Each gets a portion of the hashspace, leading to near-O(1) searchability. I'm pretty sure all the big search engines work this way, at this point -- the DB is checkpointed into RAM, but is never actually run from it.
:-)
Recent discussions about disks vs. CPU's have ignored the massive decreases in the cost of RAM. For a very long time, the secret bottleneck in PC's (in that it wasn't advertised heavily) was RAM. That's starting to disappear -- there's a gig in my laptop, and there's no discernable improvement in all but the most intense applications if I were to go beyond that.
Virtual Memory is already on the chopping block; any time it's imaginable that a system might need another gig of storage, it's probably worth going to the store and spending the hundred dollars.
But what if more RAM is indeed needed? One of the most interesting developments in this department has involved RDMA: Remote DMA over Ethernet. Effectively, with RAM several orders of magnitude faster than disk, and with Ethernet achieving disk-interface speeds of 120MB/s, we can either a) use other machines as our "VM" failover, or more interestingly, b) Directly treat remote RAM as a local resource -- a whole new class of zero copy networking. This Is Cool, though there are security issues as internal system architectures get exposed to the rough and tumble world outside the box. It'll be interesting to see how they're addressed (firewalls don't count).
What next, for the RAM itself? I don't think there's much that value in further doublings...either of capacity, or soon, of speed. What I'm convinced we're going to start seeing is some capacity for distributed computation in the RAM logic itself -- load in a couple hundred meg in one bank, a couple hundred meg in another, and XOR them together _in RAM_. It'd just be another type of read -- a "computational read". Some work's been done on this, though apparently there's massive issues integrating logic into what's some very dumb, very dense circuitry. But the logic's already done to some degree; ECC verifiers need to include adders for parity checking.
My guess...we'll probably see it in a 3D Accelerator first.
*yawns* Anyway, just some thoughts to spur discussion. I go sleep now
Yours Truly,
Dan Kaminsky
DoxPara Research
http://www.doxpara.com
Many databases contemplate database sizes in the tens to tens of thousands of gigabytes. For smaller databases, RAM is an easy thing, and even for a small number of gigabytes it might be reasonable. For larger databases RAM would be unthinkable. The fact that a database developer doesn't know what databases are used for is disturbing.
Most modern databases also make very effective use of RAM as a cache in order to speed up queries. I don't know about MySQL since I don't use it. My guess, however, is that it does not, since that would eliminate the need for this stupid measure.
As far as reliability, RAM is more vulnerable to transient things like cosmic radiation. ECC memory will take care of most single-bit problems (there are lots of them...), but all it can do for multi-bit failures is determine that yes, your data is screwed.
Also, swapping out a bad hard disk in a RAID configuration is relatively simple and has a recovery process. Suppose your RAM stick fails; what is your recourse? You've permanently lost that data, and systems with hot-swappable RAM are much more costly than ones with similar capabilities for hard drives.
Finally, consider the problem of catastrophic system failure. If the power goes out, your RAM dies but your hard disk is still safe. If it is worse (say your facility burns down) then it is much easier to recover data from the charred remnants of a hard disk than from the charred remnants of a DRAM chip.
The idea of replacing disks with DRAMs has been around for quite a while now. But disks continue to get (a bit) faster and (much) larger. Every time the morons want to replace it they get shot down. More sensible people focus on using the resources available in ways such as caches that make systems faster and more reliable.
...have nothing to do with the medium the data is stored in! What you're trying to guard against is concurrent access of resources by transactions which in cases can cause incorrect or inconsistent results in a RDBMS. I think this article is a bit obvious for most people who've had any training in how databases actually work and I think Monty was actually pretty gracious for taking the time to give the interviewer a smidgeon of clue.
There are some cool ideas there. They use two copies on disk for backup in case of system failure. Because of this they don't have to do page-latching.
In some configurations, though, this is irrelevant, because write transactions lock the whole database! Because they know all transactions will be extremely short, this is faster than locking at page or row level.
Hell, I'd just be happy if they would normalize their tables. I've seen joins across three tables which all hold essentially the same data, and working on the entire result set after downloading it to the client and then uploading the whole thing back when one thing has changed.
And they wonder why I'm crazy.
You think that I'm crazy, you should see this guy!
I'm confused. I actually haven't used MySQL much, and someone else can clarify its current ACID compliance. My application involves multiuser financial transactions. When making my DB selection a couple of years ago, at that time it was claimed that MySQL had some ACID deficiencies that made me nervous. I settled on PostgreSQL, which I'm very happy with.
But there's a lot more to ACID than just keeping RAM and disk in sync, and I don't see how RAM would make ACID that much easier, and certainly not "almost trivial". You still have all the transactional semaphores, record locking, potential deadlocks, rollbacks, etc. to worry about. In fact I don't see why you wouldn't just have the RAM pretend to be a disk and be done with it, since the disk version already has stable software. Then, if it is important to increase performance further, RAM-specific code optimization could be done over time, but slowly and carefully.
I'm sorry - I really don't want to get into a religious war here, but the interview didn't do much to bolster my confidence in MySQL for mission-critical financial stuff. Educate me.
Back in the early 90's IBM added a machine instruction to their mainframes called DIV. It treated data in a file system as if it where in virtual mememory - ie addressRecord[12345] appeared to the program as an in memory array, but was backed by disk storage - the same format that was used for paging virtual memory - brilliant. It's a shame it never caught on - it would make advances like this transparent in implementation. Well I guess you can't really say it never caught on - it was a big reason IBMs mainframe databases outperformed everyone else for so long.
Is there a similiar kind of instruction on Intel? It's probably too late though - indexed arrays have become less useful since associative array patterns have become better defined. A hardware implementation (RAM) of JDO would be interesting.
slashdot troll = you make a compelling argument I do not like the implications of.
There are components of ACIDity that would be implmented very differently for RAM-persistent databases than for disk-persistent ones. Maintaining ACIDity on disk-persistent databases requires complicated algorithms to mitigate the disatrous disk seek times. These complicated algorithms would be rendered unnecessary if disks were no longer used.
For example, disks have incredibly slow seek times and much better bandwidth; therefore it's far cheaper to write things to disk in big chunks. The purpose of write-ahead logging (or "redo logging") is to mitigate the performance impact of slow seek times by blasting all the transactions to disk at once, in the redo log, thereby avoiding the slow seeks that would be required by putting each transaction in its proper place. Putting the transaction data in its proper place is deferred until after the load has died down somewhat. This could be seen as exchanging seek times for bandwidth.
This redo log mechanism would be unnecessary for ram-persistent databases. It's a significant source of complexity that would be obviated by the removal of disks. And that's just one example of complexity required to get adequate performance from disk, a medium that has disastrously slow seek times.
I believe it surfaced a while back on /. - can't find any links at the moment, but AFAIK the entire Google index is stored in RAM.
grisha.org
Any Perl programmers in the audience may wish to check out DBD::RAM. From the CPAN documentation: "DBD::RAM allows you to import almost any type of Perl data structure into an in-memory table and then use DBI and SQL to access and modify it. It also allows direct access to almost any kind of file, supporting SQL manipulation of the file without converting the file out of its native format." More information here
First, as other have said, a properly designed RAM subsystem can be battery backed up. In terms of getting the data out, loss of power to the RAM is no more catastrophic than loss of power to the CPU, the router, the computer running the middleware, or whatever. Because RAM is a purely semiconductor approach, any battery backup system can be simple and reliable.
In fact, it should not be too difficult to design a system which, in the event of power fail, dumps data to backup disk drives. To get to that state, the main system has already failed to do a clean shutdown, so this is a last resort issue.
The next thing is error detection and correction. It's true that single bit ECC is limited, but it also takes only limited resources (7 additional bits for a 32-bit subsystem, 8 for 64). Memory subsystems could have extra columns so that if bit errors start to multiply in one column, it can be switched out for a new one. Just as with any error detection and correction strategy, single bit detection in columns can be combined with further error correction down rows, using dedicated hardware to encode and decode on the fly. Just good old basic electronics.
In the worst case, it should be possible to build an extremely reliable memory system for a bit penalty of 50% - no worse than mirroring two hard drives. It won't be quite as fast as writing direct to motherboard RAM, but we don't want to do that anyway (we want to be able to break the link on power fail, save to disk, then later on restore from disk. And we want the subsystem in its own cabinet along with the batteries. No one in their right minds is suggesting having a couple of C cells taped to this thing and held on with croc clips.)
I'd even venture to suggest that most MySQL databases are not in the terabyte range, and that most databases aren't in the gigabyte range even if they are mission critical in SMEs.
Conclusion? As usual we have the people trying to boast "My database is far too big and complicated for MySQL! So MySQL sucks! My database is too (etc.) to run in RAM! So running DBs from RAM sucks!" and ignoring the fact that there are many web databases where transactional integrity is not an issue, and the market for a RAM store for databases in the low Gbyte range might actually be rather substantial.
Panurge has posted for the last time. Thanks for the positive moderations.
We need archival storage devices that won't lose data unless physically destroyed. We don't have them. Tapes don't hold enough data any more. Disk drives don't have enough shelf life.
DVD-sized optical media in caddies for protection, maybe.
(It's annoying that CDs and DVDs went caddyless. Early CDs drives use caddies to protect the CDs, but for some idiotic reason, the caddies cost about $12 each. We should have had CDs and DVDs in caddies, with the caddy also being the storage box and the retail packaging for prerecorded media. There's no reason caddies have to be expensive. 3.5" floppies, after all, are in caddies.)
Valid points, of course :) But you have to admit that for simple home pages (and not corporate databases) MySQL is simple, to-the-point, and easy to use.
And free, although many other ones are free as well. (I wouldn't want to run Oracle@Home ... of course, postgreSQL is also free, and I hear it's more mainstream as far as true database functionality.)
"Careful" people can enforce their own data integrity - obviously, it gets harder as the size (number and complexity of tables, I mean) of the database expands.
Can you tell I use it myself? :) You sound like you have experience with other database systems, how difficult do you think it would be to port an existing MySQL+PHP system to PostGreSQL or something similar?
Modern storage solutions (like EMC) use redundant battery backed ram to buffer writes, greatly reducing perceived write latency. This gives you a lot of the performance gain of a ram only database, and also scales very well to large loads. (in fact, when choosing RAID stripe size you take into account whether writes are buffered; if not, keep stripes small for log files)
...most databases don't cleanup indexes after deletes, forcing periodic rebuilding. Other index schemes not generally considered because of poor locality prinicles could be considered. Note that Hash Indexes would probably still use Linear Hashing.
If you know that your data will always fit into available ram then there are a number of performance optimizations that can be done. I'm not sure about ACID becoming "trivial"; You still need most of the same db components: indexes, lock managers, operation journaling, etc. But many of these could be greatly simplified:
1. Page/Buffer Manager Eliminated. Since no disk IO will be required for the normal running of the db, there will be no need for a page manager. This eliminates complexity such as block prefetch and marking and replacement strategies. In fact, the data will probably not be stored on pages at all. Details such as block checksum, flip flop, log position, page latches etc can all be removed. The values in the rows would be sitting in memory in native language formats rather than packed making retrieval much faster. There would be no need for block chaining.
2. More flexible indexing. Since it is not necessary to store data in pages, traditional B-Trees are not absolutely required. Other index structures like AVL trees would be faster and might allow better concurrency. These trees would also be easier to keep balanced
3. Lock Manager Simplified. Row level locking (and MVC) are still desired features, but keeping the locks all in memory simplifies implementation. Oracle and InnoDB store lock information in the blocks (associated with transaction) to allow update transactions larger than memory.
4. Log manager simplified. You will still need journaling capability for rollback, replication, recovery from backup etc. But the implementation of the log need not be traditional. Any structure that maintains information about transactions and contains causal ordering will do. Techniques such as keeping old versions of rows adjacted to current versions that are unacceptable for disk based databases (ahem, Postgres) could be used.
Although these may seem like small things, they can add up: less code to run is faster code. A company called TimesTen offered a product that they claimed was 10x faster than Oracle using an all memory DB. Generally the corporate world doesn't care to split hairs. They want something that works, and they are willing to throw some money and iron at it. Thats why battery backed ram in the disk controller to buffer writes is probably going to be fine for now.
A last note: modern databases already know to not bother with indexes when a table is sufficiently small.
JJ