Slashdot Mirror
MySQL Creator Contemplates RAM-only Databases
Aavidwriter writes "Peter Wayner asks Michael 'Monty' Widenius of MySQL, 'When will RAM prices make disk drives obsolete for database developers?' From Monty's answers, it sounds like hard drives may be nothing but backups before long." From experience, I'd wager that RAM failure rates are less than hard drive failure rates, so it might also mean more stability from that perspective.
But also a very strong Memory Manager. We've all seen a poorly written program corrupt memory.
I remember reading somewhere that, due to things like thermal radiation and cosmic rays, every so often a bit in RAM is changed by 'accident'... isn't the ECC RAM (which, IIRC, negates the effects of such interaction) horrendously expensive though, more so than the 'normal' SDRAM variants we have these days?
but doesn't RAM need power running through it to hold its data? If this is true and we do switch to RAM for our SQL servers, all it would take is one fool to trip over a power chord (or just a power-outtage) to lose one heck of a lot of data.
With our Exchange server, we use a Platypus Qik Drive to send our retrieval times through the basement. We put the database on Qik Drives (but mirror it hourly on to HDDs)...it makes our effective Exchange bandwidth limited to the gigabit ethernet port on the server.
Q: "Why do sound techs say 'check 1, 2'?"
A: "Cause if they could count any higher they'd be lighting techs."
RAM is highly susceptible to transient faults. Things like comic radiation at high altitudes make computing a real problem. ECC helps this but it won't totally eliminate it. With a hard drive, the probability of a hard fault goes up but a soft fault goes down.
Surely a well tuned database server stores uses quite a lot of RAM for buffering?
Nobody in their right mind would have a busy database server which accesses the hard disk like crazy. A few years back I saw Oracle servers running NT with 4GB of RAM, so I guess they're using even more now.
...goes to whoever is crazy enough to put their entire database in RAM.
Now if the RAM was non-volatile and was static with the power off that would rock, but volitile RAM - are you crazy?!!
Comment removed based on user account deletion
"RAM failure rates are less than hard drive failure rates, so it might also mean more stability from that perspective" Well that is because they havnt been subjected to that sort of load as yet. RAM could pose its set of unique problems once implemented as databases.
Siggy Say, Siggy Do
If you have a database that is stored in RAM and periodically written out to the hard disk (for backup reasons) then you get better performance than if you have a database that is reading and writing most of the time.
UPS would prevent the data loss, the database could be written to disk when the power fails.
Considering the non-existent ACID support in MySQL it sounds like a good idea, it's not like MySQL will get any more errorprone than it is now...
When our site was slashdotted last year, we were able to cope with the load after putting our database into RAM. It's probably not the best solution, since the RAM would get deleted if the system crashes (or the power goes out, etc.), but it's a good temporary measure.
OLPC Australia
I guess the issue with databases is not only speed and reliability, but a totally different ballgame called 'user-perception'. Even now, tape drives are used to archive databases; despite the fact that less than 1 in 1000 of the tape media get used for actually retrieving the data during a crash. NAS devices and the like have changed this, but the temptation remains to use tape.
I guess the RAM vs disk debate is on similar lines - but there are some vital differences:
1. Disks (esp. IDE) have become a commodity item and can be accessed by different system architectures easily.
2. IDE and SCSI standards have stood the test of time - 13 and 20 years respectively, unlike RAM wihch has evolved from Parity, non-EDO, EDO, DRAM, SDRAM, DDR-RAM, RAMBUS RAM etc., and suffers several patent and copyright encumberances.
3. Although RAM prices are driving down, the h/w to interface speciality RAM banks is proprietary and hence cost-prohibitive, and comes with attendant long-term supportability risks - think Palladium, or even Server mobos over the last 10 years. TCO for RAM based systems could thus be much higher than disk-based systems.
Overall, except for apps that need super-high speeds, and users that can risk proprietary stuff, disk-based databases shall remain.
My 0.02
If you keep throwing chairs, one day you'll break windows....
And large image-bases are too much for RAM today. Remeber the 640K limit? I'd guess that more than 50% current corporate databases would fit in a single gigabyte, including indexes.
The real speed improvements, according to the guys working on projects like Bamboo on sourceforge come not from the fact that it's in RAM, they test against SQL in RAM and show that most of the performance improvements come from keeping the data in the same process space as the application operating on the data. If they're right, putting MySQL in RAM or Oracle or Microsoft SQL Server is a small improvement.
The large speed improvements come from lowering the process boundary hopping overhead--marshaling and all that good stuff.
RAM-resident Database? Yes, that would be Google -- a massive, massive cluster of x86 boxen with a couple gigs of RAM apiece. Each gets a portion of the hashspace, leading to near-O(1) searchability. I'm pretty sure all the big search engines work this way, at this point -- the DB is checkpointed into RAM, but is never actually run from it.
:-)
Recent discussions about disks vs. CPU's have ignored the massive decreases in the cost of RAM. For a very long time, the secret bottleneck in PC's (in that it wasn't advertised heavily) was RAM. That's starting to disappear -- there's a gig in my laptop, and there's no discernable improvement in all but the most intense applications if I were to go beyond that.
Virtual Memory is already on the chopping block; any time it's imaginable that a system might need another gig of storage, it's probably worth going to the store and spending the hundred dollars.
But what if more RAM is indeed needed? One of the most interesting developments in this department has involved RDMA: Remote DMA over Ethernet. Effectively, with RAM several orders of magnitude faster than disk, and with Ethernet achieving disk-interface speeds of 120MB/s, we can either a) use other machines as our "VM" failover, or more interestingly, b) Directly treat remote RAM as a local resource -- a whole new class of zero copy networking. This Is Cool, though there are security issues as internal system architectures get exposed to the rough and tumble world outside the box. It'll be interesting to see how they're addressed (firewalls don't count).
What next, for the RAM itself? I don't think there's much that value in further doublings...either of capacity, or soon, of speed. What I'm convinced we're going to start seeing is some capacity for distributed computation in the RAM logic itself -- load in a couple hundred meg in one bank, a couple hundred meg in another, and XOR them together _in RAM_. It'd just be another type of read -- a "computational read". Some work's been done on this, though apparently there's massive issues integrating logic into what's some very dumb, very dense circuitry. But the logic's already done to some degree; ECC verifiers need to include adders for parity checking.
My guess...we'll probably see it in a 3D Accelerator first.
*yawns* Anyway, just some thoughts to spur discussion. I go sleep now
Yours Truly,
Dan Kaminsky
DoxPara Research
http://www.doxpara.com
Many databases contemplate database sizes in the tens to tens of thousands of gigabytes. For smaller databases, RAM is an easy thing, and even for a small number of gigabytes it might be reasonable. For larger databases RAM would be unthinkable. The fact that a database developer doesn't know what databases are used for is disturbing.
Most modern databases also make very effective use of RAM as a cache in order to speed up queries. I don't know about MySQL since I don't use it. My guess, however, is that it does not, since that would eliminate the need for this stupid measure.
As far as reliability, RAM is more vulnerable to transient things like cosmic radiation. ECC memory will take care of most single-bit problems (there are lots of them...), but all it can do for multi-bit failures is determine that yes, your data is screwed.
Also, swapping out a bad hard disk in a RAID configuration is relatively simple and has a recovery process. Suppose your RAM stick fails; what is your recourse? You've permanently lost that data, and systems with hot-swappable RAM are much more costly than ones with similar capabilities for hard drives.
Finally, consider the problem of catastrophic system failure. If the power goes out, your RAM dies but your hard disk is still safe. If it is worse (say your facility burns down) then it is much easier to recover data from the charred remnants of a hard disk than from the charred remnants of a DRAM chip.
The idea of replacing disks with DRAMs has been around for quite a while now. But disks continue to get (a bit) faster and (much) larger. Every time the morons want to replace it they get shot down. More sensible people focus on using the resources available in ways such as caches that make systems faster and more reliable.
This is interesting. Lots of responses so far have said that putting a database into volatile memory is preposterous. But from reading the article I'm not certain if it's such a bad idea in some situations. There are often sites that have a lot of relatively static data in their databases. These sites often use a backend database because it's easier, programmatically and as far as maintenance is concerned, to do so rather than create lots of static pages. Writes to the database could be done as a pass-through so they do get written to the disk "backup". A good example may be Google's cache -- the pages do not need to be re-indexed all the time but speed is critical. If RAM can be faster and, possibly even use less power than a hard drive, then there is a benefit. In Google's case, there is no writing, only queries.
This means that in any situation where data is unchanging except for periodic updates this could be a good idea.
http://www.imperialtech.com/success_ebay.htm
The basic idea: use solid state ram drives (with separate power supply) for your busy tablespaces and your redo logs.
This leverages 'cheap ram' technology with existing (and proven and scalable) db architecture.
For ebay, for example, they might store 'active items' in 'ram-drive-backed' tablespace and 'old items' in the 'hard-drive-backed tablespace'.
These solid-state drives are expensive, but additional Oracle licenses (or moving from 'standard' to 'enterprise' or to 'clustered') are very very expensive.
bill m
There is already a leading in Memory database that is extremely fast. Check out TimesTen. That is what we use. There is also another one called Polyhedra. But the redundancy on Polyhedra doesn't appear to be as good as TimesTen, and it doesn't support Unicode either.
...have nothing to do with the medium the data is stored in! What you're trying to guard against is concurrent access of resources by transactions which in cases can cause incorrect or inconsistent results in a RDBMS. I think this article is a bit obvious for most people who've had any training in how databases actually work and I think Monty was actually pretty gracious for taking the time to give the interviewer a smidgeon of clue.
There are some cool ideas there. They use two copies on disk for backup in case of system failure. Because of this they don't have to do page-latching.
In some configurations, though, this is irrelevant, because write transactions lock the whole database! Because they know all transactions will be extremely short, this is faster than locking at page or row level.
I told my boss that it was a very Bad Thing as the stores could lose data so easily. He told me several things:
- Running entirely in RAM, the system was very very fast. When the system could smoke a more expensive IBM PC-XT running a dBase app, he could sell more systems
- Every system would have a UPS as he refused to sell them without
- He signed my paycheck, not the other way around
As best I can figure, Darwin was more interested in awarding JATO assisted drag racers back then because we got lucky and actually had more trouble with the systems using hard drives. That was back during the heyday of the small mom-and-pop video stores. The last of those RAM disk based systems that I knew of converted to a "real" system in 1993. I believe they were assimilated by one of the national chains soon after.You either believe in rational thought or you don't
Memory sounds like a good idea in theory .. but what about power failures or momentary blips .. UPS can help but not eliminate that risk.
.. with some options for active standby and hot replacement available.
.. as the number of individual ram components increases the risk of a single bit non ecc correctable fault scales up accordingly .. such that with 8 gig + arrays the chance of uncorrectable error approches 50% per time interval
.. several of the Sun Fire 6800 series machines I work with on a regular basis develop these kinds of errors occasionally .. though with Sun the hardware is smart enough to map around the error and make relevent OBP console & syslog entries.
A recent hardware write up I read from HP / Compaq has ram partitioning / raid'ing on some of the higher end x86 servers
Another little burb was that with ram
I know memory can develop stuck bits without any warning
Hell, I'd just be happy if they would normalize their tables. I've seen joins across three tables which all hold essentially the same data, and working on the entire result set after downloading it to the client and then uploading the whole thing back when one thing has changed.
And they wonder why I'm crazy.
You think that I'm crazy, you should see this guy!
Normalize until it hurts, denormalize until it works. Until you hit the really huge databases, you won't appreciate that saying.
I'm confused. I actually haven't used MySQL much, and someone else can clarify its current ACID compliance. My application involves multiuser financial transactions. When making my DB selection a couple of years ago, at that time it was claimed that MySQL had some ACID deficiencies that made me nervous. I settled on PostgreSQL, which I'm very happy with.
But there's a lot more to ACID than just keeping RAM and disk in sync, and I don't see how RAM would make ACID that much easier, and certainly not "almost trivial". You still have all the transactional semaphores, record locking, potential deadlocks, rollbacks, etc. to worry about. In fact I don't see why you wouldn't just have the RAM pretend to be a disk and be done with it, since the disk version already has stable software. Then, if it is important to increase performance further, RAM-specific code optimization could be done over time, but slowly and carefully.
I'm sorry - I really don't want to get into a religious war here, but the interview didn't do much to bolster my confidence in MySQL for mission-critical financial stuff. Educate me.
If you've got enough ram for your database to fit in, why not mmap it and do a simple search? It tends to take up much less memory than a database and you can search a whole lot of records in the time it takes to do a context switch (which is what you get when you use a socket to talk to the database program).
Here's a tip kids, when you stop playing around with the toy databases, give us a call and we'll show you why you still need hard drives.
Back in the early 90's IBM added a machine instruction to their mainframes called DIV. It treated data in a file system as if it where in virtual mememory - ie addressRecord[12345] appeared to the program as an in memory array, but was backed by disk storage - the same format that was used for paging virtual memory - brilliant. It's a shame it never caught on - it would make advances like this transparent in implementation. Well I guess you can't really say it never caught on - it was a big reason IBMs mainframe databases outperformed everyone else for so long.
Is there a similiar kind of instruction on Intel? It's probably too late though - indexed arrays have become less useful since associative array patterns have become better defined. A hardware implementation (RAM) of JDO would be interesting.
slashdot troll = you make a compelling argument I do not like the implications of.
There are components of ACIDity that would be implmented very differently for RAM-persistent databases than for disk-persistent ones. Maintaining ACIDity on disk-persistent databases requires complicated algorithms to mitigate the disatrous disk seek times. These complicated algorithms would be rendered unnecessary if disks were no longer used.
For example, disks have incredibly slow seek times and much better bandwidth; therefore it's far cheaper to write things to disk in big chunks. The purpose of write-ahead logging (or "redo logging") is to mitigate the performance impact of slow seek times by blasting all the transactions to disk at once, in the redo log, thereby avoiding the slow seeks that would be required by putting each transaction in its proper place. Putting the transaction data in its proper place is deferred until after the load has died down somewhat. This could be seen as exchanging seek times for bandwidth.
This redo log mechanism would be unnecessary for ram-persistent databases. It's a significant source of complexity that would be obviated by the removal of disks. And that's just one example of complexity required to get adequate performance from disk, a medium that has disastrously slow seek times.
the interview didn't do much to bolster my confidence in MySQL for mission-critical financial stuff. Educate me.
Cancel that financial adjective. You're thinking too narrowly.
Infuriate left and right
I believe it surfaced a while back on /. - can't find any links at the moment, but AFAIK the entire Google index is stored in RAM.
grisha.org
A caveman dreams of being us, the incalculable power and riches. We dream of being Q, then what?
Running the DB from RAM is nice, but as far as I can see this won;t require any changes to the software itself, you could just mount your DB on a RAMdisk and be done with it. Whats the big deal?
What MySQL and PostgreSQL really lack is the ability to replicate on-the-fly and to support running on clusters for *real* failover and fault tolerance.
For Postgres, this means multiple 'postmaster' processes being able to access the same database concurrently, and probably something similar for MySQL.
Being able to run a database on an OpenMOSIX cluster, for example, would make it massively scalable, and being able to run multiple independent machines with an existing HA (High Availability) monitoring system would provide a truly fault-tolerant database.
There are of course major technical difficulties involved in making databases work this way, but an Open Source DB that can compete with Oracle's 'Unbreakable' claims would be a huge shot in the arm for OSS in the business world.
I gots ta ding a ding dang my dang a long ling long
Rarely? When you work in a bank, "Rarely" is a 100 billion dollar word. You don't want that sort of damocles hanging over you. When one disk fails, generally the other disks don't. If you've got 100 copies of your stock exchange, you're not going to cry if you've only got 99. The problem is that when power fails, every single copy of your data dies.
Then again, bank dataservers are presumably distributed clusters, and taking out all of them at once is pretty damn hard. Guaranteeing power isn't all that hard -- like you said, have some batteries in a UPS type arrangement; enough to keep your entire cluster running for a minute or two while your diesel generator starts spinning. Have several UPSes, several diesel generators, and a very strict schedule for testing and replacing batteries and you should have a system that can survive anything a hard disk can.
Just don't use the word "rarely" when you're telling the CEO about something that's going to handle more money than some nation states have.
Any Perl programmers in the audience may wish to check out DBD::RAM. From the CPAN documentation: "DBD::RAM allows you to import almost any type of Perl data structure into an in-memory table and then use DBI and SQL to access and modify it. It also allows direct access to almost any kind of file, supporting SQL manipulation of the file without converting the file out of its native format." More information here
First, as other have said, a properly designed RAM subsystem can be battery backed up. In terms of getting the data out, loss of power to the RAM is no more catastrophic than loss of power to the CPU, the router, the computer running the middleware, or whatever. Because RAM is a purely semiconductor approach, any battery backup system can be simple and reliable.
In fact, it should not be too difficult to design a system which, in the event of power fail, dumps data to backup disk drives. To get to that state, the main system has already failed to do a clean shutdown, so this is a last resort issue.
The next thing is error detection and correction. It's true that single bit ECC is limited, but it also takes only limited resources (7 additional bits for a 32-bit subsystem, 8 for 64). Memory subsystems could have extra columns so that if bit errors start to multiply in one column, it can be switched out for a new one. Just as with any error detection and correction strategy, single bit detection in columns can be combined with further error correction down rows, using dedicated hardware to encode and decode on the fly. Just good old basic electronics.
In the worst case, it should be possible to build an extremely reliable memory system for a bit penalty of 50% - no worse than mirroring two hard drives. It won't be quite as fast as writing direct to motherboard RAM, but we don't want to do that anyway (we want to be able to break the link on power fail, save to disk, then later on restore from disk. And we want the subsystem in its own cabinet along with the batteries. No one in their right minds is suggesting having a couple of C cells taped to this thing and held on with croc clips.)
I'd even venture to suggest that most MySQL databases are not in the terabyte range, and that most databases aren't in the gigabyte range even if they are mission critical in SMEs.
Conclusion? As usual we have the people trying to boast "My database is far too big and complicated for MySQL! So MySQL sucks! My database is too (etc.) to run in RAM! So running DBs from RAM sucks!" and ignoring the fact that there are many web databases where transactional integrity is not an issue, and the market for a RAM store for databases in the low Gbyte range might actually be rather substantial.
Panurge has posted for the last time. Thanks for the positive moderations.
We need archival storage devices that won't lose data unless physically destroyed. We don't have them. Tapes don't hold enough data any more. Disk drives don't have enough shelf life.
DVD-sized optical media in caddies for protection, maybe.
(It's annoying that CDs and DVDs went caddyless. Early CDs drives use caddies to protect the CDs, but for some idiotic reason, the caddies cost about $12 each. We should have had CDs and DVDs in caddies, with the caddy also being the storage box and the retail packaging for prerecorded media. There's no reason caddies have to be expensive. 3.5" floppies, after all, are in caddies.)
Valid points, of course :) But you have to admit that for simple home pages (and not corporate databases) MySQL is simple, to-the-point, and easy to use.
And free, although many other ones are free as well. (I wouldn't want to run Oracle@Home ... of course, postgreSQL is also free, and I hear it's more mainstream as far as true database functionality.)
"Careful" people can enforce their own data integrity - obviously, it gets harder as the size (number and complexity of tables, I mean) of the database expands.
Can you tell I use it myself? :) You sound like you have experience with other database systems, how difficult do you think it would be to port an existing MySQL+PHP system to PostGreSQL or something similar?
...But such a condition (DB in RAM) will make his product pretty much obsolete.
The Prevayler Project is a RAM-only Java persistence project that works and is so simple not a single bug has been found in the production release.
3000 times faster that MySQL (9000 times faster than Oracle) even with the database in caches entirely in RAM simply because of the JDBC overhead that is eliminated .
The only sticking points I've seen are:
1. Normal PC's boards generally will only take 1GB of RAM. Sure there are thos expensive Sun machines...
2. Querying objects in an efficient manner.
3. Others, but I've gotta take a dump real bad...
Several companies are already doing this. As one poster stated before, Lucent is doing this, however the leader in this type of RDBMS is a company called TimesTen. http://www.timesten.com/ It was originally part of HP and it was spun off into its own company. Its a rather brilliant product. It only uses the disks for logging and disaster recovery. The unfortunate thing about the whole deal is the volatility of the ram and the fact that it is still cheaper to have 1TB of disk space for a huge database then it is to have half as much ram.
I like to call this number "software snobbery". Many people compare software applications feature for feature, paying no attention to what their requirements are. The fact is, a very large percent of the database market not need more than a single GB database for their current task.
Well, Monty started it by talking about SouthWest's booking system. Airlines are among the heaviest IT users in the world. If you want to drop names, be prepared to be called on your assertions by someone who does know. Someone asked me what a typical DB was in finance; I posted links to case studies on a couple of multi-terabyte databases. Also, SouthWest are an Oracle site, so they aren't even using MySQL for their seat reservations, so I don't know what Monty was talking about.
The fact is our MySQL deploys outnumber the Oracle deployments, and over time as MySQL and Postressql get better I'm expecting that MySQL will creep into Oracle space as well.
You assume that Oracle is a static target. It is not. PostgreSQL is a perfectly adequate database for small tasks; its SQL parser and core transaction processing is infinitely superior to MySQL. But both of them are beaten hands down by open source databases like SAP/DB and Borland Interbase (or Firebird, whatever it's called these days). I'd happily use either of those with databases of a few hundred M or even a few G.
Now, if open source advocates talked about those two when they talked database, I'd show some respect. But the constant worship of MySQL just shows that they don't know.
With everything in RAM, why not create true object servers, rather than distorting and maiming our object models by breaking encapsulation & imposing database layout restrictions?
http://www.prevayler.org
Modern storage solutions (like EMC) use redundant battery backed ram to buffer writes, greatly reducing perceived write latency. This gives you a lot of the performance gain of a ram only database, and also scales very well to large loads. (in fact, when choosing RAID stripe size you take into account whether writes are buffered; if not, keep stripes small for log files)
...most databases don't cleanup indexes after deletes, forcing periodic rebuilding. Other index schemes not generally considered because of poor locality prinicles could be considered. Note that Hash Indexes would probably still use Linear Hashing.
If you know that your data will always fit into available ram then there are a number of performance optimizations that can be done. I'm not sure about ACID becoming "trivial"; You still need most of the same db components: indexes, lock managers, operation journaling, etc. But many of these could be greatly simplified:
1. Page/Buffer Manager Eliminated. Since no disk IO will be required for the normal running of the db, there will be no need for a page manager. This eliminates complexity such as block prefetch and marking and replacement strategies. In fact, the data will probably not be stored on pages at all. Details such as block checksum, flip flop, log position, page latches etc can all be removed. The values in the rows would be sitting in memory in native language formats rather than packed making retrieval much faster. There would be no need for block chaining.
2. More flexible indexing. Since it is not necessary to store data in pages, traditional B-Trees are not absolutely required. Other index structures like AVL trees would be faster and might allow better concurrency. These trees would also be easier to keep balanced
3. Lock Manager Simplified. Row level locking (and MVC) are still desired features, but keeping the locks all in memory simplifies implementation. Oracle and InnoDB store lock information in the blocks (associated with transaction) to allow update transactions larger than memory.
4. Log manager simplified. You will still need journaling capability for rollback, replication, recovery from backup etc. But the implementation of the log need not be traditional. Any structure that maintains information about transactions and contains causal ordering will do. Techniques such as keeping old versions of rows adjacted to current versions that are unacceptable for disk based databases (ahem, Postgres) could be used.
Although these may seem like small things, they can add up: less code to run is faster code. A company called TimesTen offered a product that they claimed was 10x faster than Oracle using an all memory DB. Generally the corporate world doesn't care to split hairs. They want something that works, and they are willing to throw some money and iron at it. Thats why battery backed ram in the disk controller to buffer writes is probably going to be fine for now.
A last note: modern databases already know to not bother with indexes when a table is sufficiently small.
JJ