Slashdot Mirror
MySQL Creator Contemplates RAM-only Databases
Aavidwriter writes "Peter Wayner asks Michael 'Monty' Widenius of MySQL, 'When will RAM prices make disk drives obsolete for database developers?' From Monty's answers, it sounds like hard drives may be nothing but backups before long." From experience, I'd wager that RAM failure rates are less than hard drive failure rates, so it might also mean more stability from that perspective.
But also a very strong Memory Manager. We've all seen a poorly written program corrupt memory.
There are these things called batteries. Of course the article mentions them, but who reads the articles anyway...
RAM only needs a trickle to keep refreshed...
If you have a database that is stored in RAM and periodically written out to the hard disk (for backup reasons) then you get better performance than if you have a database that is reading and writing most of the time.
UPS would prevent the data loss, the database could be written to disk when the power fails.
Wow! $25,000 for 16 GB of RAM disk seems a tad high or widespread adoption.
It's also interesting to note that Microsoft was going to release what they called In Memory Database(IMDB) support in Windows 2000. However, this feature was removed after Windows 2000 RC2 due to technical issues.
Considering the non-existent ACID support in MySQL it sounds like a good idea, it's not like MySQL will get any more errorprone than it is now...
I guess the issue with databases is not only speed and reliability, but a totally different ballgame called 'user-perception'. Even now, tape drives are used to archive databases; despite the fact that less than 1 in 1000 of the tape media get used for actually retrieving the data during a crash. NAS devices and the like have changed this, but the temptation remains to use tape.
I guess the RAM vs disk debate is on similar lines - but there are some vital differences:
1. Disks (esp. IDE) have become a commodity item and can be accessed by different system architectures easily.
2. IDE and SCSI standards have stood the test of time - 13 and 20 years respectively, unlike RAM wihch has evolved from Parity, non-EDO, EDO, DRAM, SDRAM, DDR-RAM, RAMBUS RAM etc., and suffers several patent and copyright encumberances.
3. Although RAM prices are driving down, the h/w to interface speciality RAM banks is proprietary and hence cost-prohibitive, and comes with attendant long-term supportability risks - think Palladium, or even Server mobos over the last 10 years. TCO for RAM based systems could thus be much higher than disk-based systems.
Overall, except for apps that need super-high speeds, and users that can risk proprietary stuff, disk-based databases shall remain.
My 0.02
If you keep throwing chairs, one day you'll break windows....
RAM-resident Database? Yes, that would be Google -- a massive, massive cluster of x86 boxen with a couple gigs of RAM apiece. Each gets a portion of the hashspace, leading to near-O(1) searchability. I'm pretty sure all the big search engines work this way, at this point -- the DB is checkpointed into RAM, but is never actually run from it.
:-)
Recent discussions about disks vs. CPU's have ignored the massive decreases in the cost of RAM. For a very long time, the secret bottleneck in PC's (in that it wasn't advertised heavily) was RAM. That's starting to disappear -- there's a gig in my laptop, and there's no discernable improvement in all but the most intense applications if I were to go beyond that.
Virtual Memory is already on the chopping block; any time it's imaginable that a system might need another gig of storage, it's probably worth going to the store and spending the hundred dollars.
But what if more RAM is indeed needed? One of the most interesting developments in this department has involved RDMA: Remote DMA over Ethernet. Effectively, with RAM several orders of magnitude faster than disk, and with Ethernet achieving disk-interface speeds of 120MB/s, we can either a) use other machines as our "VM" failover, or more interestingly, b) Directly treat remote RAM as a local resource -- a whole new class of zero copy networking. This Is Cool, though there are security issues as internal system architectures get exposed to the rough and tumble world outside the box. It'll be interesting to see how they're addressed (firewalls don't count).
What next, for the RAM itself? I don't think there's much that value in further doublings...either of capacity, or soon, of speed. What I'm convinced we're going to start seeing is some capacity for distributed computation in the RAM logic itself -- load in a couple hundred meg in one bank, a couple hundred meg in another, and XOR them together _in RAM_. It'd just be another type of read -- a "computational read". Some work's been done on this, though apparently there's massive issues integrating logic into what's some very dumb, very dense circuitry. But the logic's already done to some degree; ECC verifiers need to include adders for parity checking.
My guess...we'll probably see it in a 3D Accelerator first.
*yawns* Anyway, just some thoughts to spur discussion. I go sleep now
Yours Truly,
Dan Kaminsky
DoxPara Research
http://www.doxpara.com
Many databases contemplate database sizes in the tens to tens of thousands of gigabytes. For smaller databases, RAM is an easy thing, and even for a small number of gigabytes it might be reasonable. For larger databases RAM would be unthinkable. The fact that a database developer doesn't know what databases are used for is disturbing.
Most modern databases also make very effective use of RAM as a cache in order to speed up queries. I don't know about MySQL since I don't use it. My guess, however, is that it does not, since that would eliminate the need for this stupid measure.
As far as reliability, RAM is more vulnerable to transient things like cosmic radiation. ECC memory will take care of most single-bit problems (there are lots of them...), but all it can do for multi-bit failures is determine that yes, your data is screwed.
Also, swapping out a bad hard disk in a RAID configuration is relatively simple and has a recovery process. Suppose your RAM stick fails; what is your recourse? You've permanently lost that data, and systems with hot-swappable RAM are much more costly than ones with similar capabilities for hard drives.
Finally, consider the problem of catastrophic system failure. If the power goes out, your RAM dies but your hard disk is still safe. If it is worse (say your facility burns down) then it is much easier to recover data from the charred remnants of a hard disk than from the charred remnants of a DRAM chip.
The idea of replacing disks with DRAMs has been around for quite a while now. But disks continue to get (a bit) faster and (much) larger. Every time the morons want to replace it they get shot down. More sensible people focus on using the resources available in ways such as caches that make systems faster and more reliable.
Surely a well tuned database server stores uses quite a lot of RAM for buffering?
Well, a professional database like Oracle manages its own cache, but MySQL really only relies only on the OS-level cache. The problem with that approach is that the database knows a lot more about what you're doing, so it can make much smarter decisions about what to cache, what to age out, what to prefetch, etc. On an Oracle server, you want to lock the buffer in memory, and leave as little as possible over for the OS filesystem cache. You see, if a block doesn't exist in the cache, it has to be fetched from disk into the database cache, and if it does, the db will go straight to its own cache. Another caching layer inbetween provided by the OS is just wasted.
I don't think Monty understands any of this; in the article he seems to say that ACID, rather than being a fundamental principle of relational databases, is just something you need to do because disks are slower than RAM. The only reason that you might not want full ACID and use semaphores instead, as he suggests, is because you are only updating one record in one table in a single transaction!
Further, if he is thinking in terms of a few Gb of data, then he is a little out of touch with modern database usage. SouthWest airlines do not have a database that stores 10 bytes of data for every seat on every flight, and I have a hard time figuring out why they would want to - the database of seats would be tied into the customer records, the freight/baggage handling database, the billing records, the accounting system. That is the point of relational database, that you can query data based on data that exists elsewhere. Monty thinks in terms of databases that have only a few tables, because that's all you can do in MySQL. He says that database programmers are forced to be very careful about the consistency of their data - well those using MySQL are, but those using Oracle (or any other database with real transactions and real integrity constraints) find it's all taken care of transparently.
...have nothing to do with the medium the data is stored in! What you're trying to guard against is concurrent access of resources by transactions which in cases can cause incorrect or inconsistent results in a RDBMS. I think this article is a bit obvious for most people who've had any training in how databases actually work and I think Monty was actually pretty gracious for taking the time to give the interviewer a smidgeon of clue.
Not true, you are describing Parity RAM. Let's review:
Normal RAM: no errors detected
Parity RAM: an extra bit is used to store the XOR of all the othe bits on the line. If the math doesn't work, the OS can be informed the data at a particular address is corrupt, and let the software decide how to best cope with the condition. Only odd numbers of bit errors can be detected.
ECC RAM: Uses a more complicated method and even more bits to provide single-bit error *correction* and double-bit error detection. The odds of two bits on the same line being corrupt is slim unless the ram is really bad. Single bit errors cause no problems. Double bit errors are at *least* detected. More than that and chances are the memory module is so screwed that even if the OS doesn't get it reported, the problem will be obvious to the user.
XML is like violence. If it doesn't solve the problem, use more.
I'm confused. I actually haven't used MySQL much, and someone else can clarify its current ACID compliance. My application involves multiuser financial transactions. When making my DB selection a couple of years ago, at that time it was claimed that MySQL had some ACID deficiencies that made me nervous. I settled on PostgreSQL, which I'm very happy with.
But there's a lot more to ACID than just keeping RAM and disk in sync, and I don't see how RAM would make ACID that much easier, and certainly not "almost trivial". You still have all the transactional semaphores, record locking, potential deadlocks, rollbacks, etc. to worry about. In fact I don't see why you wouldn't just have the RAM pretend to be a disk and be done with it, since the disk version already has stable software. Then, if it is important to increase performance further, RAM-specific code optimization could be done over time, but slowly and carefully.
I'm sorry - I really don't want to get into a religious war here, but the interview didn't do much to bolster my confidence in MySQL for mission-critical financial stuff. Educate me.
You really cant compare things like that for databases. AISITA (as is stated in the article) the big bottlenecks for both are similar in nature but orders of magnitude in scope. I currently work with a medical database where everything has to be logged. Disk access is a big factor for us, so we use fibre channel scsi (specifically Seagate 73.4GB 10000RPM) where the cost is more like 700 dollars for 70gb) (basically $10 per GB not the $1 per GB you are showing) Also there is the issue of supporting hardware but we will ignore that for the time being.
time for some napkin math:
1 512MB ecc reg pc2100 dim -> $ 78 or $156GB
1 70GB Fibre Channel Drive -> $700 or $ 10GB
Now lets factor in raid (for access speed and redundancy)
we typically put 8 drives in a bundle which tends to give us 36% of the total drive capacity (mirrored raid 5 aka raid 6 remember teh ram is ecc reg so this factoring is already in place for it)
8 * $700 -> $5600 for
36% * 8 * 70 = 200GB
This give me approximately 1GB for $28
now thats a factor of 5.6 (call it 6) in price from ram only. AND i still get a prolly 4 fold increase in throughput. Not bad at all in my book.
Bad Panda! No Bamboo for you! In matters of importance ACs will not be responded to. Want to say something critical,OK
There are components of ACIDity that would be implmented very differently for RAM-persistent databases than for disk-persistent ones. Maintaining ACIDity on disk-persistent databases requires complicated algorithms to mitigate the disatrous disk seek times. These complicated algorithms would be rendered unnecessary if disks were no longer used.
For example, disks have incredibly slow seek times and much better bandwidth; therefore it's far cheaper to write things to disk in big chunks. The purpose of write-ahead logging (or "redo logging") is to mitigate the performance impact of slow seek times by blasting all the transactions to disk at once, in the redo log, thereby avoiding the slow seeks that would be required by putting each transaction in its proper place. Putting the transaction data in its proper place is deferred until after the load has died down somewhat. This could be seen as exchanging seek times for bandwidth.
This redo log mechanism would be unnecessary for ram-persistent databases. It's a significant source of complexity that would be obviated by the removal of disks. And that's just one example of complexity required to get adequate performance from disk, a medium that has disastrously slow seek times.
UPS is old technology, the battery needs constant replacement, and very few have multiple redundant batteries and/or transistors to deal with wear and tear. Yes even a simple MOSFET transistor is not 100% reliable. Usually the only way to tell a battery is dead is your UPS fails when you need it (this happened to us when my MD was demonstrating our service live to customers, afterwards was the only time he's taken less than 9 months to sign off a purchase order on new equipment). A UPS also has a power cord to pull out when you recoil after burning your fingers on a Seagate Cheetah 15000RPM HD in the server room. A UPS also trips if you overload it, which again means the UPS fails when you most need it.
Other posts mention cosmic radiation at high altitude makes RAM fail. Last time I checked there were no Quad-Xeon Oracle databases on Concorde, although if the International Space Station were to use one this might pose a problem for non-ECC RAM. Anyway, somebody could always write a driver to do software-ECC with Reed-Solomon for RAM if it becomes necessary.
Huge databases (>500 Gigabyes) would benefit most from this as running a simple OUTER JOIN query on the biggest tables will require most of the database to be called into RAM.
- Small databases become slow due to HD latency problems if they do a lot of WRITE operations (the database is stored in RAM, the transaction log is appended to, COMMIT TRANS). This would benefit least FROM RAMdisk because a HD append operation is cheap, however it would benefit database speed in mid-backup
- Mid-size databases become HD-intensive due to aggregate queries/triggered operations over large '>RAM' datasets. For instance enforced cascading deletes where millions of tuples are being deleted cascaded to hundreds of other unindexed tables (in my job I go to the toilet whenever I run a query like this).
- Huge databases where 'Index size' > 'RAM size' - the simplest query would benefit hugely from more RAM or faster storage or RAM-storage. With current databases this would be a 10Gig Eth connection to a Terabyte RAMSan solid-state disk.
In the future, who knows, maybe a FPGA/ASIC DPU (Database Processing Unit) for INSTANT COMMIT like NVidia's GPU?A caveman dreams of being us, the incalculable power and riches. We dream of being Q, then what?
Actually, there are RAID memory systems out there. One of HP's high end Intel servers has it. Hot swappable RAM as well...very neat stuff
First, as other have said, a properly designed RAM subsystem can be battery backed up. In terms of getting the data out, loss of power to the RAM is no more catastrophic than loss of power to the CPU, the router, the computer running the middleware, or whatever. Because RAM is a purely semiconductor approach, any battery backup system can be simple and reliable.
In fact, it should not be too difficult to design a system which, in the event of power fail, dumps data to backup disk drives. To get to that state, the main system has already failed to do a clean shutdown, so this is a last resort issue.
The next thing is error detection and correction. It's true that single bit ECC is limited, but it also takes only limited resources (7 additional bits for a 32-bit subsystem, 8 for 64). Memory subsystems could have extra columns so that if bit errors start to multiply in one column, it can be switched out for a new one. Just as with any error detection and correction strategy, single bit detection in columns can be combined with further error correction down rows, using dedicated hardware to encode and decode on the fly. Just good old basic electronics.
In the worst case, it should be possible to build an extremely reliable memory system for a bit penalty of 50% - no worse than mirroring two hard drives. It won't be quite as fast as writing direct to motherboard RAM, but we don't want to do that anyway (we want to be able to break the link on power fail, save to disk, then later on restore from disk. And we want the subsystem in its own cabinet along with the batteries. No one in their right minds is suggesting having a couple of C cells taped to this thing and held on with croc clips.)
I'd even venture to suggest that most MySQL databases are not in the terabyte range, and that most databases aren't in the gigabyte range even if they are mission critical in SMEs.
Conclusion? As usual we have the people trying to boast "My database is far too big and complicated for MySQL! So MySQL sucks! My database is too (etc.) to run in RAM! So running DBs from RAM sucks!" and ignoring the fact that there are many web databases where transactional integrity is not an issue, and the market for a RAM store for databases in the low Gbyte range might actually be rather substantial.
Panurge has posted for the last time. Thanks for the positive moderations.
Just from curiosity, how much data are we talking about for a large corporation, say SW Air or BofA?
Impossible to put a figure on the total amount of data that exists within an organization, but a typical SAN in a major financial institution has terabytes online. UBS Warburg has 2 Tb in just its general ledger database. Acxion has 25 Tb in its data warehouse, which will mainly be used for queries, whereas the GL database will be more transaction heavy. SouthWest is an Oracle customer, but it doesn't say here how much data they have.
We need archival storage devices that won't lose data unless physically destroyed. We don't have them. Tapes don't hold enough data any more. Disk drives don't have enough shelf life.
DVD-sized optical media in caddies for protection, maybe.
(It's annoying that CDs and DVDs went caddyless. Early CDs drives use caddies to protect the CDs, but for some idiotic reason, the caddies cost about $12 each. We should have had CDs and DVDs in caddies, with the caddy also being the storage box and the retail packaging for prerecorded media. There's no reason caddies have to be expensive. 3.5" floppies, after all, are in caddies.)
Valid points, of course :) But you have to admit that for simple home pages (and not corporate databases) MySQL is simple, to-the-point, and easy to use.
And free, although many other ones are free as well. (I wouldn't want to run Oracle@Home ... of course, postgreSQL is also free, and I hear it's more mainstream as far as true database functionality.)
"Careful" people can enforce their own data integrity - obviously, it gets harder as the size (number and complexity of tables, I mean) of the database expands.
Can you tell I use it myself? :) You sound like you have experience with other database systems, how difficult do you think it would be to port an existing MySQL+PHP system to PostGreSQL or something similar?