Slashdot Mirror
eBay Deploys 100TB of SSDs, Cuts Rackspace By Half
Lucas123 writes "eBay's QA division was facing mounting performance issues related to its exponential growth of virtual servers, so instead of purchasing more 15k rpm Fibre Channel drives, the company began migrating over to a pure SSD environment. eBay said half of its 4,000 VMs are now attached to SSDs. The changeout has improved the time it takes the online site to deploy a VM from 45 minutes to 5 minutes and had a tremendous impact on its rack space requirements. 'One rack [of SSD storage] is equal to eight or nine racks of something else,' said Michael Craft, eBay's manager of QA Systems Administration."
essentially because SSD has far better IOPs, you need less units to get the same speed. So you can cut the size of the storage array in half. Ebay are clearly io bound rather than needing huge storage space. So for them, its a win.
For others, maybe not so much.
I run: Windows, OS X, Linux, FreeBSD. Just because you have a hammer, doesn't mean everything is a nail.
TFA reads like a thinly-veiled promo for Nimbus Data Systems, which I can only guess are pushing a Linux-based SAN appliance full of SSDs. Big whoop.
What I would love to know is: Why does eBay need 4000 VMs ?
-Billco, Fnarg.com
Depends on your workload. (Disclosure: I work in storage for a living.)
Sometimes, what you need is raw, bulk storage. There are two serious contenders in this space: tape, and disk. You use tape if you have a lot of data you need to store, but not much that you need to access regularly: less power, and it scales to near infinite levels of storage (at the cost of very slow access for a given piece of data.) Or you use disk if you need to access most of it reasonably regularly. SSDs are not, and never will be, a contender in this space - you're paying through the nose on a per GB basis.
On the other hand, sometimes what you need is IOs per second. Database administrators are very familiar with this - you need a bit of data from over here, and a bit of data from over there, and maybe a little bit more from somewhere in the middle, and you need it five minutes ago. Traditionally, you got this performance by building a large array across many spindles, and giving up half, three quarters, or even more of your disk space, in return for that nice, fast section of disk on the outside of the platter. Lots of running hard drives, drawing lots of power, generating lots of heat, and costing a lot of money for storage space that isn't even half used - because if you throw something else on that empty space, you completely ruin the throughput of your major production database.
In that latter space, SSD is king. Sure, it's more expensive on a dollars per GB basis, but hey, guess what - GB isn't the important metric here. You figure out which bit of data is being hammered, and you move it across to the SSD. Rather like profiling an application: pick the function that takes 90% of the time in the software, optimise the wazoo out of it, and you get a significant improvement (rather than picking something at random and optimising it to billy-oh, and getting not much return for your investment.)
So yeah - SSDs aren't going to compete in raw capacity any time soon. But in random I/O performance, they make a hell of a lot of sense. In some respects, yes, they most definitely are cheaper than traditional platters of spinning rust - see the aforementioned massive RAID set across dozens of spindles.
You discover modern hardware does virtualization real well. You get a good host software, like vSphere or something on new hardware and you have extremely near native speeds. The CPUs handle almost everything just like it was running as the host OS, and sharing the CPU resources works great. Memory is likewise real good, in fact VMs only use what they need at the time so they can have a higher memory limit collectively than the system RAM and share, so long as they don't all try to use it all at once.
You really do have a situation where you can divide down a system pretty evenly and lose nothing. Let's say you had an app that used 2 cores to the max all the time and 3GB of RAM. You'd find that it would run more or less just as well on VM server with 4 cores and 8GB of RAM, half assigned to each of two VMs, as it would on two 2 core 4GB RAM boxes. ...Right up until you talk storage, then everything falls down. You have two VMs heavily access one regular magnetic drive at the same time and performance doesn't drop in half, it goes way below that. The drive is not good at random access and that is what it gets with two VMs hitting it at the same time, even if their individual operations are rather sequential.
It is a bottleneck that can really keep things from scaling like the other hardware can handle.
At work I use VMs to maintain images for instructional labs (since they all have different, custom requirements). When I'm doing install work on multiple labs, I always do it sequentially. I have plenty of CPU, a hyper-threaded 4 core i7, plenty of RAM, 8GB, there's no reason I can't load up multiples. However since they all live on the same magnetic disk, it is slower to do the installs in parallel than sequential.
If I had an SSD, I'd fire up probably 3-4 at once and have them all do installs at the same time, as it would be faster.