Slashdot Mirror
3 Terabytes, 80 Watts
legoburner writes "The Enquirer is reporting that Capricorn have released a mini-itx based 1U-sized storage computer featuring four 750-GB hard drives and a 1-Ghz controller system with a typical power usage of an astounding 80 W per machine. A full 40U rack only uses 3.2 kW, which is less than 30 kW for an entire Petabyte!"
And they would need all that storage to record their utility bills.
At last, a chance for a rejected ask slashdot of mine... What is the structure of your file storage area / file server? How do you filter and back things up for your home file server?
Warhammer forums
As long as it's not silver, black or grey, I'm fine with adding another petabyte to my current configuration. If only my file system could handle it...
Knowing my luck, I'd probably tip the rack over and lose all the bank data to a massive head crash.
If information is power, then this thing is a perpetual motion machine.
A) Because why would you want every reality TV program at your fingertips?
B) Because we already do (See http://bittorrent.com/)
C) Because... just because.
To understand recursion, one must first understand recursion...
Not when I keep getting a new internet every few minutes. A whole internet every few minutes! Can you imagine how many libraries of congress that is? I don't know about you, but I'd have a lot of trouble stuffing an entire library of congress into one of those tubes! And since the library of congress is obviously a lot bigger than this storage computer, there's no way you could stuff it into it!
Until they come out with one of these that's bigger than the library of congress, I'm not buying!
- Senator Ted Stevens, computer guru extraordinaire
Use 'slashdot stuff' in the subject line in any email you send me if you want to get past the spam filter.
My head spinning with the amazing possibilities such an immense data storage per kW solution could be applied to...why, it was even re-kindling my interest in the mini-ITX board. Then....I read the article
"The next step up is the TB120 PetaBox, basically a rack of 40 GB3000s and an ethernet switch or two."
WOW! so far so good...then, things turn ugly
"If you need more space than that, I would say it is time to lay off the naughty pictures for a bit and seek serious help.
In any case, Capricorn is saying you can get into one to the TB120s for about $1.50 a GB, and a little math says a full rack would cost under $200K. If you think that is a lot, imagine the Tivo you could make out of one, you could have every reality TV program at your fingertips for a little less than the cost of an average house."
Yup, for a mere $200K, you too can have every reality TV program and/or naughty picture at your fingertips...and here I was thinking about mundane things like virtual libraries, genome sequencing, protien folding, etc.
I'm going to be sad now...
A goal is a dream with a deadline
What's the best SW RAID to run against a cluster of these 3TB 1U appliances? That transparently offers swappable cluster units to apps designed to write to local filesystems?
--
make install -not war
I'm not interested in no mini-itx box unless it's wedged into an adorable, panda-like case. http://www.norhtec.com/products/panda/index.html
there is no need to sign your posts. this isn't usenet. your username is right there above your post. stop it.
And they would need all that storage to record their utility bills.
Where do you live that 80 watts is a big drain on financial resources?
My CPU consumes 39 watts and I consider that loverly, compared to the old CPU which sucked 70+ watts.
A feeling of having made the same mistake before: Deja Foobar
Since it's not mentioned on their webpage or in the article, I searched for a listing of the price points and found the following.
h tml
"The PetaBox nodes and racks are available now. Base pricing for the nodes (512K RAM, 10/100 interface, and no LCD) ranges from $1,595 (GB1000) to $3,395 (GB3000)." http://products.datamation.com/dms/sc/1156440622.
The GB1000 is the 1TB node and the GB3000 is the 3TB node. I think they might mean 512MB of RAM base, but who knows. Sounds like it's a Fedora linux based product which makes me wonder what services it provides, they don't list. I would assume basic NFS/SMB/AFS services but there's no mention of backup / replication services, mirroring between twin nodes, etc that competitive products offer.
Swedish plasma phys. PhD student; MSc EE; knows maths, programming, electronics; finance interest; seeks opportunities
And, no, I'm not talking about the starship Enterprise, so can it with the "Star Trek" comments.
Obviously, this is the kind of product that companies and perhaps even data centers will possibly take a very long and desiring look at. No doubt that's exactly what Capricorn is hoping for. 3.2Kw/hr is nothing compared to the power that's eaten up by a rack that's loaded with arrays and SCSI drives.
My concern is with reliability. For the most part, the general attitiude is that SCSI, while much more expensive than IDE or SATA, is also more reliable with a larger MTBF. Whether that's really true or not is up for debate, but that's the general opinion that out there. Of course, there's also the general attitude that more spindles means more throughput and more reliability if in a proper RAID configuration. From what I've seen with other solutions, we can probably assume with a wide margin of safety that 120TB for this Capricorn system is RAID 0. If a 1U system only contains four drives and they're all independent RAID configurations, then say goodbye to 30 TB just to add a modicum of redundancy with RAID 5, whereas if there were more spindles, the amount of lost space would be greatly decreased even though there would be the increased chance of a failed drive.
Looking at this system, my gut feel is that a more-spindle configuration might be a wiser move, unless the money saved in electricity goes to a better-than-average backup system. Maybe it's my bias towards SCSI/fibre channel, but I don't know that I can yet trust a low-spindle, IDE configuration to do the same thing in an enterprise environment.
Just out of curiosity, has anyone out there in Slashdotland had good luck with enterprise IDE solutions? Who knows. Perhaps some success stories might change my pro-SCSI/fibre view.
The Overrated mod is for reversing inappropriate, positive mods, not for voicing disagreement with a post.
Technically this is a convenient way to stuff a lot of managed storage into a small space with low power consumption. Cool, but it's really nothing more than a bunch of servers in a single rack with big hard drives. If I've got a petabyte of storage to utilize I want to manage it as one large pool (or several large pools), not 40 servers, on each of which I need to run an OS and services which make a relatively small portion of that storage available. Where's my FC or iSCSI target ports?
At $1.50 per GB in a large install its about two and a half times what one could build on their own. 5U 24 drive racks go for about $2k each, add in $2k for mobo/controllers, and 24 750 GB drives at $330 ea and we're at $11,920 per 18 TB. You can fit eight of these in a rack so that's 144 TB per rack at a cost of $95,360. Add in for the rack itself, a good switch, and some miscellaneous expenses and call it an even $100k. That's a cost of $0.66/GB.
It's not raid but it is a ton of storage space. Even if you back out one drive for parity and one for spare in each enclosure the cost per GB only goes up to about $0.72.
Low-end EMC SAN boxes use SATA: http://www.emc.com/products/platforms.jsp
Can I get that petabyte in Cornflower Blue?...As long as it's not silver, black or grey, I'm fine with adding another petabyte to my current configuration. If only my file system could handle it...
You can get that in any colour you like, as long as it's Beige
muah ha ha ha haaaa!
A feeling of having made the same mistake before: Deja Foobar
I dont think it's RAID anything at all.
The PetaBox TB120 says 120TB of space on 40 nodes. That's 3TB a node, and given 4 drives per node, that's 750GB drives.
So basically the RAID selection is left up as an exercise to the reader, they're just marketing raw diskspace with a very low power consumption.
Your hair look like poop, Bob! - Wanker.
if the goal is low power i'd prefer to use more than 4 drives in the system. half the power budget is the motherboard, so an 8 drive chassis would result in a 25% reduction in power for larger installations. Clearly the focus here isn't on performance after all.
If all I wanted was 4 drives why would I care? Why would I want a 1U rack? Why wouldn't I just stick them in my PC?
While this used to be true, modern drives are the same between IDE/SATA/SCSI except for the control board the drive is strapped to. The reason SCSI is still preferred over IDE/SATA in most cases is from this old belief, most devices for enterprise level storage are still built mainly around it, SCSI still offers more devices per controller (14 per cable, rather than 2 of IDE/SATA), and SCSI is alot more hot-swap friendly. The company I work with has several storage solutions for different needs, the central and main storage is a large Fiberchannel system (3Par InServ), but our backup systems are SATA based (Nexsan SATABeast). All of them use some variant of RAID5, the 3par going so far as allowing raided volume provisioning across the array. As for enterprise level IDE/SATA, the SATABeast, and SATABoy are definately worth at least a peek.
More throughput, maybe, if setup in a RAID that allows that. Reliability, maybe as in the array as a whole, but more spindles=more parts to fail, and with more spindles, more drives WILL fail. The up side is that when a drive fails, it doesnt take as large a chunk of the redundancy with it. With the 3Par, (iirc) a whole shelf of drives (40drives) can fail or be taken offline without losing operation of the array if setup correctly, where in a 4drive RAID5 setup (3 active, 1 hot spare), one drive failure requires rebuilding the failed drive on the hot spare. Losing another requires immediate replacement of hardware. For home or small office, that might be acceptable. But for large enterprise solutions, its not. You simply cannot afford to be running around hoping drives wont fail (they will), with a rack full of these 4drive units. If 2 drives go bad in the same unit at the same time, you just lost data.
tm
Support TBI Research: http://www.raisinhope.org
They're using ITX motherboards to keep price/power down. If they used notebook HDs instead of the 3.5" 750GB ones, they'd get about 10% the storage density per host, 50% the price performance per GB, but much better power efficiency per GB. Is there a way to stuff 40 80GB notebook drives into an ITX host, for even better power efficiency at only double the price?
--
make install -not war
Here's the Motherboard Info:
Motherboard/Processor:
* 1GHz VIA C3 CPU
* VIA CLE266 Northbridge
* VIA VT8237 Southbridge
* DDR266 RAM - Up to 1GB
* 2 USB 2.0 ports
* 1 Serial port
* 1 Parallel port
* 1 VGA port
* PS2 mouse & keyboard ports
Anybody have performance numbers for these units? A 1GHZ CPU can be hard-pressed to run an OS, serve disk and support a gige connection at full throughput. I'd be weary of looking at these for a data center without knowing how fast they can serve out the disk over a single gige connection. In fact, I see a distinct lack of information about this unit functions as a "storage node". Are you buying a 1U, 3.0TB node on which you need to install an OS and fileserver? Doesn't look like it would have the horsepower to run an iSCSI driver in additional to software raid drivers and still produce any real transfer speeds.
While a rack of these sounds nice in cost/wattage terms, it appears that you would have just purchased a cluster of storage nodes. A cluster of storage nodes with no way to present the available 120TB's as any kind of coherent storage space. You might be able to run Lustre, PVFS or GFS on them, if that's even possible, but that's a level of complexity the price and performance don't warrant.
If you figure in the cost of a Storage Engineer and lack of performance, this looks less appealing at the full rack level. Doesn't mean some PHB's won't buy into the the whole "Cheap Cluster Disk!" theme though. I pity the sysadmins who get a 120TB of raw disk and 40 more nodes to admin.
The Internet has no garbage collection
Depends on what you call "mission critical", if the company relies so much on the data (Telecom and banks) that they will actually go bankrupt if the system is down for a day. Then I would go for the expensive and proven solution. A golden rule is: When you want the best, you got to pay the bill.
But for a smaller company, cost savings are significant if you dare to take a chance.
The biggest difference between SCSI and PATA configuration is throughput performance. A PATA RAID 5 is very likely to save your data in the event of a disk failure, just like a SCSI RAID will.
And since the disks are very cheap and you can afford a hot spare, why not just replace a drive once in a while?
Also remember that there have been rare and unusual cases with SCSI drives dying almost simultaneously and therefore trashing the RAID configuration, so a SCSI configuration is not a guarantee to success.
But obviously, with a 15K RPM rotation for SCSI, throughput is higher. However, SCSI disk space is more than 6 times more expensive.
So a PATA raid might be a good solution if you require a bit less performance and higher capacity.
I believe that the most important thing in any storage solution is the controller, since this is the technology that will actually need handle the any kind of failure correctly.
Just out of curiosity, has anyone out there in Slashdotland had good luck with enterprise IDE solutions? Who knows. Perhaps some success stories might change my pro-SCSI/fibre view.
Yeah, kinda. We've got a tray of PATA in our EMC Clariion. Don't ask it to perform with multi-threaded I/O, and it's certainly slower than the FC stuff, but it works okay for test and backups. Can't say we've seen a higher failure rate on the disks than we have with the FC trays. I hear that the SATA stuff is much better about handling multi-threaded I/O.
What if it is just turtles all the way down?
No it's not even close to enterprise ready! A basic dual-powersupply server with a hardware raid card and a raid5 of sata drives isn't really enterprise ready. Enterprise means no single point of failure. Redundant raid controllers, power supplies, storage networks, mirrored caches, remote administration and performance monitoring, remote snapshots or archiving. Enterprise is expensive, but for good reason.
As for your question, enterprise IDE can only be realistically used for back-up or archiving purposes where the drives are used intermittenly. Several drive makers have sata disks with fibre channel interfaces on them, termed FATA drives. IF you put a bunch of FATA drives under high load 24-hours a day, after about a week you'll start to see 1% of the drives fail EVERY DAY. I'm not joking. I had to deal with a cluster of FATA raids used for high-def video workloads, which was loosing 4-5 drives every day, out of 550 installed. We eventually junked the entire setup and installed 1300 real FC drives instead. Even those die at more than 1 per week. IDE drives work fine in your desktop because you are only loading them up 10 minutes at a time, a couple dozen times a day.
It doesn't look like Capricorn is possitioning this as an enterprise solution anyway. It looks like a workgroup NAS sort of thing, or a proxy cache of some sort. I'd file it in the "not mission-critical" folder.
They don't use RAID at all. They use RAIC (which is an acronym I just made up for a Redundant Array of Inexpensive Computers). Each individual node is a file server. Each file is distributed over a number of file servers. When a machine fails, they just swap in a new machine. It then grabs a load of files that aren't mirrored as much as they should be, and begins serving them.
I am TheRaven on Soylent News
They are very low power, reasonably easy to work on and not hot-swap. They are not going to win any speed contests, but they store data cheaply and make it accessible at reasonable rates.
The VIA based systems are PATA, so they will not be RAID5 friendly because RAID5 on master/slave is simply stupid.
They are reasonable fast at delivering the data. Having only a 100Mb connection means that it takes a really really long time to fill it. At the Internet Archive we use the nodes in JBOD and do the redundancy at the application layer.
If I were doing it at home I would probably try out ATA over Ethernet and make all of these hosts/drives targets. Mirroring is always another option.
Capricorn's unit (750GB drives): 3TB per 1U
Sun Fire X4500 (500GB drives): 24TB per 4U
Capricorn TB per 42u rack: 126TB
Sun Fire X4500 TB per 42u rack: 240TB
Capricorn watts per rack (80w/unit): 3360w
Sun Fire X4500 watts per rack (1500w/unit): 15000w
Capricorn watts per PB: 26667W
Sun Fire X4500 watts per PB: 62500W
Capricorn cost per rack: ~ $200,000
Sun Fire X4500 cost per rack: $470,995
Capricorn cost per PB: ~ $ 1,560,000
Sun Fire X4500 cost per PB: ~ $1,960,000
So yes, Capricorn's solution provides lower power usage, but also lower density (And less processing power and redundancy I'd imagine). So it's a tradeoff. Lower the power bills, but raise the rent bill and the risk.
It should be noted that for Sun's server, I'm using the 1500W rating of each of the redundant power supplies, the typical usage would actually be much less (just like how a PC with a 500w PSU might only use 300W under load). This also ignores processor power, as each Sun unit is a quad opteron. It also ignores RAID, as the Capricorn could do no more than 3 drive RAID5, while each Sun box could have a 48 drive RAIDZ or RAIDZ2, wasting a lot less for parity. And things might change if Sun put 750GB drives in their unit instead of 500GB drives. It's all about tradeoffs.
"I have an old 386 running fedora and samba on a 120GB drive with no RAID whatsoever. The machine won't fit another drive and an upgrade will involve so much hassle I've been putting it off over and over. Any reasonable upgrade would have to involve a terabyte machine because I don't want to go through the hassle of upgrading too soon after."
Yea, well, 1986 called, they want their CPU back.
Your system isn't a 386, though; old PATA IDE controllers on those things couldn't address more than 4 or 8gb (the first lip; then there were controller issues at 20, 32 or 36, and 160gb as well). Given that a real 386 won't have the PCI slot for a modern IDE controller, I call bullshit. Just spend the 400$ to get a basic system with a decent IO subsystem, spend 400$ to get the 4gb of RAM to buffer it, and then spend the 800$ to put 1Tb of disk space in (RAID5). That's a fileserver.
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.