Slashdot Mirror
SoHo NAS With Good Network Throughput?
An anonymous reader writes "I work at a small business where we need to move around large datasets regularly (move onto test machine, test, move onto NAS for storage, move back to test machine, lather-rinse-repeat). The network is mostly OS X and Linux with one Windows machine (for compatibility testing). The size of our datasets is typically in the multiple GB, so network speed is as important as storage size. I'm looking for a preferably off-the shelf solution that can handle a significant portion of a GigE; maxing out at 6MB is useless. I've been looking at SoHo NAS's that support RAID such as Drobo, NetGear (formerly Infrant), and BuffaloTech (who unfortunately doesn't even list whether they support OS X). They all claim they come with a GigE interface, but what sort of network throughput can they really sustain? Most of the numbers I can find on the websites only talk about drive throughput, not network, so I'm hoping some of you with real-world experience can shed some light here."
The network is mostly OS X and Linux with one Windows machine (for compatibility testing)
Well it looks like SMB is your best bet for compatibility. For a budget, just go with a small Linksys or Cisco device, as you can specify the hard drive and the network around it governs the speed.
FreeNAS or OpenFiler on a PC with a raid controller and GigE should work. It might even be cheaper than a NAS box.
As to OS/X support. I thought OS/X supported Windows networks out of the box. Odds are very good that if it supports Windows OS/X will work.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
You might as well build it yourself.
Go get a lowbie Core2, mobo, good amount of ram, and 4 1TB disks. Install Ubuntu on them with LVM and encryption. Run the hardening packages, install Samba, install NFS, and install Webmin.
You now have a 100% controlled NAS that you built. You can also duplicate it and use DRBD, which I can guarantee that NO SOHO hardware comes near. You also can put WINE on there and Ming on your windows machines for remote-Windows programs... The ideas are endless.
If you want decent throughput build it yourself. Seriously. I have a coworker that bought 5 different NAS devices to do a bakeoff for a small skunkworks office and they all sucked for throughput. We ended up buying a $1K NAS that still wasn't great but sure beat all the SOHO ones. Numbers were ~8MB/s max on the fastest SOHO unit vs 25MB/s on the midrange one.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
In terms of cost/benefit ratio, nothing beats a stripped down PC with a lot of drives stuffed in it or in an external esata enclosure. I run a HP NAS MV2020, and a linksys NAS200 and they both cant hold a candle to a PC in throughput. Ive heard of some commercial systems out there, but they cost a small fortune. Just my $.02
Good-bye
According to the CNET reviews I saw,located at http://reviews.cnet.com/external-hard-drives/hp-media-vault-mv2020/4505-3190_7-32104518-2.html the actual 5GB copy tests they did show it being faster than the rest, even one other system using RAID5
Sig Follows: "Suppose you were an idiot. And suppose you were a member of Congress. But I repeat myself." -- Mark Twain
Just got a drobo myself with a droboshare front end. I did some testing and can get about 15 MB/s reads with 13 MB/s writes with 4 250 gig seagate hard drives. i was hoping for more speed, but traded it for the other features of the drobo. tested on a GigE network.
I have both a dlink DNS-323 and a readynas duo (netgear), and they're dealing fairly decently on GbE.(though the netgear does quite better) Typically that's ~30MB/s with my dns-323 in RAID1, and ~40MB/s with the netgear. Still far from full GbE saturation, but seeing how cheap they are, and easy to replace/manage(as well as being compact) they're quite useful imho.
We have a ReadyNAS 1100, it's alright, but I wouldn't call it stellar. I get around 80Mb/sec to it over the network, but the management interface is IE only (as far as I can tell, since it has problems with FF and Chrome), and it has these odd delays when opening shares and browsing directories. Some of the nice features are the out-of-the-box NFS support and small, 1U size.
I worked on an older version of Buffalo's NAS. It has a good, standard design (how you would probably do it if you were rolling your own anyway) and has good throughput, but it may try to do a lot of extra work if you are uploading media to it due to some media streaming capabilities in some versions.
I have evaluated a few different products (I have a retail store) and so far I have been very happy with the DLINK DNS-323
Disclaimer: I have no affiliation with DLINK other than I stock some of their goods
DSLIP Web Design and Content Management Australia.
They have neat solutions, but their throughput is horrible. They support GigE, but the CPUs they use in their boxes are so underpowered they never achieve anything reasonably higher than 100-base-T (if that).
I'd post links, but typing "Buffalo NAS throughput" in google comes up with multiple hits of reviews complaining about throughput.
I tried every decent and legal way I could think of to resolve the issue w/the business before I rented the chicken suit
If you're going to buy a NAS we've found the QNAP ones give around 30 to 40 mbps
however
if you have os x clients if you use an os x machine then they will be able to search the drive with spotlight. any other operating system and they will have to do a normal slow search.
Addonics NAS is a $55US device with a USB on one side and RJ45 on the other. I dont know anything about it other than it was written up recently on Gizmodo
"For only $55, Addonics claims that this tiny gadget can easily turn any USB storage device into a full-fledged Network Attached Storage (NAS) server with support for both SMB and FTP access."
Infrant Ready NAS+ is very nice. I have had it on my gigaE network with OSX Ubuntu and Windows XP MCE as well as various other machines and laptops running win2k and XP professional. It is very fast and snappy with gigaE. I had been running it on a slower network and it was still able to reliably stream video content and simultaneously do large file transfers without a hiccup. With the HP procurve gigabit ethernet switch it is all just much faster. I am not a technical user, but it has been very reliable with no problems and seems very very speedy. Big thumbs up for this option.
Synology has a fair amount of good data on their website, and I've found the interface to be nice, though I'm a lite user. They're one of the relatively few companies that I've read about that has consistently good reviews. Their items are pricey, however.
Build it yourself and install Opensolaris. ZFS rocks.
I'm in the same boat as of last week. I have 2 machines, an XP box and a Linux server, both with GbE adapters from different manufacturers (Intel and Broadcom). Last week I decided to reduce a few TV programs I had captured using the firewire connection from my cable box. One was about 12 GB. Nothing like transferring a 12 GB file to clue you in to the fact that the little router you THOUGHT was doing 100 Mbps is actually only doing 10 Mbps. What a nightmare. So now I'm also looking for something that can support my GbE adapters. The router I can replace, but even if it did handle 100 Mbps traffic, even that would be too slow for these files.
I have a Terastation 2 (by Buffalo) and I am plugged into 100Mbps ethernet at work, so I can't tell you about the throughput, but I can tell you that the Terastation Mac stuff is very half-assed. I couldn't get AFP/Appletalk to work at all and while SMB is rock solid for large files, it cannot handle large amounts of small files. It chokes on directories with huge amounts of files (not sure if that's a limitation of the Finder or the Terastation's fault, though). I had a user's backup program run amok and generate millions of tiny .tmp files over the course of about a month, and I was unable to delete them from OS X, even when waiting days. I had to use Windows Explorer, which was slow but eventually worked.
The built-in webpage used for administration is pretty terrible too. It works best with IE 6 on Windows, but even with that, sometimes the columns don't line up properly. If you misclick, you could end up changing the wrong shared folder.
On the plus side, the Terastation 2 is pretty cheap. I'd give it about a B minus in terms of what I need it to do.
(-1, Raw and Uncut is the only way to read)
Great little debian server, really bad performance as a NAS. Even with Debian on there.
I like the idea of the QNAP Turbo stations - effectively a modernised NSLU2 with 256 MB of RAM and a 500MHz chip, but then I want another server rather than an actual NAS...
www.smallnetbuilder.com maintaines a NAS Chart, I find it quite complete and recent.(http://www.smallnetbuilder.com/component/option,com_nas/Itemid,190/)
Drobo v2 connected via FW800 maxes out at 50MBps reads and ~35MBps writes.
sigs are for fools and trolls. no signature is *always* appropriate. you should turn them off in your preferences.
They have the most comprehensive benchmarks and NAS's around (that I've stumbled across, at least). Also, lots of good tests showing various things like Jumbo frames, etc. Very good overall.
I frequent the site a bit, and there's a couple tricks to getting good performance out of a NAS, or LAN throughput in general.
1. Use Jumbo Frames, period.
2. Use PCI-e NIC's, onboard or PCI just can't deliver the speeds offered by GigE. You can find smiple intel PCI-e nics for under $20.
3. Drives make a big difference, obviously.
www.smallnetbuilder.com -- Good site.
Check Tom's Hardware. They reviewed the Synology DS207+ this week and have benchmarks for similar products for you to compare.
If your testing is highly automated, I can't help you as I don't have a lot of experience with high speed networking.
If your testing is reasonably manual, consider storing your data set on removable hard drives which are manually plugged into one computer, data is copied, then disconnected and moved to the other. A USB 2 interface will give you the most compatibility given the wide variety of hardware you're using, but perhaps there may even be hardware that does hot plugging E-SATA properly if you're willing to pay a premium.
Remember, for really high bandwidth physical media being shipped from one location to another is still a solution which should be considered.
These posts express my own personal views, not those of my employer
The nice thing about hardware like these is you can call up newegg, order one, play with it for 15 days, and wipe and return it if it doesn't fit your needs. I don't know exactly what your timeline is (from the question it doesn't sounds particularly time-sensitive), but unless you need the last word today, just buy the different ones you're thinking about and try them out.
Off-shelf NAS device will be not only slow but also full of various bogus bugs with which you need to wait for vendor to issue firmware update...
Just build it yourself - build a PC. You have plenty of options:
1. If you have a rack somewher buy a low end rack 2U rack server with enclosures for SATA disks and some decent RAID controller.
Or:
2. Build yourself a PC in tower enclosure. Get some Core 2 Duo mobo (cheapest), medicore ammount of RAM - SMB and NFS and AppleTalk servers with Linux operating system will eat up something like 80MB for the system and 10MB per client computer - go figure, the rest of RAM is for I/O buffers. Stuff as much as you can get SATA disks into that (like 4x 1TB). Setup it with software RAID. And you are done with it. Probably it will be much cheaper than decent NAS box (so called SoHo boxes are no worth even looking at).
Do so and you have a decent storage that is more efficent that your network.
You said about network efficency? Well - this has nothing to do with NAS box. You can have the best performing NAS box - but if your network is weak - well here goes your efficency.
So as for network buy managable switch that can cope with Linux channel bonding - with that you can bond N ethernet channels and get network transfers somewhat lower than N*interface speed.
Most of the embedded-style NAS will crap out WAY below whatever throughput you are looking for.
The trick is going to be maxing out the transfer bandwidth by identifying the bottlenecks in a Linux file server.
The most direct route I can imagine is a proper SAN and fibre channel controllers. Not cheap, but my time isn't either.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
http://www.thecus.com/ has a whole line of NAS's from SOHO to Enterprise. I believe they are linux based.
I've been VERY pleased with the reliability and speed of my Drobo. If you don't want to spend the time rolling your own, dealing with linux raid drivers and related issues, etc... Just get a Drobo.
Your best performance is likely to come by rolling your own. Off the shelf SOHO devices are built for convenience, not throughput.
Grab a PC (need not be anything top-of-the-line), a good server NIC, a decent hardware RAID card (you can usually get a good price on a Dell PERC SATA RAID on ebay), and a few SATA hard drives. Install something like FreeNAS or NexentaStor (or, if you want to go all the way, FreeBSD or Linux and Samba).
Where is the wisdom we have lost in knowledge?
Where is the knowledge we have lost in information?
Just visit small network builder and look up their NAS charts. Contains benchmark for every device they ever tested.
Building your own is still strongly recommended. If you want a quality system, buy a dirt cheap HP Proliant ML115 server and four fast >1TB SATA II drives. Use software raid 10 mode (mdadm can handle this) and it'll fly. It's also reusable, I just converted mine to run whitebox VMWare ESXi without any hassles.
Okay, unRaid is not particularly fast compared to an optimized system, but it's expandable, had redundancy, is expandable, is web managed, plays nice with windows, sets up in about 20 minutes, costs $0 for a three disc license and $69(?) for a 6 disk license.
My total unoptimized box on an utterly unoptimized Gb network (stock cards, settings, with 100 and 1000 nodes) and unmanaged switches just transferred an 8.3GB file in a hair under three minutes. From a single, cheap SATA drive to a Vista box with an old EIDE drive. Now 380Mb/s is not blazingly fast, but remember that it took almost no effort.
http://lime-technology.com/
No connection except as a happy customer with a 4TB media server that took longer to assemble the case than to get the SW running. If only my Vista Media Center install has been this easy.
Is it just my observation, or are there way too many stupid people in the world?
I have one of these with 4x750GB seagate drives. I get 35mb/sec read and about 15-20 write. I'm using OSX on AFS and SMB shares
They support all platforms, including the Apple networks, include RAID. The four drive one even has RAID 5 support.
You'll have to buy drives for the unit, but they are very nice and perform well. Try and stay away from the media sharing add-ons for it, they'll lock the unit up from time to time.
You don't want software raid and the (cheapest) MB sucks way out of data chipset also you don't have to have on board vidoe as it takes up system ram and chip set i/o even if you are useing the system for much.
you will want 2gb or more ram + dual gig-e or more in teaming + some kind of a raid card a good pci-e x4 or better one is about $250+
The type of NAS will depend on how many simultaneous transfers. It wasn't said if the large datasets were being transferred sequentially or in parallel. ATA does not handle simultaneous IO as well as SCSI/FC. If they are sequentially, a NAS with ATA disks would be sufficient and RAID3 might be more optimal. If there are many simultaneous transfers then fiber channel drives and a more expensive NAS might be needed.
If you use a single disk NAS solution and you are doing sequential reads through your files and file system, your throughput can't be greater than the read/write speed of a single disk, which is no where near GigE (1000 Gbps is about 125 MB/second ignoring network protocol overhead). So you will need RAID (multiple disks) in your NAS, and you will want to use striped RAID (RAID 0) for performance. This means that you will not have any redundancy, unless you go with the very expensive striped mirror or mirrored stripes (1+0/0+1). RAID 5 gives you redundancy, and isn't bad for read, but will not be that great for writes.
As you compare/contrast NAS device performance, be sure that you understand the disk architecture in each case and see oranges to oranges comparisons (i.e, how does each one compare with the RAID architecture that you are interested in using - NAS devices that support RAID typically offer several RAID architectures). Also be sure that the numbers that you see are based on the kind of disk activity you will be using. It doesn't do much good to get a solution that is great at random small file reads (due to heavy use of cache and read-ahead) but ends up running out of steam when faced with steady sequential reads through the entire file system where cache is drained and read-ahead can't stay ahead.
Once you get past the NAS device's disk architecture, you should consider the file sharing protocol. Supposedly (I have no authoritative testing results) CIFS/SMB (Windows file sharing) has a 10% to 15% performance penalty compared to NFS (Unix file sharing). I have no idea how Apple's native file sharing protocol (AFP) compares, but (I think) OS X can do all three, so you have some freedom to select the best one for the devices that you are using. Of course, since there are multiple implementations of each file sharing protocol and the underlying TCP stacks, there are no hard and fast conclusions that you can draw about which specific implementation is better without testing. One vendor's NFS may suck, and hence another vendors good CIFS/SMB may beat its pants off, even if the NFS protocol is theoretically faster than the CIFS/SMB protocol.
Whichever file sharing protocol you choose, its very possible it will default to operation over TCP rather than UDP. If so, you should pay attention to how you tune your file sharing protocol READ/WRITE transaction sizes (if you can), and how you tune your TCP stack (windows sizes) to get the best performance possible. If you use an implementation over UDP, you still have to pay attention to how you set your READ/WRITE buffer sizes and how your system deals with IP fragmentation if the UDP PDU size exceeds what fits in a single IP packet due to the READ/WRITE sizes you set.
Finally, make sure that your network infrastructure is capable of supporting the data transfer rates you envision. Not all gigabit switches have full wire-speed non-blocking performance on all ports simultaneously, and the ones that do are very expensive. You don't necessarily need full non-blocking backplanes based on your scenario, but make sure that whatever switch you do use has enough backplane capacity to handle your file transfers and any other simultaneous activity you will have going through the same switch.
If you check the forums on infrant's website you can get lots of real-world through-put numbers from users. I have the ReadyNAS and am quite happy with it.
If you have the money, why not go virtual? Put all the different operating systems on one box using VirtualBox, then they can all share the same hard drive.
This results in no reliance on network throughput, just disk read/write time.
Instead of migrating between boxes to get different environments, migrate between operating systems on the same box.
Disk will always be. Since disk is your slowest spot you will always be disk I/O bound. So in effect there's no real reason to worry about network throughput from the NIC. NICs are efficient enough these days to just about never get bogged down. What you would want to look at for the network side would be your physical topology -- make sure you have a nice switch with nice backplane throughput.
About disks:
Your average fibre channel drive will top out at 300 IO/s because few people sell drives that can write any faster to the spindle (cost prohibitive for several reasons). Cache helps this out greatly. SATA is slightly slower at between 240-270 IO/s depending on manufacturer and type.
Your throughput will depend totally upon what type of IO is hitting your NAS and how you have it all configured (RAID type, cache size, etc). If you have a lot of random IO, your total throughput will be low once you've saturated your cache. Reads will always be worse than writes even though prefetching helps.
If you're working with multi-gigabyte datasets, you'll want to increase the number of spindles (ie number of disks) to as high as you can go within your budget and make sure you have gobs of cache. If you decide to RAID it, which type you use will depend on how much integrity you need (we use a lot of RAID 10 with lots of spindles for many of our databases). That will speed you up significantly more than worrying about the NICs throughput. don't worry about that until you start topping a significant portion of your bandwidth -- for example, say 60MB/sec sustained over the wire.
This doesn't get fun until you start having to architect petabytes worth of disk. ;)
Hi,
I get 15 megabytes a second out of my Bufallo Linkstation live 500 Gb. The bottlneck is (I believe) the relitavely slow ARM chip in the Linkstation. For comparison, I get 6 megabytes/s when talking to the linkstation from a fast Ethernet (as opposed to gigabit) device.
That measurement is transferring large files. It is a bit slower for many small files.
Jim.
The problem with off the shelf NAS products is two fold: the linux installs aren't disk optimized and the disks are usually the slowest (eg - cheapest, eg - highest profit margin) the company can find.
Your best bet, hands down, is to build your own. But, at best, your "bursty" speed is going to max out around 300 - 4000 Mbps and long term transfer will approach, at best, around 40 Mbps if you're lucky. If you setup a mirrored and striped raid array, you might be able to come close to doubling that. If you get some really expensive 10k RPM disks and a high end RAID controller, you can probably double that.
Keep in mind that's for single transfers. Once you add in multiple people trying to do IO on a single controller, your performance will take a hit.
With that said, I have a first gen Buffalo LinkStation. I love it, it does exactly what I need. But, as everyone else has said, it is DOG SLOW. My brother used it without problem from OS X 10.3 - 10.5.
We have ours built out to 2GB with RAID mirroring.
Frankly the throughput and directory browsing speeds are disappointing. We are using 7200RPM 32MB cache drives. Large file copies or writes of large amounts of small files seem to take abnormally long, and our network is not congested in general.
Just an example - http://www.mini-itx.com/2008/12/08/via-launch-artigo-a2000-barebones-storage-server
There are lots of 'kits' you can buy so you don't really have to do much.
OpenFiler and FreeNAS work really well. Most SoHo systems don't have great throughput, and are substantially more expensive. Even the DroboApps can't compete with just installing something yourself if you want to download torrents (like satellite imagery) to it.
The Drobo is not fast.
I have built little PC do do my file serving.
I dont know why they added gigabit port when the dinky CPU was barely capable of couple MB/s of IO.
It was 250GB ATA drive and 266MHz PPC with 64MB of memory.
My advice: build cheap small PC with Linux and software RAID. (I like software RAID better because you are not tied to particular RAID controller hardware so you can move it to newer system).
It is going to be way faster than any embedded crap.
Never underestimate the bandwidth of a guy carrying a bundle of removable hard drives around the office.
Or a station wagon loaded with hard drives.
Nothing can beat them.
How many escape pods are there? "NONE,SIR!" You counted them? "TWICE, SIR!"
I've got an Thecus N2100 and the performance as a NAS isn't great. The CPU isn't powerful enough to take advantage of the gigE interface. For what you want, I'd get something more powerful which probably means an x86 box. For anyone who just wants a home server that doesn't consume too much electricity so can be left on all the time, a small ARM based box is great. I'm running Debian on it and it's really useful.
Actually, you dont want any RAID card, because it limits your upgrade and recovery options. Any modern CPU is not going to have any problems doing memcopy and XORing required for RAID.
You do want as much memory as you can afford, especially that memory is cheap now.
My little home server has 8GB of memory, it can sink huge write transfers very quickly. It uses 3 laptop SATA HDDs in RAID5 so it can take it's sweet time to write the data to HDD later because it effectively has 8GB disk cache.
What if it's for a small, say =6 person office? 4 (5400RPM) SATA drives are perfectly fine.. if they're document and spreadsheet workers. The situation is completely different if these were video editors or CAD/CAM software types. Then you need 10GBps server, hardware raid10, say 15 drives, quad-core, and max ram.
Pulling over 70 megabytes a second using a 90 AUD motherboard, with a 80 AUD amd cpu and using silicon image sata cards with 1tb WD green caviar hard drives, and running windows 2003 standard. Now you dont have to run 2003, Solaris would be your next best bet for the beauty that is ZFS, or just run something along the lines of freenas. If the os doesnt like the onboard nic (atheros L1 in my case) you can get a PCI-E intel gigabit desktop card for 70 AUD, which should run just beautifully. Like everone said, if you want speed, dont even bother with the small nas products. Its just not going to happen. In your application you need a real proper server, and a non embedded OS.
I bought a Buffalo NAS about three years ago; I bought it because of the 1000base-T interface and low cost. I persevered with it for about three months, and then demanded and got a full refund from the retailer.
get a small pc case such as one of the many small cube cases that come as a barebones. Put a dual core chip and 2GB ram. The you can install something like openfiler which will give you a nice web interface and the ability to do nfs,cifs,ftp,and iscsi. Alternatively, install solaris or opensolaris and use ZFS and have the ability to compress the files at the filesystem level and also do a raidz with 3 drives for reliability and speed.
either way you can bond two ethernet interfaces together for 2Gbit which should get you 80-100MB/s realworld bandwidth.
http://blog.olbrecht.net/2008/09/21/netgear-readynas-pro/
I have an Infrant ReadyNAS+ and it is not fast. It has a TON of features (most of which I don't use) but transfer speeds are pegged at approx 7% to 8% network utilization through a gigE switch even with jumbo frames on and an upgraded stick of ram for the NAS cache. I get the same transfer rates with 3 different computers of various types including an older laptop and a very fast gaming machine, and my transfer rates are fairly close to what others report, which tells me the bottleneck is the NAS device. There may have been some improvements in the device since infrant sold the product line, but you'll need to check their support forums to see what people are reporting with the newer ones.
If you don't need many of the easy to use features of most of the low-end NAS devices, you are probably better off rolling your own. Even using the cheap embedded raid chips on consumer mobos and what you would probably consider a bottom-end cpu (like a single core celeron) is going to get you faster transfer rates than many of the NAS devices on the market. There are a few ready to go home router linux distributions that ought to be fairly secure and feature-rich, and they will probably grossly outperform a consumer level NAS box. The only tradeoff will probably be power
consumption, however if you pick decent components, allow the cpu to throttle down, and let the drives halt when not in use, you can minimize the difference in power usage to the point where it might cost an extra $50ish/year over a purpose-built NAS device.
Try this unit.
http://www.acnc.com/02_01_jetstor_sata_405u.html
Sounds like you want and maybe need a turn key solution that is cheap. But as they say, Fast, Cheap, Reliable... Buffalo LinkStation Quad could be an appealing solution. It gives you Time Machine for your macs, but is probably a dog with datasets larger than 1GB. At a 1GB file size, smallnetbuilder.com shows write speeds to be 11.4MB/sec. If and when you do ultimately decide that soho solutions can't give you the speed you need, be sure to load the pc with as much RAM as it can take. Ample RAM to cache the file system can either maintain or reap huge throughput gains.
I have a Linksys NSLU2 (running Debian Lenny) and it maxes out at about 4 MB/sec, the dinky amount of RAM means almost no FS or other buffering is possible, and the limp CPU (266 MHz) just can't push the IO fast enough. /share filesystem via Samba.
A barebones PC is probably $200 + drives, slap OpenFiler or a real distro on it, and share out 1 big
I want to delete my account but Slashdot doesn't allow it.
QNAP 409-T.
Never let me down. Highly recommended.
See, the problem with responses like this is that they ignore the request of the original poster, and, while being valid instructions for a home-built, it is only a good solution if the time of the OP has zero value. Your instructions involve eight steps: Order (multiple) parts, wait for delivery, assemble, learn how and then install OS, learn now and install three other packages. The OP is looking for three steps: Order one thing, wait for delivery, plug in and use.
Your post has value to the DIY crowd, certainly. But for someone looking for a product recommendation, it totally missed the boat.
For example:
Best home network NAS?
http://ask.slashdot.org/article.pl?sid=07/11/21/141244&from=rss
What NAS to buy?
http://ask.slashdot.org/article.pl?sid=08/06/30/1411229
Building a Fully Encrypted NAS On OpenBSD
http://hardware.slashdot.org/article.pl?sid=07/07/16/002203
Does ZFS Obsolete Expensive NAS/SANs?
http://ask.slashdot.org/article.pl?sid=07/05/30/0135218
What the hell? Is this the new quarterly NAS discussion?
Just disrupt the deflector shield with a tachyon burst.
i have written the drivers for several
nas products. i have benchmarked several
nics (pci and pcie) at 125MB/s (1gbit).
the key is jumbo frames. even 4k jumbos
will do.
WD ShareSpace(TM) Network Storage System - 4 TB, Gigabit Ethernet http://www.wdc.com/en/products/Products.asp?DriveID=501 I found it for a little over $800. I went back and forth with several options the Drobo and MS Home Server included. I came to conclusion that, Home Server does not provide the level of fault-tolerance I wanted in a solution so that was out. The Drobo is cool and all, but I don't like the extra $200 for the networking. Since I was going to go buy WD drives anyway, why not get the whole thing from them ;)
You should check out www.smallnetbuilder.com. Lots of great reviews of SoHo type NASes. I've been happy with my Thecus N4100PRO, built around the AMD Geode processor.
How many gigabytes are "multiple" gigabytes? Seriously, moving around five GB is much easier than 50 GB and enormously easier than 500 GB.
Another thing to consider: how many consumers are there? A "consumer" is any process that requests the data. If this post is a disguised version of "how do I serve all my DVD rips to all the computers in my house" then you probably won't ever have too many consumers to worry about. On the other hand, I work for an algorithmic trading company; we store enormous data sets (real-time market data) that range anywhere from a few hundred MB to upwards of 20 GB per day. The problem is that the traders are constantly doing analysis, so they may kick off hundreds of programs that each read several files at a time (in parallel via threads).
From what I've gathered, when such a high volume of data is requested from a network store, the problem isn't the network, it's the disks themselves. I.e., with a single sequential transfer, it's quite easy to max out your network connection: disk I/O will almost always be faster. But with multiple concurrent reads, the disks can't keep up. And note that this problem is compounded when using something like RAID5 or RAID6, because not only does your data have to be read, but the parity info as well.
So the object is to actually get many smaller disks, as opposed to fewer huge disks. The idea is to get the highest number of spindles as possible.
If, however, your needs are more modest (e.g. serving DVD rips to your household), then it's pretty easy (and IMO fun) to build your own NAS. Just get:
You might also want to purse the Ars Technica Forums. I've seen a number of informative NAS-related threads there.
One more note: lots of people jump immediately to the high performance, and high cost RAID controllers. I personally prefer Linux software RAID. I've had no problems with the software itself; my only problem is getting enough SATA ports. It's hard to find a non-server grade (i.e. cheap commodity) motherboard with more than six or eight SATA ports. It's even harder to find non-PCI SATA add-on cards. You don't want SATA on your PCI bus; maybe one disk is fine, but that bus is simply too slow for multiple modern SATA drives. It's not too hard to find two port PCI express SATA cards; but if you want to run a lot of disks, two ports/card isn't useful. I've only seen a couple of four-port non-RAID PCIe SATA cards. There's one eight port gem, but it requires PCI-X, which, again, is hard to find on non-server grade boards.
These guys make fast storage units. Delta-V was announced last week, and they showed it off at SC08 a few weeks ago. I saw their 3U unit in some booth. It was attached to a Windows machine as an iSCSI target, and the guy running the demo was the company founder, and had IOmeter running on it. He got 500 MB/s across the iSCSI link. He said it hooks to anything, windows, linux, Mac, and unix.
They don't do too badly for xfer speed and are quite reliable. They seem to use less power and aren't noisy like other NAS systems (especially the RYO).
Linux is their OS and if you need to add some functionality, you can get in and do it, but it works well out of the box.
RAID 5 or 6 with the 508
I've done the Windows SMB and it sucks for maintenance and you're back at RYO - patch and crotch rub. I've built many a linux box for this and, though they work, I have better things to do with my time. I really appreciate buying a few HD and sticking them into a box and having a system that can store data, xfer data, backup themselves, etc. in a matter of minutes.
Oh yes, compatible... via CIFS with most systems. NFS with Mac and Linux if you are so inclined. rsync for backup.
If you're that concerned about speed, you might want SSD drives instead, seeing as the network transfer speed will appear slower due to conventional hard disk write speeds.
Consider Solaris + ZFS too. Especially now that Solaris 10 u6(?) now can install to ZFS root partition (HINT: Use Text installer - options 3 or 4 if memory serves).
Solaris is free as in beer, even if it isn't open source. Plus you get the benefit of some of the proprietary drives if you have older hardware. Plus, Solaris proper won't leave you in a lurch when things change in OpenSolaris and you can't do updates or run some programs. [Admittedly this problem seems to be mostly resolved, but for mostly production environment I'd suggest Solaris over OpenSolaris unless you need some particular bleeding edge feature not yet migrated from OpenSolaris into Solaris.]
I did just this a while back when looking for a storage solution for backups. The SOHO options did not have the bandwidth, even with gigabit nic ports. In the end, moving to PC hardware with SATA drives worked much better.
Due to SATA controller issues with port multipliers when I set this up a year or two ago I ended up having to switch to Linux with md. Regardless, the performance difference was dramatic and the PC based system actually worked quite well.
. 62,400 repetitions make one truth -- Brave New World, Aldous Huxley
It canhaz the standard RJ45 jack. Into the hub it plugs.
Bought this a year ago from PCConnection (post raccoons). Plug into the network and store. Works with Mac Win Lin. $300 on sale.
Just do it.
Oh, and I run at work a network with Leopard, Tiger, DOS3.3 Q&ADOS, Win98, WinXP, MintLinux, SAMBA, and the boss's stupid kids scratch the last one. Don't recommend Clarknet, stick with rolling a MintLinux.
They are a little on the high end, cost wise for consumer boxes but they are very reliable, the firmware actually works WELL, they support NTFS and their network interfaces function up to spec. And they support Mac.
They make units from 1 bay SATA up to 4 bay 1U hot swappable dual 1Gb dual power supply rackmounts.
www.synology.com
I'm going to suggest that you skip the NAS and just get a large-capacity eSata or firewire drive. Plug it into your current test machine, do your thing, unplug it and move along to the next machine. This approach sidesteps any limitations of your LAN, host machine, RAID cards, or NICs.
You might want to investigate running a data network w/ jumbo frames (change ethernet MTU size from 1500 to 9000).
1500 MTU was good when networks were 10M, but with Gig they are really just too small. The CPU gets pummelled with packet assembly/disassembly; setting the MTU to the jumbo (9000) size reduces the required CPU overhead greatly. However, all devices on the network segment must be set to jumbo (9000) byte MTU.
On a larger corporate network we'd setup a private, non-routed VLAN for NAS; all servers involved would have a second NIC attached to this VLAN at 9000 MTU. You could still manage this in a SOHO with multiple NICs on a separate LAN/switch (be sure the switch supports jumbo frames) just for storage. These days a good percentage of motherboards come with dual NICs anyway.
Jumbo frames can significantly increase ethernet throughput, particularly with large streaming I/O, which sounds like what you have.
Another option, particularly on your NAS/server, would be to setup an ethernet trunk (also called 'etherchannel' or 'nic teaming') - logically bonding two network interfaces to send/receive data as one IP address. If you have multiple clients uploading/downloading data to the server at the same time this is essential.
All 'modern' operating systems support jumbo frames and trunking; many of the cheapo SOHO ethernet switches do not, so you might have to shop around. I think I've seen models which support both jumbo frames and trunking for under $300.
You could also investigate getting TOE (TCP Offload Engine) cards which have 'packet co-processors' to perform the packet assembly/disassembly of the small 1500 byte packets - but these are usually pricey and jumbo frames often work just as well.
I researched Drobo, and found it to waste too much space, not to mention that the network add-on is expensive...and then reduced to usb transfer rates... I have a Buffalo on my desk, and it is ok, but if you need speed, that isn't a way to go. I found that I could buy a real RAID card and roll my own RAID NAS with little trouble and get great speeds out of it. Later I purchased a micro board, it has 6 sata ports, and I have a Software Raid running on Ubuntu server, with all the necessary bits for SAMBA, XFS, CIFS, etc. I have 6 disks all in raid and a Gigabit ethernet...faster than anything I have purchased previously...does a good job too...
--E--
I have several thecus NASes (http://www.thecus.com/) the N5200, the N5200Pro and I'm looking into a N7700 or a N8800. They all have Gigabit ethernet connections. They are more expensive than purpose built boxes - but they are "set it and forget it" appliances - which is where there value comes from.
I can sustain approx 25MB/s write and 40 MB/s on my N5200. NOTE: That's bytes not bits. In otherwords 200Mbit read throughput and 320Mbit write throughput. This is far better than the other domestic of SOHO NASes - which is why thecus tends to win awards http://www.thecus.com/news_contentx.php?nid=827 , http://www.tomshardware.com/reviews/holiday-gift-guide,2065-12.html
If you contrast this with one of my windows RAID arrays (100+MB/s read 100/MB write - array to array or just streaming) it looks rather poor.
If you want something that you can just forget about and will give reasonable performance then TheCus is what you want ... if you want better performance than a custom built server is what you want (Windows or Linux).
I had their TeraStation a few years ago. I bought it from Newegg, whose site (at the time) said that the TeraStation came with a 2 year warranty. 1 year after I bought it, it started acting funny. I called up BuffaloTech, only to be informed (after a near 2 hour wait on hold), that the TeraStation warranty is in fact only ONE YEAR and that they DO NOT repair TeraStations out of warranty. Yes, you heard that right... they won't even repair it and bill you. The jackass had the audacity to tell me that I should buy another one to get my data off. I told him to fuck off. I plugged the hard drives into my linux box and got the data off myself. Assholes.
P.S.--Newegg saved the day. At first, they told me to go fly a kite. After asking them very nicely to ask their manager, they said OK and issued me a RMA number. I got my money back minus a small restocking fee (which is reasonable considering I didn't have the original box anymore). Newegg FTW.
That's what I meant. Gawd, it's weird. Just roll the standard Samba.
You say test, do they change the data set?
I do not know if you have automated anything yet because from your description it sounds like it is still a very manual system. The Linux box I am assuming is a test box. Also it sounds like you are looking for a centralized storage location. It also sounds like this is using a mixed wired and wireless system. In my opinion, there has been very little thought put into how your network and future expansion.
For speed which it sounds like is your immediate concern, Current OS-X boxes have Gigabyte Ethernet capability. Most likely you have a single Gateway for the entire office. If this is the case, an upgrade is highly recommended. I would also look into your Windows and Linux box network cards. Default hardware by Dell and other major distributors tends to be cheap equipment. Cheaper gateways will reduce the speed of the network to the slowest entity. Which is your wireless device. Having separate Gateway and wireless devices may be something else to look into. A more advanced Gateway/Router will be able to allow you to easily change speed.
As far as a networked hard drive and Lan gateway, please don't do that. You are playing with fire. It is good for consumers for light duty use, but companies that need throughput and speed, it will die within the year. Many people that posted to create your own Linux storage box are correct. A single box is easily expanded into a larger server. You can buy an off the shelf system. I would also suggest that you do not use a test machine. Also you can create a Raid disk array to make swapping drives that die as painless as possible.
I did this professionally for many years. To conclude, upgrade your gateway and add a new computer for data storage.
Ah, wrong.
This guy is talking about SOHO type NAS boxes, their cpu and network throughput is their bottleneck.
If he was talking about 'real' NAS, then that is very different (although it is still trivial to get a NAS that can saturate GBit for many workloads).
Our 16/32 drive Raid6 SATA raid arrays easily sustain 400MB/sec locally for moderately non-random workloads - there are workloads for which this of course does not apply, but since he is apparently moving around GByte lumps, it would not be his case.
SOHO NAS devices normally run out of grunt at around 6MB/secish, even for long linear reads, some do better at up to 25.
I am thinking your workload is TPC type database loads, dont assume everyones is (we have a mix of video files and software development, very different..). TPC type disk loads are a corner case.
We also love ATAOE but that is DEFINITELY not what he is looking for.
Okay, unRaid is not particularly fast compared to an optimized system, but it's expandable, had redundancy, is expandable
Good thing your post has redundancy as well, otherwise we wouldn't know it was expandable.
Enigma
im not helping you and your warez
thats what killed the economy
I also work for a smaller company and I have 2 Drobos in service for backups. They are working fine but you are limited to USB 2.0 speed. This is true when you use the associated DroboShare network appliance as the connection from the Droboshare to the Drobo is USB. BackupExec does not like the DroboShare so I have them directly connected to a server. This arrangement keeps us from running all that data across the network as well. The only downside to the Drobo (this is unsubstantiated as I have not tested it) is they are supposed to be susceptable to data loss resulting from loss of power. If you are operating a data center with adequate UPS and a backup generator this would not apply. 4TB of redundant SAN for less than 1K is hard to beat, we'll see how long they last.
Of all the things I've lost, I miss my mind the most. Or do I?
http://www.smallnetbuilder.com/component/option,com_nas/Itemid,190/
"This test measures the average filesystem performance of NASes at the Ethernet connection speed"
I have the Netgear ReadyNAS pro. It has a intel core duo processor running Linux on it. A fairly nice interface for managing the device from the web browser. Plus if you want (and I did), you can install a module provided by Netgear to get root access to the box. You risk voiding your warranty depending on what you do, but you can then tweak the box to do more, etc.
Performance has been good. I've got a 5 port Netgear switch (1Gb). From my MacBook pro 2.4ghz via AFP, I can get a sustained write of 41 megabyte / sec. I don't know if it can go faster or not writing, as that might be my laptops read max. Reading from it, I was only able to get 39 megabyte / sec...but that again might be the max write speed of my drive...?
My ReadyNAS pro has three 1TB drives in it. There was also a rebate on it for a $400 video camera making the incentive to get it a little better.
I've been very happy with the speeds of it.
--Ben
http://sfbay.craigslist.org/pen/sys/955430088.html
Dual-Core Intel box with 12 SATA drives for 3TB total storage. 3Ware 12-port raid card.
$1000
Since it's for a business, it probably makes sense. For my home, too loud, big & power hungry.
The QNAP TS-509 looks interesting for my home network, but it's expensive and I'm unsure about the software....
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
The computer your currently using, then you can get a better one!
Solaris and ZFS do wonders making PC's into NAS servers.
Sorry folks: if you want grunty corporate grade equipment with high performance then don't expect to buy it at a retail shop.
Build your own or spend up on professional grade equipment.
Engineering is the art of compromise.
The Buffalo Terastation uses a software RAID, which slows it considerably, with the side benefit of being nearly impossible to recover if it crashes.
It does support SMB, NFS, and AFS out of the box though.
These boxes are cheap crap, and have a very limited useful lifespan. Our company lost a good deal of information when ours crapped out after 366 days. (Yes, we had backups, No they weren't perfect. They happened to be with me halfway around the globe at the time...)
Really seems like the product offerings in this space are limited usability, poor reliability, imperfect implementations, and grossly overpriced. Doing it over again, I would go for a build-it-yourself box hands down.
Thecus 5200b Pro - currently the best soho NAS i can find. I use it with Raid 5 and it never drops below 30MB/s throughput. Highly recommended!
and it's also expandable
Take an older Mac, spend some $$ on a Drobo and OSX Server and you have a easily managed, robust server with redundant storage. If you have the FW800 Drobo, your throughput will be more than typical gigabit can handle. I have this setup on an old G4 PowerBook with OSX 10.5 server. Bonus: With OS X Server, all attached drives can be enabled for Time Machine backups. Way better and faster than a Time Capsule. (Too bad it is 5x the price...)
What you're expecting is really beyond the capability of common SOHO NAS equipment. These devices lack the RAM and CPU to approach the capacity of GB Ethernet.
Unless you're willing to roll your own, you should consider a better class of gear and spend your time arguing for the funds to pay for it (a NetApp S550, perhaps.) If you are willing to roll your own, you can get there for $1-2k using all new hardware.
Beware reusing older hardware; many GB NICs can't approach GBE saturation, either due to PCI bus contention or low end, low cost implementation. Yes, in some cases older hardware can get there, but this will require careful configuration and tuning.
You want a PCI-E bus, a decent 'server' class NIC, recent SATA disks, a modern CPU (practically any C2D is sufficient) and enough RAM (2-4 GB). Personally I stick to Intel based MB chipsets and limit myself to the SATA ports provided by Intel (as opposed to the third party provided by jaton, silcon image, et al.) Linux, md raid 10. Will saturate a GBE port all day long, provided your switch can handle it...
You're serving desktops so jumbo frames are probably impractical (because some legacy hardware on that LAN will not tolerate it.) If your managed (?) switch can provide VLANs you can multihome your critical workstations and use jumbo frames. This will get you more performance with less CPU load for 'free'.
Lurking at the bottom of the gravity well, getting old
I'm going to go out on a limb and say that the original poster might not be thrilled with the responses... which are the same sort of responses that usually get posted in response to a question like this.
I'd like to ask the exact same question again, while adding the additional condition that I ABSOLUTELY DO NOT UNDER ANY CIRCUMSTANCES WANT TO BUILD AND MAINTAIN YET ANOTHER SERVER BOX TO DO THIS!!!!!
Yes, I know it's probably cheaper, faster, and (most importantly) geekier, but at the moment, I'm already up to my big fat overworked ass in machines that need to be managed, both at work and at home, and what *I* need at home is a little box full of hard drives that I can plug in and share files across the Linux, Macos, and Windows platforms with WITH AS CLOSE TO ZERO ADMINISTRATIVE OVERHEAD AS IS FEASIBLE.
I have a little DLink DNS-323, but it kind of sucks; it is very slow, cannot effectively use my gigabit home network, and I do not trust it.
I've heard good things about the Infrant... er, now Netgear, I guess? I've heard a few good things about Drobo. I've never used the Buffalo stuff, but I bought a couple of their external hard drives a couple years back, and they were crap, so I'm disinclined to buy a higher-end item with them.
I have tried FreeNAS, and for what it is, it's actually fairly nice, and for those inclined, I'd recommend investigating it, particularly if they've finished the administrative interface for groups and stuff, which they hadn't when I last tried it. But, to reiterate, that sort of solution is NOT WHAT I AM LOOKING FOR!
So, can anyone recommend a little toaster-size box that sits quietly and shares files, and doesn't require me to build the hardware and then install software? Hmm?
We've installed a couple of the Infrant/Netgear ReadyNAS boxes at work for various people, and haven't had many complaints. So I'm guessing that's what I'll go with, someday. They're just a little pricey; and I don't know what the performace is like.
Since you are probably not creating umpteen gigs of data every day, would it be possible to subdivide the data into smaller managable files and use version control to sync things up?
CVS over NFS is pretty fast. Subversion might be better for binary files.
While it is true that the outside of the disk is spinning faster than the inner portion, in a modern HDD there are also several times more sectors in those outer rings. So while strictly speaking the read times might be faster, the seek times are not, and may even be slower. The sectors might even be interleaved, making any such comparison almost meaningless.
However, as you say, benchmarking is the only way to really tell. Highly recommended.
Maybe - but software raid has some advantages. 1) It's not tied to a particular piece of hardware to function. Meaning in the event of a catastrophic failure (motherboard dies, or raid controller card) Anything I have will work. 2) It is usable while rebuilding the raid, which the raid controller I have most certainly is not (And takes a great deal more time to build)
No, I am not an English major. My posts are subject to typos and incorrect grammar. Do not expect perfection.
And I don't mean the shipping company.
This is something I have been researching recently to store my photography, and I found that Toms Hardware has some good reviews and benchmarks using Intel's NAS performance toolkit. This review http://www.tomshardware.com/reviews/DS207-synology-nas,2081-5.html shows a comparison of the Synology DS-207 to similar lower priced units. From what I've seen, most of the inexpensive NAS's don't have very good throughput either due to low power cpu's, or slower bus speeds. I would like to have a nice appliance, but for the best price/performance it looks like I'll be building a box with FreeNAS or OpenFiler.
... to say that software RAID is almost invariably a poor solution. It is woefully slow compared to even a slow hardware RAID implementation.
Spend a few bucks and get the right hardware. It is not expensive these days.
This may have been true years ago, but it's not anymore. Modern CPUs can handle parity computations without a problem. As long as your controllers can support the throughput needed, there is no need for hardware RAID. After all, we have ZFS.
Storage is undergoing a massive paradigm shift and folks like EMC are being caught with their pants down. Their spindle cost and price per GB is just too high.
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
if it says gigabit ethernet, for me that usually means anywhere between 200-800Mbps
And you would be one of those 'suckers' born every minute.
I tried a 1Ge that allowed an internal 2-disk RAID-0, setup. The best it would do was about 12MB/s read, 7-8MB/s write.
That's about the performance I'd expect on a 100Mb ethernet -- they just added Gb-E, as a marketing gimmick and because with current volumes -- GbE are probably getting chearper than 100bE.
Do not assume 1Gb ethernet is >= 100Mb ethernet for throughput unless you have numbers to back it up.
It's a very sad state of NAS units -- it's been only recently that they started offering 1Gb, but I have yet to see any that really support those speeds.
On my contemporary $100 Intel desktop motherboard, with $170 CPU and three $90 sata drives, I get 165 MB/s via linux software raid5, for local access to large data files (such as I think the OP is mentioning for moving around test data). This is a 1.5 TB system costing me less than $600 USD, including the RAM and cheap case plus PSU. I use it for my DVR and it is absurdly over-provisioned.
People need to understand that small file access is significantly affected by round-trip latency when using naive programs that do sequential access. The LAN-based access protocols will not pipeline file access and keep the LAN saturated if your program isn't issuing pipelined access to a stream of these small files...
And those little SOHO NAS boxes are a joke, they cannot saturate the 100baseT speed, and are essentially false advertising when they add gigabit link speed and pretend it is better than the older model.
The days of network being faster than disk are over.
Gig-E speed is about 30MB/s in the real world. This is with a crossover cable, machine to machine. I've tested and verified this over a number of platforms, including expensive server systems.
Cheap terabyte single disks these days can do 80MB/s.
The only reason to go with raid is for redundancy, or better handling of simultaneous connections.
Read this for a more in depth analysis: http://www.codinghorror.com/blog/archives/000339.html
Photos.
Wrong. Go do your homework.
I can't speak for your config. Mine is obviously different, cheap parts on my side. The only name brand anything are the Seagate disks.
Sustained single disk speeds are around 12Mbps. Using RAID, you can increase that a little depending on the width of your array and the amount of disk cache RAM you have available.
My SMB server is a Ubuntu 6.xx LTS desktop. It runs all sorts of other processes, but also has an external RAID5 array (4x320GB) connected via infiniband. The network is 1GB switched.
I use Linux software RAID, not hardware RAID due to driver issues with my cheap Promise RAID card and the added flexibility it provides. Here's an excellent article for why using software RAID is a good idea http://linux.yyz.us/why-software-raid.html
The server has only 2GB of RAM. Most of the RAM goes towards disk buffering - no tuning needed. I backup via rsync over ssh about 10GB of data every morning. 50Mbps throughput until the buffer gets filled, then it is limited by the disk performance on the source systems, usually around 12Mbps. Using network testing tools, I was able to see 550+Mbps throughput for pure network tests between Ubuntu clients and servers, so the network isn't an issue. WinXP client throughput was only slightly less over samba. I run WinXP in a Vbox VM under Vista, so that isn't really fair.
As for Samba performance... I have an OpenSolaris server which I use with a 2.5 TB disk stack in a Raid-5z configuration as a NAS for my SOHO setup. I set it up to use as Mac OS/X time-machine server for my Mac machines. With Samba, the initial backup took about 5 hours and consumed 1 of my two CPUs. I switched over to the native CIFS server in OpenSolaris and tried it from scratch again. This time it took under 2 hours and consumed less than 1% total CPU. I had similar experiences when I backed up my Windows machines. Sun still has the fastest NFS stack if that's the way you want to go as well. But, I typically use dirvish (rsync based) to back up my Unix/Linux machines nightly to the server. In 3 years of heavy use, I've never had a single hiccup.
Yes and the MAKE webzine is pretty cool too!
What makes me happy is a CentOS 5.2 server running a Promise ST EX8650 RAID-1 and RAID-5 solution (6 HDs on the SATA300 capable controller - oh, it does SAS too and is capable of addressing >200 HDs). You have to use ES.2 Seagate or similar drives otherwise you don't have their vendor support, but they run FAST and with my GiGe (Gigabit Ethernet) it runs at about 26 MBps. Currently my box is running Samba, NFS, and 3 VMWare systems because it has a nice AMD AM2 Dual Proc and a decent amount of RAM.
This allows you to run server apps in the virtual systems without worrying about affecting your priceless data. Just turned up my Azureus virtual system recently and am able to turn off the main system in my room at night (save a little electricity now ^_^). With my setup, I actually added 2 10/100 cards and have them set to be bridged to the actual virtual machines (direct IP to the network).
Specs
AMD Athlon(tm) 64 X2 2.4 Ghz
EPoX AM2+ Mobo
1.5 GB DDR2 RAM
1.1 TB Logical Drive 1
1.4 TB Logical Drive 2
2 GB Ethernet
2 10/100 Ethernet
Cost me around $800 (HDs and RAID card, the rest I already pretty well had)
Jonny5 'ko derf'
I'm gonna call bullshit on this right now. General purpose processors have been dropping in price and going up in speed and core count much faster than hardware raid solutions. Take for example your average 12 port areca or 3ware or lsi raid card which will cost you 700 bucks, now instead of spending 700 on that card buy three 4 port lsi sata cards and spend the 300 bucks you save on a faster proc or one with a higher core density (4 vs 2 for example). You'll get not only faster or at least comparable performance, but you'll also get performance that scales up as you upgrade the rest of the system instead of being stuck with the perf that you get from your raid card. Another bonus is greater compatibility with different OS's as raid card drivers are traditionally notorious for having poor support in the long term and also have less tunability of the underlying logic. Finally you get better data portability, with a HW raid card you are stuck with the type of card you have used to create the array, with zfs or linux raid you are only limited to the OS you used; which sounds silly at first, but 3-4 years down the line when you are trying to find someone who stocks a defunct HW raid card to replace your burned out card you'll quickly appreciate. This is the exact same paradigm that makes things like SSL offload accelerators a poor value for a system which will be in use for more than a year or two, the more general use hardware will always outpace the performance of the purpose built stuff and by the nature of it being general use it will be far more flexible in the long run.
-*The above statement is printed entirely on recycled electrons*-
Avoid Netgear at all costs.... I've got two, once GigE, and one 100/E. They suck, not because of the throughput, but because of the super crappy Z-san service that you have to use to access them across the network.
I've also got a couple of Western Digital My Book World II's, both GigE, and they work just peachy. I just finished tranferring 1Tb of data across my network, and it took about 36 hours. Bunches of 350 and 700 Mb files. They each take about 3 mins to transfer across the network.
I'm not crazy,I'm actively irresponsible.
I have been researching NAS appliances for some time and the top performers that I have found are made by Qnap http://www.qnap.com/product-index.asp
Several sites have benchmarked some of their high end arrays around 60-70MB/s
http://www.sun.com/storage/disk_systems/unified_storage/
I have the buffalo you speak of and I only get 13MB up and 13.5MB down over gigE with 3 year old desktop with xp.
The Galactic Constant for data transfer on a cheap X86 system seems to be 32MB/s. I've tested lots of cheap machines and they all turn out to max out at that speed, so good luck with your quest.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
EMC is actually doing quite well, and Netapp is kicking ass as well. The only shops that actually look at cost/GB as a measuring stick are small shops, or shops with very specific needs.
Large corporations, government and high tech companies are usually more concerned with management costs, retention, migration and so forth.
Not to mention getting into things like actual storage utilization - often thin provisioning and deduplication (see: Netapp) cuts your utilization so far down that you can purchase systems two or three models below what you were looking at before hand.
I cannot imagine managing multiple sites with multiple TB of data inside of just a Linux or Solaris box. Talk about a nightmare.
http://www.theregister.co.uk/2008/12/08/idc_q308_disk_storage_numbers/
"Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe." --Albert Einstein
I looked at the pricing and could not stomach it. A full blown Athlon 64 dual core box with 2GB of RAM costs less than a NAS box with no disks. And it can do a heck of a lot more. So that's what I ended up doing - bought an "energy efficient" version of Athlon 64, a mobo with integrated graphics, and stuffed it with SATA hard drives. RAID5, web server, file server, torrent, I even run scientific computations on it every now and then. I was running a VIA Epia box before it, but that one needed a SATA card to run RAID, since it only had two SATA ports. Best of all, I know that if the box goes titsup, I'll just replace the mobo and it will be able to run my RAID array again. The same can't be guaranteed with NAS boxes, since they use "proprietary" algorithms.
I have Thecus N5200BR with 5 x 1TB WD Greenpower drives in a RAID 5 array. It uses a Linux based software raid but has a nice, relatively beefy CPU to keep speeds up. Supports Gigabit ethernet with jumbo frames. I believe you can couple the LAN and WAN ports to double your network bandwidth. You can also open it up and upgrade the RAM if you're adventurous enough. It supports the major file sharing services and I've had no problem with it on my home network (Vista 64 and 32, XP, OS X Leopard, OS X Tiger, Ubuntu). Works beautifully out the box, but you can install various "modules" as well to give you shell access, media servers, etc. All in all a fantastic high performance NAS.
NAS Reviews can be found at http://www.smallnetbuilder.com/content/blogcategory/50/75/
It's Linux, damnit! Pay no attention to renaming attempts by self-aggrandizing blowhards.
The only shops that actually look at cost/GB as a measuring stick are small shops, or shops with very specific needs.
Large corporations, government and high tech companies are usually more concerned with management costs, retention, migration and so forth.
This is simply not true. There are plenty of commodity storage requirements that do not require Fibre Channel or even NetApp level NAS. On the other end of the spectrum, cost/GB might not be a huge factor, but the cost of getting necessary IOPS is certainly a factor.
I work on Wall St. and we have multiple PB of storage. We have tons of EMC. However, things like the Sun X4500 and similar products from HP are changing the game. Couple that with being able to do 48 ports of line-rate 10GigE in a 1 RMU stackable, per priority pause coming into use, and Data Center Ethernet down the road and you have many reasons to seriously reconsider the scope of your fibre channel deployment.
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
I get 950 Mbps with iperf (w/o jumbo frames). Actual disk to disk, or even disk to memory transfer speeds are at 30MB/s. If you'd read the article I linked to you'd know that.
Actual protocols are a LOT slower than just slamming as many bits as possible through a pipe.
Photos.
Go with something like Infiniband, or 10GB ether (should give you access to FCOE via ISCSI for instance. Fiber SCSI SAN the sucker, and there you go.
Look up netapp.
Enjoy.
I work in an imaging department (fMRI). We're moving around usually gigabytes in datasets. If you need fast, simple, cheap storage for OS X we went with the Apple XRAID's which have now been replaced by the Promise machines. They're dirt-cheap for what they have to offer, they're plenty fast and provide all the safeguards (battery backup etc.) and tools you need. I am sometimes pumping 500 Mbps per machine over NFS and we are still on dual PowerPC G5's, the Intel's go even further.
If you need specifics on our configuration let me know, just contact me, I made some tweaks to the TCP/IP settings to get this type of throughput.
If you really want to go cheaper, consider the NORCO DS-1220. It's a 12-bay eSATA enclosure. It needs a bit tweaking to get it to work properly but it's plenty fast and dirt cheap (enclosure is under $800 + 10 disks) Only use the 10 disks on the expanders though if you're limited in a 1U server. You can use the 4-channel eSATA controller to drive 2 enclosures that way (20 drives)
Custom electronics and digital signage for your business: www.evcircuits.com
My ex-company had a TeraStation Pro. Throughput was not good. It was also unable to do decent authentication. Flashed with the latest firmware 1.04 at that time.
... to say that software RAID is almost invariably a poor solution. It is woefully slow compared to even a slow hardware RAID implementation.
Please cure your ignorance here.
Or simply test some systems before you talk nonsense.
Spend a few bucks and get the right hardware. It is not expensive these days.
Are you selling raid controllers or are you one who actually believe their marketing bullshit?
I know, I know, it's not a NAS, but I worked in the broadcast video world for a while and I started using them as NAS boxes since I needed to sustain 300 megabytes per second over bonded ethernet. It worked well.
But on the other hand, an HP rack server with at least a quad core Xeon and OpenFiler is a GREAT/AWESOME option.
Let me just point out what hasn't been said here so far. For sustained high speed data transfer, Xeon blows the doors off of Core 2 since the focus of those chips performance wise has been on bus speeds.
To answer the poster's question, I have a Thecus N5200 Pro that performs pretty well - RAID0 should break 40MB/s - not amazing, but better than any other SOHO NAS's around, barring the new N7700, which looks like it'll hit Gig-E limits (finally a cheapish NAS that approaches 100MB/s for reads/writes). That said, you're gonna pay for the performance - the N5200 is about $700 and the N7700 is $1100 (enclosure's only). eAegis sells them w/ burned in drives as well - that's where I got my N5200. The hot-swap and automatic RAID rebuilding works as well and it has built in FTP, SMB, AFP, and NFS and is pretty good for a plug and play system.
That being said, you're definitely paying a premium, and you could easily throw together a multi-terabyte system that would max out your GigE for about the same price as what you'd pay for the N5200 enclosure. My only big recommendation there is that you get a hotswap rack w/ that - makes things much more pleasant when replacing drives.
One other thing to consider is power consumption. My N5200 Pro idles at 80W - if you built a low-power mini-ITX system you could probably get something pretty close to that, but a regular PC would probably be closer to 150-200W. Depending on your electric billing, you could be talking about a $100-200/yr difference there.
Some other comments:
PROS: At least until the last OS iteration from Netgear they used a Debian based system. So you could just use apt to get new stuff installed. The Netgear forums are busy and provide useful answers to questions. The Duo specifically is small and silent, and was therefore approved by the girlfriend. Can be programmed to turn/off on specific time periods of the day, which is ideal for a home file server.
CONS: The processor (sparc arch) is quite slow. The Debian dist they are based on is "sarge" (not maintained anymore), you can't upgrade the system (with another Debian system), and their own ReadyNas software is installed in a subsystem so changing its options through the command line remotely with ssh is not straightforward.
I have numbers to back it up: D-LINK DNS-323, 2x 500gb 5400rpm Samsung drives in Raid-1 configuration. I don't know the exact model, but I certainly selected these for low noise, low energy consumption and low heat output. So they're absolutely no high performers, but in regular, day-to-day operations, the Gigabit adapter manages a throughput at a steady 15 percent of 1000mbit push and pull from/to medium performance Windows workstations.
This NAS unit is on the market for well over a year and it took several firmware revisions before other problems were worked out - but raw speed above 100mbit was never an issue. I don't have any real high performance client workstations, so I cannot say if these steady 150mbit throughput is limited by client or the NAS itself, but it certainly is enough to max out any and all WiFi links, which is enough for many applications except full disk backups, which take some hours in any case.
I researched for a while before buying and got pretty much what other users described. I suggest you do the same so you can avoid the bad apples in the crowd of NAS units.
I second those that suggest building your own box. Just make sure you get all disk controllers on PCIe together with the NIC itself. Also forget about the integrated NIC and get a quality one, Broadcom or Intel with the latest drivers. Jumbo frames and TCP Offloading enabled will get your data across faster. Finally, if the switch supports it, put two or more NICs and enable bonding across them. As other posters suggested analyze your workload, the level of concurrent access and decide wheter to get many small spindles or less higher capacity.
That was my -.02
Mi domando chi à il mandante di tutte le cazzate che faccio - Altan
Hi There,
I have been using a network appliance storevault in our environment that has similar requirements to what you seem to have. The device is a little more expensive when compared to the other netgear etc products you mention but for large sequential file transfers we will almost always get line-rate gigabit performance. It is great because it also gives you a gateway to experience the data ontap software from the netapp range. We paid just under £5000 for the particular setup we have but I have never looked back. Everyone in the organization is extremely happy with the products performance and reliability and at the end of the day isn't that what it is all about?
just get one of these things and stick it next to the fridge in the kitchen.
http://h18006.www1.hp.com/products/storageworks/xp24000/index.html
It sounds like a perfect case for this except I'm not sure if there is a MacOS version. But you could still run it on the remaining Linux & Windows boxes and share it out from there.
costs $0 for a three disc license and $69(?) for a 6 disk license.
I can't speak for anyone else, but you lost me there...
unRAID is very, very picky with hardware. I've had so many issues both times I tried to setup unRAID that I've given up in it. I know people who can't say enough good things about it, but all I ever got were hard hangs and reiserfs errors.
I used 2 LaCies for a while, but they both had a throughput of 10MB/s (the NAS with XP as OS) and 6MB/s (LaCie with Linux).
Then I switched to Synology DS408. Mine has 4x Seagate 1.5TB HDs, RAID 5, so I have around 4TB of space.
The network throughput maxes out at around 60MB/s(!). But this might be due to my not-so-good switch. It's all on a Gbps-Network.
I used it only with Mac OS X (iMac, MBP, MBA, MB) with AFP. I haven't tested performance with SMB or NFS, but should be as fast as AFP (probably even faster).
One thing, which really convinced me of Synology, was their support. Since the Seagate 1.5TB HDs have some problems (make sure you buy those with Firmware >=SD1A), I had a lot of issues at the beginning and thought that it's a problem with the NAS. I even thought I lost data. When I contacted Synology, they offered to log-on on to the NAS and try recovery, local check and everything - for free. And in the end, they found the problem with the Seagate HDs, proposed the solution and I am now even more happy then before.
And no, I'm not working at Synology...
If you want the fastest possible speed, and if money is not the object, and you have a reasonably low amount of "streams" in and out, look at a Bright SAN. In the film world you can get multiple streams of uncompressed 2k dpx's playing off these without dropping frames, and they have gone a long way to solving the fragmetation issue that affects similar systems, particularly as the have written the FS.
I have always found software RAID to be faster than hardware. Apparently CPUs can XOR like crazy.
I recently assembled a 4 disk software RAID-5 system for a friend. It read at 300 MBytes/sec, and no, it wasn't in cache. I thought that was pretty impressive.
Instead of FreeNAS, I've tried . I managed to configure an iSCSI target with DRBD as the datastore for my VMware ESX 3.5 server.
OpenFiler is neat and easy to use. Check it out too.
w00t
Thecus are good. You can get one that does i-scsi or has a GigE router.
Qnap is very solid.
Synology if you like features.
Roll your own, a box with Direct attached storage is hard to beat.
Instead of using nc and pv to test the network throughput between a server and a client, try iperf.
w00t
I looked through the SoHo NAS market pretty thoroughly before settling on a QNAP TS-109 for my needs. It does NOT have the kind of bandwidth you are looking for, it is better than many of the other low-power boxes, but all of the low power boxes are CPU limited in terms of bandwidth.
For the same money as a dedicated NAS box, you can get a low-end PC with Linux that will kick the NAS box to the curb on performance. The drawbacks of the Linux box are:
Size may not be a problem for you, and if you want 4 disk 7200RPM SATA II Raid 0 performance (or better), you're not going to be in a particularly small box anyway. Same goes for power consumption - 4 drives are starting to be quite a load in their own right. Which leaves: configuration effort - the NAS boxes are mostly plug and play, you might spend a couple of hours installing Debian (or similar) with Samba support, but after the initial setup, maintenance should be similar (backups, regular prayers to the spindle bearing and head crash deities).
The place where a Linux box can get out of hand is its flexibility and configurability, you can always slap a new application server on there for some purpose or another - if you resist that temptation and keep it simple as a NAS, the Linux box should be every bit as reliable and trouble free as a NAS.
If the NAS supports the non-routable NetBeui protocal.
Install the optional "Netbeui" protocal stack located on the XP install disk. (same add-on will also work on Vista.)
Don't forget to disable (uncheck) the "QOS Packet Scheduler", it will limit you to 20-25% of max link speed.
Lastly, one must also disable the NetBIOS over TCP/IP, if it connects first you won't see any performance boost. (Option located in the TCP/IP Advanced/WINS dialog).
The older/non-routable NetBeui protocal stack in the NT/W2K days was roughly 10x more CPU efficient per byte than NetBios over TCP/IP.
In XP/Vista environments it's still 5x more CPU eff than NetBios over TCP/IP.
You can try snap server, it has enterprise level speed and reliability, flexibility... supports almost all the file sharing protocols and authentication protocols out there...
I currently own a Buffalo Linkstation NAS, it gets about 20 MB (megabytes) of traffic/sec on reads max, which isn't all that great but at least it's into GigE speeds. The web interface is decent, and since it supports samba and ftp it should work fine with OSX.
I also have tried the HP Media Vault (their small linux NAS), but I found that when I put a second drive in it would over-heat and turn off. The user interface sucked too. I didn't get a chance to benchmark its speeds before I returned it.
What, at $69? How much time do you think it costs to set up a linux machine from scratch, including all the sharing necessary for a mixed environment? How much do you think a development person costs? In billable hours, most of these guys waste $69 worth of time on coffee breaks or taking a dump. I've set up two of these, and each one took about 30 minutes (exclusive of the hardware build, which was zero for the old Dell I had). The last time I installed ubuntu it took 30 minutes just to get to the startup boot.
Free is not the end-all, be-all, especially in business. This would be $0 if he didn't need more than, say, 2TB of storage (3x1TB, one for parity), but $69 is not really a heavy cost. There are a lot of things I could do instead of paying for, but I don't, because even at my pedestrian $130/hr billing rate it's better for me to pay for a widget that fixes my problem than spend 4 hours troubleshooting it and getting it fixed myself.
Is it just my observation, or are there way too many stupid people in the world?
The only shops that actually look at cost/GB as a measuring stick are small shops, or shops with very specific needs.
cough Google cough
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
I don't think speed is the right reason to go with a hardware raid card like 3Ware. The real reason to go with a hardware raid card is for the battery backed cache. A UPS can help mitigate that issue, but that doesn't help with kernel panics. You still have to trust the software running on the raid card to do the right thing in the event of a crash of the OS, but being able to trust that the data you sent to memory on the card is on 'stable storage', run the drives with the write cache on (unsure how safe that is, even with the battery backed cache), and still be sure of your data is _huge_.
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
I took an old Dell GX280 that we were getting rid of and turned it into a FreeNAS. The OS is installed on a 1GB MicroSD card in a tiny USB adapter and I stuck in a WD6400AAKS (WD 640GB SATA). Using the onboard NIC, I transferred a multiple-GB ISO between it and my PC (Athlon X2 6000+, 4GB, nForce 590, Seagate 500GB 7200.10). I sustained over 300Mb/s on the transfer. Also, my TrendNET switch between the two doesn't support Jumbo Frames, so this is with standard packet sizes. Not bad for two PCs with single SATA drives and onboard NICs, on a ~$120 NAS built in less than an hour (including hardware and software install).
If I had to build a NAS/SAN on the cheap for work it would be something based off of OpenSolaris/ZFS. The amount of features you'll get out of ZFS/Opensolaris for free can't be beat. Really worth a look. A few products that I would say to look at are: http://www.pogolinux.com/nexenta.php http://www.nexenta.com/corp/index.php?option=com_frontpage&Itemid=148 http://www.nexenta.org/os and of course: http://www.opensolaris.com/
Is the theoritical limit on a Gig-E network still 119MBps with Jumbo-Frames turned on (MTU=9000)? Or does that theoritical limit go up to more (6x if liniar)? I ask as I just purchased a Jumbo-Frame capable Gig-E switch to use for a SAN: Ubuntu NFS box (amd 1.2GHz/1GB/4x600GB connected to a CPU-based PCI-4channel-Sata card.) Although it looks like I will also be PCI-limited in terms of throughput...
Typically you can expect between 10-20MB/s from a Marvell Orion powered consumer NAS, which is barely quicker than fast ethernet.
Most consumer NAS these days use this SOC, with the exception of Thecus who cant seem to decide what vendor to standardise upon.
If your storage must be NAS, build your own for better performance.
Even when you go up to 4 bay NAS orion/powerpc based NAS they are still slow, and costing about £400-£500 which is a total rip off for the performance they give - for that you can build a very nice little storage box possible even with real hardware RAID.
It sounds to me like you need to better define the criteria which require a NAS. Whenever looking into new equipment it's always helpful to define the ideal situation in which your new equipment would function... for me based on what you mentioned I would say the ideal would be:
-0 wait time for transfer
-Portability of data
At no point did you mention other people needing the data while you're running the tests. Based on those criteria I would shy away from a NAS. Unless you're willing to shell out a grand for a Linux solution, a 1 TB external drive with an eSata connection will be ~$900 cheaper and possibly perform better to those criteria above. When you're not doing the testing you can simply plug it into an idle machine for people access the data over the network and to perform backups.
I'm a big fan of home-brewed Linux Raid NAS solutions, I setup and maintain a few myself for several different organizations and I regularly transfer 100+ gig dumps of files... however I would avoid doing all of this if I could. Keep it simple... it's much easier to just carry the data with you from machine to machine if your files are exceedingly large... after all HD's are cheap now-a-days.
On another note if you do insist on a NAS, one thing I've noticed is that Filesizes can easily affect transfer rates. The smaller the average filesize, the larger the individual overhead is in comparison to the file. In other words if you're transferring 100meg files and the overhead is a theoretical 1kb then the majority of time is spent moving the file but if you're transferring 1kb files and the overhead is still 1kb then you spend just as much time on the overhead as on the files themselves.
One trick I've learned to speed large transfers of small files along is to turn your files into a storage-level-compression tarball on the fly with a blowfish cipher and pipe the output over ssh to the other machine, where it is disassembled on the fly as well. This means that you're only transferring one file across the pipe and there is less overhead to the total transfer. This trick keeps the transfer rate on my machines steady at around 10-20mb/s as opposed to 6-7mb/s and those numbers are on a standard configuration with no special hd's or raid implemented. I can only expect the numbers to be better with raid. Just things to think about.
1. Go to www.google.com
2. Type in the search box "nas throughput chart"
3. Hit "I'm Feeling Lucky"
There, question answered, you have to really master google search to find esoteric tech info such as this.
(Notice that the charts displayed are on 100Mbsp network, you'll want to change the drop-down to 1000Mbps)
First of all for the purposes mentioned in the original post, a USB HDD is by far the fastest, cheapest and most reliable solution. Transfer rates of 500MB in as long as it takes to walk from one desk to another.
But if you are looking for a network based solution (and have $10k), I am surprised no one has mentioned CoRaid. CoRaid is a AoE SAN hardware company but they offer NAS solutions too (basically a 1u linux box connected to their SAN) - for about $8-9k ($6.6k+15 1TB drives) you can have a 15TB (unraided capacity) NAS with advertised throughput of 100MB/60MB (read/write).
-Em
RelevantElephants: A Somatic WebComic...
Use a SAN and you just zone the storage to the server that needs it when they need it. No transfering of GBs of data required. :)
If performance is your goal, don't look at the "buy-it-at-Frys" level of NAS, or at "roll-your-own."
If you build your own, you'll end up bottle-necked by the performance of the particular OS you use, plus SAMBA or NFS (depending on your needs.) Plus, there's the time factor in putting it together, tuning it, and maintaining it. Granted, this isn't a lot if you're already a tech-head, but your time isn't free.
If you buy one of the consumer-level NAS boxes, what you're getting is the equivalent of building your own, without the ability to tune it, since most are based on the same open-source software you would use yourself. Pretty much every NAS device I've ever seen has the same cyclical bursty transfer profile as a build-it-yourself.
If you want better performance and buzzwords, every major PC vendor now has a SAN solution. You get the benefit of a team of people whose job it is to maximize disk performance, and a nice management system. However, your system head still has the same problems as before - using a general purpose OS and/or open-source software.
If you want pure performance, look at Network Appliance. They've been in the game for a long time, and their hardware/software combination allows them to control/tune the whole environment. To a first approximation, all the cool things in ZFS were done ten years ago by NetApp. You get the benefit of a whole company whose job it is to maximize disk and network performance. You can look at a performance review from earlier this year showing about 30k SPC-1 IOPS.
Personal anecdotes:
Note: My only relation with NetApp is being a very satisfied customer.
For what it's worth, I'm using a Netgear readynas+ here at work with 4 500GB drives in it (RAID5). I get usable throughput, but it's no PC. Reads are about 20-25MB/s on GigE without jumbo frames, writes are at around 12-15MB/s. One thing I really like about the box is the ability to SSH to it and manipulate files locally, I use it for holding security camera storage, and being able to mv 1000's of small files around is nice. Cost is high for what it is though, if I remember correctly, it's about $900 for the device with 2 drives in it. User interface is decent.
As posted previously, we've tested a Thecus NAS and it was hell. We've since replaced it with a SansDigital MN4L+B, with 4x 750GB Seagate ES.2, and it works like a charm. It's the fastest NAS you'll find below 1000$ (beside building your own). It has a Celeron 1.5GHz processor, extra slots for expanding DDR2 RAM up to 2GB, and it's made of metal, some some cheap plastic junk like Thecus') It's also one of the very few NAS that support NFS and iSCSI. Yes, we were surprised indeed to find out that a LOT of NAS out there don't have NFS support. We have it configured as RAID 10, and it's as fast as you can get on the GbE link. It has also rebuilt our 1.3TB array in a matter of hours after a drive was replaced. So far, we're very happy with it. Note that I'm not comparing with the more expensive solutions (Synology, Intel, Dell...) although I doubt they bring much more to the table other than extra storage space ans SCSI support.
Doesn't stuff like ZFS pretty much make that a moot point?
Opera, Proxomitron-Grypen,GPG 0x0A1C6EE3
Who's doing 48 ports of 10GbE? That's pretty damn impressive.
Lime-Technology makes a really interesting piece of software. It's a bootable NAS using their raid implementation called unRAID, which is similar to RAID4 except that each disk has an independent filesystem. So, you can pull a single drive out and it's readable. Also, you can mix and match any drive sizes you like, the only condition being the largest drive is used for parity.
Also the basic version is free
http://lime-technology.com/
4 drives and eSATA for expansion. Details here
This uses a 1.6GHz Celeron, which, while low end for a general cpu, is pretty fast compared to most cheap NAS units and will come closer to utilizing the gig port than most.
I don't think so, if my mail server issues a write to ZFS, and it returns that the data is on 'stable storage' (fsync symantics), then issues the "250 message received" to the client, then immediately crashes; either 1 the message is lost if the data wasn't really on 'stable storage', or 2 ZFS had to wait for the write to complete to the platters or the battery-backed cache on the raid card. I'm guessing just having to get to the battery-backed cache on the raid card would speed things up quite a bit...
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
I'll also recommend the Synology boxes, based on several years with them.
They are typical SOHO NAS solutions, so they don't compete in performance with custom roll-your-own solutions or with a midrange (expensive) NAS. However, they do come with built-in RAID support. Most of them have built-in webserver, photo station, media server, and print server functions.
The down side is that every Synology NAS assumes all of its clients are either Windows or Mac boxes. The supplied backup and p2p download functions only work with those clients. However, Synology provides free add-ons to upgrade their boxes to use NFS instead of SMB and add telnet and ssh support. We've had no problems using them in a Linux-only home.
The administration interface works fine in Opera, Epiphany, and Firefox. It is supposed to work with Internet Explorer, but I cannot personnally confirm that.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
My homebuilt NAS (not really NAS, just a desktop with an external RAID5 array) just finished receiving backup mirrors from another machine in the house (a Xen box with 6 VMs running). These transfers are over rsync/ssh with no pre-existing file to help with speed up. Only the seagate drives are branded, everything else is cheap. Software RAID5 on the target with about 1GB of disk buffer RAM, single disk on the source. GigE network with either built-in NICs or $9 cards. Ubuntu and JFS on both sides.
crm46-20081217.tgz 478555089 100% 49.85MB/s 0:00:09 (xfer#1, to-check=45/54)
dms44-20081217.tgz 3043217581 100% 16.49MB/s 0:02:55 (xfer#2, to-check=36/54)
mon45-20081217.tgz 464314500 100% 43.80MB/s 0:00:10 (xfer#3, to-check=27/54)
pki42-20081217.tgz 369984893 100% 40.45MB/s 0:00:08 (xfer#4, to-check=18/54)
xen41-20081217.tgz 1546169689 100% 14.87MB/s 0:01:39 (xfer#5, to-check=9/54)
zcs43-20081217.tgz 2496573639 100% 15.32MB/s 0:02:35 (xfer#6, to-check=0/54)
total: matches=0 hash_hits=0 false_alarms=0 data=8398815391
49MB/s = 392 Mbps
15MB/s = 120 Mbps
Not bad for cheap solution that's been working 3 years now. Cost was about $550 total. Check out the Addonics - http://addonics.com/products/raid_system/mst4.asp
I know this doesn't exactly answer your question... but unless your machines are widely geographically separated, why not cut out the middleman and just get a hard drive and a usb2 enclosure? You know exactly what you are getting (instead of all the debate about "real world" performance). Sneakernet ftw
The Answer
dunno, but cisco have this: http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/ps9512/Data_Sheet_C78-437757.html 32 ports of 10Gb with 80Gb fabric.
software RAID ... is woefully slow compared to even a slow hardware RAID
Wrong. Go do your homework.
Well, maybe... what I've found over the years is that RAID-0, RAID-1, RAID-10 all perform extremely well as Software RAID without taxing the CPU.
But RAID-5 and RAID-6 do tend to become CPU-bottlenecked given enough spindles.
So if you're going with RAID-5 or RAID-6, makes sure to purchase a CPU with a high single-core performance (Linux Software RAID is currently not multi-threaded enough to split the calculations across multiple cores yet for a single array). The last RAID-6 box that I built, we made the mistake of going with a 1.8GHz CPU instead of a 2.4GHz or 2.6GHz CPU, which cut our performance by about 1/4 to 1/3 of what it could have been.
The spindles are definitely not at 100% utilization, but you can see the software RAID process pegging one of the CPUs at 100%. (Fairly modern motherboard, PCIe-based, pair of Opteron dual-core, 8GB RAM, 15 SATA drives.)
Wolde you bothe eate your cake, and have your cake?
Well, that sounds better.
Yes, RAID5 is taxing the CPU but then again you don't normally use RAID5 in a box that is supposed to host I/O intensive apps.
About a year ago we benchmarked a few hardware RAID cards (3ware, LSI, adaptec) versus linux mdraid and found that only the very high end cards ($1000 range) could actually saturate our 16 SCA disks.
mdraid had no problems maxing out the disks (on Dual Xeons iirc), admittedly under significant CPU load.
It's supposed to tolerate these faults for modifying data because it writes additional or replacement information somewhere else and then updates the index. So - you still might lose the data, but you won't have your index pointing at something corrupt due to a partial write.
No, I am not an English major. My posts are subject to typos and incorrect grammar. Do not expect perfection.
Firm rule: those who malign hardware RAID aren't using the right controllers. The three you list are nowhere near the top of the heap. Try a HP Smart controller for some hardware RAID pleasure.
Then again you were defending the Sun x4100 servers last time I saw you, which also has a POS RAID controller. And crappy disk bandwidth in general.
HBI's Law: Frequency of calling others Nazis is directly correlated with the likelihood of the accuser being Communist.
Well, I wouldn't even malign hardware raid for performance alone. Hardware raid is a bad idea for much better reasons, such as reliance on single vendors, non-portable on-disk formats, unknown failure modes etc. And RAID5, soft or hard, is a horrible idea with today's disk sizes anyways.
And what's wrong with the PCIe performance in the xfires? We haven't seen any problems so far.
If you buy their POS controller instead of your "HP pleasure" then who's to blame?
100% GPL - download the source
N5200B Pro - easily over 70MB/s up to 7.5TB
N7700/N8800 - easily over 100MB/s up to 10.5TB/12TB & ZFS & XFS
Supports everything: Windows, OSX, Linux, BSD, Unix
Supports everything: CIFS/SMB, AFP 3, NFSv3, FTP, HTTP, HTTPs
Who's doing 48 ports of 10GbE? That's pretty damn impressive.
Arista Networks. Formally known as Arastra.
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
Arista Networks. Formally known as Arastra.
Self-correction: That should be formerly.
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
Certainly you can do decent Raid via software, but (normall) why would you want to? That is to say, why load down the CPU when you can offload the task to dedicated hardware?
If it is a box dedicated to JUST network storage, I could understand... and I suppose that might be what he had in mind. But otherwise you are misusing your resources.
Again that's fine, IF you want to dedicate your general-purpose processors for such tasks. But that is like shooting an ant with a cannon.
I just did not want to get into the really gory details.
But in fact if a drive is physically interleaved (which as you point out is invisible to the outside), the latency is NOT identical. A drive with interleaved sectors takes more than one physical revolution to read all the sectors on the "virtual" track; how many revolutions depends on the particular interleaving scheme.
As you point out, the latency is related to how long it takes a disk to do one revolution. But on an interleaved track, a "virtual" revolution may be several physical revolutions.
This might not be a common scheme anymore, but again, a benchmark is the only way to really tell.
Good lord thats amazing. I had no idea that kind of port density existed.
Right, the filesystem wouldn't be corrupted, but data would be lost. Lots of people find that unacceptable :-)
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
Did you read what I wrote? IF you save money on the disk subsystem and use that money to buy more processor THEN you can use that additional processor to do a better job of managing your disk subsystem than the dedicated hardware could. So if you play it like I've outlined then you end up with faster disk when you need it and you also end up with more processor for other tasks when disk isn't taxing it.
-*The above statement is printed entirely on recycled electrons*-
You are right: it doesn't matter... today. It used to. I can cite one situation where the latency most definitely was not the same, though: when the interleave was incorrect. It could take many revolutions to find the correct sector.
:o) Although I did see some instances of it.
But that was due to improper low-level formatting and doesn't really count.
Do not assume 1Gb ethernet is >= 100Mb ethernet for throughput unless you have numbers to back it up.
The problem with ethernet at 1Gbyte speeds was the limit of 1500-byte ethernet frame. So they added 9KB jumbo frames, but these are not supported by early devices (switches or cards). So if you do not use jumbo frames, you can easily bring down your cheap switch.
I can tell you that I am using one router (PC with linux) with 1 laptop 7200 HDD (for torrents) and my gaming rig with no RAID in any of them. I can transfer 20-25MB/s because that is the limit of the 2.5" 7200rpm HDD. So my switch (TP-link 8-port all-gigabit) throught which all my network is connected has the bandwidth. But this is with big files (100+MB).
For small files the file-system overhead and incomplete ethernet/IP packets have a big penalty to network speeds AND HDD reads/writes. Also, the busier the network is, the worst the performance is. Keep in mind that the higher latency you have, the lower speed you're gonna get. Especially with windows file sharing (SMB). While experimenting with a local cascading 100Mbps network, I saw that SMB was affected if you communicated through 2 switches (10MB/s) vs. 4 switches (7-8MB/s). The Server (test source) was the same but the client did not affect the result, because for FTP I got same speeds (10MB/s at 2 or 4 switches). There was one test file around 600MB in size.
I tried a 1Ge that allowed an internal 2-disk RAID-0, setup. The best it would do was about 12MB/s read, 7-8MB/s write.
From what I understand only one "device" had RAID setup. So maybe the other end (which was not RAID) could not handle more speed. Remember that when you copy, you have the speed of the weakest link. You would not expect to copy at RAID speed from a CD/DVD.
I wounder if you could use the GPU on that board to off load checksums and maybe encryption?
This is definitely a *BAD* idea.
Although AMD is active a lot inside the OpenCL development - which mean you could easily have a nice C99-like environment to develop your GPGPU-accelerated RAID system - there are 4 main reasons NOT to do it :
That mean you have to use twice the bandwith :
once to load the data from harddisk to main RAM, and then a second time to upload the data from main RAM to Video RAM.
Although, probably, with some clever programming trick you could peer-2-peer connect the GPU to the disk controller over PCIe, but this goes beyond the basic "throw a couple of code lines in C". And I don't know how the onboard GPU is internally connected to the rest of the bus in a 780G (does it have 16 dedicated PCIe lane - in which case it should be able to hold the load ? Or does it have a smaller connect which will bring an additional bottle neck to your setup ?)
But for something like RAID, whose *purpose* is to ensure reliability, and whose main enemy is bitrot, adding yet another layer that can break once in a while isn't the best practice.
Although you can offload checksum *verifications* to the GPU and use the failures not to mark failed blocks, but blocks which will have to be double-checked on the CPU.
Then, on the other hand, if your purpose isn't to speed up the process, but simply unload the CPU, you probably will manage to have more CPU time available while the GPU takes care of it.
With software RAID you have the absolute certainty that the copy you have in RAM is correct (and that copy will be kept subsequently correct thank to ECC RAM).
With a hardware RAID, you have the risk that errors appear between the place were intergrity was checked (the hardware controller) and the place where it will be used (CPU + RAM) : and not all bus have error checking. PCI and PCI-X do not. The full PCIe has 32bit CRC, but not the "Graphics PCIe" used by most consumer material.
Then again it depends on the degree of reliability you expect from your RAID configuration
So, for a home server whose purpose is mainly to host the collection of photos and videos, a GPGPU RAID implementation might be a fun project, but I would seriously avoid this on a system requiring high reliability.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
On non NAS devices (like from a linux server), I get 200-400Mb/s Xfer over 1G (w/non-raid disks). I don't see that on NAS devices.
Max speed I was able to get with standard packets (1500 bytes) was about 700Mbs -- that's expectable/acceptable. I would "love" to use jumbo frames, but Jumbo Frames usage is flawed as currently implemented (IMO).
Currently, all equipment must run at same frame size -- you can't upgrade "some cards", and switch over to larger packetsizes, slowly as you need to replace things...its all 1 size or nothing. The bad thing is that when larger packet devices would try to talk to smaller devices, they would work find for small packet xfers -- like 'ssh' -- but just "drop" larger packets on a TCP connection --
no error sent back to sending device to break packet down into smaller chunks -- packet is just dropped.
There was no negotiation (or no "correct negotiation") of max packet size -- if a host was set to allow use of large packets, it would use them talking to every client -- was MAJORLY bummed when I figured this out -- there was no way to tell the host to use different packet sizes based on some agreed max-MTU size -- all or nothing.
I might have been able to use multiple virtual IP interfaces and cut up the network into different virtual, overlapping IP spaces based on their MAX MTU size, but that was way too much headache to enable Jumbo frames. Having to move clients around on a virtual subnet based on what card they were running and what Max-MTU they were capable of... There needs to be some auto-negotiation as is done for link-speed&duplex.
Under test conditions, though, Jumbo packets only gave me about a max of 15% more throughput. In practice, it wasn't noticeable, but admittedly, could have been if my server had been using RAID or if more of the clients were linux (instead of Windows -- Windows stack was notably slower).
I don't know the details of the jumbo frames (like negotiations, if any), but I do know that at least the communicating points (hosts and switch) need to "know" it. But I see it as mainly a "switch helper". Think of an 8-point switch with 8 hosts and all of them transferring at maximum speed. So if you can send 1 big packet instead of 6, you can help the switch very much. Although the ehternet checksum is done on the same size (wether it's 6 1.5K packets or 1 9K), the overhead and switching are done 6-times less.
So the jumbo frames will not give you much higher speeds (mainly the overhead 18 bytes/packet of the 5 packets that are sent if jumbo frames are disabled) unless your network is congested.
But like you said: compatibility within the network is questionable.
In my previous post I said that there was a problem (although I did not say it was about usage and not bandwidth), but not that the jumbo frames will give you always a speed boost.
I guess it has more to do with what you're storing. Only recent data would be lost, so you wouldn't lose the entire document - just the latest revision. Really - that's an issue for any pending writes during a shutdown.
No, I am not an English major. My posts are subject to typos and incorrect grammar. Do not expect perfection.
S550 from NetApp should serve your needs if it fits in your budget.
But RAID-5 and RAID-6 do tend to become CPU-bottlenecked given enough spindles.
Even a 10+ year old 300Mhz Pentium 2 class CPU has a RAID5/6 checksumming speed well into the hundreds of MB/sec. It's not going to be a limiting factor.
So if you're going with RAID-5 or RAID-6, makes sure to purchase a CPU with a high single-core performance (Linux Software RAID is currently not multi-threaded enough to split the calculations across multiple cores yet for a single array). The last RAID-6 box that I built, we made the mistake of going with a 1.8GHz CPU instead of a 2.4GHz or 2.6GHz CPU, which cut our performance by about 1/4 to 1/3 of what it could have been.
The checksumming speed of the CPU was almost certainly not the problem (unless you were seriously loading down the machine with other processing). You don't mention what kind of CPU, but even a 1.8Ghz Pentium 4 will be able to compute RAID5/6 checksums in excess of a couple of gigabytes/sec. Most likely you were bus-limited, or had crappy disk controllers that couldn't shift data fast enough or were generating storms of interrupts.
The "high CPU usage" of parity-based RAID is a myth that needs to be laid to rest (although it is good for identifying which people really do know what they're talking about when it comes to RAID - I often use it as an interview question).