"urging device manufacturers to start immediate testing with its pre-beta release" - Translation: Get on the ball and do our work for us.
How is it MS' job to write drivers for 3rd party hardware?
"'There is not another WinHEC planned before Windows 7 is released,' Microsoft has warned them." - Translation: We have you by the balls. Don't make us squeeze. We want you to do things for our benefit, and we're unwilling to wait, or even to ask nicely.
This is just ignorant. Do you have any idea how much money and effort it takes to put on something like PDC or WinHEC?
Do you have any idea how much it drastically disrupts those same people that are building windows 7 to have these? The same people doing preso's workshops and support during winhec are the same ones building windows 7. The more winhec's you have, the slow the development happens.
Windows 7 is being released, and soon. Yeah, we screwed the pooch with Vista. But we'd like to fix things, and we'd like your help. Towards that end we are making a pre-release version of Windows 7 beta available to developers so we can make something that has the promise of Vista, but actually delivers. And we'll be holding several WinHEC sessions, to help you, our valued partners make this next Windows the best product it can be.
Fortunately the company I worked for at the time dumped Windows for Linux shortly before Vista was released, because trying to port the drivers from XP to Vista was hell, particularly when so many of the changes were really only needed for supporting DRM.
This makes absolutely no sense.
What drivers were these that you were trying to port in-house? It cant be a product you sell, because just abandoning the windows market for software/hardware vendor is suicide right now if done in the manner you describe.
And why would you port drivers in-house?
And if moving to Vist was so hard, why did you not just do... NOTHING.
And specifically what exact changes are you referring to that were 'only needed for supporting DRM'?
If it was video drivers, most of the pain was moving the driver to userspace. That had nothing to do with DRM.
Not to mention that hardware vendors were under NO obligation to support any DRM in drivers, unless you needed HDCP or similar labels attached to the packaging.
Make sure you sue netgear, or whoever sold you a card and counterfeited to make it look like netgear.
MS certification on the 'designed for vista' absolutely, no exceptions, requires both 32-bit and 64-bit drivers to be available when the device is sold.
Someone scammed you by erroneously slapping a 'designed by vista' sticker on a box that wasnt.
Or better yet: they could demand all devices conform to a set standard and then produce drivers for standard hardware only.
There is no reason for printers to all have different ways to talk to the OS. Same goes for scanners. This could all be standardized.
This all exists already in the windows ecosystem.
There are highly generic printer drivers that work with a substantial chunk of printers in existence, but has no extra features like duplex printing, quality levels, etc. And thats NOT include PS print drivers.
If a company who makes a scanner just wants minimal functionality for a scanner, they can also just ship twain drivers (or whatever the current version is) and force the end-user to use the MS scanning software that ships with windows.
You can get a generic laserjet driver that has no GUI, no fancy features from HP that works with nearly every laserjet printer they have ever made. But only businesses use it.
Hardware makers use the 'helper software' as a way to differentiate their products.
It's unfortunate, but makes a lot of sense from the HW maker point of view. They want people to KNOW that they're using a Lexmark scanner. Plus they often provide value-added features, like scanning multi-page documents automatically (using the sheet feeder) and automatically producing a PDF at the end. You dont get this with generic TWAIN drivers.
In other words, this isnt a Microsoft problem, or a windows problem. This is a business problem.
FWIW, driver signing doesnt cost anything that you pay towards Microsoft. And for a business its trivial, its a couple hundred bucks a year that you pay to Verisign or similar. You dont pay anything to Microsoft in any way.
Now it DOES cost you to send your stuff to the compatibility labs and get the higher level of signage you can use, but thats also paid out to a third party company who actually does the work. No one (other than that third party company) is making any money there.
I still think microsoft should have said no to backwards compatibility for apps as well as drivers. New OS, new apps and drivers for that OS.
Yeah, thats pretty much exactly what MS did with Vista. Thats why nearly every hardware manufacturer had to put out new drivers and did such a crappy job of it for so long.
That is like 60% of the complaints about Vista, is that tons of hardware wouldnt work right away with it until/unless manufacturers release new drivers.
Good. Maybe I can buy Window NT 7 and skip over Vista (NT6) completely.
Be aware that Windows 7 is a minor release from Vista, which is why its not NT7, its NT6.1.
It's a polish, speed, and stability release.
This is actually a good thing though, as by now the ecosystem (ie, hardware, software and drivers) has largely matured, and will be even more so then.
Your buddy/brother/whatever that had a very slow Vista box.... it was likely the manufacturer's fault that it was as slow as it was. It took manufacturers a _long_ time to ship machines that were reasonably stable in Vista, due to so much of their trialware, drivers, and OOBE stuff being just flat not stable under Vista.
But right now, if you (for example) go to HP and buy a nice business class engineering laptop (Compaq 8510w or similar), it ships with clean & shiny Vista AND Vista x64 drivers, and has very very minimal crap from HP, and generally runs quite well.
You do realize that you can login while you are responding, right?
On the same page/form even.
You dont _have_ use the login form on the sidebar, you can just hit Reply, pen your response, then put your user/pass in the fields and hit Preview to log you in.
I sure hope it's more than something like 32 gigs. If you can support an insanely high number, do it.
In the 64-bit version, yes. IIRC its 32GB for the desktop versions.
As an aside, is there any way to increase the 4GB RAM limit in XP Pro without upgrading to 64 bit edition?
No. MS had a version of XP that allowed you to flip PAE on, but that was like XP sp1 I believe, so only a brief period.
The problem is that a large percentage of consumer-level drivers puke and die if they encounter a PAE situation. This (reportedly) caused such a support nightmare for MS that they just shut off the ability to use PAE in 32-bit windows.
Mind you, even XPx64 allows a relatively large amount of memory, something like 32- or 64GB.
The problem is that 'securing windows' consists 80% of making sure its patched to current, and another 15% not having the users run as admin.
If you're not patching, you're ignoring the single biggest thing you can do to protect your systems, not to mention the easiest.
All that this has to do is be incorporated into a drive-by-install that targets IE, and if any of your users trip over a website that contains this, then it'll eat everything on your LAN in a few seconds.
Frankly, not patching for 2-3 months is possibly one of the worst possible things you can do.
You'd be better off patching quickly, and not running AV or firewalls.
The problem is when a worm for this exploit starts being incorporated into drive-by-download attacks into IE.
So someone browses to a site, IE promptly downloads and runs the worm behind the scenes, in about 4 seconds every machine on the LAN has been compromised.
Unless they're making it easy for people to replace Windows AD servers with Samba servers running on Linux, this is not a big deal.
Did you not read the article you're posting about?
The bulk of the article is about precisely that. He's the lead dev on samba building an AD server, and talking about the wonderful level of support ms is (now) giving them.
He specifically mentioned that since they're the only ones working on an AD server replacement, at one of the 'plug fests' at MS' redmond campus, they were the only ones there to take advantage of it.
In addition, he mentions that the team on samba for building an AD server is short on developers, and is asking for help.
To achieve the same storage size with smaller disks (comparing like with like) then you need more of them, and mathematically, there's more chance that more of them will start going at any one time the more you have, and during a rebuild procedure. Your redundancy will need to increase exponentially with that the more drives you have, as your storage requirements increase - and everyone's has over the past few years.
What you describe isnt how it works in the real world though. You just dont keep adding drives to a single array, you build pools of storage based on redundant arrays in the sweet spot size (ie, 5-7 spindles, in my experience). So no, the risk of drive failure does _not_ keep going up. The risk is at a 'shelf' level, and thats the level you build redundancy into. Then you pool multiple shelves together.
This is how it actually works in the real world in many, many systems. And works well, and highly reliably.
Also, in that situation, the redundancy does NOT grow exponentially. It grows linearly within ranges, and then you have a discontiguous cost jump as you move to a bigger class of storage. Then it grows linearly within that group again.
I don't trust hot spares because having a drive sitting around doing nothing that you won't know anything about until you have a failure is not a great idea.
Thats why you have patrols/scrubbing/whatever. The drives are constantly being tested and beat on, even if they're not being used. Including the hot spares.
The problem is, to get an equivalent storage size (comparing like with like) you will need to use more disks, statistically there is more chance of more drives going through failures and you will spend just as much time, if not more, rebuilding the array (or multiple arrays). The array might take less time to rebuild as a one-off task, even allowing for the fact that you have more disks to rebuild (increasing wear and tear incidentally), but you will do it more often and spend more time rebuilding overall. That has only got worse as array sizes have increased.
In actuality, since you have your redundancy in a storage unit that is sized right for your situation, it deosnt actually work out as you describe. Yes, you may end up replacing more disks as your storage size grows, but you lose disks at a fairly predictable rate within a redundant unit of storage, and the likelihood of losing 2 disks in one unit is very small, due to small arrays, small disks.
Mind you, expensive SCSI and SAS drives are only for the gullible and I have seen no evidence that they are more reliable. The storage size of a drive doesn't correlate well to the reliability of the drive. A 150GB drive is not inherently more reliable than a 1TB drive, other than you have perhaps slightly more chance with more storage space of seeing a failure.
There are differences. Less so now than there were in the IDE/PATA vs. SCSI days.
There are MANY consumer class drives that are literally not rated to be run 24x7. The designer says so clearly on the box (ie, the ibm death-stars).
The differences are less now, but in the past the cheaper consumer drives used a different kind of bearing that didnt hold up so well.
The biggest thing you're paying for typically in consumer-class vs. enterprise class is the amount of circuitry/smarts on the drive. Again, in the SCSI/IDE days, this was very distinct. SCSI drives had all sorts of smarts, could do re-ordering and optimizations via TCQ, etc. IDE drives were just dumb slaves for the controller, as a comparison.
Nowadays, even some SATA drives have NCQ, which can be quite useful in certain kinds of workloads.
But heck, just pick up an enterprise class drive in one hand, and a typical consumer drive in another. The former will weigh several times the latter, as it contains much more metal. Built to tighter tolerances, better heat dissipation due to the she
I completely agree, and talked about that a bit more in my response to segedunum above.
So yes, agreed, the way to make bigger storage available is to do as you did and keep the drives reasonably sized, use more spindles, and make arrays built of other smaller raid arrays (ie, just like you said).
Variations of what you describe is how you get it from NetApp and many SAN vendors as well. Pools are built from shelves which each have their own internal redundancy, and in some systems you also have fully hot-spare shelves available too.
It's the total storage that is being talked about in the article, not the size of the disks, so no, the article doesn't say that. The problem is not the size of drives but the storage requirements that exist today. Regardless of how many drives you have then you are very, very likely to have a failure somewhere due to the amount of data that is being stored.
I'm not sure how you get that from the article.
The core issue that its talking about (which is old news) is the ratio between rebuild time in a disk failure and error rates.
With very large disks, even with a very low error rate, if you read the entire disk worth of content, you're bound to hit an error (statistically). The total number of spindles also increases failure rate, but that doesnt matter. If you keep your spindles reasonable sized, then your rebuild time for a single disk failure is low, and your risk of a two-drive failure is low.
The article's core argument isnt about having errors, its about the likelihood of getting a second drive failure while the array is rebuilding as a result of the first. Mathematically, this has absolutely nothing to do with total storage size, and ONLY has to do with the spindle size of the failed drive.
However, if you are storing terabyte upon terabyte of data with smaller disks then a not insubstantial rebuild time is going to be seen there as well and you've got a greater chance of it happening with more disks.
How exactly do you get this? Rebuild time has zero to do with total storage size and everything to do with the size of the failed disk (and the controller/bus/drive speed, but we assume thats equal).
It's a simple function of the failed disk size, the variable of total storage size isnt even part of the equation.
No, that's not the reason at all. The reason is because such drives are more expensive to produce, have much higher spindle speeds and need greater performance.
That is absolutely the reason. Even if Dell or HP offered servers with 1TB SAS drives, I would never buy them that way. It's a newb mistake to build important raid arrays out of drives that size, due to the rebuild time and overall performance. In a raid array you get performance out of additional spindles and being able to serve multiple simultaneous read requests off of that array (due to some reads only having to hit one or two disks).
If you fill it with 1TB or 2TB drives, then not only will your rebuild time be unacceptably high, but you lose performance gains, as your contention against the same spindle for a set of requests goes up.
This dramatically reduces the kinds of storage you can have. If you have terabytes of data kicking around then you are going to need a lot more drives here, and as such, the chances of you encountering a failure go up as you increase the number of drives to achieve the storage requirement you have.
Which is why when you need really large storage, you build pools of volumes made out of multiple arrays.
There's a reason why all your high end nas and san vendors build their systems this way. They're made up of reasonable sized disks, grouped in redundant arrays of 5-7 (mostly), and then these groups are all pooled together.
Its done this way not only because that makes for more expensive products, but because it works! It's the practical way to build these things.
What we need is a more reliable storage technology within disk drives at an affordable price because that's where the problem is. The biggest candidate for that are SSDs.
I do agree that we need something better, but I would say new kinds of storage software is the way of it, like what ZFS has done, and what google does with their massively redundant storage. SSD's may help (though I'm not sure how their real-world failure rate compares to spinning disks), but I think these sorts of new approaches will be the important jump.
You seem to misunderstand the article. They are saying that if you need 12T of storage RAID 5 is not reliable. You would be better off with a single 12T disk if such a thing existed.
Thats not what the article says at all.
The article says that if you build your RAID arrays from the biggest disks available (which no one with half a brain does) like 1-3TB drives, and you have them filled, then the numbers come out as presented.
But there's a reason why no one on the planet builds important raid arrays out of 1TB drives. Rebuild time is too long.
This is also one of the big reasons why you see so many 73GB and 140GB SAS/SATA drives in raid arrays, and why server storage drives dont grow anything like as fast as consumer garbage drives.
Slashdot Mirror
And in addition to what vux984 said below, needing access to the source is bad software design.
Hardware drivers SHOULD be written to a spec or an API. NEVER to the internal implementation, which is what you get with source.
This is like software programming 101.
"urging device manufacturers to start immediate testing with its pre-beta release" - Translation: Get on the ball and do our work for us.
How is it MS' job to write drivers for 3rd party hardware?
"'There is not another WinHEC planned before Windows 7 is released,' Microsoft has warned them." - Translation: We have you by the balls. Don't make us squeeze. We want you to do things for our benefit, and we're unwilling to wait, or even to ask nicely.
This is just ignorant. Do you have any idea how much money and effort it takes to put on something like PDC or WinHEC?
Do you have any idea how much it drastically disrupts those same people that are building windows 7 to have these? The same people doing preso's workshops and support during winhec are the same ones building windows 7. The more winhec's you have, the slow the development happens.
Windows 7 is being released, and soon. Yeah, we screwed the pooch with Vista. But we'd like to fix things, and we'd like your help. Towards that end we are making a pre-release version of Windows 7 beta available to developers so we can make something that has the promise of Vista, but actually delivers. And we'll be holding several WinHEC sessions, to help you, our valued partners make this next Windows the best product it can be.
Yeah. Thats pretty much exactly what they said.
And wow, wouldnt it be nice if they made all the session from winhec available online for you.
Your comment shows a gross ignorance of actually running a business of this nature.
Fortunately the company I worked for at the time dumped Windows for Linux shortly before Vista was released, because trying to port the drivers from XP to Vista was hell, particularly when so many of the changes were really only needed for supporting DRM.
This makes absolutely no sense.
What drivers were these that you were trying to port in-house? It cant be a product you sell, because just abandoning the windows market for software/hardware vendor is suicide right now if done in the manner you describe.
And why would you port drivers in-house?
And if moving to Vist was so hard, why did you not just do ... NOTHING.
And specifically what exact changes are you referring to that were 'only needed for supporting DRM'?
If it was video drivers, most of the pain was moving the driver to userspace. That had nothing to do with DRM.
Not to mention that hardware vendors were under NO obligation to support any DRM in drivers, unless you needed HDCP or similar labels attached to the packaging.
Make sure you sue netgear, or whoever sold you a card and counterfeited to make it look like netgear.
MS certification on the 'designed for vista' absolutely, no exceptions, requires both 32-bit and 64-bit drivers to be available when the device is sold.
Someone scammed you by erroneously slapping a 'designed by vista' sticker on a box that wasnt.
Sue them.
Or better yet: they could demand all devices conform to a set standard and then produce drivers for standard hardware only.
There is no reason for printers to all have different ways to talk to the OS. Same goes for scanners. This could all be standardized.
This all exists already in the windows ecosystem.
There are highly generic printer drivers that work with a substantial chunk of printers in existence, but has no extra features like duplex printing, quality levels, etc. And thats NOT include PS print drivers.
If a company who makes a scanner just wants minimal functionality for a scanner, they can also just ship twain drivers (or whatever the current version is) and force the end-user to use the MS scanning software that ships with windows.
You can get a generic laserjet driver that has no GUI, no fancy features from HP that works with nearly every laserjet printer they have ever made. But only businesses use it.
Hardware makers use the 'helper software' as a way to differentiate their products.
It's unfortunate, but makes a lot of sense from the HW maker point of view. They want people to KNOW that they're using a Lexmark scanner. Plus they often provide value-added features, like scanning multi-page documents automatically (using the sheet feeder) and automatically producing a PDF at the end. You dont get this with generic TWAIN drivers.
In other words, this isnt a Microsoft problem, or a windows problem. This is a business problem.
FWIW, driver signing doesnt cost anything that you pay towards Microsoft. And for a business its trivial, its a couple hundred bucks a year that you pay to Verisign or similar. You dont pay anything to Microsoft in any way.
Now it DOES cost you to send your stuff to the compatibility labs and get the higher level of signage you can use, but thats also paid out to a third party company who actually does the work. No one (other than that third party company) is making any money there.
I still think microsoft should have said no to backwards compatibility for apps as well as drivers. New OS, new apps and drivers for that OS.
Yeah, thats pretty much exactly what MS did with Vista. Thats why nearly every hardware manufacturer had to put out new drivers and did such a crappy job of it for so long.
That is like 60% of the complaints about Vista, is that tons of hardware wouldnt work right away with it until/unless manufacturers release new drivers.
Good. Maybe I can buy Window NT 7 and skip over Vista (NT6) completely.
Be aware that Windows 7 is a minor release from Vista, which is why its not NT7, its NT6.1.
It's a polish, speed, and stability release.
This is actually a good thing though, as by now the ecosystem (ie, hardware, software and drivers) has largely matured, and will be even more so then.
Your buddy/brother/whatever that had a very slow Vista box .... it was likely the manufacturer's fault that it was as slow as it was. It took manufacturers a _long_ time to ship machines that were reasonably stable in Vista, due to so much of their trialware, drivers, and OOBE stuff being just flat not stable under Vista.
But right now, if you (for example) go to HP and buy a nice business class engineering laptop (Compaq 8510w or similar), it ships with clean & shiny Vista AND Vista x64 drivers, and has very very minimal crap from HP, and generally runs quite well.
You do realize that you can login while you are responding, right?
On the same page/form even.
You dont _have_ use the login form on the sidebar, you can just hit Reply, pen your response, then put your user/pass in the fields and hit Preview to log you in.
I sure hope it's more than something like 32 gigs. If you can support an insanely high number, do it.
In the 64-bit version, yes. IIRC its 32GB for the desktop versions.
As an aside, is there any way to increase the 4GB RAM limit in XP Pro without upgrading to 64 bit edition?
No. MS had a version of XP that allowed you to flip PAE on, but that was like XP sp1 I believe, so only a brief period.
The problem is that a large percentage of consumer-level drivers puke and die if they encounter a PAE situation. This (reportedly) caused such a support nightmare for MS that they just shut off the ability to use PAE in 32-bit windows.
Mind you, even XPx64 allows a relatively large amount of memory, something like 32- or 64GB.
The problem is that 'securing windows' consists 80% of making sure its patched to current, and another 15% not having the users run as admin.
If you're not patching, you're ignoring the single biggest thing you can do to protect your systems, not to mention the easiest.
All that this has to do is be incorporated into a drive-by-install that targets IE, and if any of your users trip over a website that contains this, then it'll eat everything on your LAN in a few seconds.
Frankly, not patching for 2-3 months is possibly one of the worst possible things you can do.
You'd be better off patching quickly, and not running AV or firewalls.
The problem is when a worm for this exploit starts being incorporated into drive-by-download attacks into IE.
So someone browses to a site, IE promptly downloads and runs the worm behind the scenes, in about 4 seconds every machine on the LAN has been compromised.
If you're still using XP in 2014 when they stop issuing security patches, then the problem may be on your end.
There is not a mechanism for this to happen in windows unless you (or your sa's) specifically configured it to do so.
So this wasnt MS doing anything to you ... this is you setting something to happen and forgetting that you did so, or your SA's setting it to happen.
Buy from a different manufacturer.
All of our systems from HP (Compaq corporate level stuff, not consumer garbage) come with 5 discs:
XP 32-bit recovery
XP drivers disc
Vista 32-bit install
Vista 64-bit install
Vista drivers disc
The XP disc isnt a real windows disc, its a recovery disc, but the Vista ones are real OS discs.
Did you not read the article? They _are_ working together, extensively.
Most of the article was about just that, how supportive and helpful MS is nowadays for the Samba team.
NFS is a relatively small feature subset of AD+SMB/CIFS+MSRPC, which is what Samba is replacing.
It's not a reasonable comparison.
Unless they're making it easy for people to replace Windows AD servers with Samba servers running on Linux, this is not a big deal.
Did you not read the article you're posting about?
The bulk of the article is about precisely that. He's the lead dev on samba building an AD server, and talking about the wonderful level of support ms is (now) giving them.
He specifically mentioned that since they're the only ones working on an AD server replacement, at one of the 'plug fests' at MS' redmond campus, they were the only ones there to take advantage of it.
In addition, he mentions that the team on samba for building an AD server is short on developers, and is asking for help.
You mean like this phrase:
Disable the Server and Computer Browser services
In the section titled: "workarounds".
Yeah, it would be great if they would share that with us.
You make it a regular practice to shut down the Server and Computer Browser service?
The only place I can even imagine this working is in homes with only 1 PC. Any other situation, and thats going to be a real pain.
Without these services you cant share files on the networks, cant share printers, and cant browse the local network for CIFS/SMB stuff.
Many things expect the server service to be running.
To achieve the same storage size with smaller disks (comparing like with like) then you need more of them, and mathematically, there's more chance that more of them will start going at any one time the more you have, and during a rebuild procedure. Your redundancy will need to increase exponentially with that the more drives you have, as your storage requirements increase - and everyone's has over the past few years.
What you describe isnt how it works in the real world though. You just dont keep adding drives to a single array, you build pools of storage based on redundant arrays in the sweet spot size (ie, 5-7 spindles, in my experience). So no, the risk of drive failure does _not_ keep going up. The risk is at a 'shelf' level, and thats the level you build redundancy into. Then you pool multiple shelves together.
This is how it actually works in the real world in many, many systems. And works well, and highly reliably.
Also, in that situation, the redundancy does NOT grow exponentially. It grows linearly within ranges, and then you have a discontiguous cost jump as you move to a bigger class of storage. Then it grows linearly within that group again.
I don't trust hot spares because having a drive sitting around doing nothing that you won't know anything about until you have a failure is not a great idea.
Thats why you have patrols/scrubbing/whatever. The drives are constantly being tested and beat on, even if they're not being used. Including the hot spares.
The problem is, to get an equivalent storage size (comparing like with like) you will need to use more disks, statistically there is more chance of more drives going through failures and you will spend just as much time, if not more, rebuilding the array (or multiple arrays). The array might take less time to rebuild as a one-off task, even allowing for the fact that you have more disks to rebuild (increasing wear and tear incidentally), but you will do it more often and spend more time rebuilding overall. That has only got worse as array sizes have increased.
In actuality, since you have your redundancy in a storage unit that is sized right for your situation, it deosnt actually work out as you describe. Yes, you may end up replacing more disks as your storage size grows, but you lose disks at a fairly predictable rate within a redundant unit of storage, and the likelihood of losing 2 disks in one unit is very small, due to small arrays, small disks.
Mind you, expensive SCSI and SAS drives are only for the gullible and I have seen no evidence that they are more reliable. The storage size of a drive doesn't correlate well to the reliability of the drive. A 150GB drive is not inherently more reliable than a 1TB drive, other than you have perhaps slightly more chance with more storage space of seeing a failure.
There are differences. Less so now than there were in the IDE/PATA vs. SCSI days.
There are MANY consumer class drives that are literally not rated to be run 24x7. The designer says so clearly on the box (ie, the ibm death-stars).
The differences are less now, but in the past the cheaper consumer drives used a different kind of bearing that didnt hold up so well.
The biggest thing you're paying for typically in consumer-class vs. enterprise class is the amount of circuitry/smarts on the drive. Again, in the SCSI/IDE days, this was very distinct. SCSI drives had all sorts of smarts, could do re-ordering and optimizations via TCQ, etc. IDE drives were just dumb slaves for the controller, as a comparison.
Nowadays, even some SATA drives have NCQ, which can be quite useful in certain kinds of workloads.
But heck, just pick up an enterprise class drive in one hand, and a typical consumer drive in another. The former will weigh several times the latter, as it contains much more metal. Built to tighter tolerances, better heat dissipation due to the she
I completely agree, and talked about that a bit more in my response to segedunum above.
So yes, agreed, the way to make bigger storage available is to do as you did and keep the drives reasonably sized, use more spindles, and make arrays built of other smaller raid arrays (ie, just like you said).
Variations of what you describe is how you get it from NetApp and many SAN vendors as well. Pools are built from shelves which each have their own internal redundancy, and in some systems you also have fully hot-spare shelves available too.
Anyway, I'm completely agreeing with you.
It's the total storage that is being talked about in the article, not the size of the disks, so no, the article doesn't say that. The problem is not the size of drives but the storage requirements that exist today. Regardless of how many drives you have then you are very, very likely to have a failure somewhere due to the amount of data that is being stored.
I'm not sure how you get that from the article.
The core issue that its talking about (which is old news) is the ratio between rebuild time in a disk failure and error rates.
With very large disks, even with a very low error rate, if you read the entire disk worth of content, you're bound to hit an error (statistically). The total number of spindles also increases failure rate, but that doesnt matter. If you keep your spindles reasonable sized, then your rebuild time for a single disk failure is low, and your risk of a two-drive failure is low.
The article's core argument isnt about having errors, its about the likelihood of getting a second drive failure while the array is rebuilding as a result of the first. Mathematically, this has absolutely nothing to do with total storage size, and ONLY has to do with the spindle size of the failed drive.
However, if you are storing terabyte upon terabyte of data with smaller disks then a not insubstantial rebuild time is going to be seen there as well and you've got a greater chance of it happening with more disks.
How exactly do you get this? Rebuild time has zero to do with total storage size and everything to do with the size of the failed disk (and the controller/bus/drive speed, but we assume thats equal).
It's a simple function of the failed disk size, the variable of total storage size isnt even part of the equation.
No, that's not the reason at all. The reason is because such drives are more expensive to produce, have much higher spindle speeds and need greater performance.
That is absolutely the reason. Even if Dell or HP offered servers with 1TB SAS drives, I would never buy them that way. It's a newb mistake to build important raid arrays out of drives that size, due to the rebuild time and overall performance. In a raid array you get performance out of additional spindles and being able to serve multiple simultaneous read requests off of that array (due to some reads only having to hit one or two disks).
If you fill it with 1TB or 2TB drives, then not only will your rebuild time be unacceptably high, but you lose performance gains, as your contention against the same spindle for a set of requests goes up.
This dramatically reduces the kinds of storage you can have. If you have terabytes of data kicking around then you are going to need a lot more drives here, and as such, the chances of you encountering a failure go up as you increase the number of drives to achieve the storage requirement you have.
Which is why when you need really large storage, you build pools of volumes made out of multiple arrays.
There's a reason why all your high end nas and san vendors build their systems this way. They're made up of reasonable sized disks, grouped in redundant arrays of 5-7 (mostly), and then these groups are all pooled together.
Its done this way not only because that makes for more expensive products, but because it works! It's the practical way to build these things.
What we need is a more reliable storage technology within disk drives at an affordable price because that's where the problem is. The biggest candidate for that are SSDs.
I do agree that we need something better, but I would say new kinds of storage software is the way of it, like what ZFS has done, and what google does with their massively redundant storage. SSD's may help (though I'm not sure how their real-world failure rate compares to spinning disks), but I think these sorts of new approaches will be the important jump.
Yes, good catch. Thats exactly what I meant.
You seem to misunderstand the article. They are saying that if you need 12T of storage RAID 5 is not reliable. You would be better off with a single 12T disk if such a thing existed.
Thats not what the article says at all.
The article says that if you build your RAID arrays from the biggest disks available (which no one with half a brain does) like 1-3TB drives, and you have them filled, then the numbers come out as presented.
But there's a reason why no one on the planet builds important raid arrays out of 1TB drives. Rebuild time is too long.
This is also one of the big reasons why you see so many 73GB and 140GB SAS/SATA drives in raid arrays, and why server storage drives dont grow anything like as fast as consumer garbage drives.