Is this supposed to be some logical follow up to a real question?
Errr, yer. Most businesses don't stay away at night thinking about spindle size, and managing those 48 disks is going to more than outweigh the benefits.
People have been buying EMC, NetAPP, HP and IBM's expensive hardware for years as well and in higher volumes than those that do homebrew solutions. What is your point there?
Sun isn't making it cheaper and is flying right over the heads of small businesses.
Sun's strategy: Over engineer the fuck out of a machine where people won't use most of what is in it, and use that as justification to charge an exorbitant price.
No, but Sun is with this overpriced bucket of bolts, right back to where we were with them a few years ago. They just don't get it.
For most companies, even "enterprises", hardware reliability gives you a better bang for the buck, because you can't bloody afford multiple data centers.
I wish people would stop using that stupid phrase 'bang for the buck'. Repeating it isn't going to make this value for money, as it's all about Sun working out ways of charging what they have always charged and pretending the last ten years never happened. This provides barely a whimper, let along a bang. The vast majority, and the people who Sun say they are targeting with this, don't need to spend money on this or on 'data centres' although when you get beyond a certain size you will need a data centre regardless. You're going to need a data centre to store these damn things when you have them though.
Sun targets a *slightly* different market with their device (think: databases, mission critical, pink slips).
Which is non-existent. Sun is trying to target small business with this which is laughable, and many running the sort of workloads you described switched to commodity hardware and storage years ago. To assume that they will then switch back is even more funny, but then, Sun is always game for a laugh.
It's not like you can just grab 3 1TB SATA drives, throw them into RAID-5 and say that you've got 2TB of production ready storage. Well, you can, but you'd be an idiot.
That's exactly what Google and many others do, and they spend their money, and significantly less than this, on managing that storage effectively. It works. When it boils down to it, you can have all the exorbitantly expensive and brilliant 'enterprise ready' tools you want but the bottom line is you need redundancy - and that's pretty much it.
Your "home brew" solution will not meet any of the objectives Sun are achieving with this product.
Sun say they are targeting small businesses, and they have lost already with this poor showing. They have advanced no further than when they stiffed all the Cobalt Cube customers and withdrew the product, who then went out and bought Windows SBS servers;-). If you think people are going to jack them in for this then you need a stiff drink.
Your spindle count will suck, so concurrent access will be slow.
Ahhh, shit. I'm heart broken. What I'd like to know is how a small business will handle a behemoth like that, how they'll fund the electricity for all those drives and who'll manage it all. I expect that will be an ongoing cost to Sun support;-).
Keep to building crappy 3 or 4 disk RAID-5 systems using extremely large drives for storing your music, movies and pr0n on, but don't ever ever ever ever think about using those in any situation where your financial livelihood depends on that data.
I have news for you. People have been doing it for years, and the reason why Sun's business has gone down the toilet to commodity servers, Linux and Windows, especially with small businesses, for the past ten years is exactly for this reason.
Sun need to stop pretending that they can package up some commodity shit with some features very, very, very, very, very few need (and is waaaaaaaaaaaaay outside their target market) and label it as 'enterprise ready', which they think justifies an exorbitant price tag and ongoing support. They lost with this strategy with x86 and Solaris where they tried to protect SPARC, they lost with the exodus from SPARC after the dot com boom and they will keep on losing.
The Great Train Robbery was a great book turned into a marvellous film. Truly excellent. He was able to step effortlessly outside of the usual Sci-Fi and tech writing.
I feel for you. Man, those Christian fundamentalists have really screwed up conservative values. I'm a Christian myself, and I think we can all agree that some spiritual grounding and much of the stuff in the Bible are good things.
The problem I have with the Christian Conservatives is they display little in the way of Christian understanding and compassion, and the way they literally interpret the Bible and think they are good Christians scares the shit out of me. Dare I say it, they sound just as bad, if not worse, than the Islamic fundamentalists they rail against.
He. While you're on the subject of the constitution, there is nothing in there giving people the right to carry arms nor to go out into the middle of nowhere and blow away some innocent animals. It talks about an armed and controlled militia and nothing else.
Additionally, the constitution states that no one apart from Congress shall be responsible for minting coins and dictating national monetary policy. For some reason, you have a Federal Reserve of private banks that prints its own notes and does what it wants that got you into this mess in the first place, and why you have a $10 trillion national debt - because they're the ones lending the money!
But, you know, you just interpret the constitution the way you want. The constitution does not say "Do whatever you want and don't let the government interfere". It lays out some ground rules as to where it is sensible for the government to have a social conscience to allow things to move along. It's been correct for a couple of hundred years that anything involving bankers and finance can't be trusted and the country needed to be protected. That's why the constitution was written and why it has been ignored so often.
What has lost the debate on the war in Iraq for the Republicans is Palin's stupid insistence that the whole 'surge' has been a success. It hasn't and it is storing up a massive amount of trouble. The war lords and local leaders who control various communities have been bought off with troops handing out grants and other incentives, and so violence has lessened. What happens when that money runs out? They will fight for control all over again. It makes a withdrawal close to impossible as everyone in the region fights to fill the vacuum.
I don't understand people who think that if they encrypt something it automatically becomes secure. For that data to be of any use to someone it will need to be decrypted and relevant people given access, so that destroys the notion of defacto encryption for security right there.
Encryption assumes that bad people are going to get access to your data whatever happens, and if you are using whole disk encryption then you really need to be seriously asking yourself who has physical access to your disks and where your data is located. That needs to be sorted out first, and once it is with data held centrally, I doubt whether disk encryption will be needed. You will probably need some form of encryption between the data and the remote users though. Using full disk encryption gives you something else to go wrong, is a variable in performance impairment you probably can't account, is something else to support for and will almost certainly be unnecessary once you've taken other steps first.
If you're keeping confidential patient information where it would be a Bad Thing(tm) if it ever got mislaid (even if it is encrypted, you don't want a computer with stuff on it lost I assume), in the name of all that is holy, please centralise your data and vet access. Stop people from passing around Excel spreadsheets of data, regardless of when and how it is encrypted.
I really am aghast as to how stupid people are about how and where their data needs to be protected. PGP is the wrong solution here, if you can call it a solution.
Defining and checking for screen scraping is a very grey area because browsers are effectively screen scrapers. If you make a web site publicly available then you must accept public access, unless the load is excessive.
What I worry about is this:
Part requires scraping most of the site, parsing the data and presenting it as our own without human intervention.
I can't see you being able to do that without getting into hot water later. Even if copyright is OK it is still dodgy.
The other is, for lack of better words, a "load balancing" part that requires using multiple free accounts instead of purchasing space and CPU time for less than $2,000 USD per month. The boss sees it as "distributed" computing when in reality it's "parasitic".
Your boss is a fucking idiot. Pay the money. It will all be blamed on you when it is inevitably unavailable one of these days.
My question is am I wrong about the ethics? If I do need to walk how best can I handle it without damaging my reputation and future employment opportunities?
The fact that you are phrasing that in terms of stepping non eggshells tells me that you need to be looking for another job and reducing your stress. These ideas that you have presented from your boss are s-t-u-p-i-d, and they will almost certainly come back and bite you and not him. I'm afraid you'll have to be honest and blunt.
Not only have they not tried to sink Mono with patent lawsuits. I can't think of ANYONE they've EVER attacked with patents.
It hasn't stopped Ballmer and others at Microsoft talking about it incessantly, and it hasn't stopped them and won't stop them grabbing patents on fundamental.Net technology and telling everyone they need a patent grant. Most of Microsoft's patents thus far quite cleverly tell you that it applies to anything running within a CLR at the top, which means that they are not the general things everyone tells you they are when this is brought up. The only reason why the ECMA stuff is made available under RAND terms is because the ECMA requires it, but there is no guarantee that things will stay that way at all.
That's the state of play. Once.Net has reached critical mass they have the luxury of killing off any compatibility through technological changes, telling everyone that they now need a license grant, and if push comes to shove, patents that apply to.Net technology and nothing else if it needs to be enforced. Microsoft does not want people using their technology unless money ends up flowing into their coffers, and Microsoft employees on more than one occasion have called this a give-get scenario, where you give now and get more later. I am aghast that people haven't grokked that yet.
I'm tired of hearing this lame FUD scenario from the Slashdot crowd every time MS dabbles in open source.
Heh. Regardless of what the Slashdot crowd says, I don't know if you've noticed but Microsoft has had a certain degree of scepticism, and at times, outright hostility to this whole open source thing from their own lips. Are you seriously trying to tell me that you haven't noticed that yet and are you seriously wondering why people are sceptical right back?
The article's core argument isnt about having errors, its about the likelihood of getting a second drive failure while the array is rebuilding as a result of the first. Mathematically, this has absolutely nothing to do with total storage size
To achieve the same storage size with smaller disks (comparing like with like) then you need more of them, and mathematically, there's more chance that more of them will start going at any one time the more you have, and during a rebuild procedure. Your redundancy will need to increase exponentially with that the more drives you have, as your storage requirements increase - and everyone's has over the past few years. There is a point at which that, in these multi-terabyte RAID array times (which is why the article talks about 1TB disks), becomes counter-productive unless you start using larger drives.
Ultimately, that is why the article talks about 1TB drives and up, but I think the example of using a 7 or 8 drive RAID 5 array is, quite frankly, stupid. Even in a 4 disk array you would want at least RAID 6 or some form of RAID 5EE with a decent hot space. I don't trust hot spares because having a drive sitting around doing nothing that you won't know anything about until you have a failure is not a great idea.
How exactly do you get this? Rebuild time has zero to do with total storage size and everything to do with the size of the failed disk (and the controller/bus/drive speed, but we assume thats equal).
The problem is, to get an equivalent storage size (comparing like with like) you will need to use more disks, statistically there is more chance of more drives going through failures and you will spend just as much time, if not more, rebuilding the array (or multiple arrays). The array might take less time to rebuild as a one-off task, even allowing for the fact that you have more disks to rebuild (increasing wear and tear incidentally), but you will do it more often and spend more time rebuilding overall. That has only got worse as array sizes have increased.
That is absolutely the reason. Even if Dell or HP offered servers with 1TB SAS drives, I would never buy them that way. It's a newb mistake to build important raid arrays out of drives that size, due to the rebuild time and overall performance.
You buy SAS and SCSI drives of that size for performance and reliability, and you buy them for a specific workload that matches that. The trade-off for that performance and reliability is that you spend money on that rather than the storage size. Mind you, expensive SCSI and SAS drives are only for the gullible and I have seen no evidence that they are more reliable. They could end up making a 1TB drive of that quality, but it would be colossally expensive and no one would buy it. The storage size of a drive doesn't correlate well to the reliability of the drive. A 150GB drive is not inherently more reliable than a 1TB drive, other than you have perhaps slightly more chance with more storage space of seeing a failure. The more storage you have and the more disks you have then you have an even greater chance of seeing multiple failures.
Personally, I find that 'enterprise' SCSI and SAS drives are simply a rip-off for the gullible who are willing to pay the premium, get less storage and have to buy more drives and build more arrays to get an equivalent level of storage. Google certainly doesn't use them. You're buying snake oil basically.
In a raid array you get performance out of additional spindles and being able to serve multiple simultaneous read requests off of that array (due to some reads only having to hit one or two disks).
Indeed you do, but in a much larger array more disks (and more arrays) become counter productive from a management point of view. What you need are larger drives.
Which is why when you need really large storage, you build pool
The article says that if you build your RAID arrays from the biggest disks available (which no one with half a brain does) like 1-3TB drives, and you have them filled, then the numbers come out as presented.
It's the total storage that is being talked about in the article, not the size of the disks, so no, the article doesn't say that. The problem is not the size of drives but the storage requirements that exist today. Regardless of how many drives you have then you are very, very likely to have a failure somewhere due to the amount of data that is being stored. In reality, if you have a few 1TB disks versus several smaller drives then it really is swings and roundabouts whether you are more at risk from having a failure because of having more drives. It's not inconceivable that you are at greater risk.
You can't get away from this by having smaller drives in larger quantities. It's the storage requirements that are now a problem.
But there's a reason why no one on the planet builds important raid arrays out of 1TB drives. Rebuild time is too long.
RAID arrays with lots and lots and lots of disks, yes - just about. Rebuild time is too long. However, RAID arrays with a few drives can get away with it comfortably and mitigate some risk with 5EE or 6. However, if you are storing terabyte upon terabyte of data with smaller disks then a not insubstantial rebuild time is going to be seen there as well and you've got a greater chance of it happening with more disks.
This is also one of the big reasons why you see so many 73GB and 140GB SAS/SATA drives in raid arrays
No, that's not the reason at all. The reason is because such drives are more expensive to produce, have much higher spindle speeds and need greater performance. This dramatically reduces the kinds of storage you can have. If you have terabytes of data kicking around then you are going to need a lot more drives here, and as such, the chances of you encountering a failure go up as you increase the number of drives to achieve the storage requirement you have.
Either way, storage requirements today are increasing the chances of failure quite a bit regardless of whether you make up your multi-terabyte storage from fewer larger disks or more smaller disks. What we need is a more reliable storage technology within disk drives at an affordable price because that's where the problem is. The biggest candidate for that are SSDs.
ZFS has checksums and will find errors, but only will be able to self-heal the errors in a redundant configuration. On a single disk, ZFS will find the error thanks to checksums but will not be able to recover your data. Since ZFS was mainly designed for systems that will use redundant configurations, it may have sense there, but desktops are not never going to do such things.
I find this checksumming and self-healing interesting, but the real question is what do you actually do to really solve it? With ZFS, an awful lot of people over at OpenSolaris get excited about detecting 'bit rot', but answers are a bit thin on the ground when you ask what can be done about it or what some of the errors actually mean. Yer, you're a bit less likely to get data loss, but you can only really avoid that if you have redundancy. Also, most of the problems ZFS has detected that I have seen have, at a best guess, probably been caused by a Solaris device driver doing something no one had known about. The filesystem can't help you there, no matter how advanced it is.
The problem is our current storage technology, and more needs to be done where the problems occur - within disk drives themselves. I'm hoping SSDs will end up giving us a better fundamental starting point when it comes to storage.
Hmmmmmm. I'm just wondering why Sun didn't manage to do the dual licensing thing for ZFS and Solaris that they did for Java;-). That would really have meant no license issues whatsoever.
ZFS is present in both Mac OSX and FreeBSD, thank you! They have no license issues whatsoever.
Good for them (if indeed Apple manages to actually use ZFS). Unfortunately, they get far fewer contributions of source code, and there'll be even less, if any, flowing into OpenSolaris;-).
And if we ask people which is better monolithic or microkernel, they reply microkernel.
Do they?
But ZFS using FUSE would be like a microkernel driver, so which is it?
Running a next generation Linux filesystem purely from userspace is simply not an option. The obvious performance issues kill it for one thing, not to mention integration with VFS, RAID and LVM.
Solaris is nicer than Linux in a lot of ways. dtrace for example. Better performance under high loads. Better throughput. Linux was designed for consumer level x86 hardware and it's improved a lot since then (and will continue to do so) but Solaris kicks its ass on beefed up hardware.
This is the problem I have had with Sun, and Sun consultants trying to talk to me, for the past ten years. You have refuted nothing of what was written. The answer is yes, you will need to recompile, and support for the compiler tools and runtime environment, as well as the applications, will need to be good enough otherwise this doesn't mean all that much. Applications are everything, which is where OpenSolaris is playing major catch-up.
People are not going to run Solaris on a zSeries for Dtrace or even ZFS, no matter how much people start jumping up and down and now matter how much Sun people look at you in disbelief that you might have other priorities. Your comment smacksof everything that is wrong with Sun for the best decade - the disbelief that you could run anything on a tin-pot consumer level OS and kernel like Linux, disbelief that anyone would not run a real Unix like Solaris or real hardware like SPARC and the mythical notion that although Linux might have claimed this area Solaris runs better on some undefined beefed up hardware.
Linux runs on everything from consumer level hardware like x86 (the same hardware that has ate SPARC's breakfast for about eight years incidentally) right up to the very same mainframe hardware that Solaris has only now been ported to.
Better under high loads and better throughput? Unsubstantiated and backed up with nothing, and it's the same old story Sun's consultants have been talking to a brick wall about since the turn of the century.
I would laugh if it wasn't so sad. The truth is, Sun has everything it needs to make as much money as it wants if it would only ditch this cultural nonsense about Solaris and SPARC that pervades it.
Of course, in the same discussion people don't bother to explain why the same argument also holds for SPARC CPUs.
Decent analysis, apart from this one tiny flaw. SPARCs have traditionally competed in areas where there is a lot of crossover. Indeed, a lot of SPARC's business post-2000 has been eaten for breakfast by x86, and x86 running on Linux. This is why Sun has suffered more than most at the hands of x86 and Linux. SPARC simply cannot keep up with x86 for performance, and although PowerPCs and other processors have backed themselves into a decent enough niche that you have adequately explained, the clock is ticking. Why? Because there is nowhere else for those processors and hardware platforms to go. They're reliable enough in lots of areas that really matter, but they aren't going to improve elsewhere at anywhere near the same rate.
You said they are giving a $700b blank check to Henry Paulson. Well...If there's a name and an amount, then what's blank on the check?
It is blank. What people don't realise is that this is a $700 billion credit facility - i.e. $700 billion at any one time, but potentially trillions can wash through this.
I'm not entirely sure how talking about alien conspiracies can be equated to what the OP posted on actual earth-based historical events, such as the obvious scaremongering that has gone on, the somewhat dodgy Suez events of the fifties, and the fact that America's national debt is largely backed and borrowed from China these days.
Oh, yer. I get it. It's an attempt to discredit what was said by equating it with outlandish and unrelated conspiracy theories.
Slashdot Mirror
Errr, yer. Most businesses don't stay away at night thinking about spindle size, and managing those 48 disks is going to more than outweigh the benefits.
Sun isn't making it cheaper and is flying right over the heads of small businesses.
Sun's strategy: Over engineer the fuck out of a machine where people won't use most of what is in it, and use that as justification to charge an exorbitant price.
No, but Sun is with this overpriced bucket of bolts, right back to where we were with them a few years ago. They just don't get it.
I wish people would stop using that stupid phrase 'bang for the buck'. Repeating it isn't going to make this value for money, as it's all about Sun working out ways of charging what they have always charged and pretending the last ten years never happened. This provides barely a whimper, let along a bang. The vast majority, and the people who Sun say they are targeting with this, don't need to spend money on this or on 'data centres' although when you get beyond a certain size you will need a data centre regardless. You're going to need a data centre to store these damn things when you have them though.
Which is non-existent. Sun is trying to target small business with this which is laughable, and many running the sort of workloads you described switched to commodity hardware and storage years ago. To assume that they will then switch back is even more funny, but then, Sun is always game for a laugh.
That's exactly what Google and many others do, and they spend their money, and significantly less than this, on managing that storage effectively. It works. When it boils down to it, you can have all the exorbitantly expensive and brilliant 'enterprise ready' tools you want but the bottom line is you need redundancy - and that's pretty much it.
Sun say they are targeting small businesses, and they have lost already with this poor showing. They have advanced no further than when they stiffed all the Cobalt Cube customers and withdrew the product, who then went out and bought Windows SBS servers ;-). If you think people are going to jack them in for this then you need a stiff drink.
Ahhh, shit. I'm heart broken. What I'd like to know is how a small business will handle a behemoth like that, how they'll fund the electricity for all those drives and who'll manage it all. I expect that will be an ongoing cost to Sun support ;-).
I have news for you. People have been doing it for years, and the reason why Sun's business has gone down the toilet to commodity servers, Linux and Windows, especially with small businesses, for the past ten years is exactly for this reason.
Sun need to stop pretending that they can package up some commodity shit with some features very, very, very, very, very few need (and is waaaaaaaaaaaaay outside their target market) and label it as 'enterprise ready', which they think justifies an exorbitant price tag and ongoing support. They lost with this strategy with x86 and Solaris where they tried to protect SPARC, they lost with the exodus from SPARC after the dot com boom and they will keep on losing.
The Great Train Robbery was a great book turned into a marvellous film. Truly excellent. He was able to step effortlessly outside of the usual Sci-Fi and tech writing.
"Why did you commit such a despicable crime?"
"Because I wanted the money."
I feel for you. Man, those Christian fundamentalists have really screwed up conservative values. I'm a Christian myself, and I think we can all agree that some spiritual grounding and much of the stuff in the Bible are good things.
The problem I have with the Christian Conservatives is they display little in the way of Christian understanding and compassion, and the way they literally interpret the Bible and think they are good Christians scares the shit out of me. Dare I say it, they sound just as bad, if not worse, than the Islamic fundamentalists they rail against.
He. While you're on the subject of the constitution, there is nothing in there giving people the right to carry arms nor to go out into the middle of nowhere and blow away some innocent animals. It talks about an armed and controlled militia and nothing else.
Additionally, the constitution states that no one apart from Congress shall be responsible for minting coins and dictating national monetary policy. For some reason, you have a Federal Reserve of private banks that prints its own notes and does what it wants that got you into this mess in the first place, and why you have a $10 trillion national debt - because they're the ones lending the money!
But, you know, you just interpret the constitution the way you want. The constitution does not say "Do whatever you want and don't let the government interfere". It lays out some ground rules as to where it is sensible for the government to have a social conscience to allow things to move along. It's been correct for a couple of hundred years that anything involving bankers and finance can't be trusted and the country needed to be protected. That's why the constitution was written and why it has been ignored so often.
Well of course it's bang on target. They have a six month release cycle where they release come-what-may without a feature list.
What has lost the debate on the war in Iraq for the Republicans is Palin's stupid insistence that the whole 'surge' has been a success. It hasn't and it is storing up a massive amount of trouble. The war lords and local leaders who control various communities have been bought off with troops handing out grants and other incentives, and so violence has lessened. What happens when that money runs out? They will fight for control all over again. It makes a withdrawal close to impossible as everyone in the region fights to fill the vacuum.
I don't understand people who think that if they encrypt something it automatically becomes secure. For that data to be of any use to someone it will need to be decrypted and relevant people given access, so that destroys the notion of defacto encryption for security right there.
Encryption assumes that bad people are going to get access to your data whatever happens, and if you are using whole disk encryption then you really need to be seriously asking yourself who has physical access to your disks and where your data is located. That needs to be sorted out first, and once it is with data held centrally, I doubt whether disk encryption will be needed. You will probably need some form of encryption between the data and the remote users though. Using full disk encryption gives you something else to go wrong, is a variable in performance impairment you probably can't account, is something else to support for and will almost certainly be unnecessary once you've taken other steps first.
If you're keeping confidential patient information where it would be a Bad Thing(tm) if it ever got mislaid (even if it is encrypted, you don't want a computer with stuff on it lost I assume), in the name of all that is holy, please centralise your data and vet access. Stop people from passing around Excel spreadsheets of data, regardless of when and how it is encrypted.
I really am aghast as to how stupid people are about how and where their data needs to be protected. PGP is the wrong solution here, if you can call it a solution.
What I worry about is this:
I can't see you being able to do that without getting into hot water later. Even if copyright is OK it is still dodgy.
Your boss is a fucking idiot. Pay the money. It will all be blamed on you when it is inevitably unavailable one of these days.
The fact that you are phrasing that in terms of stepping non eggshells tells me that you need to be looking for another job and reducing your stress. These ideas that you have presented from your boss are s-t-u-p-i-d, and they will almost certainly come back and bite you and not him. I'm afraid you'll have to be honest and blunt.
It hasn't stopped Ballmer and others at Microsoft talking about it incessantly, and it hasn't stopped them and won't stop them grabbing patents on fundamental .Net technology and telling everyone they need a patent grant. Most of Microsoft's patents thus far quite cleverly tell you that it applies to anything running within a CLR at the top, which means that they are not the general things everyone tells you they are when this is brought up. The only reason why the ECMA stuff is made available under RAND terms is because the ECMA requires it, but there is no guarantee that things will stay that way at all.
.Net has reached critical mass they have the luxury of killing off any compatibility through technological changes, telling everyone that they now need a license grant, and if push comes to shove, patents that apply to .Net technology and nothing else if it needs to be enforced. Microsoft does not want people using their technology unless money ends up flowing into their coffers, and Microsoft employees on more than one occasion have called this a give-get scenario, where you give now and get more later. I am aghast that people haven't grokked that yet.
That's the state of play. Once
Heh. Regardless of what the Slashdot crowd says, I don't know if you've noticed but Microsoft has had a certain degree of scepticism, and at times, outright hostility to this whole open source thing from their own lips. Are you seriously trying to tell me that you haven't noticed that yet and are you seriously wondering why people are sceptical right back?
To achieve the same storage size with smaller disks (comparing like with like) then you need more of them, and mathematically, there's more chance that more of them will start going at any one time the more you have, and during a rebuild procedure. Your redundancy will need to increase exponentially with that the more drives you have, as your storage requirements increase - and everyone's has over the past few years. There is a point at which that, in these multi-terabyte RAID array times (which is why the article talks about 1TB disks), becomes counter-productive unless you start using larger drives.
Ultimately, that is why the article talks about 1TB drives and up, but I think the example of using a 7 or 8 drive RAID 5 array is, quite frankly, stupid. Even in a 4 disk array you would want at least RAID 6 or some form of RAID 5EE with a decent hot space. I don't trust hot spares because having a drive sitting around doing nothing that you won't know anything about until you have a failure is not a great idea.
The problem is, to get an equivalent storage size (comparing like with like) you will need to use more disks, statistically there is more chance of more drives going through failures and you will spend just as much time, if not more, rebuilding the array (or multiple arrays). The array might take less time to rebuild as a one-off task, even allowing for the fact that you have more disks to rebuild (increasing wear and tear incidentally), but you will do it more often and spend more time rebuilding overall. That has only got worse as array sizes have increased.
You buy SAS and SCSI drives of that size for performance and reliability, and you buy them for a specific workload that matches that. The trade-off for that performance and reliability is that you spend money on that rather than the storage size. Mind you, expensive SCSI and SAS drives are only for the gullible and I have seen no evidence that they are more reliable. They could end up making a 1TB drive of that quality, but it would be colossally expensive and no one would buy it. The storage size of a drive doesn't correlate well to the reliability of the drive. A 150GB drive is not inherently more reliable than a 1TB drive, other than you have perhaps slightly more chance with more storage space of seeing a failure. The more storage you have and the more disks you have then you have an even greater chance of seeing multiple failures.
Personally, I find that 'enterprise' SCSI and SAS drives are simply a rip-off for the gullible who are willing to pay the premium, get less storage and have to buy more drives and build more arrays to get an equivalent level of storage. Google certainly doesn't use them. You're buying snake oil basically.
Indeed you do, but in a much larger array more disks (and more arrays) become counter productive from a management point of view. What you need are larger drives.
It's the total storage that is being talked about in the article, not the size of the disks, so no, the article doesn't say that. The problem is not the size of drives but the storage requirements that exist today. Regardless of how many drives you have then you are very, very likely to have a failure somewhere due to the amount of data that is being stored. In reality, if you have a few 1TB disks versus several smaller drives then it really is swings and roundabouts whether you are more at risk from having a failure because of having more drives. It's not inconceivable that you are at greater risk.
You can't get away from this by having smaller drives in larger quantities. It's the storage requirements that are now a problem.
RAID arrays with lots and lots and lots of disks, yes - just about. Rebuild time is too long. However, RAID arrays with a few drives can get away with it comfortably and mitigate some risk with 5EE or 6. However, if you are storing terabyte upon terabyte of data with smaller disks then a not insubstantial rebuild time is going to be seen there as well and you've got a greater chance of it happening with more disks.
No, that's not the reason at all. The reason is because such drives are more expensive to produce, have much higher spindle speeds and need greater performance. This dramatically reduces the kinds of storage you can have. If you have terabytes of data kicking around then you are going to need a lot more drives here, and as such, the chances of you encountering a failure go up as you increase the number of drives to achieve the storage requirement you have.
Either way, storage requirements today are increasing the chances of failure quite a bit regardless of whether you make up your multi-terabyte storage from fewer larger disks or more smaller disks. What we need is a more reliable storage technology within disk drives at an affordable price because that's where the problem is. The biggest candidate for that are SSDs.
Just think. If they weren't paying all that money to Oracle, for not all that much it seems to me, then they could have paid you even more ;-).
I find this checksumming and self-healing interesting, but the real question is what do you actually do to really solve it? With ZFS, an awful lot of people over at OpenSolaris get excited about detecting 'bit rot', but answers are a bit thin on the ground when you ask what can be done about it or what some of the errors actually mean. Yer, you're a bit less likely to get data loss, but you can only really avoid that if you have redundancy. Also, most of the problems ZFS has detected that I have seen have, at a best guess, probably been caused by a Solaris device driver doing something no one had known about. The filesystem can't help you there, no matter how advanced it is.
The problem is our current storage technology, and more needs to be done where the problems occur - within disk drives themselves. I'm hoping SSDs will end up giving us a better fundamental starting point when it comes to storage.
Good for them (if indeed Apple manages to actually use ZFS). Unfortunately, they get far fewer contributions of source code, and there'll be even less, if any, flowing into OpenSolaris ;-).
Funny, but not the sort of thing that should be on a Wikipedia page describing filesystems.
Do they?
Running a next generation Linux filesystem purely from userspace is simply not an option. The obvious performance issues kill it for one thing, not to mention integration with VFS, RAID and LVM.
This is the problem I have had with Sun, and Sun consultants trying to talk to me, for the past ten years. You have refuted nothing of what was written. The answer is yes, you will need to recompile, and support for the compiler tools and runtime environment, as well as the applications, will need to be good enough otherwise this doesn't mean all that much. Applications are everything, which is where OpenSolaris is playing major catch-up.
People are not going to run Solaris on a zSeries for Dtrace or even ZFS, no matter how much people start jumping up and down and now matter how much Sun people look at you in disbelief that you might have other priorities. Your comment smacksof everything that is wrong with Sun for the best decade - the disbelief that you could run anything on a tin-pot consumer level OS and kernel like Linux, disbelief that anyone would not run a real Unix like Solaris or real hardware like SPARC and the mythical notion that although Linux might have claimed this area Solaris runs better on some undefined beefed up hardware.
I would laugh if it wasn't so sad. The truth is, Sun has everything it needs to make as much money as it wants if it would only ditch this cultural nonsense about Solaris and SPARC that pervades it.
Decent analysis, apart from this one tiny flaw. SPARCs have traditionally competed in areas where there is a lot of crossover. Indeed, a lot of SPARC's business post-2000 has been eaten for breakfast by x86, and x86 running on Linux. This is why Sun has suffered more than most at the hands of x86 and Linux. SPARC simply cannot keep up with x86 for performance, and although PowerPCs and other processors have backed themselves into a decent enough niche that you have adequately explained, the clock is ticking. Why? Because there is nowhere else for those processors and hardware platforms to go. They're reliable enough in lots of areas that really matter, but they aren't going to improve elsewhere at anywhere near the same rate.
It is blank. What people don't realise is that this is a $700 billion credit facility - i.e. $700 billion at any one time, but potentially trillions can wash through this.
I'm not entirely sure how talking about alien conspiracies can be equated to what the OP posted on actual earth-based historical events, such as the obvious scaremongering that has gone on, the somewhat dodgy Suez events of the fifties, and the fact that America's national debt is largely backed and borrowed from China these days.
Oh, yer. I get it. It's an attempt to discredit what was said by equating it with outlandish and unrelated conspiracy theories.
Hmmmm. More girl-on-girl action then?