I'm certain he meant "spayed". Nice to know he considers 3rd world people to be cats and dogs. Vasectomies would work fine if that were the right answer. I'd save neutering for the parent poster.
incidently, his calculation of 20% was off by a factor of 5x (should have been 4%). Sounds bad, but the server drive rating was lower not only because of the improved rating but because the drive was less than 40% the size! Lesson is only use small drives if you want reliability. Right;)
The guy has a vesting interest in producing FUD and has done so. Anyone believe that their drive is going to fail after 5 surface scans?
take the word of someone with a vested interest in selling you enterprise drives, but i stopped reading when he claimed a 20% likelihood of an unrecoverable read error on a single sweep. what bullshit!
Yes I did but only recently. It was dead earlier. If the claim is that hardware will solve that for you then I don't know. Any solution that mostly works is broken, especially for storage. Unfortunately not all agree:(
"ATA is not a perfectly fine protocol for block storage. It works, true. But it's far from optimal. Its methods of handling error signalling is poor. Its methods of handling delayed error signalling is very poor."
In what way? Are you confusing it with SCSI error handling and its contingent allegiance conditions? Talk about bullshit! Never, ever, EVER had a problem dealing with ATA error handling. Besides, that's all subject to reimplementation with a new physical layer.
"I think the idea of using ATA and all its cruft to save a couple pennies on a big-tyme storage system doesn't make sense to me. If I can buy an optical drive for $30 that has SCSI (ATAPI) in it, I think the benefits of SCSI can be affordably brought to a large storage system too."
Again, what cruft? SCSI has 1000x the cruft of ATA. Have you ever programmed either of these devices? The purpose of using ATA here is that the command set is ENTIRELY adequate for disks and it's much leaner than the bloated SCSI command set.
"Do you actually have an ATA RAID array that can perform 10 Gigabits/s full-duplex? I would love to see that, I really don't think those disks really exist though (maybe a couple or 10 10Krpm WD ones..:)"
I meant that it's hard to do 10GigE without a TOE. In rough terms 1GigE is 100MB/s and it's easy to swamp that with a modest number of disks. For servers you want 10GigE. Sorry for the confusion.
Regarding a dedicated port and switch, I agree that it's what you want to do in any case.
AoE forces nonroutability because they want the efficiency of of avoiding TCP, not for security reasons;) I would assume that the ethernet protocol would be efficient enough to not require an offload engine.
In this context, the OS has data to store and it wants to decide where to store it. Do you want the OS to have the ability to make that decision above the filesystem or do you want it to happen at the block level? Obviously, blocks are the wrong place.
Once you realize that, you understand that it would be better to have the part on the MB even though it could clearly be in the drive. The reason drive makers are lobbying the other way is that they get to sell and profit from it. Obviously having it in the drive allows for transparent retrofits into existing systems much like the USB drive does.
Same could be said to you. Good to see you've finally used standardized terminology for the first time. Who said you couldn't learn anything?
Since you didn't define system, you can't define, after the fact, what constitutes a "failure". It is commonly accepted that reliability refers to the unlikelihood of failure. When you buy a system or a component, that characteristic would be specified using MTBF and does not consider redundancy. In a redundant system, a different measure (MTDL) is used to indicate the likelihood of "failure" you refer to. Servicably is measured through a third term, MTTR.
"If you have a server with redundant PSUs, and one of them goes out, but the other one takes over completely without interruption, then one of the PSUs has failed. The server has absolutely not failed."
Depends on what you mean. The system remains available but there has been a failure and there is a degraded state. That's why additional language is valuable and exists. If I paid my money and the PSU died I'd be inclined to say the system failed even though that would be naiive.
Furthermore, with your definition the backup component doesn't even have to take over without interruption since it would only be a requirement if the system design dictated that it was. A PSU is a bad example of this, but automatic rollover isn't always implemented when propagating a failure upstream can suffice. When that is considered there may, in fact, be an interuption in availability but the "system" (cluster) remains functional. An extreme example of this is Google. They let failed nodes die in place and never get serviced or removed from the rack.
"If you have to down the server to replace the bad PSU because you can't hot-swap it, then the system has lesser availability than one in which you can hot-swap."
It means that the system is not designed for a nonstop application. How MTTR is calculated for a hotplug device I don't know, but if it's 0 then you are right. Percentage-wise, not many applications require no scheduled downtime.
"This is entirely orthogonal."
What is?
There is a common, decades-old language that describes these matters. If you were as knowledgable in these matters as you let on, one would think you would speak in those terms.
The dedicated controller and switch are exactly what it's intended for. Anything else is foolish. Servers SHOULD connect to their storage that way, not through a shared network.
Since when do you do offsite redundancy that way? Your link to the offsite location is slow relative to your local connection and you want your software to know that.
No way is AoE intended for the home NAS market (it isn't even NAS) but it may eventually find a use there.
Direct DMA (zero buffer) is hard to do, not easy. Yes, NIC's have DMA capability but that's not what I'm referring to. Ideally you would like a data request to be satisfied between the storage device and the data buffer itself, even if it's a buffer in the app, without any intervening copies. Currently data is copied many times. IB offers this. Current filesystems don't.
None of this is being done for the home user, so if you see this from that perspective you aren't really understanding it.
iSCSI, and now AoE, were done as competitors to FC and InfiniBand. The argument for ethernet there is obviously cost and ubiquity. Problem was that 10GigE would REQUIRE TOE's in order to get the throughput so the assumption that TOE's will be optional is just that. AoE is obviously intended to eliminate the requirement of TOE's. Supporters of IB, me for one, aren't terribly excited about using ethernet for the purpose. Lots of things will need reinventing and it sets back the industry years.
yes, removing routing takes away that option but a fully redundant connection doesn't really need that. what it needs is fully independant paths and multiple routes aren't needed there.
The holy grail of SAN's is unified SAN management and the number one complaint of SAN's is that management is too complicated and causes vendor lockin. No doubt that a centralized place for storage management is frequently desirable. If it weren't SANs would never exist in the first place.
"If a subsystem component fails in such a way that the overall system continues to function as designed, no system failure has occured."
Because you say so. If a system is "designed" to crash when it's hard drive fails then no system failure occurs then according to your definition.
You asked for the "relative system reliability" but never defined what that was. You still haven't, yet you declare your correctness without ever demonstating that you even understand common terminology.
From http://www.sun.com/products-n-solutions/hardware/d ocs/html/817-3881-11/ras.html:
"Reliability, availability, and serviceability (RAS) are aspects of a system's design that affect its ability to operate continuously and to minimize the time necessary to service the system. Reliability refers to a system's ability to operate continuously without failures and to maintain data integrity. System availability refers to the ability of a system to recover to an operational state after a failure, with minimal impact. Serviceability relates to the time it takes to restore a system to service following a system failure. Together, reliability, availability, and serviceability features provide for near continuous system operation."
Fact is that your original question cannot be answered because you never defined "system", never gave MTBF numbers for drives, never gave costs of SCSI or ATA drives, never offered configurations, and never provided MTBF numbers for common components. All you specified was $1000 in drives for either case. It appeared that you didn't know what you were talking about and it still does.
Now, if your point was that a more useful system can be constructed with ATA drives than SCSI for the same price then I might agree with you. Reliability differences between SCSI and ATA are mainly artificial and imagined anyway.
Drives should be classified as enterprise and non-enterprise, not ATA and SCSI. Protocol is meaningless. Once that is accepted, the original RAID paper makes your point well enough.
ATA is a perfectly fine protocol for block storage and is much leaner that SCSI. The SCSI protocol was used for ATAPI because it already existed and was needed to support a wide variety of devices besides disk. It makes no sense to put your imaginary "brains" back in.
I'm pretty confident that you can prevent unintended data corruption. TCP/IP manages it so there's your proof of concept:)
GigE is not a good choice for disk attachment since it is easily outrun by a small number of disks. 10GigE is where it's at.
Apple is not the end-all-be-all of server manufacturers.
I was just saying that I hadn't read the article since it's unavailable. I would assume that the problems with performance under 10GigE don't apply because TCP isn't used. I would also assume that the offload engines can't be taken advantage of with this protocol.
I could see benefit in using this over iSCSI for 10G NIC's without TOE's but I wonder if there will be any of those. There real advantage of running anything over ethernet is to get the benefit of huge volumes, so I would suspect all 10G Nics will have TOE's.
One of the problems of TCP is it's very hard to make go fast over 10G. All those issues become moot for AoE since it doesn't use that. GigE ethernet isn't really interesting for attaching large numbers of disks.
Trouble is that I'd assume all 10GigE NIC's will come with offload engines anyway so there's no savings.
There is no functional problem with making the product non-routable. Servers need physical security and physical proximity. What you seem to think is a liability is not one.
while I agree with you, the issue of multipath is moot. Any device with only one port has that port as a single point of failure. You solve that problem with two ports and redundant switches. It is no different for SAN's, iSCSI or FC.
SAN's management capability is also it's downfall. Expensive, complicated and vendor-specific.
I think you mean availability. Redundancy doesn't improve reliability, in fact it lessens it. What redundancy does is offer availability, the ability for a system to remain available after a failure.
The answer to your question is that, assuming there are more ATA drives of assumed lower reliability, the ATA system will be less reliable. You, sir, total fail it.
considering that it's non-routable, in what way is this "a great approach to networked storage"? Ethernet simply takes the place of a SCSI cable here and the device protocols differ. It lacks some of the availability characteristiccs of SATA/SAS and fails to solve and sharability issues because it's still a block interface. "From a comp sci point of view" it's a total dud.
The problem with ethernet is that it's hard to make go fast. We have 1G now but 10G is difficult because of all the processing involved and the offload engines that come with that. IF I could read the article perhaps I might understand if this is better in that respect, but the future of networked storage does not lie in block device protocols. To make this interesting we need 10G links and file symantics.
how is that relevant to the discussion of protocols?
reliability of SCSI versus ATA is largely imagined and the rest is intentional. drive manufacturers want you to believe their enterprise drives are more reliable and right now those drives are largely SCSI.
If that were realistic then you wouldn't need OS support at all. The drive is the worst location because it lies underneath the filesystem where much of the knowledge of what is being done is lost. Flash storage is additional nonvolatile storage that is made available to the operating system and hiding it in the drive is the worst place. If you wanted to do that you'd be better off using RAM and just enough battery back to ensure the data gets flushed.
Slashdot Mirror
I'm certain he meant "spayed". Nice to know he considers 3rd world people to be cats and dogs. Vasectomies would work fine if that were the right answer. I'd save neutering for the parent poster.
The world is "loosing" it's ability to spell...
I'd be interested to know why they didn't use UDP as well. I don't think routing was a priority or even desirable for them, right or wrong.
incidently, his calculation of 20% was off by a factor of 5x (should have been 4%). Sounds bad, but the server drive rating was lower not only because of the improved rating but because the drive was less than 40% the size! Lesson is only use small drives if you want reliability. Right ;)
The guy has a vesting interest in producing FUD and has done so. Anyone believe that their drive is going to fail after 5 surface scans?
take the word of someone with a vested interest in selling you enterprise drives, but i stopped reading when he claimed a 20% likelihood of an unrecoverable read error on a single sweep. what bullshit!
"Did you read the article?"
:(
Yes I did but only recently. It was dead earlier. If the claim is that hardware will solve that for you then I don't know. Any solution that mostly works is broken, especially for storage. Unfortunately not all agree
"ATA is not a perfectly fine protocol for block storage. It works, true. But it's far from optimal. Its methods of handling error signalling is poor. Its methods of handling delayed error signalling is very poor."
In what way? Are you confusing it with SCSI error handling and its contingent allegiance conditions? Talk about bullshit! Never, ever, EVER had a problem dealing with ATA error handling. Besides, that's all subject to reimplementation with a new physical layer.
"I think the idea of using ATA and all its cruft to save a couple pennies on a big-tyme storage system doesn't make sense to me. If I can buy an optical drive for $30 that has SCSI (ATAPI) in it, I think the benefits of SCSI can be affordably brought to a large storage system too."
Again, what cruft? SCSI has 1000x the cruft of ATA. Have you ever programmed either of these devices? The purpose of using ATA here is that the command set is ENTIRELY adequate for disks and it's much leaner than the bloated SCSI command set.
"Do you actually have an ATA RAID array that can perform 10 Gigabits/s full-duplex? I would love to see that, I really don't think those disks really exist though (maybe a couple or 10 10Krpm WD ones.. :)"
;) I would assume that the ethernet protocol would be efficient enough to not require an offload engine.
I meant that it's hard to do 10GigE without a TOE. In rough terms 1GigE is 100MB/s and it's easy to swamp that with a modest number of disks. For servers you want 10GigE. Sorry for the confusion.
Regarding a dedicated port and switch, I agree that it's what you want to do in any case.
AoE forces nonroutability because they want the efficiency of of avoiding TCP, not for security reasons
I agree though, not very interesting.
It does, but it's RAM not flash.
In this context, the OS has data to store and it wants to decide where to store it. Do you want the OS to have the ability to make that decision above the filesystem or do you want it to happen at the block level? Obviously, blocks are the wrong place.
Once you realize that, you understand that it would be better to have the part on the MB even though it could clearly be in the drive. The reason drive makers are lobbying the other way is that they get to sell and profit from it. Obviously having it in the drive allows for transparent retrofits into existing systems much like the USB drive does.
Same could be said to you. Good to see you've finally used standardized terminology for the first time. Who said you couldn't learn anything?
Since you didn't define system, you can't define, after the fact, what constitutes a "failure". It is commonly accepted that reliability refers to the unlikelihood of failure. When you buy a system or a component, that characteristic would be specified using MTBF and does not consider redundancy. In a redundant system, a different measure (MTDL) is used to indicate the likelihood of "failure" you refer to. Servicably is measured through a third term, MTTR.
"If you have a server with redundant PSUs, and one of them goes out, but the other one takes over completely without interruption, then one of the PSUs has failed. The server has absolutely not failed."
Depends on what you mean. The system remains available but there has been a failure and there is a degraded state. That's why additional language is valuable and exists. If I paid my money and the PSU died I'd be inclined to say the system failed even though that would be naiive.
Furthermore, with your definition the backup component doesn't even have to take over without interruption since it would only be a requirement if the system design dictated that it was. A PSU is a bad example of this, but automatic rollover isn't always implemented when propagating a failure upstream can suffice. When that is considered there may, in fact, be an interuption in availability but the "system" (cluster) remains functional. An extreme example of this is Google. They let failed nodes die in place and never get serviced or removed from the rack.
"If you have to down the server to replace the bad PSU because you can't hot-swap it, then the system has lesser availability than one in which you can hot-swap."
It means that the system is not designed for a nonstop application. How MTTR is calculated for a hotplug device I don't know, but if it's 0 then you are right. Percentage-wise, not many applications require no scheduled downtime.
"This is entirely orthogonal."
What is?
There is a common, decades-old language that describes these matters. If you were as knowledgable in these matters as you let on, one would think you would speak in those terms.
Another reference to reliability vs. availability: http://www.barringer1.com/ar.htm
This one tends to speak of reliability only in terms failure rates of components.
"Next time, either gracefully admit fault, or don't post in the first place."
Thanks for the advice and same to you, AC.
I agree there. Unfortunately, iSCSI fans believes it will. FC needs replacement and IB has had trouble getting traction since the market crash.
I would like to see a cheap, high performance NAS solution. TCP and SMB/NFS isn't it.
The dedicated controller and switch are exactly what it's intended for. Anything else is foolish. Servers SHOULD connect to their storage that way, not through a shared network.
Since when do you do offsite redundancy that way? Your link to the offsite location is slow relative to your local connection and you want your software to know that.
No way is AoE intended for the home NAS market (it isn't even NAS) but it may eventually find a use there.
the Newton was a PDA but the UMPC is not. Referring to the devices as PDA's shows a builtin bias.
ok, they may call it that but it's not the same as the cache in the drive. it's the write cache of the OS.
Direct DMA (zero buffer) is hard to do, not easy. Yes, NIC's have DMA capability but that's not what I'm referring to. Ideally you would like a data request to be satisfied between the storage device and the data buffer itself, even if it's a buffer in the app, without any intervening copies. Currently data is copied many times. IB offers this. Current filesystems don't.
None of this is being done for the home user, so if you see this from that perspective you aren't really understanding it.
iSCSI, and now AoE, were done as competitors to FC and InfiniBand. The argument for ethernet there is obviously cost and ubiquity. Problem was that 10GigE would REQUIRE TOE's in order to get the throughput so the assumption that TOE's will be optional is just that. AoE is obviously intended to eliminate the requirement of TOE's. Supporters of IB, me for one, aren't terribly excited about using ethernet for the purpose. Lots of things will need reinventing and it sets back the industry years.
yes, removing routing takes away that option but a fully redundant connection doesn't really need that. what it needs is fully independant paths and multiple routes aren't needed there.
The holy grail of SAN's is unified SAN management and the number one complaint of SAN's is that management is too complicated and causes vendor lockin. No doubt that a centralized place for storage management is frequently desirable. If it weren't SANs would never exist in the first place.
"If a subsystem component fails in such a way that the overall system continues to function as designed, no system failure has occured." Because you say so. If a system is "designed" to crash when it's hard drive fails then no system failure occurs then according to your definition. You asked for the "relative system reliability" but never defined what that was. You still haven't, yet you declare your correctness without ever demonstating that you even understand common terminology. From http://www.sun.com/products-n-solutions/hardware/d ocs/html/817-3881-11/ras.html:
"Reliability, availability, and serviceability (RAS) are aspects of a system's design that affect its ability to operate continuously and to minimize the time necessary to service the system. Reliability refers to a system's ability to operate continuously without failures and to maintain data integrity. System availability refers to the ability of a system to recover to an operational state after a failure, with minimal impact. Serviceability relates to the time it takes to restore a system to service following a system failure. Together, reliability, availability, and serviceability features provide for near continuous system operation."
Fact is that your original question cannot be answered because you never defined "system", never gave MTBF numbers for drives, never gave costs of SCSI or ATA drives, never offered configurations, and never provided MTBF numbers for common components. All you specified was $1000 in drives for either case. It appeared that you didn't know what you were talking about and it still does.
Now, if your point was that a more useful system can be constructed with ATA drives than SCSI for the same price then I might agree with you. Reliability differences between SCSI and ATA are mainly artificial and imagined anyway.
Drives should be classified as enterprise and non-enterprise, not ATA and SCSI. Protocol is meaningless. Once that is accepted, the original RAID paper makes your point well enough.
ATA is a perfectly fine protocol for block storage and is much leaner that SCSI. The SCSI protocol was used for ATAPI because it already existed and was needed to support a wide variety of devices besides disk. It makes no sense to put your imaginary "brains" back in.
:)
I'm pretty confident that you can prevent unintended data corruption. TCP/IP manages it so there's your proof of concept
GigE is not a good choice for disk attachment since it is easily outrun by a small number of disks. 10GigE is where it's at.
Apple is not the end-all-be-all of server manufacturers.
I was just saying that I hadn't read the article since it's unavailable. I would assume that the problems with performance under 10GigE don't apply because TCP isn't used. I would also assume that the offload engines can't be taken advantage of with this protocol.
I could see benefit in using this over iSCSI for 10G NIC's without TOE's but I wonder if there will be any of those. There real advantage of running anything over ethernet is to get the benefit of huge volumes, so I would suspect all 10G Nics will have TOE's.
One of the problems of TCP is it's very hard to make go fast over 10G. All those issues become moot for AoE since it doesn't use that. GigE ethernet isn't really interesting for attaching large numbers of disks.
Trouble is that I'd assume all 10GigE NIC's will come with offload engines anyway so there's no savings.
There is no functional problem with making the product non-routable. Servers need physical security and physical proximity. What you seem to think is a liability is not one.
while I agree with you, the issue of multipath is moot. Any device with only one port has that port as a single point of failure. You solve that problem with two ports and redundant switches. It is no different for SAN's, iSCSI or FC.
SAN's management capability is also it's downfall. Expensive, complicated and vendor-specific.
to go fast?
what are the benefits of SCSI commands for storage? Oh yeah, command queuing. ATA has that.
"Given the cost of main CPU cycles, though, it seems to me that most systems will have the cycles to spare on TCP overhead."
not at 10G they won't.
the question isn't whether iSCSI or AoE is better but rather why you would use either of them.
Will there be any 10G NIC's that don't offer offload engines? If not, what is the advantage architecturally of AoE?
What about direct DMA and zero-buffer?
I think you mean availability. Redundancy doesn't improve reliability, in fact it lessens it. What redundancy does is offer availability, the ability for a system to remain available after a failure.
The answer to your question is that, assuming there are more ATA drives of assumed lower reliability, the ATA system will be less reliable. You, sir, total fail it.
considering that it's non-routable, in what way is this "a great approach to networked storage"? Ethernet simply takes the place of a SCSI cable here and the device protocols differ. It lacks some of the availability characteristiccs of SATA/SAS and fails to solve and sharability issues because it's still a block interface. "From a comp sci point of view" it's a total dud.
The problem with ethernet is that it's hard to make go fast. We have 1G now but 10G is difficult because of all the processing involved and the offload engines that come with that. IF I could read the article perhaps I might understand if this is better in that respect, but the future of networked storage does not lie in block device protocols. To make this interesting we need 10G links and file symantics.
how is that relevant to the discussion of protocols?
reliability of SCSI versus ATA is largely imagined and the rest is intentional. drive manufacturers want you to believe their enterprise drives are more reliable and right now those drives are largely SCSI.
If that were realistic then you wouldn't need OS support at all. The drive is the worst location because it lies underneath the filesystem where much of the knowledge of what is being done is lost. Flash storage is additional nonvolatile storage that is made available to the operating system and hiding it in the drive is the worst place. If you wanted to do that you'd be better off using RAM and just enough battery back to ensure the data gets flushed.