The main corner case in your suggested "unit file" implementation is where someone is overwriting a file too large for the filesystem to contain two copies of it. You have to truncate when this happens to fit the new one, you can't just keep the old one around until it's replaced. This makes it impossible to meet the spec you're asking for in all cases. The best you can do is try to keep the original around until disk space runs out, and only truncate it when forced to. However, if that's how the implementation works, then applications can't just blindly rely on the filesystem to always do the right thing and "give you that for free". They've still got to create the new file and confirm it got written out before they touch the original if they want to guarantee never losing the original good copy, so that they bomb with a disk space error rather than risk truncating the original. That's why this whole path doesn't go anywhere useful; better to work on poplarizing an API for atomic rewrites or something.
As for your "managed files" case, that won't work for all database approaches. For example, in PostgreSQL, only writes to the database write-ahead log are done with O_SYNC/O_DIRECT. The main data block updates (and writes that are creating new data blocks) are written out asynchronously, and then when internal checkpoints reach their end any unwritten blocks are forced to disk with fsync if they're still in the OS cache. You'd be hard pressed to detect which of your suggested modes was the appropriate one for just the obvious behavior there, and there's still more weird corner cases to worry about buried in there too (like what the database does with the data blocks and the WAL to repair corruption after a crash).
Both these highlight that it's hard to make improvements here at just the filesystem level. Some of the really desirable behavior is hard to do unless applications are modified to do something different too. That hasn't really been going well for ext4 this year, and how that played out highlights how hard an issue this is to crack.
There is a real concern here, I just think the explanation was too far removed from the dedupe case for it to be obvious. Imagine the following: you have a 100GB drive. You copy a 75GB file onto it. You now you copy the same file, but with a different name, onto the disk again. This takes up some tiny amount of marginal space, because all of the blocks are duplicates, and it looks like there is still around 25GB free.
Now, imagine someone updates most of one of the two copies in-place, not realizing the shared situation (maybe they can't even see the other copy). Since the file is already on the drive and they're updating it, not creating a new file, the user would have no reason to believe this operation could fail due to running out of disk space. But it will--as they update blocks, copy-on-write is going to force allocating new space. The disk will run out of space around 1/3 into the update operation, leaving you with a junk file containing neither the original (unless you took a snapshot first, which is not normally a user operation) nor a good copy of the updated file. That's a business disaster right there. Quotas don't improve this situation. And I'd bet against that whatever application was in the middle of the update will be able to recover after more space is added to the pool, so that it can complete updating the file.
Since applications and users will sometimes check for disk space and/or available quota, in this same situation without a deduped setup the second file would have obviously never fit in the first place, and it would have had to go somewhere else instead. You can easily see how some would consider this preferable to failing oddly is rare (but easily plausible) situations. Deduping is ultimately a form of disk space oversubscription, and it's unrealistic to expect applications or users are going to consider all of the ramifications of how that might go wrong in a subtle way.
I've seen the same thing happen on a X4500. People need to understand that SATA disks and chipsets are fundamentally weak at error reporting and recovery. There's only so much you can do about that at the driver or OS level if a problem drives the chipset crazy. You really need hardware optimized for that purpose, like a mature and battle-tested RAID controller. The controller hardware on Sun's big SATA storage servers are just not customized enough for that purpose to handle all of the weird things dying drives can do to take out the other devices sharing resources on each drive bus.
How alarmist and uninformed; borderline FUD. The reality is as follows...
First, you can't remove a vdev yet, but development is in progress, and support is expected very soon now.
The bug report for this problem goes back to at least April of 2003. With that background, and that I've been hearing ZFS proponents suggesting this is coming "very soon now" for years without a fix, I'll believe it when I see it. Raising awareness that Sun's development priorities clearly haven't been toward any shrinking operation isn't FUD, it's the truth. Now, to be fair, that class of operations isn't very well supported on anything short of really expensive hardware either, but if you need these capabilities the weaknesses of ZFS here do reduce its ability to work for every use case.
I'm glad you're visiting dictionary.com to improve your reading comprehension. While you were occupied insulting me for no good reason, you seem to have missed that the discussion context here was comparing among the somewhat similar MLC drives on the market right now. The post I replied to might lead a reader to believe that the superior write speed of the OCZ and Patriot drives translated into them being better overall than Intel's design. I was pointing out that while they excelled at that category, no question about that, their performance in a broader context wasn't so exceptional. Their overall real-world performance is in fact quite ordinary compared to the other drives on the market. (And look at that, there's one of the words on the list!)
That wording might have been a bit generous, because as you point out Intel's drives have a 3X advantage in some benchmarks. I was giving the benefit of the doubt to my parent poster (something you could use some practice at). They certainly might prioritize one of the workloads where Intel's drives don't have a clear advantage, which certainly exist. I can't say they made the wrong decision for them, I was just pointing out that the logic used for that purchase might not extrapolate very well to others. It certainly doesn't apply to what I do, work which is heavy on random I/O, making the Intel drives the only reasonable product available in this segment--write capped or not.
And considered in the correct context here, it would be wrong to call the performance of the Velociraptor or Momentus drives mundane. Only achieving 6% of good performance isn't mundane at all. Those drives are downright terrible performers by the standards of the MLC SSDs we were comparing. (terrible: extremely bad, horrible)
I hope this clears up your vocabulary questions for today.
Yes, I was replying to what you meant, not what you said. A week ago, I'd have been completely with you; the 80MB/s cap was obviously a locked speed rather than a design limitation, as pointed out back in April. I'm not so sure there's as much headroom in the current 100MB/s limit though. While we know that it's possible to hit closer to 160MB/s in raw speed here based on similar units, the approach Intel is using to deliver its higher random performance has to hurt sequential performance too; that's the trade-off I was alluding to, the things the design is good at has some amount of overhead. I can't seem to find it right now, but one of the reviews I was reading about the new firmware suggested that Intel admitted to product segmentation by artificial limits in the past, but that didn't think there was any significant throttling left to unlock here. It's quite possible they just can't go past 100MB/s while still keeping the rest of their design and its background operations going to spec.
Regardless, I still wouldn't by any other drive on the market but Intel's. For the kind of apps I run, sequential write performance is secondary to the random work that their drives excel at, and the increase from 80MB/s to 100MB/s means they're at least now competitive with decent regular hard drives.
There are extremely few read-world apps that require sequential writes at speeds even reaching 80MB/s, much less 100MB/s. Sure, you can do it with a file copy, but how often does that come up as something you're waiting for? You certainly can't reach that speed with anything network-based. The fact that Intel optimized for smaller writes instead was absolutely the right thing. Sure, the Patriot or OCZ drives manage fast write speeds just as you describe. But they're mundane on random reads and writes, and on everything but the heaviest workloads Intel dominates. They made the right trade-offs here for most situations compared to the competition.
As alluded to in the link, modern hard drives transparently remap sectors when they go bad, so that when you ask to read the known bad one you actually get something from a completely different area altogether (wherever the spare sector space is at). All that happens via a closed-source firmware implementation you can't debug, improve, or fix if it goes awry. We're already being presented with a logical view of our drive hardware that maps to underlying physical hardware in a way we have no control over, been like that for many years now.
You should also recognize that were it not possible to spread the product development costs for SSDs across people with any operating system, they would still be in cost no object land. The world at large is running Windows, and the manufacturer has to make things work even with a stupid OS to sell enough volume. If you've solved that problem well, so that the hardware can do all the work, there's a lot less value to exposing the internals of that working design for people to mess with. The issues improved by TRIM were the last of the major issues here that can be fixed by firmware alone, and I expect to see much more stable firmware (from Intel at least) now that they're moving past that development.
Regardless, this Intel SSD problem isn't any different than the recent disaster with Seagate's bad firmware on their 1.5TB drives, or countless other examples. As I know multiple people who lost data on those drives due to that particular bug, or similar firmware issues in other regular hard drives, I don't see any real difference between the SSDs and where we were already at. These devices all suck, you're a fool if you think of applying an update without a full backup, and twice a fool if you decide to be the first on your block to try a brand new release. Firmware is hard to get right because there's so many different types of devices it has to talk to in the field, and it always will be fragile. Thinking whether or not you have access to the underlying filesystem implementation changes that is pretty naive. And the idea that a group like the Linux kernel hackers will do a better QA job than Intel seems pretty unlikely, given how they've basically pushed all QA and stability testing toward the distributors in the 2.6 series.
The other possibility is that the testing failed to consider what would happen if you still had debris left over from an earlier OS X install. I doubt that Snow Leopard testing considered the case of someone who started with 10.0 and upgraded to every version in between along the way for example.
Apple has really gotten a break that their users accept very little backward compatibility testing beyond a release or two really. It's completely reasonable for application developers to say "only runs on Leopard" right now and still sell stuff, while an app that only runs on Vista would be unavailable to a large chunk of its target market. They've done it with reasonable upgrade pricing, lack of upgrade DRM to make the users who don't want to pay suffer, and by adding useful APIs to every release that developers want to take advantage of. Quite enviable compared to the backward compatibility situation in Windows land.
3DMark had some limited value back when there weren't enough 3D games on the market for any one title to really push the hardware in multiple performance aspects. Nowadays, let's say you have two cards, one gets higher 3DMark scores, while the other posts higher benchmarks on the types of games you play. Aren't you going to ignore the 3DMark results in that case and buy based on real game performance? If so, why did you even consider 3DMark in the first place?
Now that you bring it up, why should we trust The Tech Report? I think it's pretty clear they've been bought by NVIDIA as payback for their chipset issues with Intel. (tightens tin foil hat)
special optimisations for benchmarks which don't affect real applications is getting pretty close to the line
But it does impact some real applications; unlike some earlier "cheats" here, this one really does work on a short list of games, too. The whole thing smells like a QA process to me, rather than an attempt to cheat on the benchmark. They've identified a small number of titles the optimization is known to work on, they add new titles to the list as they confirm they work with the new acceleration approach, and that keeps them from breaking apps they haven't tested yet. Once that list is large enough and they've gained more confidence in the new technology, one would presume they'd flip the switch to make it global for all titles.
There are all sorts of database applications that depend on a single CPU to run really fast. The sort of big reports people usually run overnight are an example. Niagara systems fail to work well if you any such requirement in your app, single threaded apps are way too slow to compared to Intel/AMD solutions. Niagara hardware is decent for applications that always have lots of users going at once, but they're only good for that, and that limits the market you can sell them into. It's certainly not the case that all database apps are of that type.
It's easy to get Linux to stop completely dead when something is writing heavily to disk. I wrote a blog entry on one such example where the server became completely unresponsive to anything for several seconds at a time, even though only a little over 40% of the RAM was being used. That I was able to improve using a tunable that's now the default in current kernels, but the root problem is still there, and as system RAM capacities go up it gets worse rather than better.
The kernel developers enjoy breaking modules that aren't part of the kernel tree just to punish developers who write them; that fact is so obvious that regular contributor Alan Cox dubbed it the war on binary modules and Theodore Tso commented on how they even target open-source solutions with that tactic. I lose a virtualization software package or two quite regularly when trying to upgrade kernels, even when said packages are open-source like Virtualbox (example). The only solution that will work is to push the kernel developers to accept the modules everyone is working on.
You forgot to mention that you can speed up the break-in if you play the Sheffield XLO Burn In CD in your DVD drive. Make sure to outline the edge with a green marker first.
That's an article from 2004 that's not fully relevant anymore. The GNOME IPC method described there, Bonobo, was mostly replaced by D-Bus in GNOME 2.4, with a full deprecation planned for GNOME 3.0; there are some lingering accessibility issues left to resolved before all the Bonobo is gone.
With KDE also adopting D-Bus some time ago, Open Office is the slow kid here. Everybody else is jumping on D-Bus as the way to handle cross-application message passing on Linux systems, because it's not like anybody ever really like all those awful CORBA-derived message bits anyway--that's some of the worst software ever designed. People have been suggesting that the UNO object model only used by OO be replaced by a D-Bus based approach for at least some things for years now. But since that object model is central to so much of Open Office and the code base is huge, that's a really hard bit of refactoring to do. Frankly, I'm surprised that the OO Clipboard works as well as it does, and I'd rather see people continue hammering away at the data problems there first before moving on to message passing.
Netapp released a study as well that included enterprise targeted drives, and in some cases the fiber-channel drives had significantly better error rates. The other interesting studies here are from Google and Carnegie Mellon
It's not silent when those errors get caught by their "deduplicate" step. You can certainly still get corruption with ZFS (imagine bad RAM flipping a bit after the checksum is verified but before it's sent over the network), so you can't rely just on it to keep your data safe. Since you need a higher-level integrity check anyway, it's completely reasonable to design a system such that all of the integrity checks are at the application level instead, particularly if you plan to store duplicate copies of the data.
The QA tasks they're only doing on the old hardware, and they don't have to re-run those again every time they change something that only impacts newer models. If a new model they might still be issuing updates to shipped with the backward compatibility feature, they'd have to add the whole PS2 testing back into to their QA matrix again. And they'd have to plan for an uptick in the support cases related to that feature too. I find it hard to imagine why Sony would go through all that just to pick up the small incremental market share that comes from people who want backward compatibility but didn't buy a model that had it back when they were available.
The usual excuse I hear from people in that category for why they didn't buy one of those is "they were too expensive", which also does not bode well for how much revenue Sony can expect from those customers anyway. They satisfied the big spenders and early adopters already, and it's unrealistic to think the remaining market is worth much very much to them. The costs of serving that market would need to be very tiny for it to make sense, and product costs in software aren't just based on manufacturing and development costs.
First off, there is still plenty of coding involved to support PS2 titles even when one has the Emotion Engine chip present only on earlier PS3s. The chip doesn't just work on its own without associated driver software that needs to be maintained, and there's specific features like the PS2 upscaling involved too.
And product development resources aren't just coding. The work that was going into the PS2 Support List wasn't trivial either, and in a real product every feature you advertise needs its own dedicated QA. My guess is that the QA effort here is why there hasn't been any success on the long-rumoured software-only PS2 emulation; it's just a giant support mess for Sony relative to what it's worth to them in terms of product sales.
Backward compatibility costs Sony significant development resources for a feature that only impacts a small number of people. The real diehards all bought the early PS3 models that had support for PS2 games. While vocal, the people who are still left waiting backward compatibility but not willing to pay for it in the past are a tiny number, and I can't imagine any scenario where Sony starts caring about you. Like those of us who wanted a PS3 when they played SACDs, a feature cut in the same period as backward compatibility, you should just recognize that Sony doesn't feel your sale is worth what it would cost to produce the product you want, and they never will.
Slashdot Mirror
The main corner case in your suggested "unit file" implementation is where someone is overwriting a file too large for the filesystem to contain two copies of it. You have to truncate when this happens to fit the new one, you can't just keep the old one around until it's replaced. This makes it impossible to meet the spec you're asking for in all cases. The best you can do is try to keep the original around until disk space runs out, and only truncate it when forced to. However, if that's how the implementation works, then applications can't just blindly rely on the filesystem to always do the right thing and "give you that for free". They've still got to create the new file and confirm it got written out before they touch the original if they want to guarantee never losing the original good copy, so that they bomb with a disk space error rather than risk truncating the original. That's why this whole path doesn't go anywhere useful; better to work on poplarizing an API for atomic rewrites or something.
As for your "managed files" case, that won't work for all database approaches. For example, in PostgreSQL, only writes to the database write-ahead log are done with O_SYNC/O_DIRECT. The main data block updates (and writes that are creating new data blocks) are written out asynchronously, and then when internal checkpoints reach their end any unwritten blocks are forced to disk with fsync if they're still in the OS cache. You'd be hard pressed to detect which of your suggested modes was the appropriate one for just the obvious behavior there, and there's still more weird corner cases to worry about buried in there too (like what the database does with the data blocks and the WAL to repair corruption after a crash).
Both these highlight that it's hard to make improvements here at just the filesystem level. Some of the really desirable behavior is hard to do unless applications are modified to do something different too. That hasn't really been going well for ext4 this year, and how that played out highlights how hard an issue this is to crack.
There is a real concern here, I just think the explanation was too far removed from the dedupe case for it to be obvious. Imagine the following: you have a 100GB drive. You copy a 75GB file onto it. You now you copy the same file, but with a different name, onto the disk again. This takes up some tiny amount of marginal space, because all of the blocks are duplicates, and it looks like there is still around 25GB free.
Now, imagine someone updates most of one of the two copies in-place, not realizing the shared situation (maybe they can't even see the other copy). Since the file is already on the drive and they're updating it, not creating a new file, the user would have no reason to believe this operation could fail due to running out of disk space. But it will--as they update blocks, copy-on-write is going to force allocating new space. The disk will run out of space around 1/3 into the update operation, leaving you with a junk file containing neither the original (unless you took a snapshot first, which is not normally a user operation) nor a good copy of the updated file. That's a business disaster right there. Quotas don't improve this situation. And I'd bet against that whatever application was in the middle of the update will be able to recover after more space is added to the pool, so that it can complete updating the file.
Since applications and users will sometimes check for disk space and/or available quota, in this same situation without a deduped setup the second file would have obviously never fit in the first place, and it would have had to go somewhere else instead. You can easily see how some would consider this preferable to failing oddly is rare (but easily plausible) situations. Deduping is ultimately a form of disk space oversubscription, and it's unrealistic to expect applications or users are going to consider all of the ramifications of how that might go wrong in a subtle way.
I've seen the same thing happen on a X4500. People need to understand that SATA disks and chipsets are fundamentally weak at error reporting and recovery. There's only so much you can do about that at the driver or OS level if a problem drives the chipset crazy. You really need hardware optimized for that purpose, like a mature and battle-tested RAID controller. The controller hardware on Sun's big SATA storage servers are just not customized enough for that purpose to handle all of the weird things dying drives can do to take out the other devices sharing resources on each drive bus.
How alarmist and uninformed; borderline FUD. The reality is as follows...
First, you can't remove a vdev yet, but development is in progress, and support is expected very soon now.
The bug report for this problem goes back to at least April of 2003. With that background, and that I've been hearing ZFS proponents suggesting this is coming "very soon now" for years without a fix, I'll believe it when I see it. Raising awareness that Sun's development priorities clearly haven't been toward any shrinking operation isn't FUD, it's the truth. Now, to be fair, that class of operations isn't very well supported on anything short of really expensive hardware either, but if you need these capabilities the weaknesses of ZFS here do reduce its ability to work for every use case.
I'm glad you're visiting dictionary.com to improve your reading comprehension. While you were occupied insulting me for no good reason, you seem to have missed that the discussion context here was comparing among the somewhat similar MLC drives on the market right now. The post I replied to might lead a reader to believe that the superior write speed of the OCZ and Patriot drives translated into them being better overall than Intel's design. I was pointing out that while they excelled at that category, no question about that, their performance in a broader context wasn't so exceptional. Their overall real-world performance is in fact quite ordinary compared to the other drives on the market. (And look at that, there's one of the words on the list!)
That wording might have been a bit generous, because as you point out Intel's drives have a 3X advantage in some benchmarks. I was giving the benefit of the doubt to my parent poster (something you could use some practice at). They certainly might prioritize one of the workloads where Intel's drives don't have a clear advantage, which certainly exist. I can't say they made the wrong decision for them, I was just pointing out that the logic used for that purchase might not extrapolate very well to others. It certainly doesn't apply to what I do, work which is heavy on random I/O, making the Intel drives the only reasonable product available in this segment--write capped or not.
And considered in the correct context here, it would be wrong to call the performance of the Velociraptor or Momentus drives mundane. Only achieving 6% of good performance isn't mundane at all. Those drives are downright terrible performers by the standards of the MLC SSDs we were comparing. (terrible: extremely bad, horrible)
I hope this clears up your vocabulary questions for today.
Yes, I was replying to what you meant, not what you said. A week ago, I'd have been completely with you; the 80MB/s cap was obviously a locked speed rather than a design limitation, as pointed out back in April. I'm not so sure there's as much headroom in the current 100MB/s limit though. While we know that it's possible to hit closer to 160MB/s in raw speed here based on similar units, the approach Intel is using to deliver its higher random performance has to hurt sequential performance too; that's the trade-off I was alluding to, the things the design is good at has some amount of overhead. I can't seem to find it right now, but one of the reviews I was reading about the new firmware suggested that Intel admitted to product segmentation by artificial limits in the past, but that didn't think there was any significant throttling left to unlock here. It's quite possible they just can't go past 100MB/s while still keeping the rest of their design and its background operations going to spec.
Regardless, I still wouldn't by any other drive on the market but Intel's. For the kind of apps I run, sequential write performance is secondary to the random work that their drives excel at, and the increase from 80MB/s to 100MB/s means they're at least now competitive with decent regular hard drives.
There are extremely few read-world apps that require sequential writes at speeds even reaching 80MB/s, much less 100MB/s. Sure, you can do it with a file copy, but how often does that come up as something you're waiting for? You certainly can't reach that speed with anything network-based. The fact that Intel optimized for smaller writes instead was absolutely the right thing. Sure, the Patriot or OCZ drives manage fast write speeds just as you describe. But they're mundane on random reads and writes, and on everything but the heaviest workloads Intel dominates. They made the right trade-offs here for most situations compared to the competition.
As alluded to in the link, modern hard drives transparently remap sectors when they go bad, so that when you ask to read the known bad one you actually get something from a completely different area altogether (wherever the spare sector space is at). All that happens via a closed-source firmware implementation you can't debug, improve, or fix if it goes awry. We're already being presented with a logical view of our drive hardware that maps to underlying physical hardware in a way we have no control over, been like that for many years now.
You should also recognize that were it not possible to spread the product development costs for SSDs across people with any operating system, they would still be in cost no object land. The world at large is running Windows, and the manufacturer has to make things work even with a stupid OS to sell enough volume. If you've solved that problem well, so that the hardware can do all the work, there's a lot less value to exposing the internals of that working design for people to mess with. The issues improved by TRIM were the last of the major issues here that can be fixed by firmware alone, and I expect to see much more stable firmware (from Intel at least) now that they're moving past that development.
Regardless, this Intel SSD problem isn't any different than the recent disaster with Seagate's bad firmware on their 1.5TB drives, or countless other examples. As I know multiple people who lost data on those drives due to that particular bug, or similar firmware issues in other regular hard drives, I don't see any real difference between the SSDs and where we were already at. These devices all suck, you're a fool if you think of applying an update without a full backup, and twice a fool if you decide to be the first on your block to try a brand new release. Firmware is hard to get right because there's so many different types of devices it has to talk to in the field, and it always will be fragile. Thinking whether or not you have access to the underlying filesystem implementation changes that is pretty naive. And the idea that a group like the Linux kernel hackers will do a better QA job than Intel seems pretty unlikely, given how they've basically pushed all QA and stability testing toward the distributors in the 2.6 series.
The other possibility is that the testing failed to consider what would happen if you still had debris left over from an earlier OS X install. I doubt that Snow Leopard testing considered the case of someone who started with 10.0 and upgraded to every version in between along the way for example.
Apple has really gotten a break that their users accept very little backward compatibility testing beyond a release or two really. It's completely reasonable for application developers to say "only runs on Leopard" right now and still sell stuff, while an app that only runs on Vista would be unavailable to a large chunk of its target market. They've done it with reasonable upgrade pricing, lack of upgrade DRM to make the users who don't want to pay suffer, and by adding useful APIs to every release that developers want to take advantage of. Quite enviable compared to the backward compatibility situation in Windows land.
3DMark had some limited value back when there weren't enough 3D games on the market for any one title to really push the hardware in multiple performance aspects. Nowadays, let's say you have two cards, one gets higher 3DMark scores, while the other posts higher benchmarks on the types of games you play. Aren't you going to ignore the 3DMark results in that case and buy based on real game performance? If so, why did you even consider 3DMark in the first place?
Now that you bring it up, why should we trust The Tech Report? I think it's pretty clear they've been bought by NVIDIA as payback for their chipset issues with Intel. (tightens tin foil hat)
special optimisations for benchmarks which don't affect real applications is getting pretty close to the line
But it does impact some real applications; unlike some earlier "cheats" here, this one really does work on a short list of games, too. The whole thing smells like a QA process to me, rather than an attempt to cheat on the benchmark. They've identified a small number of titles the optimization is known to work on, they add new titles to the list as they confirm they work with the new acceleration approach, and that keeps them from breaking apps they haven't tested yet. Once that list is large enough and they've gained more confidence in the new technology, one would presume they'd flip the switch to make it global for all titles.
Wiping out a patent troll only after they've extorted money from Microsoft: priceless
There are all sorts of database applications that depend on a single CPU to run really fast. The sort of big reports people usually run overnight are an example. Niagara systems fail to work well if you any such requirement in your app, single threaded apps are way too slow to compared to Intel/AMD solutions. Niagara hardware is decent for applications that always have lots of users going at once, but they're only good for that, and that limits the market you can sell them into. It's certainly not the case that all database apps are of that type.
It's easy to get Linux to stop completely dead when something is writing heavily to disk. I wrote a blog entry on one such example where the server became completely unresponsive to anything for several seconds at a time, even though only a little over 40% of the RAM was being used. That I was able to improve using a tunable that's now the default in current kernels, but the root problem is still there, and as system RAM capacities go up it gets worse rather than better.
The kernel developers enjoy breaking modules that aren't part of the kernel tree just to punish developers who write them; that fact is so obvious that regular contributor Alan Cox dubbed it the war on binary modules and Theodore Tso commented on how they even target open-source solutions with that tactic. I lose a virtualization software package or two quite regularly when trying to upgrade kernels, even when said packages are open-source like Virtualbox (example). The only solution that will work is to push the kernel developers to accept the modules everyone is working on.
You forgot to mention that you can speed up the break-in if you play the Sheffield XLO Burn In CD in your DVD drive. Make sure to outline the edge with a green marker first.
That's an article from 2004 that's not fully relevant anymore. The GNOME IPC method described there, Bonobo, was mostly replaced by D-Bus in GNOME 2.4, with a full deprecation planned for GNOME 3.0; there are some lingering accessibility issues left to resolved before all the Bonobo is gone.
With KDE also adopting D-Bus some time ago, Open Office is the slow kid here. Everybody else is jumping on D-Bus as the way to handle cross-application message passing on Linux systems, because it's not like anybody ever really like all those awful CORBA-derived message bits anyway--that's some of the worst software ever designed. People have been suggesting that the UNO object model only used by OO be replaced by a D-Bus based approach for at least some things for years now. But since that object model is central to so much of Open Office and the code base is huge, that's a really hard bit of refactoring to do. Frankly, I'm surprised that the OO Clipboard works as well as it does, and I'd rather see people continue hammering away at the data problems there first before moving on to message passing.
Netapp released a study as well that included enterprise targeted drives, and in some cases the fiber-channel drives had significantly better error rates. The other interesting studies here are from Google and Carnegie Mellon
It's not silent when those errors get caught by their "deduplicate" step. You can certainly still get corruption with ZFS (imagine bad RAM flipping a bit after the checksum is verified but before it's sent over the network), so you can't rely just on it to keep your data safe. Since you need a higher-level integrity check anyway, it's completely reasonable to design a system such that all of the integrity checks are at the application level instead, particularly if you plan to store duplicate copies of the data.
The QA tasks they're only doing on the old hardware, and they don't have to re-run those again every time they change something that only impacts newer models. If a new model they might still be issuing updates to shipped with the backward compatibility feature, they'd have to add the whole PS2 testing back into to their QA matrix again. And they'd have to plan for an uptick in the support cases related to that feature too. I find it hard to imagine why Sony would go through all that just to pick up the small incremental market share that comes from people who want backward compatibility but didn't buy a model that had it back when they were available.
The usual excuse I hear from people in that category for why they didn't buy one of those is "they were too expensive", which also does not bode well for how much revenue Sony can expect from those customers anyway. They satisfied the big spenders and early adopters already, and it's unrealistic to think the remaining market is worth much very much to them. The costs of serving that market would need to be very tiny for it to make sense, and product costs in software aren't just based on manufacturing and development costs.
First off, there is still plenty of coding involved to support PS2 titles even when one has the Emotion Engine chip present only on earlier PS3s. The chip doesn't just work on its own without associated driver software that needs to be maintained, and there's specific features like the PS2 upscaling involved too.
And product development resources aren't just coding. The work that was going into the PS2 Support List wasn't trivial either, and in a real product every feature you advertise needs its own dedicated QA. My guess is that the QA effort here is why there hasn't been any success on the long-rumoured software-only PS2 emulation; it's just a giant support mess for Sony relative to what it's worth to them in terms of product sales.
Well duh, obviously the people looking for consoles with a really reliable drive buy an XBox instead.
Backward compatibility costs Sony significant development resources for a feature that only impacts a small number of people. The real diehards all bought the early PS3 models that had support for PS2 games. While vocal, the people who are still left waiting backward compatibility but not willing to pay for it in the past are a tiny number, and I can't imagine any scenario where Sony starts caring about you. Like those of us who wanted a PS3 when they played SACDs, a feature cut in the same period as backward compatibility, you should just recognize that Sony doesn't feel your sale is worth what it would cost to produce the product you want, and they never will.
But that experience is easy to get any day just by visiting the Apple Store, listening to what they tell you, and buying an expensive system there.