A pinhole camera has infinite depth of field. Of course it has some other problems, diffraction, sensitivity, etc.
If you have enough light, fast film, and shoot with a tight aperture, you can get very wide depth of field. Just two or three "layers" would be enough for effectively infinite depth of field even at film resolution. However compositing the layers would be a bit of a chore. For a feature length film, the compositing process would need to be automatic, perhaps assisted with something like a scanning laser rangefinder.
No, you didn't. We're talking about your software running on someone else's computer. For example, Linus' software running on millions of TiVos, virtually none of which belong to him.
Think what you want, but I described the way things actually happened, while you're describing a gloss of revisionist history. Itanium was intended to kill off all other high end RISC development programs, and it very nearly succeeded despite being an "unsuccessful" and desperately late product.
SGI/MIPS canceled two high end CPUs, Beast and Capitan, specifically because of the threat of Itanium. I was there, I saw it happen.
Obviously, Itanium redirected HP's focus away from PA-RISC since it was a HP/Intel project.
Itanium failed to completely derail SPARC, but it caused a great deal of controversy inside SUN about the future of the SPARC architecture and disrupted SPARC development for a year or two.
IBM's Power architecture was perhaps the least affected by Itanium. IBM was pretty skeptical about Itanium and kept the Power program very much alive. As a result, they are the only RISC family which still has a significant presence on the Top500 supercomputing list.
I have no idea whether all these other CPU families would have been successful in the marketplace without Itanium. However, the fact is they were killed due to one of the most influential vaporware announcements in the history of the computer industry.
Itanium is a lesson in how not to handle technological transitions. Itanium was picked by geeks who had no idea of what the market wanted or needed, and Intel marketing and management blindly believed what they were hearing from the geeks.
Actually, Itanium was a wildly successful product. Mere rumors of Itanium's capabilities were sufficient to kill DEC Alpha, drive SGI/MIPS out of the high end processor market and disrupt SPARC and PA-RISC development programs. Intel virtually eliminated the threat of competitive RISC architectures for years with the announcement of Itanium.
(Another company that works like that is Microsoft, which is why they keep churning out such bad software.)
So, just because you can't find something on Wikipedia you're willing to impugn someone's knowledge and intellectual honesty?
Incidentally, Wikipedia does have an entry for "interference fit". You incorrectly parsed "molecular interference fit" as "molecular interference fit" rather than "molecular interference fit."
For the record I have no idea whether such a phenomenon actually exists but it sounds plausible enough to me that I'm not going to denounce it without doing more than a Wikipedia search.
In blind testing of audio equipment, it is critical to match volume levels within a fraction of a decibel. That is because people have a strong tendency to prefer a slightly louder source. In blind testing, listeners will describe the louder source as better in all sorts of subjective ways that have nothing to do with loudness: brighter, richer, warmer, etc. This happens with any kind music, from chamber music to stadium rock.
I think the article oversimplifies somewhat by casting this as a matter of taste for loud rock music, rather than a more subtle issue of psychoacoustics.
The conclusion is that the cost of nuclear power falls in the range: "3 cents per kilowatt hour to nearly 14 cents per kilowatt hour".
Note the word "cost."
Solar power is about 3x the cost of the most expensive nuclear power.
Nope. Ontario is paying 42 cents per kWh. That does not mean that solar power generation costs 42 cents per kWh. Indeed, I highly doubt that the investors in this project would be interested if there is no room for profit.
According to this link, actual costs for solar power generation are currently 20 to 30 cents per kWh:
The purpose of the U.S. government is to do what's best for its citizens
Wrong! We live in a democracy, not a benevolent dictatorship. The purpose of the U.S. government is to do what its citizens want it to do, not what the government deems to be best. If a lot of Americans want to help the rest of the world, great; the government should not stand in the way. I doubt that this is actually the case, but it's worth pointing out the distinction.
In particular, you can't stifle disagreement by saying, "No, you're wrong, that's not the job of government." Government's job description is what we vote on every year or two, and spend the rest of the time arguing about. Nothing's off the table.
Personally, I want our government to facilitate free trade, globalization, and offshoring for at least three reasons:
First, I don't see any reason why Americans in general (even myself) should get preferential treatment over any other person. If I can't compete for a job, I need to improve my competitiveness or find another career, rather than trying to tilt the playing field in my favor.
Second, I believe globalization is in the direct and immediate economic interests of America, for reasons that are too complex to go into here; take a course in macroeconomics.
Third, I believe that by correcting extreme inequalities of wealth, globalization will lead to a more peaceful planet. This has obvious long term benefits to everyone, but particularly to America, which (for better or worse) gets involved in most of the world's conflicts.
Any politician foolish enough to espouse such views is likely to get my vote.:)
Did you even read past the first sentence of my post? Do you know the difference between operating margin and profit margin? I agree with you that Facebook's operating margin could be not just higher than Costco's, but easily 10 times higher. However it is mathematically impossible to have an operating margin 50 times higher than Costco's as the parent claimed.
As far as Facebook's profit margin is concerned, I believe it's probably negative. The reason I'd say that is that they were lining up venture financing just 8 months ago. I'll certainly grant that their profit margin could have become positive since then, even to the tune of 10 or 15 percent, though I doubt it. It's rare for a company at that stage to even want to be profitable, they would probably consider that a sign that they are underinvesting. The main focus is cash flow break even, because that's where they can stop diluting themselves.
Costco earns a profit margin of about 2%. The profit margin of Facebook is probably negative.
Costco earns an operating margin of about 3%. The operating margin of Facebook is unlikely to be higher than about 30%.
So, whatever metric you use, your statement is almost certainly incorrect, whereas Mr. Dagres made a statement that is plausibly correct.
He's not doing bullshit financial analysis. In fact it is textbook financial analysis to use cash flow statements as a bullshit detector. The cash flow statement is a good reality check that will show up problems in a business that aren't obvious in the income statement. As quoted in the article -- "I'll take cash flow over gross margin -- I can eat cash flow."
Yup. But in a goodly number of these cases, the extra expense of replacing the petroleum-based product is out of line.
Well naturally, or they'd already have been replaced!
The problem I am interested in is this: if the price of oil quadruples in the next decade, which is entirely possible, where will we still be using oil no matter the cost? Certainly not for fixed power generation. Certainly not for heating new homes. Probably less for plastics. But we're still going to have to use it as a transportation fuel. There is not (yet) a reasonable substitute.
Transportation accounts for a large majority of the petroleum used in the United States. Even so, it doesn't really matter what we are using today; the important thing is which uses can be substituted and which can't.
Petroleum makes extremely good transportation fuels which are very difficult to substitute. They provide very high energy density, and reasonable safety and efficiency at a low cost. In order to compete, new technologies have extremely challenging technical barriers to overcome.
Almost everything else petroleum is used for is either comparatively easy to replace with alternative technologies or represents a comparatively small consumption. Meeting our electrical demand without fossil fuels is a challenge certainly, but it's not beyond current technology. Whereas, flying a passenger plane at 500mph without petroleum fuel is a problem so difficult that virtually nobody is even making a token effort to solve it.
Home heating oil has alternatives in natural gas and electricity. Lubricants are a tiny market compared with transportation fuels. Asphalt is a low value by-product of refining, with perfectly acceptable substitutes should it become too expensive. Plastics can be substituted in many uses, recycled in some others. Although we will probably be making plastic from petroleum long into the future, transportation is the real problem we need to be worrying about.
I see the confusion. You are thinking about sequential reads, where you are correct, the asymptotic bandwidth penalty is 1.5x for a 4 disk RAID-6. With small random reads the penalty is 3x because you have to read both data chunks and one parity chunk from the stripe even though you only need one of the data chunks. On any practical workload the penalty is likely to fall somewhere between those limits.
Memory bandwidth isn't growing as fast as CPU power, but they are both growing so much faster than practical drive throughput that they have become effectively infinite by comparison. The only way you can become memory bound doing disk IO is to have a pure sequential access pattern and dozens of spindles per CPU. And as long as we are stuck with hard disks for mass storage, that imbalance is only going to get more extreme. Memory bandwidth is one of the last things a modern filesystem designer needs to lose sleep over.
The bottom line is that ZFS establishes and maintains true data integrity from the filesystem layer down at a very reasonable cost. Neither RAID of any flavor, nor any widely used filesystem can say that. This has been an interesting discussion but I don't think there's really any more that needs to be said to defend this position.
In theory you could add protections for all those things, but it doesn't seem very useful to me; why not just test the system an to empirically discover such faults?
Yes; that's exactly what ZFS does: test the integrity of your data to empirically discover faults. On the other hand, most RAID implementations and filesystems don't test the integrity of your data; that is left up to the application layer and hardware.
I can't even guess at where you got the 2/3 number; the one I come up with is more like 1/3 worst-case-scenario, and that's only at the physical disk level, not at the host-interface level.
I came up with 2/3 because you need to read the data and both parity stripes. Actually it's worse than that, you need to read all the stripes to validate any given block, just as you do in a write operation. For example in the minimal 4 disk RAID-6 array, reading and verifying will be 4 times slower than reading without verifying. That's why nobody does it. (Also because in any level lower than RAID-6 you are still screwed; you don't know whether the data or the parity is correct.)
It's also worth noting that, while the checksum in ZFS is small, the entire data set much be read from memory and processed to calculate the checksum, in addition to whatever you plan to do with the data after you've done the integrity check.
ZFS calculates checksums on a (variable size) block level. Compared with IO bandwidth, CPU and memory bandwidth are virtually free and getting cheaper all the time.
If the data-integrity benefits of ZFS require that ZFS be implemented perfectly and only protects me from implementation errors in other, generally simpler systems, I'm not seeing a lot of practical benefit here.
You know, there are so many problems with that statement that I don't even know where to start. If you believe that implementing a reliable filesystem is on the same order of magnitude of difficulty as perfectly error-hardening every piece of firmware and hardware that will ever be used with that filesystem, there's nothing I can do to convince you otherwise.
md5sum comes to mind -- that provides file-level data integrity checking, which is superior even to the filesystem-level checking that ZFS provides. And it's pretty commonly used in an automated fashion to verify that executables and archives are intact.
ZFS has an enormous advantage over md5sum: it is completely transparent to the user. I'd say that md5sum is the problem, and ZFS is the solution.
(Of course md5sum is primarily used to solve higher level problems.)
Have you read the link I posted? It is getting old to keep repeating Jeff Bonwick's arguments when you can just read what I already pointed you to.
The underlying hardware will not necessarily notice errors. Hard drives are only designed to error protect the magnetic domains on the disk. There are all sorts of other places in the increasingly long datapath to disk where data can be lost, and, in fact, routinely does get lost.
The choice to verify every read is purely an implementation decision
RAID-6 does not verify every read because it is a stupid way to achieve data integrity. It wastes two thirds of your aggregate IO read bandwidth when you can just use checksums virtually for free. CPU cycles for checksumming is dirt cheap whereas IO bandwidth is extremely expensive.
I was just arguing the novelity you seem to think ZFS has -- using checksums to verify data integrity is not exactly cutting-edge computer science.
Yet strangely there aren't any other widely available storage solutions that provide transparent data integrity from the filesystem down.
I just don't think this particular feature is unique or superior to other available solutions for the same problem.
Then name another one. I think we've already shown that vanilla RAID does not qualify.
But RAID 5 and 6 are both capable of verifying the primary data source against the parity data and transparently correcting errors that occur on less than the critical number of disks.
With RAID-5, as with RAID-1, the critical number of disks is one. RAID-5 cannot transparently correct errors that occur on even a single stripe, unless you know a priori which stripe is affected.
With RAID-6 you can automatically correct errors that occur on a single stripe, but it still does not automatically detect such errors on read the way ZFS and RAID-Z do.
Or, you know, a checksum.
That's a great idea. Too bad those ZFS guys didn't think of it. Oh, wait.:)
Hard drives silently losing data is a problem solved by RAID.
That is profoundly wrong. Vanilla RAID will not discover and cannot automatically correct silent data loss. The reason is that RAID has no way of knowing which data is correct. For example, if two mirrored copies disagree on the contents of a block, the data is unrecoverable without manual intervention or external knowledge. Furthermore, in normal operation your RAID subsystem will simply read data from whichever drive is idle at the time the read request comes in; it does not ordinarily compare the two mirrors. The data will remain corrupted until the user notices a problem, at which point they have no practical recourse. Essentially the same problem occurs with parity RAID.
There is no dedicated hardware in your system that provides the end to end data integrity that ZFS does. I honestly suggest you learn more about it before airing your opinions. Here is a start:
Let me tell you a cautionary tale. Some six months ago I decided to do exactly what you're suggesting for exactly the same reason. I got OpenSolaris and installed it on a spare drive on my fileserver. It was a pain in the ass figuring out how to administer a Solaris system, which despite all the documentation seemed opaque and baroque, but I persisted. Unfortunately, the damn thing wouldn't configure the network. I couldn't figure it out. I was reduced to tracing through the init scripts line by line figuring out where they failed. It turned out that the file which contained the hostname had a trailing carriage return. I'd pressed "enter" an extra time when I was editing it. As a result the system decided it's hostname was the empty string, which it couldn't find in the hosts database and gave up. At that point I nerfed the partition in a fit of pique and went back to Linux + software RAID + reiserfs. Any software which is that brittle and administrator-hostile honestly does not deserve to live, certainly not on my server, no matter how nice its filesystem.
If I did dd for whatever reason over the partition I could just restore from the BACKUP.
If you were aware that the data was corrupted. Hard drives can silently lose data for a variety of reasons, ZFS will discover that damage whereas Linux filesystems will silently ignore it.
Honestly, read up on ZFS. Your ignorance is palpable.
This defeats TiVO's DRM that was used to prevent Linux and Mac users from watching shows on their PC.
TiVo's DRM prevented Linux and Mac users from watching their shows only as a side effect. The intent of TiVo's DRM is to prevent people who don't know the MAK from watching the content.
Please stop replying if you have no idea what you're talking about.
Ahem.:)
This is a nice piece of reverse engineering, but no encryption scheme was cracked.
HDTV is a classic case of chicken and egg. Without an installed base, the industry has no incentive to produce and broadcast HD content. Without content, on the other hand, there will be no installed base. You can't blame the broadcasters for following their financial incentives, any more than you can blame consumers for rejecting high priced HDTV hardware on which they had nothing to watch.
Fortunately, broadcasters, unlike consumers, are beholden to federal regulators and can be coerced. The FCC saw this chicken-and-egg problem coming and mandated terrestrial broadcast of HD content in the US. The Canadians should do the same. If you broadcast SD, you have to broadcast HD as well.
Anyway, none of this matters anymore. HDTV is finally a done deal. Between the US tuner mandate, HD capable enabled game consoles, and the price trajectory of LCD flat panels, consumer adoption of HDTV is unstoppable. Advertisers and broadcasters will be dragged along soon enough.
Broadcast TV is quite relevant to HDTV. For years now, HDTV early adopters have been depending on terrestrial broadcast. That's because premium cable and satellite providers weren't particularly interested in HDTV, for a variety of reasons. Terrestrial broadcasters weren't interested in HDTV either, but the FCC forced them to broadcast in HD anyway. Which, of course, is what the Canadian authorities should do with CBC.
As for DVDs and downloaded content, those are generally standard definition or lower.
I've heard arguments like that over and over again concerning functionality that is now implemented in software, sometimes with new dedicated CPU facilities. Graphics will be no different, it is just a matter of time. It may start with "typical" business users and laptop hardware, but sooner rather than later, the architectural and cost advantages of integration and generality will always outweigh putting huge amounts of custom logic on the wrong side of an IO bus.
The handwriting is on the wall if you know where to look. CPU and graphics architectures are converging.
First the CPU side. More and more of the work being done on desktop CPUs is media related: HDTV and DIVX encoding/decoding, digital photography, audio DSP, game physics, software radio. General purpose superscalar CPUs are horribly inefficient for this class of problems. At the same time, new chip processes are providing more transistors than CPU designers know what to do with. At first they just added more cores, but that is already running into scaling problems. They're clearly tempted to add some sort of highly parallel SIMD engine that is better suited for DSP type problems. Look at MMX in all its guises. Look at the Cell architecture. Both are steps in this direction.
Then the graphics side. The clear direction here is away from specialized hardware for each stage of the graphics pipeline, and toward large numbers of more general computation elements in a SIMD architecture. Look at NVidia's latest chip and you'll see something that is completely unlike traditional graphics pipelines. However it's almost exactly like what CPU designers want to put in desktop chips anyway, to solve the problems that superscalar processors are bad at.
Finally, look at some of the other stuff we are putting on the wrong side of the IO bus: Creative Labs X-Fi, AGEIA PhysX. These are also examples of convergence toward highly parallel and relatively general DSP architectures.
Even if power gamers like to upgrade their graphics card every few months, there are other markets which are increasingly driving system architecture. Game consoles are a huge market, and the next generation of consoles is poised to take advantage of the convergence I'm talking about. Vista is a big market for 3d graphics on business PCs, and that market is all about higher integration and lower cost. Neither of these markets will ever upgrade graphics independent of CPU, but they are paying for super high bandwidth IO busses and duplicate memory interfaces which they don't need.
Graphics hardware on an IO bus has about 5 years to live, in my opinion.
The history of computer architecture is a ceaseless march toward higher integration and higher generality. That is, what was once special purpose hardware in a seperate unit is now implemented in firmware as part of a more flexible, general purpose processor.
This march is littered with those standing by the wayside saying things like, "Who needs floating point in the CPU? Leave it on a seperate chip!" or "I want to be able to upgrade my CPU without buying a new memory controller!" or "If you integrate sound on the motherboard, I'll just have to buy a seperate sound card."
So, while I'm sure you're not the only one who thinks GPU integration is a bad idea, that doesn't mean it is a bad idea.
Slashdot Mirror
No camera has an infinite depth of field
A pinhole camera has infinite depth of field. Of course it has some other problems, diffraction, sensitivity, etc.
If you have enough light, fast film, and shoot with a tight aperture, you can get very wide depth of field. Just two or three "layers" would be enough for effectively infinite depth of field even at film resolution. However compositing the layers would be a bit of a chore. For a feature length film, the compositing process would need to be automatic, perhaps assisted with something like a scanning laser rangefinder.
Martin
Because I paid for it.
No, you didn't. We're talking about your software running on someone else's computer. For example, Linus' software running on millions of TiVos, virtually none of which belong to him.
Martin
Think what you want, but I described the way things actually happened, while you're describing a gloss of revisionist history. Itanium was intended to kill off all other high end RISC development programs, and it very nearly succeeded despite being an "unsuccessful" and desperately late product.
p lans/2100-1001_3-210024.html
SGI/MIPS canceled two high end CPUs, Beast and Capitan, specifically because of the threat of Itanium. I was there, I saw it happen.
http://news.com.com/Silicon+Graphics+scraps+MIPS+
Compaq killed Alpha before the HP merger, before Carly, with the intention of moving their high end business to IA-64:
http://en.wikipedia.org/wiki/DEC_Alpha
Obviously, Itanium redirected HP's focus away from PA-RISC since it was a HP/Intel project.
Itanium failed to completely derail SPARC, but it caused a great deal of controversy inside SUN about the future of the SPARC architecture and disrupted SPARC development for a year or two.
http://news.com.com/2100-1001_3-237583.html
IBM's Power architecture was perhaps the least affected by Itanium. IBM was pretty skeptical about Itanium and kept the Power program very much alive. As a result, they are the only RISC family which still has a significant presence on the Top500 supercomputing list.
http://www.top500.org/stats/list/29/procfam
http://www.top500.org/stats/list/13/procfam
I have no idea whether all these other CPU families would have been successful in the marketplace without Itanium. However, the fact is they were killed due to one of the most influential vaporware announcements in the history of the computer industry.
Itanium is a lesson in how not to handle technological transitions. Itanium was picked by geeks who had no idea of what the market wanted or needed, and Intel marketing and management blindly believed what they were hearing from the geeks.
Actually, Itanium was a wildly successful product. Mere rumors of Itanium's capabilities were sufficient to kill DEC Alpha, drive SGI/MIPS out of the high end processor market and disrupt SPARC and PA-RISC development programs. Intel virtually eliminated the threat of competitive RISC architectures for years with the announcement of Itanium.
(Another company that works like that is Microsoft, which is why they keep churning out such bad software.)
To much the same effect.
So, just because you can't find something on Wikipedia you're willing to impugn someone's knowledge and intellectual honesty?
Incidentally, Wikipedia does have an entry for "interference fit". You incorrectly parsed "molecular interference fit" as "molecular interference fit" rather than "molecular interference fit."
For the record I have no idea whether such a phenomenon actually exists but it sounds plausible enough to me that I'm not going to denounce it without doing more than a Wikipedia search.
In blind testing of audio equipment, it is critical to match volume levels within a fraction of a decibel. That is because people have a strong tendency to prefer a slightly louder source. In blind testing, listeners will describe the louder source as better in all sorts of subjective ways that have nothing to do with loudness: brighter, richer, warmer, etc. This happens with any kind music, from chamber music to stadium rock.
I think the article oversimplifies somewhat by casting this as a matter of taste for loud rock music, rather than a more subtle issue of psychoacoustics.
The conclusion is that the cost of nuclear power falls in the range: "3 cents per kilowatt hour to nearly 14 cents per kilowatt hour".
Note the word "cost."
Solar power is about 3x the cost of the most expensive nuclear power.
Nope. Ontario is paying 42 cents per kWh. That does not mean that solar power generation costs 42 cents per kWh. Indeed, I highly doubt that the investors in this project would be interested if there is no room for profit.
According to this link, actual costs for solar power generation are currently 20 to 30 cents per kWh:
http://www.solarbuzz.com/StatsCosts.htm
Martin
Writing something disturbing is enough to cost you your right to own a gun?
No. Disorderly conduct is a misdemeanor, not a felony. You don't lose your right to own a gun for a misdemeanor.
The purpose of the U.S. government is to do what's best for its citizens
:)
Wrong! We live in a democracy, not a benevolent dictatorship. The purpose of the U.S. government is to do what its citizens want it to do, not what the government deems to be best. If a lot of Americans want to help the rest of the world, great; the government should not stand in the way. I doubt that this is actually the case, but it's worth pointing out the distinction.
In particular, you can't stifle disagreement by saying, "No, you're wrong, that's not the job of government." Government's job description is what we vote on every year or two, and spend the rest of the time arguing about. Nothing's off the table.
Personally, I want our government to facilitate free trade, globalization, and offshoring for at least three reasons:
First, I don't see any reason why Americans in general (even myself) should get preferential treatment over any other person. If I can't compete for a job, I need to improve my competitiveness or find another career, rather than trying to tilt the playing field in my favor.
Second, I believe globalization is in the direct and immediate economic interests of America, for reasons that are too complex to go into here; take a course in macroeconomics.
Third, I believe that by correcting extreme inequalities of wealth, globalization will lead to a more peaceful planet. This has obvious long term benefits to everyone, but particularly to America, which (for better or worse) gets involved in most of the world's conflicts.
Any politician foolish enough to espouse such views is likely to get my vote.
Did you even read past the first sentence of my post? Do you know the difference between operating margin and profit margin? I agree with you that Facebook's operating margin could be not just higher than Costco's, but easily 10 times higher. However it is mathematically impossible to have an operating margin 50 times higher than Costco's as the parent claimed.
As far as Facebook's profit margin is concerned, I believe it's probably negative. The reason I'd say that is that they were lining up venture financing just 8 months ago. I'll certainly grant that their profit margin could have become positive since then, even to the tune of 10 or 15 percent, though I doubt it. It's rare for a company at that stage to even want to be profitable, they would probably consider that a sign that they are underinvesting. The main focus is cash flow break even, because that's where they can stop diluting themselves.
Costco earns a profit margin of about 2%. The profit margin of Facebook is probably negative.
Costco earns an operating margin of about 3%. The operating margin of Facebook is unlikely to be higher than about 30%.
So, whatever metric you use, your statement is almost certainly incorrect, whereas Mr. Dagres made a statement that is plausibly correct.
He's not doing bullshit financial analysis. In fact it is textbook financial analysis to use cash flow statements as a bullshit detector. The cash flow statement is a good reality check that will show up problems in a business that aren't obvious in the income statement. As quoted in the article -- "I'll take cash flow over gross margin -- I can eat cash flow."
Yup. But in a goodly number of these cases, the extra expense of replacing the petroleum-based product is out of line.
Well naturally, or they'd already have been replaced!
The problem I am interested in is this: if the price of oil quadruples in the next decade, which is entirely possible, where will we still be using oil no matter the cost? Certainly not for fixed power generation. Certainly not for heating new homes. Probably less for plastics. But we're still going to have to use it as a transportation fuel. There is not (yet) a reasonable substitute.
Transportation accounts for a large majority of the petroleum used in the United States. Even so, it doesn't really matter what we are using today; the important thing is which uses can be substituted and which can't.
Petroleum makes extremely good transportation fuels which are very difficult to substitute. They provide very high energy density, and reasonable safety and efficiency at a low cost. In order to compete, new technologies have extremely challenging technical barriers to overcome.
Almost everything else petroleum is used for is either comparatively easy to replace with alternative technologies or represents a comparatively small consumption. Meeting our electrical demand without fossil fuels is a challenge certainly, but it's not beyond current technology. Whereas, flying a passenger plane at 500mph without petroleum fuel is a problem so difficult that virtually nobody is even making a token effort to solve it.
Home heating oil has alternatives in natural gas and electricity. Lubricants are a tiny market compared with transportation fuels. Asphalt is a low value by-product of refining, with perfectly acceptable substitutes should it become too expensive. Plastics can be substituted in many uses, recycled in some others. Although we will probably be making plastic from petroleum long into the future, transportation is the real problem we need to be worrying about.
I see the confusion. You are thinking about sequential reads, where you are correct, the asymptotic bandwidth penalty is 1.5x for a 4 disk RAID-6. With small random reads the penalty is 3x because you have to read both data chunks and one parity chunk from the stripe even though you only need one of the data chunks. On any practical workload the penalty is likely to fall somewhere between those limits.
Memory bandwidth isn't growing as fast as CPU power, but they are both growing so much faster than practical drive throughput that they have become effectively infinite by comparison. The only way you can become memory bound doing disk IO is to have a pure sequential access pattern and dozens of spindles per CPU. And as long as we are stuck with hard disks for mass storage, that imbalance is only going to get more extreme. Memory bandwidth is one of the last things a modern filesystem designer needs to lose sleep over.
The bottom line is that ZFS establishes and maintains true data integrity from the filesystem layer down at a very reasonable cost. Neither RAID of any flavor, nor any widely used filesystem can say that. This has been an interesting discussion but I don't think there's really any more that needs to be said to defend this position.
In theory you could add protections for all those things, but it doesn't seem very useful to me; why not just test the system an to empirically discover such faults?
Yes; that's exactly what ZFS does: test the integrity of your data to empirically discover faults. On the other hand, most RAID implementations and filesystems don't test the integrity of your data; that is left up to the application layer and hardware.
I can't even guess at where you got the 2/3 number; the one I come up with is more like 1/3 worst-case-scenario, and that's only at the physical disk level, not at the host-interface level.
I came up with 2/3 because you need to read the data and both parity stripes. Actually it's worse than that, you need to read all the stripes to validate any given block, just as you do in a write operation. For example in the minimal 4 disk RAID-6 array, reading and verifying will be 4 times slower than reading without verifying. That's why nobody does it. (Also because in any level lower than RAID-6 you are still screwed; you don't know whether the data or the parity is correct.)
It's also worth noting that, while the checksum in ZFS is small, the entire data set much be read from memory and processed to calculate the checksum, in addition to whatever you plan to do with the data after you've done the integrity check.
ZFS calculates checksums on a (variable size) block level. Compared with IO bandwidth, CPU and memory bandwidth are virtually free and getting cheaper all the time.
If the data-integrity benefits of ZFS require that ZFS be implemented perfectly and only protects me from implementation errors in other, generally simpler systems, I'm not seeing a lot of practical benefit here.
You know, there are so many problems with that statement that I don't even know where to start. If you believe that implementing a reliable filesystem is on the same order of magnitude of difficulty as perfectly error-hardening every piece of firmware and hardware that will ever be used with that filesystem, there's nothing I can do to convince you otherwise.
md5sum comes to mind -- that provides file-level data integrity checking, which is superior even to the filesystem-level checking that ZFS provides. And it's pretty commonly used in an automated fashion to verify that executables and archives are intact.
ZFS has an enormous advantage over md5sum: it is completely transparent to the user. I'd say that md5sum is the problem, and ZFS is the solution.
(Of course md5sum is primarily used to solve higher level problems.)
Have you read the link I posted? It is getting old to keep repeating Jeff Bonwick's arguments when you can just read what I already pointed you to.
The underlying hardware will not necessarily notice errors. Hard drives are only designed to error protect the magnetic domains on the disk. There are all sorts of other places in the increasingly long datapath to disk where data can be lost, and, in fact, routinely does get lost.
The choice to verify every read is purely an implementation decision
RAID-6 does not verify every read because it is a stupid way to achieve data integrity. It wastes two thirds of your aggregate IO read bandwidth when you can just use checksums virtually for free. CPU cycles for checksumming is dirt cheap whereas IO bandwidth is extremely expensive.
I was just arguing the novelity you seem to think ZFS has -- using checksums to verify data integrity is not exactly cutting-edge computer science.
Yet strangely there aren't any other widely available storage solutions that provide transparent data integrity from the filesystem down.
I just don't think this particular feature is unique or superior to other available solutions for the same problem.
Then name another one. I think we've already shown that vanilla RAID does not qualify.
But RAID 5 and 6 are both capable of verifying the primary data source against the parity data and transparently correcting errors that occur on less than the critical number of disks.
:)
With RAID-5, as with RAID-1, the critical number of disks is one. RAID-5 cannot transparently correct errors that occur on even a single stripe, unless you know a priori which stripe is affected.
With RAID-6 you can automatically correct errors that occur on a single stripe, but it still does not automatically detect such errors on read the way ZFS and RAID-Z do.
Or, you know, a checksum.
That's a great idea. Too bad those ZFS guys didn't think of it. Oh, wait.
Hard drives silently losing data is a problem solved by RAID.
_ data
That is profoundly wrong. Vanilla RAID will not discover and cannot automatically correct silent data loss. The reason is that RAID has no way of knowing which data is correct. For example, if two mirrored copies disagree on the contents of a block, the data is unrecoverable without manual intervention or external knowledge. Furthermore, in normal operation your RAID subsystem will simply read data from whichever drive is idle at the time the read request comes in; it does not ordinarily compare the two mirrors. The data will remain corrupted until the user notices a problem, at which point they have no practical recourse. Essentially the same problem occurs with parity RAID.
There is no dedicated hardware in your system that provides the end to end data integrity that ZFS does. I honestly suggest you learn more about it before airing your opinions. Here is a start:
http://blogs.sun.com/bonwick/entry/zfs_end_to_end
Let me tell you a cautionary tale. Some six months ago I decided to do exactly what you're suggesting for exactly the same reason. I got OpenSolaris and installed it on a spare drive on my fileserver. It was a pain in the ass figuring out how to administer a Solaris system, which despite all the documentation seemed opaque and baroque, but I persisted. Unfortunately, the damn thing wouldn't configure the network. I couldn't figure it out. I was reduced to tracing through the init scripts line by line figuring out where they failed. It turned out that the file which contained the hostname had a trailing carriage return. I'd pressed "enter" an extra time when I was editing it. As a result the system decided it's hostname was the empty string, which it couldn't find in the hosts database and gave up. At that point I nerfed the partition in a fit of pique and went back to Linux + software RAID + reiserfs. Any software which is that brittle and administrator-hostile honestly does not deserve to live, certainly not on my server, no matter how nice its filesystem.
If I did dd for whatever reason over the partition I could just restore from the BACKUP.
If you were aware that the data was corrupted. Hard drives can silently lose data for a variety of reasons, ZFS will discover that damage whereas Linux filesystems will silently ignore it.
Honestly, read up on ZFS. Your ignorance is palpable.
The MAK isn't the key to the encrypted stream-
:)
Yes it is... RTFWiki.
This defeats TiVO's DRM that was used to prevent Linux and Mac users from watching shows on their PC.
TiVo's DRM prevented Linux and Mac users from watching their shows only as a side effect. The intent of TiVo's DRM is to prevent people who don't know the MAK from watching the content.
Please stop replying if you have no idea what you're talking about.
Ahem.
This is a nice piece of reverse engineering, but no encryption scheme was cracked.
HDTV is a classic case of chicken and egg. Without an installed base, the industry has no incentive to produce and broadcast HD content. Without content, on the other hand, there will be no installed base. You can't blame the broadcasters for following their financial incentives, any more than you can blame consumers for rejecting high priced HDTV hardware on which they had nothing to watch.
Fortunately, broadcasters, unlike consumers, are beholden to federal regulators and can be coerced. The FCC saw this chicken-and-egg problem coming and mandated terrestrial broadcast of HD content in the US. The Canadians should do the same. If you broadcast SD, you have to broadcast HD as well.
Anyway, none of this matters anymore. HDTV is finally a done deal. Between the US tuner mandate, HD capable enabled game consoles, and the price trajectory of LCD flat panels, consumer adoption of HDTV is unstoppable. Advertisers and broadcasters will be dragged along soon enough.
Broadcast TV is quite relevant to HDTV. For years now, HDTV early adopters have been depending on terrestrial broadcast. That's because premium cable and satellite providers weren't particularly interested in HDTV, for a variety of reasons. Terrestrial broadcasters weren't interested in HDTV either, but the FCC forced them to broadcast in HD anyway. Which, of course, is what the Canadian authorities should do with CBC.
As for DVDs and downloaded content, those are generally standard definition or lower.
I've heard arguments like that over and over again concerning functionality that is now implemented in software, sometimes with new dedicated CPU facilities. Graphics will be no different, it is just a matter of time. It may start with "typical" business users and laptop hardware, but sooner rather than later, the architectural and cost advantages of integration and generality will always outweigh putting huge amounts of custom logic on the wrong side of an IO bus.
The handwriting is on the wall if you know where to look. CPU and graphics architectures are converging.
First the CPU side. More and more of the work being done on desktop CPUs is media related: HDTV and DIVX encoding/decoding, digital photography, audio DSP, game physics, software radio. General purpose superscalar CPUs are horribly inefficient for this class of problems. At the same time, new chip processes are providing more transistors than CPU designers know what to do with. At first they just added more cores, but that is already running into scaling problems. They're clearly tempted to add some sort of highly parallel SIMD engine that is better suited for DSP type problems. Look at MMX in all its guises. Look at the Cell architecture. Both are steps in this direction.
Then the graphics side. The clear direction here is away from specialized hardware for each stage of the graphics pipeline, and toward large numbers of more general computation elements in a SIMD architecture. Look at NVidia's latest chip and you'll see something that is completely unlike traditional graphics pipelines. However it's almost exactly like what CPU designers want to put in desktop chips anyway, to solve the problems that superscalar processors are bad at.
Finally, look at some of the other stuff we are putting on the wrong side of the IO bus: Creative Labs X-Fi, AGEIA PhysX. These are also examples of convergence toward highly parallel and relatively general DSP architectures.
Even if power gamers like to upgrade their graphics card every few months, there are other markets which are increasingly driving system architecture. Game consoles are a huge market, and the next generation of consoles is poised to take advantage of the convergence I'm talking about. Vista is a big market for 3d graphics on business PCs, and that market is all about higher integration and lower cost. Neither of these markets will ever upgrade graphics independent of CPU, but they are paying for super high bandwidth IO busses and duplicate memory interfaces which they don't need.
Graphics hardware on an IO bus has about 5 years to live, in my opinion.
The history of computer architecture is a ceaseless march toward higher integration and higher generality. That is, what was once special purpose hardware in a seperate unit is now implemented in firmware as part of a more flexible, general purpose processor.
This march is littered with those standing by the wayside saying things like, "Who needs floating point in the CPU? Leave it on a seperate chip!" or "I want to be able to upgrade my CPU without buying a new memory controller!" or "If you integrate sound on the motherboard, I'll just have to buy a seperate sound card."
So, while I'm sure you're not the only one who thinks GPU integration is a bad idea, that doesn't mean it is a bad idea.