Domain: wikipedia.org
Stories and comments across the archive that link to wikipedia.org.
Stories · 7,048
-
Jonathan Koomey Answers Your Questions
A couple weeks ago, you asked questions of Stanford professor Jonathan Koomey about what has been dubbed Koomey's Law — the idea that the energy efficiency of computing doubles every 1.5 years. Read on for Professor Koomey's answers to the questions you raised. What makes this a non-trivial extension?
by Anonymous
What makes your law a non-trivial extension of Moore's Law, which states that the transistor count would double every 18 months due to an increase in density? E&M theory states that if you cut a wire's length in half, it's resistance cuts in half. Granted density in this case is a 2 dimensional expansion and wire resistance is a 1 dimensional formula, but what makes this different from what a freshman in college can infer from an R = (resistivity * length)/cross sectional area?
Jonathan Koomey: First, it’s important to note that we assessed these trends empirically, using measured power data for each computer system in our dataset, and it’s often valuable to confirm with actual measurements what theory implies. Just because the result sounds intuitive to you after the fact doesn’t mean that it isn’t valuable to confirm with real data that the trends actually exist. And of course we discuss in the paper the driving forces behind the reductions in power use per logical switch (and they involve more than just reductions in I squared R losses in the wires). I’ve pasted below two relevant paragraphs from the article:
For vacuum tube computers, both computational speed and reliability issues encouraged computer designers to reduce power use. Heat reduces reliability, which was a major issue for tube-based computers. In addition, increasing computation speeds went hand in hand with technological changes (like reduced capacitive loading, lower currents, and smaller tubes) that also reduced power use. And the economics of operating a tube-based computer led to pressure to reduce power use, although this issue was probably a secondary one in the early days of electronic computing.
For transistorized and microprocessor based computers, the driving factor for power reductions was (and is) the push to reduce the physical dimensions of transistors, which reduces the cost per transistor. In order to accomplish this goal, power used per transistor also must be reduced; otherwise the power densities on the silicon rapidly become unmanageable. Per transistor power use is directly proportional to the length of the transistor between source and drain, the ratio of transistor length to mean free path of the electrons, and the total number of electrons in the operating transistor, as Feynman (2001) pointed out. Shrinking transistor size therefore resulted in improved speed, reduced cost, and reduced power use per transistor (see also Bohr (2007) and Carver Mead’s thinking in the late 1960s, as summarized in Brock (2006, pp. 98-100)).In addition, the fact that the trends have now been confirmed empirically means that people can get on with considering the implications of these trends, which I think are under-appreciated. The idea that we’ll be able to use ever more efficient computing technology in distributed applications will revolutionize data collection, communications, and control of processes, and people are only now starting to think about what may become possible.
As one of many examples showing the potential of ultra low power computing, consider the wireless no-battery sensors created by Joshua R. Smith of Intel and the University of Washington (coverage in the NY Times and the Economist). These sensors scavenge energy from stray television and radio signals, and they use so little power (60 microwatts in this example) that they don’t need any other power source. Stray light, motion, or heat can also be converted to meet slightly higher power needs, perhaps measured in milliwatts. The contours of this exciting design space are only beginning to be explored, and they are enabled by the trends identified in our paper.
I wouldn’t underestimate the importance of a shift in industry focus from raw performance to power efficiency for mobile devices. Some of the best engineers will be drawn to the problems of ultra low power computing in the same way as they’ve were drawn to high performance computing (HPC) in the past (no doubt terrific technologists will also continue to focus on HPC, but anytime a new hot area opens up there’s a migration of talent to that new topic).
Finally, I would add that the truly unexpected result was that the trend in computational efficiency extends for a longer period than Moore’s law, all the way back to Eniac in 1946. So these trends in computational efficiency are an inherent characteristic of computers that use electrons for switching, and are not limited to the microprocessor era. I, for one, did not expect that.
Your Take on Futurists?
by eldavojohn
What is your take on the interpretation of Futurists -- like Raymond Kurzweil -- in regards to extrapolating these 'laws' out to extreme distances?JK: The physicist Neils Bohr once famously said “Prediction is very difficult, especially about the future.” It’s important to be careful in making long-term extrapolations, even if some technological trend has continued for some time. I think it’s fair to say that Moore’s law (and the trends in computational efficiency we identify) have more years to run, given how far we are from theoretical limits, but exactly when we’ll hit a real roadblock it will take someone more brash than me to say. I discuss the theoretical limit based on Feynman’s calculations below, and we will eventually reach that, but there may be ways to sidestep those limits. We’ll have to see how clever we can be!
Lets work this backwards ...
by PPH
... and see where the Babbage Engine fits on the curve.JK: Since the Babbage engine never operated, I’m not sure how we could do this. I believe that some parts of the engine have been created using modern machining practices, but I don’t think anyone has ever made one in complete form. If someone has, I’d be interested to measure its electricity use and estimate its performance (of course, it was designed before the era of electricity). Nordhaus (2007) reports that
Early calculators were “dumb” machines that essentially relied on incrementation of digits. An important step in the development of modern computers was mechanical representation of logical steps. The first commercially practical information-processing machine was the Jacquard loom, developed in 1804. This machine used interchangeable punched cards that controlled the weaving and allowed a large variety of patterns to be produced automatically. This invention was part of the inspiration of Charles Babbage, who developed one of the great precursor inventions in computation. He designed two major conceptual breakthroughs, the “Difference Engine” and the “Analytical Engine.” The latter sketched the first programmable digital computer. Neither of the Babbage machines was constructed during his lifetime. An attempt in the 1990s by the British Museum to build the simpler Difference Engine using early-nineteenth-century technologies failed to perform its designed tasks. (reference: Swade, Doron. The Difference Engine. New York: Viking Press, 2000.)
Nordhaus, William D. 2007. "Two Centuries of Productivity Growth in Computing." The Journal of Economic History. vol. 67, no. 1. March. pp. 128-159. [http://nordhaus.econ.yale.edu/recent_stuff.html]
Nordhaus does attempt to estimate the speed of computation possible by hand calculations as well as abacuses, to compare to more automatic methods.
Infinity w/ reversible computing?
by DriedClexlerThis one doesn't seem to have fundamental physical limits, so long as we eventually transition to reversible computing, in which the computer does not use up useful energy because every process it uses is fully reversible (i.e. the original state could be inferred).
All the limits on computation (except regarding storage) that you hear about (e.g. Landauer limit) are on irreversible computing, which is how current architecture works. It is the irreversibility of an operation that causes it to increase entropy.
Could the whole process be bypassed by the near-infinite efficiency of reversible computers?
JK:Here’s the flip answer: Only if you can afford to wait infinitely long for your answer.
Here’s the more serious answer: in principle, reversible computing could have a revolutionary impact, if we could figure out how to do it, and some folks are working on this. But I haven’t seen any near term applications of such devices—if you know of any, please let me know.
Multicore or System on a Chip Speed bumps?
by eldavojohn
A lot of consumer grade machines have begun focusing on multicore chips with a lower frequency to provide the same or better perceived computing performance than a high frequency single core chip. What happens when a technology like this subverts our craving for higher transistor density? Can you argue that your "law" is immune to researchers focusing on some hot new technology like a thousand core processor or a beefed up system on a chip in order to improve end user experience over pure algorithm crunching speed?JK: First, I would call it (like Moore implied in his own papers) an empirical observation rather than a law.
But in any case, I don’t think that the transition to multicore has “subverted our craving for higher transistor density,” we’re just using the transistors in a different way. The density of chips (measured in components per square centimeter or equivalent metric) will continue to increase, it’s just that the scaling of clock speeds that drove performance increases for so long is no longer possible (mainly because of high leakage currents inside the chip). So that means we need to make many cores and then modify software to capture that performance.
At the end of the day, WHAT you choose to do with the computing power is unrelated to the trends we identify, but I would argue the focus of device and software design is inevitably moving towards enhancing the end-user experience because these trends in efficiency are allowing ever more mobile devices to serve people’s immediate needs in an ever more personal way.
How will this affect programmers?
by AnonymousWhen we eventually hit the physical limits of atoms, will programmers eventually stop their autistic quest for more and more layers, more and more complexity and more and more languages to move a number from one address to another?
How will programmers affect this?
by skidsWhile sarcastic, the above question is an important one: as computing power has increased, the tendency of coders to just ride over badly coded underlayers rather than redesign them competently and efficiently has increased. Why bother cutting out bloat that causes an 80% penalty on system efficiency when you can just use a more efficient chipset to get the same result?
So my question is whether you have put any thought into similarly quantifying the opposing software bloat factor, and what he sees the total balance of system works out to.
JK: Software bloat is a real issue, and I agree with your analysis that the ever-improving hardware picture has allowed poor coding practices to continue. But with the shift to multicore, there’s been at least some burden on programmers to change their ways—they have to modify their code to take advantage of multicore performance, so their skills are actually needed to capture increased performance (which is new, or at least a throwback to the early days of computing, when the programmers had so few hardware resources to work with that they had to be extremely parsimonious in their coding).
In the paper, we write:
Whether performance per CPU can grow for many years more at the historical pace is an ongoing subject of debate in the computer industry (Bohr 2007), but near-term improvements are already “in the pipeline”. Continuing the historical trends in performance (or surpassing them) is at this juncture dependent on significant new innovation comparable in scale to the shift from single core to multi-core computing. Such innovation will also require substantial changes in software design (Asanovíc et al. 2006), which is a relatively new development for the IT industry and it is another reason why whole system redesign is so critical to success.
This really doesn’t address the serious issue you raise about bloatware, which I think is a generic problem that other people more skilled in software design than me can address much better than I can. It’s hard to quantify it because it is so situation specific, but someone at a university somewhere may have tried to do this—I just don’t know.
Applied to Other Kinds of Computing?
by Anonymous
How well does Koomey's Law fit other kinds of computing? For instance, has the energy efficiency of cell phone microprocessors followed the same trend as desktop computers and servers? What about embedded systems like routers and car engine controllers, or specialized hardware like game consoles?JK: These are all excellent questions (which we raise in the article) and I’m actively seeking data, but I don’t have anything new to report on this yet. I’m also interested in trends in data transmission power efficiencies, because that’s a key limitation on these mobile devices. And I’m digging around for battery capacity data over time as well.
Moral/Ethical
by vlm
Here is the list of moral / ethical arguments about the path we're on, as seen in your law. You saw the path clearly enough to define a time based law. Are there any issues I'm not seeing on our current path?
1) Lower energy consumption at point of use
2) Higher energy consumption at manufacturing point
3) faster cpu = bigger programs = more bugs = lower quality of life
4) faster cpu = stronger DRM possibilities
5) Better processing * battery life = better medical devices
6) Better processing * battery life = better 1984 style totalitarian devices
7) Lower energy consumption = less air conditioning demand = decreasing average latitude of data centers = population shifts or whatever or something?
8) More money required for both hw and sw development = good for big corps and bad for the little guy
JK: Hmmm, I’m not quite sure where you are going with this. There are pluses and minuses to all technological innovations, but I’m pretty sure the benefits will outweigh the costs in this case (as long as we put proper restraints on how collected data can be accessed by the authorities).
Batter Capacity vs. Processor Speed
by vlm
Have you run into a law relating battery capacity (either per Kg or L) vs processor speed over time? I bet there is some kind of interesting curve for mobile devices. Or, maybe not — that’s why I'm asking a guy with previous success at data analysis in a closely related field...JK: Great questions. I haven’t seen any quantitative regularity in how battery power densities vary over time, but am actively looking for data. I hope to have something to report about that (along with the other trends I’m investigating, as I describe above). If you know of any good data sources, please let me know.
Queen of Hearts
by Anonymous
What do you think about the following observation: that every X years the amount of computing operations we use to perform basic calculations doubles (by virtue of doing those calculations with more complex software, slower languages...), so when you factor in Moore's law (and your own), the amount of useful calculations we do with computers remain more or less constant.JK: This is related to the bloatware question above. I haven’t seen any quantitative estimates of the real cost from bloatware, but computing is becoming more widely distributed throughout the society, and it’s hard to believe that will the proliferation of more and more mobile devices and all the chips now incorporated in embedded systems that we’re doing less useful computing work than in the past. Some folks have tried to quantify total computational work being done, but it’s hard to do: Hilbert, Martin, and Priscila López. 2011. "The World's Technological Capacity to Store, Communicate, and Compute Information." Science. vol. 332, no. 6025. April 1. pp. 60-65
Feynman Quote
by yakolevMr. Koomey, if we take your numbers from the attached article, which may not have been quoted correctly
Feynman indicated that there was approximately 100 billion times efficiency improvement possible, and 40,000 times improvement has happened so far.
If we take Feynman's number at face value, this means that if computing efficiency improvements continue at the current rate (doubling every 18 months,) we will reach the theoretical maximum in 2043.
Based on that, do you believe that we will see a dramatic reduction in efficiency improvements in the next 10-20 years as we approach the theoretical limit, or do you think Feynman was conservative in his estimate?
JK: Your math is correct, as is the quotation of those numbers. If computing efficiency doubles every 1.5 years, it will take 21.3 doublings before we reach the theoretical limits identified by Feynman, which means will hit that limit in 32 years (i.e. in 2043).
Here’s what Feynman had to say in the book I cited:
Of course there is a limitation, the practical limitation anyway, that the bits must be of the size of an atom and a transistor 3 or 4 atoms; the quantum mechanical gate I used has 3 atoms. (I would not try to write my bits on to nuclei, I’ll wait till the technological development reaches the atoms before I need to go any further!) That leads us just with (a) the limitations in size to the size of atoms, (b) the energy requirements depending on the time as worked out by Bennett, (c) and the feature that I did not mention concerning the speed of light; we can’t send the signals any faster than the speed of light. Those are the only physical limitations that I know on computers.
If we make an atomic size computer, somehow, it would mean that the dimension, the linear dimension is a thousand to ten thousands times smaller than those very tiny chips that we have now. It means that the volume of the computer is 100 billionth, 1011 of the present volume, because the transistor is that much smaller 1011 , than the transistors that we make today. The energy requirement for a single switch is also about eleven orders of magnitude smaller than the energy required to switch the transistor today, and the time to make the transitions will be at least ten thousands times faster per step of calculation. So there is plenty of room for improvement in the computer and I leave you, practical people who work on computers, this as an aim to get to. (Feynman, Richard P. 2001. The Pleasure of Finding Things Out: The Best Short Works of Richard P. Feynman. London, UK: Penguin Books.)So the calculation Feynman did was based on a transistor using just three atoms. In theory, one could use individual nuclei (as Feynman suggests) or there may be another as yet totally unknown way to crack this nut. But using Feynman’s calculation as the ultimate limit, in about three decades (and probably before that) we’re going to hit some kind of limit using our current methods.
But even given that, we’ve got at least another decade of improvements (that’s what my friends at Intel tell me) and probably more. Every decade means a factor of 100 improvement in the power efficiency of computing (doubling every 1.5 years) but there are also vast improvements we can make in our software as well as our implementation of power savings in the standby power of these devices (which turns out to be a much bigger power drain than the active power, given that almost all computers have very low average utilization). Hitting these limits may actually force the software designers to get more efficient (we’ll see!). And we’re just at the beginning of using the technologies enabled by these trends to accomplish human goals, so I’m hopeful we’ll be clever and figure out loads of important applications that will become possible with a factor of 100 or 1000 improvements in efficiency over the next 15 years.
Haven`t we already fallen behind?
by Anonymous
The Pentium M (which is powering the computer that I`m using to type this) came out eight years ago. Let`s call it 7.5 and make our "Koomey factor" 2^5=32. The ULV chip ran at 1.1GHz and ate 6.4W, and we can add on the power of the 855PM northbridge which would make the total 8.2W. I don`t see any products on the market that are anywhere close to a 32x improvement on performance per watt. Do you?JK: Our focus is on system power, not chip power alone. And you need to calculate what your current system is capable of in computations per kWh (which you can calculate from performance per watt) so you can compare to our numbers. But I’ll wager that the current crop of laptops (or the new Mac Mini) will blow away your old machine in terms of computations per kWh at maximum performance (which is what we measure).
-
Jonathan Koomey Answers Your Questions
A couple weeks ago, you asked questions of Stanford professor Jonathan Koomey about what has been dubbed Koomey's Law — the idea that the energy efficiency of computing doubles every 1.5 years. Read on for Professor Koomey's answers to the questions you raised. What makes this a non-trivial extension?
by Anonymous
What makes your law a non-trivial extension of Moore's Law, which states that the transistor count would double every 18 months due to an increase in density? E&M theory states that if you cut a wire's length in half, it's resistance cuts in half. Granted density in this case is a 2 dimensional expansion and wire resistance is a 1 dimensional formula, but what makes this different from what a freshman in college can infer from an R = (resistivity * length)/cross sectional area?
Jonathan Koomey: First, it’s important to note that we assessed these trends empirically, using measured power data for each computer system in our dataset, and it’s often valuable to confirm with actual measurements what theory implies. Just because the result sounds intuitive to you after the fact doesn’t mean that it isn’t valuable to confirm with real data that the trends actually exist. And of course we discuss in the paper the driving forces behind the reductions in power use per logical switch (and they involve more than just reductions in I squared R losses in the wires). I’ve pasted below two relevant paragraphs from the article:
For vacuum tube computers, both computational speed and reliability issues encouraged computer designers to reduce power use. Heat reduces reliability, which was a major issue for tube-based computers. In addition, increasing computation speeds went hand in hand with technological changes (like reduced capacitive loading, lower currents, and smaller tubes) that also reduced power use. And the economics of operating a tube-based computer led to pressure to reduce power use, although this issue was probably a secondary one in the early days of electronic computing.
For transistorized and microprocessor based computers, the driving factor for power reductions was (and is) the push to reduce the physical dimensions of transistors, which reduces the cost per transistor. In order to accomplish this goal, power used per transistor also must be reduced; otherwise the power densities on the silicon rapidly become unmanageable. Per transistor power use is directly proportional to the length of the transistor between source and drain, the ratio of transistor length to mean free path of the electrons, and the total number of electrons in the operating transistor, as Feynman (2001) pointed out. Shrinking transistor size therefore resulted in improved speed, reduced cost, and reduced power use per transistor (see also Bohr (2007) and Carver Mead’s thinking in the late 1960s, as summarized in Brock (2006, pp. 98-100)).In addition, the fact that the trends have now been confirmed empirically means that people can get on with considering the implications of these trends, which I think are under-appreciated. The idea that we’ll be able to use ever more efficient computing technology in distributed applications will revolutionize data collection, communications, and control of processes, and people are only now starting to think about what may become possible.
As one of many examples showing the potential of ultra low power computing, consider the wireless no-battery sensors created by Joshua R. Smith of Intel and the University of Washington (coverage in the NY Times and the Economist). These sensors scavenge energy from stray television and radio signals, and they use so little power (60 microwatts in this example) that they don’t need any other power source. Stray light, motion, or heat can also be converted to meet slightly higher power needs, perhaps measured in milliwatts. The contours of this exciting design space are only beginning to be explored, and they are enabled by the trends identified in our paper.
I wouldn’t underestimate the importance of a shift in industry focus from raw performance to power efficiency for mobile devices. Some of the best engineers will be drawn to the problems of ultra low power computing in the same way as they’ve were drawn to high performance computing (HPC) in the past (no doubt terrific technologists will also continue to focus on HPC, but anytime a new hot area opens up there’s a migration of talent to that new topic).
Finally, I would add that the truly unexpected result was that the trend in computational efficiency extends for a longer period than Moore’s law, all the way back to Eniac in 1946. So these trends in computational efficiency are an inherent characteristic of computers that use electrons for switching, and are not limited to the microprocessor era. I, for one, did not expect that.
Your Take on Futurists?
by eldavojohn
What is your take on the interpretation of Futurists -- like Raymond Kurzweil -- in regards to extrapolating these 'laws' out to extreme distances?JK: The physicist Neils Bohr once famously said “Prediction is very difficult, especially about the future.” It’s important to be careful in making long-term extrapolations, even if some technological trend has continued for some time. I think it’s fair to say that Moore’s law (and the trends in computational efficiency we identify) have more years to run, given how far we are from theoretical limits, but exactly when we’ll hit a real roadblock it will take someone more brash than me to say. I discuss the theoretical limit based on Feynman’s calculations below, and we will eventually reach that, but there may be ways to sidestep those limits. We’ll have to see how clever we can be!
Lets work this backwards ...
by PPH
... and see where the Babbage Engine fits on the curve.JK: Since the Babbage engine never operated, I’m not sure how we could do this. I believe that some parts of the engine have been created using modern machining practices, but I don’t think anyone has ever made one in complete form. If someone has, I’d be interested to measure its electricity use and estimate its performance (of course, it was designed before the era of electricity). Nordhaus (2007) reports that
Early calculators were “dumb” machines that essentially relied on incrementation of digits. An important step in the development of modern computers was mechanical representation of logical steps. The first commercially practical information-processing machine was the Jacquard loom, developed in 1804. This machine used interchangeable punched cards that controlled the weaving and allowed a large variety of patterns to be produced automatically. This invention was part of the inspiration of Charles Babbage, who developed one of the great precursor inventions in computation. He designed two major conceptual breakthroughs, the “Difference Engine” and the “Analytical Engine.” The latter sketched the first programmable digital computer. Neither of the Babbage machines was constructed during his lifetime. An attempt in the 1990s by the British Museum to build the simpler Difference Engine using early-nineteenth-century technologies failed to perform its designed tasks. (reference: Swade, Doron. The Difference Engine. New York: Viking Press, 2000.)
Nordhaus, William D. 2007. "Two Centuries of Productivity Growth in Computing." The Journal of Economic History. vol. 67, no. 1. March. pp. 128-159. [http://nordhaus.econ.yale.edu/recent_stuff.html]
Nordhaus does attempt to estimate the speed of computation possible by hand calculations as well as abacuses, to compare to more automatic methods.
Infinity w/ reversible computing?
by DriedClexlerThis one doesn't seem to have fundamental physical limits, so long as we eventually transition to reversible computing, in which the computer does not use up useful energy because every process it uses is fully reversible (i.e. the original state could be inferred).
All the limits on computation (except regarding storage) that you hear about (e.g. Landauer limit) are on irreversible computing, which is how current architecture works. It is the irreversibility of an operation that causes it to increase entropy.
Could the whole process be bypassed by the near-infinite efficiency of reversible computers?
JK:Here’s the flip answer: Only if you can afford to wait infinitely long for your answer.
Here’s the more serious answer: in principle, reversible computing could have a revolutionary impact, if we could figure out how to do it, and some folks are working on this. But I haven’t seen any near term applications of such devices—if you know of any, please let me know.
Multicore or System on a Chip Speed bumps?
by eldavojohn
A lot of consumer grade machines have begun focusing on multicore chips with a lower frequency to provide the same or better perceived computing performance than a high frequency single core chip. What happens when a technology like this subverts our craving for higher transistor density? Can you argue that your "law" is immune to researchers focusing on some hot new technology like a thousand core processor or a beefed up system on a chip in order to improve end user experience over pure algorithm crunching speed?JK: First, I would call it (like Moore implied in his own papers) an empirical observation rather than a law.
But in any case, I don’t think that the transition to multicore has “subverted our craving for higher transistor density,” we’re just using the transistors in a different way. The density of chips (measured in components per square centimeter or equivalent metric) will continue to increase, it’s just that the scaling of clock speeds that drove performance increases for so long is no longer possible (mainly because of high leakage currents inside the chip). So that means we need to make many cores and then modify software to capture that performance.
At the end of the day, WHAT you choose to do with the computing power is unrelated to the trends we identify, but I would argue the focus of device and software design is inevitably moving towards enhancing the end-user experience because these trends in efficiency are allowing ever more mobile devices to serve people’s immediate needs in an ever more personal way.
How will this affect programmers?
by AnonymousWhen we eventually hit the physical limits of atoms, will programmers eventually stop their autistic quest for more and more layers, more and more complexity and more and more languages to move a number from one address to another?
How will programmers affect this?
by skidsWhile sarcastic, the above question is an important one: as computing power has increased, the tendency of coders to just ride over badly coded underlayers rather than redesign them competently and efficiently has increased. Why bother cutting out bloat that causes an 80% penalty on system efficiency when you can just use a more efficient chipset to get the same result?
So my question is whether you have put any thought into similarly quantifying the opposing software bloat factor, and what he sees the total balance of system works out to.
JK: Software bloat is a real issue, and I agree with your analysis that the ever-improving hardware picture has allowed poor coding practices to continue. But with the shift to multicore, there’s been at least some burden on programmers to change their ways—they have to modify their code to take advantage of multicore performance, so their skills are actually needed to capture increased performance (which is new, or at least a throwback to the early days of computing, when the programmers had so few hardware resources to work with that they had to be extremely parsimonious in their coding).
In the paper, we write:
Whether performance per CPU can grow for many years more at the historical pace is an ongoing subject of debate in the computer industry (Bohr 2007), but near-term improvements are already “in the pipeline”. Continuing the historical trends in performance (or surpassing them) is at this juncture dependent on significant new innovation comparable in scale to the shift from single core to multi-core computing. Such innovation will also require substantial changes in software design (Asanovíc et al. 2006), which is a relatively new development for the IT industry and it is another reason why whole system redesign is so critical to success.
This really doesn’t address the serious issue you raise about bloatware, which I think is a generic problem that other people more skilled in software design than me can address much better than I can. It’s hard to quantify it because it is so situation specific, but someone at a university somewhere may have tried to do this—I just don’t know.
Applied to Other Kinds of Computing?
by Anonymous
How well does Koomey's Law fit other kinds of computing? For instance, has the energy efficiency of cell phone microprocessors followed the same trend as desktop computers and servers? What about embedded systems like routers and car engine controllers, or specialized hardware like game consoles?JK: These are all excellent questions (which we raise in the article) and I’m actively seeking data, but I don’t have anything new to report on this yet. I’m also interested in trends in data transmission power efficiencies, because that’s a key limitation on these mobile devices. And I’m digging around for battery capacity data over time as well.
Moral/Ethical
by vlm
Here is the list of moral / ethical arguments about the path we're on, as seen in your law. You saw the path clearly enough to define a time based law. Are there any issues I'm not seeing on our current path?
1) Lower energy consumption at point of use
2) Higher energy consumption at manufacturing point
3) faster cpu = bigger programs = more bugs = lower quality of life
4) faster cpu = stronger DRM possibilities
5) Better processing * battery life = better medical devices
6) Better processing * battery life = better 1984 style totalitarian devices
7) Lower energy consumption = less air conditioning demand = decreasing average latitude of data centers = population shifts or whatever or something?
8) More money required for both hw and sw development = good for big corps and bad for the little guy
JK: Hmmm, I’m not quite sure where you are going with this. There are pluses and minuses to all technological innovations, but I’m pretty sure the benefits will outweigh the costs in this case (as long as we put proper restraints on how collected data can be accessed by the authorities).
Batter Capacity vs. Processor Speed
by vlm
Have you run into a law relating battery capacity (either per Kg or L) vs processor speed over time? I bet there is some kind of interesting curve for mobile devices. Or, maybe not — that’s why I'm asking a guy with previous success at data analysis in a closely related field...JK: Great questions. I haven’t seen any quantitative regularity in how battery power densities vary over time, but am actively looking for data. I hope to have something to report about that (along with the other trends I’m investigating, as I describe above). If you know of any good data sources, please let me know.
Queen of Hearts
by Anonymous
What do you think about the following observation: that every X years the amount of computing operations we use to perform basic calculations doubles (by virtue of doing those calculations with more complex software, slower languages...), so when you factor in Moore's law (and your own), the amount of useful calculations we do with computers remain more or less constant.JK: This is related to the bloatware question above. I haven’t seen any quantitative estimates of the real cost from bloatware, but computing is becoming more widely distributed throughout the society, and it’s hard to believe that will the proliferation of more and more mobile devices and all the chips now incorporated in embedded systems that we’re doing less useful computing work than in the past. Some folks have tried to quantify total computational work being done, but it’s hard to do: Hilbert, Martin, and Priscila López. 2011. "The World's Technological Capacity to Store, Communicate, and Compute Information." Science. vol. 332, no. 6025. April 1. pp. 60-65
Feynman Quote
by yakolevMr. Koomey, if we take your numbers from the attached article, which may not have been quoted correctly
Feynman indicated that there was approximately 100 billion times efficiency improvement possible, and 40,000 times improvement has happened so far.
If we take Feynman's number at face value, this means that if computing efficiency improvements continue at the current rate (doubling every 18 months,) we will reach the theoretical maximum in 2043.
Based on that, do you believe that we will see a dramatic reduction in efficiency improvements in the next 10-20 years as we approach the theoretical limit, or do you think Feynman was conservative in his estimate?
JK: Your math is correct, as is the quotation of those numbers. If computing efficiency doubles every 1.5 years, it will take 21.3 doublings before we reach the theoretical limits identified by Feynman, which means will hit that limit in 32 years (i.e. in 2043).
Here’s what Feynman had to say in the book I cited:
Of course there is a limitation, the practical limitation anyway, that the bits must be of the size of an atom and a transistor 3 or 4 atoms; the quantum mechanical gate I used has 3 atoms. (I would not try to write my bits on to nuclei, I’ll wait till the technological development reaches the atoms before I need to go any further!) That leads us just with (a) the limitations in size to the size of atoms, (b) the energy requirements depending on the time as worked out by Bennett, (c) and the feature that I did not mention concerning the speed of light; we can’t send the signals any faster than the speed of light. Those are the only physical limitations that I know on computers.
If we make an atomic size computer, somehow, it would mean that the dimension, the linear dimension is a thousand to ten thousands times smaller than those very tiny chips that we have now. It means that the volume of the computer is 100 billionth, 1011 of the present volume, because the transistor is that much smaller 1011 , than the transistors that we make today. The energy requirement for a single switch is also about eleven orders of magnitude smaller than the energy required to switch the transistor today, and the time to make the transitions will be at least ten thousands times faster per step of calculation. So there is plenty of room for improvement in the computer and I leave you, practical people who work on computers, this as an aim to get to. (Feynman, Richard P. 2001. The Pleasure of Finding Things Out: The Best Short Works of Richard P. Feynman. London, UK: Penguin Books.)So the calculation Feynman did was based on a transistor using just three atoms. In theory, one could use individual nuclei (as Feynman suggests) or there may be another as yet totally unknown way to crack this nut. But using Feynman’s calculation as the ultimate limit, in about three decades (and probably before that) we’re going to hit some kind of limit using our current methods.
But even given that, we’ve got at least another decade of improvements (that’s what my friends at Intel tell me) and probably more. Every decade means a factor of 100 improvement in the power efficiency of computing (doubling every 1.5 years) but there are also vast improvements we can make in our software as well as our implementation of power savings in the standby power of these devices (which turns out to be a much bigger power drain than the active power, given that almost all computers have very low average utilization). Hitting these limits may actually force the software designers to get more efficient (we’ll see!). And we’re just at the beginning of using the technologies enabled by these trends to accomplish human goals, so I’m hopeful we’ll be clever and figure out loads of important applications that will become possible with a factor of 100 or 1000 improvements in efficiency over the next 15 years.
Haven`t we already fallen behind?
by Anonymous
The Pentium M (which is powering the computer that I`m using to type this) came out eight years ago. Let`s call it 7.5 and make our "Koomey factor" 2^5=32. The ULV chip ran at 1.1GHz and ate 6.4W, and we can add on the power of the 855PM northbridge which would make the total 8.2W. I don`t see any products on the market that are anywhere close to a 32x improvement on performance per watt. Do you?JK: Our focus is on system power, not chip power alone. And you need to calculate what your current system is capable of in computations per kWh (which you can calculate from performance per watt) so you can compare to our numbers. But I’ll wager that the current crop of laptops (or the new Mac Mini) will blow away your old machine in terms of computations per kWh at maximum performance (which is what we measure).
-
Ask Slashdot: How Do You View the Wall Street Protests?
__roo writes "The New York Times reports that the Occupy Wall Street movement has inspired hundreds of Facebook pages, Twitter posts, and Meetup events, and that 'blog posts and photographs from all over the country are popping up on the WeArethe99Percent blog on Tumblr from people who see themselves as victims of not just a sagging economy but also economic injustice.' What do Slashdotters think? Do you relate to the 99% stories? Do they make you angry — either at the system, or at the protesters? If it's at the protesters, is it rational or a just-world effect?" -
Oracle To Bring Dtrace To Linux
mvar writes "Dtrace co-author Adam Leventhal writes on his blog about Dtrace for Linux: 'Yesterday (October 4, 2011) Oracle made the surprising announcement that they would be porting some key Solaris features, DTrace and Zones, to Oracle Enterprise Linux. As one of the original authors, the news about DTrace was particularly interesting to me, so I started digging. Even among Oracle employees, there's uncertainty about what was announced. Ed Screven gave us just a couple of bullet points in his keynote; Sergio Leunissen, the product manager for OEL, didn't have further details in his OpenWorld talk beyond it being a beta of limited functionality; and the entire Solaris team seemed completely taken by surprise. Leunissen stated that only the kernel components of DTrace are part of the port. It's unclear whether that means just fbt or includes sdt and the related providers. It sounds certain, though, that it won't pass the DTrace test suite which is the deciding criterion between a DTrace port and some sort of work in progress.'" -
The Games Programmers Play
An anonymous reader writes "Cort Stratton, a developer who has worked on graphics code for many first-party PS3 games, wrote an article about the kinds of games that appeal to programmers. He covers coding-friendly games of varying depth, mentioning basics like RoboRally, RoboSport and Frozen Synapse before moving on to more complex options. Quoting: 'On the surface, SpaceChem has nothing to do with programming; it's merely a futuristic puzzle game in which you build factories that convert one or more input molecules into one or more output molecules. Each factory contains a pair of independent molecule manipulators (the game calls them "waldos") which follow a fixed path through the work area. Waldos can grab, drop, and rotate molecules, make and break chemical bonds between atoms, request new input molecules and submit output molecules. ... Don't be fooled! This isn't a game about chemistry; it's actually the closest thing I've ever seen to a low-level SPU programming simulator! Each factory is an SPU running a single task. The two waldos are the SPU's dual execution pipelines. Moving and editing molecules is analogous to reading, writing and operating on data in local store.'" -
Graphene Creates Electricity When Struck By Light
MrSeb writes with news out of MIT about another interesting and potentially useful property of graphene. Researchers have known for several years that graphene generates electricity when exposed to sunlight, but incorrectly attributed it to the photovoltaic effect. A new paper shows that the current is actually generated from the much more unusual 'hot-carrier' response. Quoting: "The material’s electrons, which carry current, are heated by the light, but the lattice of carbon nuclei that forms graphene’s backbone remains cool. It’s this difference in temperature within the material that produces the flow of electricity. ... Such differential heating has been observed before, but only under very special circumstances: either at ultralow temperatures (measured in thousandths of a degree above absolute zero), or when materials are blasted with intense energy from a high-power laser. This response in graphene, by contrast, occurs across a broad range of temperatures all the way up to room temperature, and with light no more intense than ordinary sunlight." It will take more work to determine what new applications are reasonable from an efficiency perspective, but it does broaden graphene's already-impressive capabilities. -
Indian Mathematician Takes Shot At Proving Riemann Hypothesis
First time accepted submitter jalfreize writes "Indian Mathematician Rohit Gupta (known by the moniker @fadesingh on twitter) has announced an online workshop which he intends to 'conclude by attacking an important problem in front of (the participants), in public view.' The problem is the Riemann Hypothesis, first proposed in 1859. Rohit outlines his approach based on quasicrystals first outlined by Freeman Dyson. His audacious plan, coupled with this recent news about quasicrystals, has kicked up a storm of interest in the Indian twitterverse." -
Indian Mathematician Takes Shot At Proving Riemann Hypothesis
First time accepted submitter jalfreize writes "Indian Mathematician Rohit Gupta (known by the moniker @fadesingh on twitter) has announced an online workshop which he intends to 'conclude by attacking an important problem in front of (the participants), in public view.' The problem is the Riemann Hypothesis, first proposed in 1859. Rohit outlines his approach based on quasicrystals first outlined by Freeman Dyson. His audacious plan, coupled with this recent news about quasicrystals, has kicked up a storm of interest in the Indian twitterverse." -
Phelps Clan Tweets Intent To Picket Jobs Funeral Via iPhone
It comes as no surprise that Margie Phelps of the infamous Westboro Baptist Church has already declared the church's intention to picket Steve Jobs's funeral. What is interesting, is that she did so using an iPhone. The 142 characters of wrath read: "Westboro will picket his funeral.He[sic] had a huge platform; gave God no glory & taught sin. MT @AP: Apple co-founder Steve Jobs has died at 56." -
Does Italian Demo Show Cold Fusion, or Snake Oil?
An anonymous reader writes "Today, Wired.co.uk is running a story, 'Cold fusion rears its head as "E-Cat" research promises to change the world.' It gives an overview of the technology that claims to fuse hydrogen and nickel into copper, with no radioactive by-products, to produce copious amounts of heat, inexpensively, with a 1 megawatt plant scheduled to come on line later this month. Apparently, Wired was not aware that today is a big test in Italy by scientists from around the world, who will be observing the technology in operation, including self-looped mode. A real-time update page has been set up at PESWiki, which has been a primary news provider of this technology since it was announced last January." Wired's article is remarkably optimistic. I'd love for this to be true, but many decades of scientific-looking free-energy machine scams make it hard to be other than cynical; the claim of a secret catalyst which "can be produced at low cost," controlled-access for outside observers, the lack of published science to explain the claimed effect, and skepticism even from the free-energy world — along with a raft of pro-E-Cat websites registered anonymously earlier this year — all make it sound like this follows the marketing style of previous "over unity" / perpetual motion machines. I invite Andrea Rossi to take part in a Slashdot interview, if he's willing to answer readers' questions about his claims. -
Dan Shechtman Wins Chemistry Nobel For Quasicrystals
Stirling Newberry writes with word that Dan Shechtman of Israel's Technion has won the Nobel prize in chemistry for his discovery of quasicrystals, and provides a short description of why quasicrystals are exciting: "Quasicrystals fill space completely, but do not repeat, even though they show self-similar patterns, the way pi has order, but doesn't repeat. That is, they tessellate in an ordered way, but do not have repeating cells. In art, Girih tiles showed the essential property of being able to cover an infinite space, without repeating. In mathematics, Hao Wang came up with a set of tiles that any Turing Machine could be represented by, and conjectured that they would eventually always repeat. He turned out to be wrong, and over the next decades, tiles that did not repeat, but showed order, were discovered, most famously, though not first, by Penrose. Physically, when x-rays diffract, that is are scattered, from a crystal, they form a discrete lattice. Quasicrystals also have an ordered diffraction pattern, and it tiles the way ordered non-repeating tiles do. Quasicrystal patterns were known before Shechtman labelled them. So why care? Because crystals have only certain symmetries, and that determines their physical properties. Quasicrystals can have different symmetries, and do not bind regularly, and so different physical properties – which means new kinds of materials. Some examples: highly ductile steel, and, in something that is a bit of a by-word among people who study them, cooking utensils." -
2011 Nobel Prize In Physics
brindafella writes "Thirteen years ago, two teams of astronomers and physicists independently made the same stark discovery: Not only is the universe expanding like a vast inflating balloon, but its expansion is speeding up. The two teams have now been recognized with the 2011 Nobel Prize in Physics. Half of the prize will go to Saul Perlmutter of Lawrence Berkeley National Laboratory and the University of California, Berkeley, who led the Supernova Cosmology Project. The other half will be shared by Brian Schmidt of the Australian National University's Research School of Astronomy and Astrophysics, who led the High-z Supernova Search Team, and Adam Riess of Johns Hopkins University and the Space Telescope Science Institute in Baltimore, Maryland, who worked on High-z. In essence, they proved that Einstein's 'biggest mistake' (the cosmological constant, to create a 'stable universe') was actually a clever theoretical prediction that there was something else happening — dark energy." -
2011 Nobel Prize In Physics
brindafella writes "Thirteen years ago, two teams of astronomers and physicists independently made the same stark discovery: Not only is the universe expanding like a vast inflating balloon, but its expansion is speeding up. The two teams have now been recognized with the 2011 Nobel Prize in Physics. Half of the prize will go to Saul Perlmutter of Lawrence Berkeley National Laboratory and the University of California, Berkeley, who led the Supernova Cosmology Project. The other half will be shared by Brian Schmidt of the Australian National University's Research School of Astronomy and Astrophysics, who led the High-z Supernova Search Team, and Adam Riess of Johns Hopkins University and the Space Telescope Science Institute in Baltimore, Maryland, who worked on High-z. In essence, they proved that Einstein's 'biggest mistake' (the cosmological constant, to create a 'stable universe') was actually a clever theoretical prediction that there was something else happening — dark energy." -
Italian Wikipedia May Shut Down Due To New Legislation
An anonymous reader writes "Proposed legislation under debate in Italy has Wikipedia warning of a shutdown for the Italian version of the site. They say the law would create 'a requirement to all websites to publish, within 48 hours of the request and without any comment, a correction of any content that the applicant deems detrimental to his/her image.' They further explain. 'Unfortunately, the law does not require an evaluation of the claim by an impartial third judge — the opinion of the person allegedly injured is all that is required, in order to impose such correction to any website. Hence, anyone who feels offended by any content published on a blog, an online newspaper and, most likely, even on Wikipedia can directly request the removal of such contents and its permanent replacement with a "corrected" version, aimed to contradict and disprove the allegedly harmful contents, regardless of the truthfulness of the information deemed as offensive, and its sources.'" -
R7RS Scheme Progress Report
John Cowan recently gave a talk on the progress of R7RS (slides), the next revision of the Scheme language standard, at LispNYC. After the R6RS debacle, the community stepped back and is now basing the next standard on R5RS; the work has been split into two languages — R7RS-Small and R7RS-Large. The first working group is preparing to issue a final draft of the R7RS-Small language (PDF; clocking in at 73 pages vs. R5RS's 50) within the next few weeks. Read on for a summary of the planned changes to R7RS (more or less in the order of presentation). The talk details a number of improvements over R5RS and R6RS, and is divided into two portions. The majority of the talk discuses the status of the small language, with the last portion giving a quick update on the future intent of the large language group.
First on the list of major new features is a mandatory library system designed to be easily implemented atop existing module systems. R6RS's library system proved to tackle too much at once and be incompatible with everything already in a use (a persistent concern in the design of the R7RS-Small language). The R7RS system, on the other hand, is static, simple, and just powerful enough to promote implementation of portable libraries.
Exceptions are stripped down from R6RS (which proved too incompatible with existing implementations for practical use). guard (similar to try...catch in other languages) and with-exception-handler are supported: the latter runs the handler in the context from which the error was triggered (permitting recovery from the error like invoke-restart ) Unlike r6rs, exceptions lack a strictly defined hierarchy and can be any object (you could e.g. throw 4 if you really felt like it).
Dynamically scoped variables, in the form of parameters, are now part of the standard. Unlike Common Lisp special variables, parameters are first-class objects bound to expressions rather than symbols that are declared dynamically scoped. For reasons of simplicity it was decided to make parameters immutable (i.e. any "mutation" has to be done by rebinding). This has the (intentional) side-effect of making parameters play more nicely with threads (when mutation is permitted, setting a parameter that has not been rebound in the current thread requires synchronizing all threads and can have unexpected results).
As expected, R7RS includes bytevectors to complement strings. The small language standard only permits accessing bytevectors as ordered sets of unsigned 8-bit values (the large language standard will offer more flexible access). Binary I/O is implemented via a set of parallel procedures (open-binary-input-file vs. open-input-file, etc.) in contrast to the incredibly complicated dual binary/text ports provided by R6RS. Additionally, string and bytevector ports similar to SRFI-6 are provided instead of the incompatible string ports provided by R6RS.
Taylor Campbell noted that integer division in most languages is insufficiently expressive, and so R7RS will provide Euclidean division and centered division in addition to the usual suspects. Mathematicians rejoice!
As with R6RS, Unicode support is mandated. Unlike R6RS, the only characters that must be supported are those present in ASCII. For supported characters, Unicode case mapping and normalization are mandatory. One interesting diversion from Unicode and R6RS is that string order comparison is implementation-dependent: this gives implementers latitude with the internal encoding of strings.
Any user of Scheme knows that the language strives for consistent and obvious names for bindings. R7RS furthers this goal by resolving the long-standing inconsistency with core data structure creation, copying, mutation, mapping, etc. Lists, strings, and vectors now have a consistent set of each: make-TYPE, TYPE-copy, TYPE-set!, etc. Conversion procedures between all three types are also provided.
Finally, number of minor improvements were made. Most notable: record types are compatible with SRFI-6 (widely in use today); case sensitivity is the default (with optional insensitivity via include-ci); s-expression and nested block comments were added; IEEE infinities and NaNs have read syntax; strings may contain C-style escape sequences; Common Lisp circular list notation is supported; and common extensions to syntax-rules were standardized.
The final 20 minutes of the talk were about the status of R7RS-Large.
The R7RS Large language is currently on hold, mostly because all the small language members are on the large language committee as well, so there is a lack of time to work on both simultaneously. Work is planned to resume upon release of the final draft of the small language. Some work, however, has been completed.
The main focus of R7RS-Large is providing Scheme "with batteries included." John Cowan started the process by looking beyond the Lisp and Scheme communities (Python is mentioned) to figure out which libraries modern programmers expect their language to include.
This resulted in a list of around 250 packages that was narrowed down to around 80 packages through an initial voting process. It was decided then than some desirable packages (e.g. a foreign function interface) had to be omitted due to complexity. It is expected that implementers will continue experimenting and gradually come to a consensus on the larger packages using the existing SRFI process, and perhaps another revision of the standard down the road.
Of the packages planned for inclusion, the most prominent are: networking, threads, regular expressions, delimited continuations, URI handling, date and time parsing/arithmetic/formatting, hash tables, ambient environment access, file system directory access, gettext (i18n support), and pattern matching.
Most will be optional; packages will only be made mandatory if a number of the other packages require them. A compliant R7RS-Large implementation will only have to either provide a package fully or not at all (half implementations are forbidden).
Interestingly, R7RS large with all packages will be even larger than Common Lisp.
To avoid getting bogged down in stylistic discussions, a decision was made to focus on functionality above other concerns. The resulting packages may not be aesthetically pleasing to everyone, but will provide useful functionality. Users who disagree with naming, scope of functionality, argument ordering, etc. will be free to use the library system to import only the bindings they want, rename functions, wrap things into the style they want, etc. Basically, a compromise between the MIT approach and "Worse is Better" is being sought.
Here a call for volunteers is made: Since the focus will be on functionality over pure aesthetics, developers outside of the Lisp and Scheme communities are actively encouraged to participate in the R7RS-Large language process. No fixed time commitment would be required; the goal is to get a lot of people involved with a few core members maintaining momentum and guiding the process. The R7RS-Large language is most definitely a language designed by and for developers. So, make your voice heard!
Overall R7RS is shaping up to be the standard R6RS should have been (which, of course, could not have happened without the lessons of R6RS). The split between an elegant core language with each design issue meticulously fretted over and voted upon and a looser library standard should, hopefully, result in a core language that will stand the test of time with a standard library that can be used to get actual work done. -
Belgian ISP Ordered to Block The Pirate Bay; Telecomix and TPB Offer Workarounds
bs0d3 writes "Today a court in Belgium overruled an earlier judgment and ordered an ISP to block The Pirate Bay. The type of block to be used by the ISP is a simple DNS filter, which is similar to ones used before in Denmark. In Denmark the DNS block was extremely easy to circumvent, and the attention to The Pirate Bay actually increased Danish site traffic after the block. Today a hacktivist group called Telecomix, which is more recently known for helping to establish communications during the Internet blackout in Egypt, is offering their help. Their custom made 'censorship proof' DNS service is designed for situations just like this. ISP customers facing a block can simply use Telecomix's DNS server instead of the ISP-provided one to access blocked sites such as The Pirate Bay." The Pirate Bay also has suggestions for getting around the DNS block. -
China's Alibaba Interested In Buying Yahoo
jfruhlinger writes "Alibaba is a company that most Americans probably haven't heard of, but it's a hugely important Internet player in China, owning the Yahoo! China site as well as a host of other marketplace Websites. It's 40 percent owned by Yahoo, but now, in what seems a bit like a snake eating its own tail, Alibaba CEO Jack Ma has declared his interest in buying the embattled Internet portal outright." The San Francisco Chronicle has a Bloomberg News article with more details; they report that Alibaba is actually one of three parties looking into a joint bid for Yahoo, the others being the equity firm Silver Lake and Russian tech investor Digital Sky Technologies. -
Ask William Shatner Whatever You'd Like
He's Canadian, he's proven himself a successful comedic actor and writer, filmmaker, and musician, but (no matter what else he does) in many people's minds he will always be James Tiberius Kirk, captain of the USS Enterprise. Now, William Shatner has agreed to answer your questions. We'll pass on to him a selection of the best reader questions; you might want to read up on Shatner's official home page (and the Wikipedia link above) to knock out some of the most obvious ones. We'll pass on to him a selection of the best questions. Note: it's tempting to pile them on, but please try to follow the interview question guidelines by posting one question per post — ask as many questions as you'd like, though. Shatner is on vacation right now, but will work on answering your questions when he gets back. -
BerliOS Software Repository Will Close At Year's End
An anonymous reader writes with some sad news from Germany, as posted on the BerliOS front page, and sent by email to developers as well. An excerpt: "As an European, non-proprietary project BerliOS pursued the goal to support the various open-source players and provide a neutral mediator function. In 2011 over 4710 projects have been hosted on BerliOS, with 50,000 registered users and over 2.6 million file downloads each month. We are proud that with BerliOS we have brought the idea of an OSS repository to Europe. Meanwhile, the concept has prevailed and there are many good alternatives. Unfortunately, as a research institute Fraunhofer FOKUS has only few opportunities to operate a repository like BerliOS. Such a project will only work with a follow-up financing, or with sponsors or partners taking over the repository. In the field of OSS this is a difficult undertaking. In a recent survey the community indicated some support in funds and manpower which we would like to thank you for. Unfortunately, the result is not enough to put the project on a sustainable financial basis. In addition the search for sponsors or partners was unsuccessful. ... As a developer, you should export your BerliOS project into another repository." BerliOS is slated to close on December 31st. -
Ask Slashdot: How to Exploit Post-Cataract Ultraviolet Vision?
xmas2003 writes "I recently had cataract surgery with a Crystalens implant. With my cloudy yellowing (UV-filtering) natural lens removed, I see the world in a new light (more on that in a moment) as everything is brighter and colors are more vivid ... plus in focus. As a typical Slashdot reader, I've been myopic since childhood, so it's wonderful not to have to wear glasses/contacts for distance. One interesting oddity is that I can now see ultraviolet light — it seems that there are a few people who have photoreceptors sensitive below 400nm into the UV spectrum. I've done some testing with a Black Light and UV filter to confirm this but would love to do more conclusive testing such as using a Monochromator — anyone in the Boulder, Colorado area have access to one? And any suggestions from Slashdot readers on how I can further explore this phenomenon? While I can't see dead people, I guess I have a 'superpower' ... although I'm not sure a middle-aged suburbanite dad should don purple tights and cape to become a crime-fighter!" -
Psystar Loses Appeal In Apple Case
The dispute between Mac cloner Psystar and Apple has been a long and twisty one; now, reader UnknowingFool writes that "Last week, the U.S. Ninth Circuit Court of Appeals ruled mostly against Psystar in their appeal of their case with Apple. The Court found for Apple in that they did not misuse copyright by having conditions in the OS X license. Psystar won on one point in which some of the court orders should have not been sealed." -
Security By Obscurity — a New Theory
mikejuk writes "Kerckhoffs' Principle suggests that there is no security by obscurity — but perhaps there is. A recent paper by Dusko Pavlovic suggests that security is a game of incomplete information and the more you can do to keep your opponent in the dark, the better. In addition to considering the attacker's computing power limits, he also thinks it's worth considering limits on their logic or programming capabilities (PDF). He recommends obscurity plus a little reactive security in response to an attacker probing the system. In this case, instead of having to protect against every possible attack vector, you can just defend against the attack that has been or is about to be launched." -
The Cult of DevOps
packetrat writes "I was at OmniTI's Surge conference today, which turned out to be, among other things, a meeting of the cult of DevOps. Ars Technica covered the keynote and some of the presentations, but some of the best stuff is in the comments. Google CIO Ben Fried told the tale of a really poorly engineered trading application at Morgan Stanley that he was associated with, and how the way IT was structured there contributed to that engineering and to its spectacular failure, costing the bank untold millions in stock trade processing fees from its institutional customers. He said what he learned from cleaning up the mess has informed how Google runs its IT operations, and a culture that promotes generalist skills. A lot of how he describes Google's approach sounds like the DevOps kool-aid a lot of the other speakers were serving, but it also sounds like common sense — are most IT organizations really that poorly run that developers are totally unaware their software is sending messages that are generating network storms, or network engineers are clueless enough about QoS to route leased lines into their data center through their public-facing Internet?" -
China Launches Space Station Laboratory Module
wisebabo writes with news from CNN that "China's first space laboratory module launched Thursday, according to state-run media, an important milestone in China's plan to build a space station." The module, known as Tiangong-1, features sleeping areas and exercise equipment. Writes wisebabo: "In another universe (Arthur C. Clarke's 2011), it would be on its way to Europa by now. Anyone know what orbital plane/altitude it's at? Can it be reached by NASA/Soyuz? Are the docking ports compatible? How about the air pressure/breathing mix?" -
Congress May Permit Robot Calls To Cell Phones
TCPALaw writes "While many hoaxes have circulated in the past about cell phone numbers being opened up to telemarketers, it now may actually happen. A bill, HR 3035 (PDF), has been introduced in Congress, that would create numerous exceptions to the Telephone Consumer Protection Act, which banned autodialed and prerecorded robot calls to cell phone numbers. If passed, HR 3035 would permit a wide range of autodialed and prerecorded calls to cell phones that are currently prohibited, and would preempt practically all state laws providing similar protections. This is being applauded by debt collectors and banks (PDF) ... as if the bailouts weren't enough, now they get to make you pay for their calls to you." -
Graphene and Quantum Hall Effect Could Help Redefine Metrics
eldavojohn writes "The National Physical Laboratory has published research in Nature that could lead to redefining two of our most commonly used metrics. There's been a lot of trouble stemming from defining an exact Kilogram as some lump of platinum-iridium sitting in a glass case somewhere, so the proposal was put forth to study the quantum hall effect with different materials. Enter the Nobel prize winning, super strong, silicon usurping graphene. NPL now says you can add quantum resistance metrology to the list of graphene's many conquests as they say the quantum hall effect in graphene is 'very robust and easy to measure.' With this at their disposal, the Kilogram may be redefined in terms of h, the Planck constant, and the Ampere may be redefined in terms of e, the electron charge (alias Elementary charge or the charge of a proton). You can find the full paper here." -
Graphene and Quantum Hall Effect Could Help Redefine Metrics
eldavojohn writes "The National Physical Laboratory has published research in Nature that could lead to redefining two of our most commonly used metrics. There's been a lot of trouble stemming from defining an exact Kilogram as some lump of platinum-iridium sitting in a glass case somewhere, so the proposal was put forth to study the quantum hall effect with different materials. Enter the Nobel prize winning, super strong, silicon usurping graphene. NPL now says you can add quantum resistance metrology to the list of graphene's many conquests as they say the quantum hall effect in graphene is 'very robust and easy to measure.' With this at their disposal, the Kilogram may be redefined in terms of h, the Planck constant, and the Ampere may be redefined in terms of e, the electron charge (alias Elementary charge or the charge of a proton). You can find the full paper here." -
Graphene and Quantum Hall Effect Could Help Redefine Metrics
eldavojohn writes "The National Physical Laboratory has published research in Nature that could lead to redefining two of our most commonly used metrics. There's been a lot of trouble stemming from defining an exact Kilogram as some lump of platinum-iridium sitting in a glass case somewhere, so the proposal was put forth to study the quantum hall effect with different materials. Enter the Nobel prize winning, super strong, silicon usurping graphene. NPL now says you can add quantum resistance metrology to the list of graphene's many conquests as they say the quantum hall effect in graphene is 'very robust and easy to measure.' With this at their disposal, the Kilogram may be redefined in terms of h, the Planck constant, and the Ampere may be redefined in terms of e, the electron charge (alias Elementary charge or the charge of a proton). You can find the full paper here." -
MRI Magnets Cause Nystagmus
Hitting the main page for the first time, tibit writes "In an interesting twist on 'it's so old it's new again,' Johns Hopkins researchers led by Dale Roberts found what must have been causing much confusion for doctors the world over: strong external magnetic fields can stimulate the semicircular canals, causing vertigo and nystagmus (pendular eye motion). It's a textbook case of the Lorentz force in action: our angular rate gyros, the semicircular canals in the middle ear, filled with endolymph, have a ionic current flowing across. In a magnetic field, the current produces a force that pushes the lymph along the channel, causing stimulation of the cupula — a pressure sensor at the end of the channel. This is interpreted by the brain as rotation of the head in space, and causes a nystagmus that's supposed to stabilize the image on the retina. Of course, the subject is laying down and not spinning in space, and the mismatch between inertial measurements coming from the ear and the real situation causes vertigo." -
Ask Slashdot: Best ccTLD To Avoid Confiscation?
First time accepted submitter Pete McCann writes "Given the recent spate of domain seizures by the U.S. government, it seems that registrations in any U.S.-hosted registry (like the gTLDs .com, .net, and .org) aren't stable places to put content that the U.S. government might find objectionable. I am wondering, are there any ccTLD registries out there that have an open registration policy and are willing to stand up to censorship demands from the USG? There is this list of ccTLDs with open registration policies, and the current MAFIAAFIRE redirection list looks very Tuvalu-heavy. Where would you register a site for maximum resistance to confiscation?" -
Ask Jennifer Granick About Computer Crime Defense
Attorney Jennifer Granick has defended many high profile hackers, including researcher Christopher Soghoian, creator of a fake boarding pass generator (2006); Michael Lynn versus Cisco/ISS (2005); Jerome Heckenkamp; and Luke Smith and Nelson Pavlosky in Online Policy Group v. Diebold Election Systems (now Premier Election Solutions), a copyright misuse case related to electronic voting. Granick also won an exemption from the U.S. Copyright Office in 2006 allowing phone unlocking despite the anti-circumvention provisions of the Digital Millennium Copyright Act, which set the stage for renewal of the exemption and for the jailbreaking exemption in 2009. At Stanford, Granick worked with Lawrence Lessig on constitutional copyright cases and taught six years worth of law students about computers, technology and civil liberties. While Civil Liberties Director at the EFF, Granick started the Coders' Rights Project and participated in litigation against ATT and the federal government for violation of surveillance regulations. Now an attorney at ZwillGen PLLC, Granick assists individuals and companies creating new products and services. And now, she's graciously agreed to answer your questions. Please, as usual, ask as many questions as you'd like, but confine each question to a separate post. -
Inferno OS Running On Android Phones
New submitter Digi-John writes "Employees at Sandia National Labs have put the Inferno OS on Android-based phones, replacing the default Java UI. Applications are written in Limbo rather than Java. The full announcement is at the bitbucket repository, and a short video demonstrates some of its capabilities." -
Wolfenstein Ray Traced and Anti-Aliased, At 1080p
An anonymous reader writes "After Intel displayed their research demo Wolfenstein: Ray Traced on Tablets, the latest progress at IDF focuses on high(est)-end gaming now running at 1080p. Besides image-based post-processing (HDR, Depth of Field) there is now also an implementation of a smart way of calculating anti-aliasing through using mesh IDs and normals and applying adaptive 16x supersampling. All that is powered by the 'cloud,' consisting of a server that holds eight Knights Ferry cards (total of 256 cores / 1024 threads). A lot of hardware, but the next iteration of the 'Many Integrated Core' (MIC) architecture, named Knights Corner (and featuring 50+ cores), might be just around the corner." -
Wild Parrots Learning To Talk From Escaped Pet Birds
bazzalunatic writes "Be careful what you teach a parrot. Some chatty pet parrots that have escaped back into the wild have taught wild parrots to talk. Seems the phenomenon could be integrated into the flock through generations. From the article: 'The evolution of language could well be passed on through the generations, says Ken. "If the parents are talkers and they produce chicks, their chicks are likely to pick up some of that," he says. This phenomenon is not unique; some lyrebirds in southern Australia still reproduce the sounds of axes and old shutter-box cameras their ancestors once learnt.'" While this doesn't reach the amazing level of Washoe the chimpanzee teaching sign language, it is still interesting and reminiscent of something out of an Alfred Hitchcock movie. -
Italian Hacker Publishes 0day SCADA Hacks
mask.of.sanity writes "An Italian security researcher, Luigi Auriemma, has disclosed a laundry list of unpatched vulnerabilities and detailed proof-of-concept exploits that allow hackers to completely compromise major industrial control systems. The attacks work against six SCADA systems, including one manufactured by U.S. giant Rockwell Automation. The researcher published step-by-step exploits that allowed attackers to execute full remote compromises and denial of service attacks. Auriemma appeared unrepentant for the disclosures in a post on his website." -
Book Review: Metasploit The Penetration Tester's Guide
eldavojohn writes "The Metasploit Framework has come a long way and currently allows just about anyone to configure and execute exploits effortlessly. Metasploit: The Penetration Tester's Guide takes current documentation further and provides a valuable resource for people who are interested in security but don't have the time or money to take a training class on Metasploit. The highlights of the book rest on the examples provided to the reader as exercises in exploiting several older versions of operating systems like Windows XP and Ubuntu while at the same time avoiding triggering antivirus or detection. The only weak point of this book is that a couple chapters refer the reader to external texts (on stacks and registers) in order to meet requirements for crafting exploits. The book also gives the reader a brief warning on ethics as many of these exploits and techniques would most likely work on many sites and networks. If you're wondering how seemingly inexperienced groups like lulzsec constantly claim victims, this would be an excellent read." Keep reading for the rest of eldavojohn's review. Metasploit The Penetration Tester's Guide author David Kennedy, Jim O'Gorman, Devon Kearns, and Mati Aharoni pages 300 publisher No Starch Press, Inc. rating 10/10 reviewer eldavojohn ISBN 978-1593272883 summary A thorough guide to penetration testing with the Metasploit Framework. In 2007, Metasploit was migrated from Perl to Ruby. The book opens with a brief history of the framework and mentions this but does not address any complaints of performance loss. Instead, the authors argues that this increased contributions and adoptions. As a result, all the code in this book (which the exception of some SQL payloads) is written in Ruby. If you don't know Ruby but you know many other languages, it's a fairly simple language to pick up.
The first chapter of this book clearly indicates that the objective is to empower white hat hackers and researchers. They lay down a predefined set of phases that one takes while pen testing a target. They are Pre-engagement Interactions, Intelligence Gathering, Threat Modeling, Vulnerability Analysis, Exploitation, Post Exploitation and Reporting. Chapter two covers the terminology that is used across the Metasploit Framework so if you're unfamiliar with concepts like 'shellcode' or 'payload' this chapter will set you straight. It also mentions a UI for Metasploit called Armitrage but my personal tastes kept me using the minimal MSFConsole and MSFcli.
Chapter three begins to cover intelligence gathering and covers everything from the basic whois tool to writing your own custom scanner. The chapter does a great job of carefully explaining in detail the difference between passive and active scanning. The stealth TCP scan that nmap provides was a new thing for me and the chapter also details how Metasploit can use several database technologies to record and store the results of your scans to be used later on. The chapter shows how to use Metasploit to scan ports, server message blocks, MS SQL servers, SSH servers, FTP and simple network management protocol sweeping. Most of these techniques are a few quick commands in Metasploit's console and with Ruby mixins the chapter illustrates how to write your own scanner for use in Metasploit in about 20 lines of code. But all of this is just to get a grasp of what's up and running on the server.
Chapter four starts to get interesting with actual vulnerability scanning. Banner grabbing is an important technique in pen testing and the book suggests using NeXpose community edition (also a Rapid7 tool). This is covered in more detail in the appendix but NeXpose is a web GUI interface for scanning, storing and managing site scans. This provides great reporting features, it's intuitive and reduces everything to point-and-click for the user. But luckily this tool can also be run from the console (something I preferred). The chapter also covers another popular scanner called Nessus and shows to import the results to Metasploit for use. The chapter also includes noisy options like SMB login scanning or just looking for open VNC or X11 servers. Mentioned here first (but also frequently later in the book) is Back|Track for connecting to such targets. Something neat about this chapter is that if you don't care that your target knows you're attacking them, you can just move from these results collected with NeXpose, Nessus or OpenVAS and drop them into the 'autopwn' tool in Metasploit. It's three commands on the console and apparently works more often than it should.
Chapter five familiarizes the reader with the MSFConsole and its basic commands like showing all the exploits, payloads and targets available in the Metasploit Framework installed. These are constantly updated and maintained so they often change. With that information, the chapter proceeds to step the reader through an exploit in a Windows XP SP2 (MS08-067) and then a Samba exploit in Ubuntu 9.04.
Chapter six spices things up by introducing Meterpreter that extends the Metasploit Framework to serve a shell to the exploited system and from there perform additional attacks. The chapter shows how to brute force an MS SQL server and use the stored procedure xp_cmdshell to gain remote access. Meterpreter has a lot of neat features like keystroke logging, capturing screenshots and dumping password hashes (including the pass-the-hash technique). Simple commands in meterpreter can allow the user to easily and effortlessly accomplish many things: privilege escalation, token impersonation, pivoting to another system, process migration, killing antivirus software, system scraping, the list goes on. The chapter finishes by briefly mentioning an intriguing tool called Railgun that I wish they had spent more time on.
Chapter seven covers avoiding antivirus detection through tools like msfencode (to avoid your exploit being fingerprinted). Even better is encoding it many many times. If you know what antivirus your target uses, you can simply run the antivirus on your encoded exploit on your local machine to see if it's picked up. The chapter also covers the basics on continuing normal execution of a backdoored executable and packers that compress an executable for you with decompression code built in.
The book gets progressively more technical with chapter eight focusing on client side attacks. The chapter covers the NOP slide technique and also introduces the Immunity Debugger. It covers the Internet Explorer Aurora Exploit (MS10.002) as an end of chapter exercise for the reader to do. Chapter nine takes a quite look at Metasploit's auxiliary modules that allow the user to do many other things than just exploits. They run through the source of a mischievous Foursquare Location Poster that can make you appear to be everywhere on Foursquare. They also cover heap spraying attacks in web browsers — a topic that was particularly discomforting for me considering how long I often leave my browser open for.
Chapter ten was probably one of the more boring for me but a very important tool for pen testers. It shows how to turn the Metasploit Framework into a social exploitation tool that can be used to send templated e-mails to distribution lists. The intent of this, of course, is to get one user in a large company to click on a site that looks like their company's homepage and perhaps enter their credentials. By just selecting from lists of options, you can create java applet exploits that appear to be legitimately signed, clone a website like gmail and harvest credentials, tabnabbing, webjacking, man-left-in-the-middle and finally mixing those all together in a multipronged attack. The next chapter is just more exploits via Fast-Track (an open source Python based tool that builds on top of Metasploit).
Chapter twelve covers Karmetasploit, a Metasploit implementation of the wireless security tool Karma. The strategy of this exploit is to present your machine as a wireless access point. When a user connects, you can use karmetasploit to host fake webpages and grab their credentials or even gain shell access through a client side attack. Knowing how frequently people attach to anything in coffee shops and airports, this sort of attack could be particularly brutal and extremely easy to execute given Metasploit's simplicity for users.
The final chapters do an okay job of showing you how to first build your own module for Metasploit in chapter thirteen. Then in fourteen, the book looks at building your own exploit and goes into detail about fuzzing applications on your local machine and using the Immunity Debugger to look at what's happening given the fuzzed input. What follows is a lengthy discussion of the Structured Exception Handler (SEH) and the Next SEH (NSEH) and how you can manipulate registers and utilize JMPs to hit a NOP slide into your shellcode. This is one of the longest and most complicated chapters with probably the most technically intensive writing. I would like to see further editions of this book expand on things like this as it was important for me as a software developer to understand how these attacks are manufactured.
Chapter fifteen was similar to fourteen but showed how to port exploits to the metasploit framework. This chapter covers more so the general guidelines for writing exploits for the metasploit framework and doing it so that you leverage metasploit's flexibility. Chapter sixteen covers the scripting abilities of meterpreter and customizing that to execute further exploits once you have access to a target machine with meterpreter.
The final chapter brings the key steps together for a simulated penetration testing of a preconfigured system with web server (the book lists the Pirate Bay as a source of this torrent). As you work through this chapter, the phases of pen testing are exercised with all the aforementioned strategies employed.
This book was a real eye opener to read for a software developer. I haven't done formal pen testing aside from testing my own code so a lot of these advanced concepts were new to me. I enjoyed how the code was laid out with circled numbers marking code (instead of every line being numbered) that were referenced later in the text. I hope future editions of this book provide progressively more and more material as there's clearly a lot of sections that are condensed into a few paragraphs but could be expanded upon almost endlessly. I'm glad this sort of tool didn't exist during my younger more mischievous years as this book demonstrates that it could be used for gaining access to just about anything (depending on how much free time and skill you have).
You can purchase Metasploit: The Penetration Tester's Guide from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
Book Review: Metasploit The Penetration Tester's Guide
eldavojohn writes "The Metasploit Framework has come a long way and currently allows just about anyone to configure and execute exploits effortlessly. Metasploit: The Penetration Tester's Guide takes current documentation further and provides a valuable resource for people who are interested in security but don't have the time or money to take a training class on Metasploit. The highlights of the book rest on the examples provided to the reader as exercises in exploiting several older versions of operating systems like Windows XP and Ubuntu while at the same time avoiding triggering antivirus or detection. The only weak point of this book is that a couple chapters refer the reader to external texts (on stacks and registers) in order to meet requirements for crafting exploits. The book also gives the reader a brief warning on ethics as many of these exploits and techniques would most likely work on many sites and networks. If you're wondering how seemingly inexperienced groups like lulzsec constantly claim victims, this would be an excellent read." Keep reading for the rest of eldavojohn's review. Metasploit The Penetration Tester's Guide author David Kennedy, Jim O'Gorman, Devon Kearns, and Mati Aharoni pages 300 publisher No Starch Press, Inc. rating 10/10 reviewer eldavojohn ISBN 978-1593272883 summary A thorough guide to penetration testing with the Metasploit Framework. In 2007, Metasploit was migrated from Perl to Ruby. The book opens with a brief history of the framework and mentions this but does not address any complaints of performance loss. Instead, the authors argues that this increased contributions and adoptions. As a result, all the code in this book (which the exception of some SQL payloads) is written in Ruby. If you don't know Ruby but you know many other languages, it's a fairly simple language to pick up.
The first chapter of this book clearly indicates that the objective is to empower white hat hackers and researchers. They lay down a predefined set of phases that one takes while pen testing a target. They are Pre-engagement Interactions, Intelligence Gathering, Threat Modeling, Vulnerability Analysis, Exploitation, Post Exploitation and Reporting. Chapter two covers the terminology that is used across the Metasploit Framework so if you're unfamiliar with concepts like 'shellcode' or 'payload' this chapter will set you straight. It also mentions a UI for Metasploit called Armitrage but my personal tastes kept me using the minimal MSFConsole and MSFcli.
Chapter three begins to cover intelligence gathering and covers everything from the basic whois tool to writing your own custom scanner. The chapter does a great job of carefully explaining in detail the difference between passive and active scanning. The stealth TCP scan that nmap provides was a new thing for me and the chapter also details how Metasploit can use several database technologies to record and store the results of your scans to be used later on. The chapter shows how to use Metasploit to scan ports, server message blocks, MS SQL servers, SSH servers, FTP and simple network management protocol sweeping. Most of these techniques are a few quick commands in Metasploit's console and with Ruby mixins the chapter illustrates how to write your own scanner for use in Metasploit in about 20 lines of code. But all of this is just to get a grasp of what's up and running on the server.
Chapter four starts to get interesting with actual vulnerability scanning. Banner grabbing is an important technique in pen testing and the book suggests using NeXpose community edition (also a Rapid7 tool). This is covered in more detail in the appendix but NeXpose is a web GUI interface for scanning, storing and managing site scans. This provides great reporting features, it's intuitive and reduces everything to point-and-click for the user. But luckily this tool can also be run from the console (something I preferred). The chapter also covers another popular scanner called Nessus and shows to import the results to Metasploit for use. The chapter also includes noisy options like SMB login scanning or just looking for open VNC or X11 servers. Mentioned here first (but also frequently later in the book) is Back|Track for connecting to such targets. Something neat about this chapter is that if you don't care that your target knows you're attacking them, you can just move from these results collected with NeXpose, Nessus or OpenVAS and drop them into the 'autopwn' tool in Metasploit. It's three commands on the console and apparently works more often than it should.
Chapter five familiarizes the reader with the MSFConsole and its basic commands like showing all the exploits, payloads and targets available in the Metasploit Framework installed. These are constantly updated and maintained so they often change. With that information, the chapter proceeds to step the reader through an exploit in a Windows XP SP2 (MS08-067) and then a Samba exploit in Ubuntu 9.04.
Chapter six spices things up by introducing Meterpreter that extends the Metasploit Framework to serve a shell to the exploited system and from there perform additional attacks. The chapter shows how to brute force an MS SQL server and use the stored procedure xp_cmdshell to gain remote access. Meterpreter has a lot of neat features like keystroke logging, capturing screenshots and dumping password hashes (including the pass-the-hash technique). Simple commands in meterpreter can allow the user to easily and effortlessly accomplish many things: privilege escalation, token impersonation, pivoting to another system, process migration, killing antivirus software, system scraping, the list goes on. The chapter finishes by briefly mentioning an intriguing tool called Railgun that I wish they had spent more time on.
Chapter seven covers avoiding antivirus detection through tools like msfencode (to avoid your exploit being fingerprinted). Even better is encoding it many many times. If you know what antivirus your target uses, you can simply run the antivirus on your encoded exploit on your local machine to see if it's picked up. The chapter also covers the basics on continuing normal execution of a backdoored executable and packers that compress an executable for you with decompression code built in.
The book gets progressively more technical with chapter eight focusing on client side attacks. The chapter covers the NOP slide technique and also introduces the Immunity Debugger. It covers the Internet Explorer Aurora Exploit (MS10.002) as an end of chapter exercise for the reader to do. Chapter nine takes a quite look at Metasploit's auxiliary modules that allow the user to do many other things than just exploits. They run through the source of a mischievous Foursquare Location Poster that can make you appear to be everywhere on Foursquare. They also cover heap spraying attacks in web browsers — a topic that was particularly discomforting for me considering how long I often leave my browser open for.
Chapter ten was probably one of the more boring for me but a very important tool for pen testers. It shows how to turn the Metasploit Framework into a social exploitation tool that can be used to send templated e-mails to distribution lists. The intent of this, of course, is to get one user in a large company to click on a site that looks like their company's homepage and perhaps enter their credentials. By just selecting from lists of options, you can create java applet exploits that appear to be legitimately signed, clone a website like gmail and harvest credentials, tabnabbing, webjacking, man-left-in-the-middle and finally mixing those all together in a multipronged attack. The next chapter is just more exploits via Fast-Track (an open source Python based tool that builds on top of Metasploit).
Chapter twelve covers Karmetasploit, a Metasploit implementation of the wireless security tool Karma. The strategy of this exploit is to present your machine as a wireless access point. When a user connects, you can use karmetasploit to host fake webpages and grab their credentials or even gain shell access through a client side attack. Knowing how frequently people attach to anything in coffee shops and airports, this sort of attack could be particularly brutal and extremely easy to execute given Metasploit's simplicity for users.
The final chapters do an okay job of showing you how to first build your own module for Metasploit in chapter thirteen. Then in fourteen, the book looks at building your own exploit and goes into detail about fuzzing applications on your local machine and using the Immunity Debugger to look at what's happening given the fuzzed input. What follows is a lengthy discussion of the Structured Exception Handler (SEH) and the Next SEH (NSEH) and how you can manipulate registers and utilize JMPs to hit a NOP slide into your shellcode. This is one of the longest and most complicated chapters with probably the most technically intensive writing. I would like to see further editions of this book expand on things like this as it was important for me as a software developer to understand how these attacks are manufactured.
Chapter fifteen was similar to fourteen but showed how to port exploits to the metasploit framework. This chapter covers more so the general guidelines for writing exploits for the metasploit framework and doing it so that you leverage metasploit's flexibility. Chapter sixteen covers the scripting abilities of meterpreter and customizing that to execute further exploits once you have access to a target machine with meterpreter.
The final chapter brings the key steps together for a simulated penetration testing of a preconfigured system with web server (the book lists the Pirate Bay as a source of this torrent). As you work through this chapter, the phases of pen testing are exercised with all the aforementioned strategies employed.
This book was a real eye opener to read for a software developer. I haven't done formal pen testing aside from testing my own code so a lot of these advanced concepts were new to me. I enjoyed how the code was laid out with circled numbers marking code (instead of every line being numbered) that were referenced later in the text. I hope future editions of this book provide progressively more and more material as there's clearly a lot of sections that are condensed into a few paragraphs but could be expanded upon almost endlessly. I'm glad this sort of tool didn't exist during my younger more mischievous years as this book demonstrates that it could be used for gaining access to just about anything (depending on how much free time and skill you have).
You can purchase Metasploit: The Penetration Tester's Guide from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page. -
New BIOS Exploiting Rootkit Discovered
First time accepted submitter mtemar writes with a Symantec analysis of an interesting new trojan/virus. From the article:"There are more and more known viruses that infect the MBR. Symantec Security Response has published a blog to demonstrate this trend last month. However, we seldom confront with one that infects the BIOS. One of them, the notorious CIH, appeared in 1999, which infected the computer BIOS and thus harmed a huge number of computers at that time. Recently, we met a new threat named Trojan.Mebromi that can add malicious components into Award BIOS which allows the threat to take control of the system even before MBR." -
NASA Unveils Design for New Space Launch System
wooferhound writes with an article in the Orlando Sentinel about NASA's Deep Space Exploration project. From the article: "After months of debate, NASA has settled on plans for its next spaceship — a space shuttle hybrid that will fly twice in the next decade and cost $30 billion through 2021, according to senior administration officials and internal NASA documents. That NASA decided to recycle elements of the shuttle is not unexpected. Last year, Congress and the White House agreed NASA should reuse equipment from old programs and the new design — which includes a giant fuel tank and two booster rockets — largely reflects that compromise. The most noticeable change is the plane-like orbiter will be replaced by an Apollo-like crew capsule atop the tank." The Space Launch System will be powered by a combination of the Shuttle main engine for the core launch stage, and the J-2 engine (from the Saturn V project) for the upper stage. The same solid booster rockets used for Shuttle missions will be used for at least the initial unmanned launch in 2017, but NASA will have a design contest to replace them for the 2021 crewed launch and beyond. -
NASA Unveils Design for New Space Launch System
wooferhound writes with an article in the Orlando Sentinel about NASA's Deep Space Exploration project. From the article: "After months of debate, NASA has settled on plans for its next spaceship — a space shuttle hybrid that will fly twice in the next decade and cost $30 billion through 2021, according to senior administration officials and internal NASA documents. That NASA decided to recycle elements of the shuttle is not unexpected. Last year, Congress and the White House agreed NASA should reuse equipment from old programs and the new design — which includes a giant fuel tank and two booster rockets — largely reflects that compromise. The most noticeable change is the plane-like orbiter will be replaced by an Apollo-like crew capsule atop the tank." The Space Launch System will be powered by a combination of the Shuttle main engine for the core launch stage, and the J-2 engine (from the Saturn V project) for the upper stage. The same solid booster rockets used for Shuttle missions will be used for at least the initial unmanned launch in 2017, but NASA will have a design contest to replace them for the 2021 crewed launch and beyond. -
NASA Unveils Design for New Space Launch System
wooferhound writes with an article in the Orlando Sentinel about NASA's Deep Space Exploration project. From the article: "After months of debate, NASA has settled on plans for its next spaceship — a space shuttle hybrid that will fly twice in the next decade and cost $30 billion through 2021, according to senior administration officials and internal NASA documents. That NASA decided to recycle elements of the shuttle is not unexpected. Last year, Congress and the White House agreed NASA should reuse equipment from old programs and the new design — which includes a giant fuel tank and two booster rockets — largely reflects that compromise. The most noticeable change is the plane-like orbiter will be replaced by an Apollo-like crew capsule atop the tank." The Space Launch System will be powered by a combination of the Shuttle main engine for the core launch stage, and the J-2 engine (from the Saturn V project) for the upper stage. The same solid booster rockets used for Shuttle missions will be used for at least the initial unmanned launch in 2017, but NASA will have a design contest to replace them for the 2021 crewed launch and beyond. -
Whither Moore's Law; Introducing Koomey's Law
Joining the ranks of accepted submitters, Beorytis writes "MIT Technology review reports on a recent paper by Stanford professor Dr. Jon Koomey, which claims to show that the energy efficiency of computing doubles every 1.5 years. Note that efficiency is considered in terms of a fixed computing load, a point soon to be lost on the mainstream press. Also interesting is a graph in a related blog post that really highlights the meaning of the 'fixed computing load' assumption by plotting computations per kWh vs. time. An early hobbyist computer, the Altair 8800 sits right near the Cray-1 supercomputer of the same era." -
Whither Moore's Law; Introducing Koomey's Law
Joining the ranks of accepted submitters, Beorytis writes "MIT Technology review reports on a recent paper by Stanford professor Dr. Jon Koomey, which claims to show that the energy efficiency of computing doubles every 1.5 years. Note that efficiency is considered in terms of a fixed computing load, a point soon to be lost on the mainstream press. Also interesting is a graph in a related blog post that really highlights the meaning of the 'fixed computing load' assumption by plotting computations per kWh vs. time. An early hobbyist computer, the Altair 8800 sits right near the Cray-1 supercomputer of the same era." -
Ask Slashdot: Best Use For a New Supercomputing Cluster?
Supp0rtLinux writes "In about 2 weeks time I will be receiving everything necessary to build the largest x86_64-based supercomputer on the east coast of the U.S. (at least until someone takes the title away from us). It's spec'ed to start with 1200 dual-socket six-core servers. We primarily do life-science/health/biology related tasks on our existing (fairly small) HPC. We intend to continue this usage, but to also open it up for new uses (energy comes to mind). Additionally, we'd like to lease access to recoup some of our costs. So, what's the best Linux distro for something of this size and scale? Any that include a chargeback option/module? Additionally, due to cost contracts, we have to choose either InfiniBand or 10Gb Ethernet for the backend: which would Slashdot readers go with if they had to choose? Either way, all nodes will have four 1Gbps Ethernet ports. Finally, all nodes include only a basic onboard GPU. We intend to put powerful GPUs into the PCI-e slot and open up the new HPC for GPU related crunching. Any suggestions on the most powerful Linux friendly PCI-e GPU available?" -
Skein Hash... In Bash
First time accepted submitter Matt16060936 writes "...Last night (err.. 3am this morning) I finished an implementation of the Skein 512-512 hash algorithm (version 1.3). I'm a fan of Skein and hope it wins the SHA-3 competition next year. One of the nice things about Skein is how quickly it's been adopted by many platforms and implemented in many languages. To that end, I present Skein 512-512 implemented in Bash." -
Type Safety Coming To DB Queries
An anonymous reader writes "A new type-safe query language for the popular full-text search platform Solr, called Slashem (a Rogue-like), has just been released. Slashem is implemented as a domain-specific language in Scala, providing compile time type-safety, allowing you do things like date range queries against date fields but keeping you from trying to do a date range query against a string field. Hopefully this trend catches on, resulting in fewer invalid queries exploding at runtime." -
Has Cleverbot Passed the Turing Test?
kruhft writes "It seems that Cleverbot, the chatbot so ready to admit that it was a unicorn during a discussion with itself, has passed the Turing test. This past Sunday, the 1334 votes from a Turing test held at the Techniche festival in Guwahati, India were released. They revealed that Cleverbot was voted to be human 59.3% of the time. Real humans did only slightly better and were assumed to be humans 63.3% of the time." As the Wikipedia link above points out, though, there's no single, simple "Turing Test," per se — many systems have successfully convinced humans over the years. Perhaps Cleverbot would consent to taking part in a Slashdot interview, to be extra-convincing. -
Has Cleverbot Passed the Turing Test?
kruhft writes "It seems that Cleverbot, the chatbot so ready to admit that it was a unicorn during a discussion with itself, has passed the Turing test. This past Sunday, the 1334 votes from a Turing test held at the Techniche festival in Guwahati, India were released. They revealed that Cleverbot was voted to be human 59.3% of the time. Real humans did only slightly better and were assumed to be humans 63.3% of the time." As the Wikipedia link above points out, though, there's no single, simple "Turing Test," per se — many systems have successfully convinced humans over the years. Perhaps Cleverbot would consent to taking part in a Slashdot interview, to be extra-convincing. -
The Linux Counter Relaunches
psychonaut writes "Long-term readers of Slashdot may be familiar with The Linux Counter, which attempts to measure (through surveys and statistics) the number of people using GNU/Linux operating systems. The project started in 1993 and shot to fame six years later, largely as a result of three Slashdot articles (two of which brought the Counter to its knees). After four years of stagnation, project founder Harald Tveit Alvestrand has handed over the reins to a new maintainer, Alexander Mieland. Over the past few months, Mieland has completely redeveloped the project, with a modernized design and support facilities (including a bug tracker, mailing list, RSS feed, and Twitter account). The New Linux Counter is now up and running, with all the data for active users from the old counter. The old site will continue to operate for a time but will soon be shut down and requests redirected to the new site." -
New Skeleton Finds May Revamp History of Human Evolution
brindafella links to a series of articles published yesterday in the journal Science "on Australopithecus sediba, explaining that skeletons found in the Malapa cave in the World Heritage listed 'Cradle of Civilisation' push back to 1.97 million years the oldest known tool-using, ape-like pre-humans." As is typical, the full Science articles are paywalled, but the abstracts are interesting. (If you're a university student — or, in some cases, an alumni club member — you may have full journal access and not even realize it.) NPR has a nice article on the find as well.