Slashdot Mirror
IBM Scientists Measure the Heat Emitted From Erasing a Single Bit
ananyo writes "In 1961, IBM physicist Rolf Landauer argued that to reset one bit of information — say, to set a binary digit to zero in a computer memory regardless of whether it is initially 1 or 0 — must release a certain minimum amount of heat, proportional to the ambient temperature. New work has now finally confirmed that Landauer was right. To test the principle, the researchers created a simple two-state bit: a single microscopic silica bead held in a 'light trap' by a laser beam. (Abstract) The trap contains two 'valleys' where the particle can rest, one representing a 1 and the other a 0. It could jump between the two if the energy 'hill' separating them is not too high. The researchers could control this height by changing the power of the laser, and could 'tilt' the two valleys to tip the bead into one of them by moving the physical cell containing the bead slightly out of the laser's focus. By monitoring the position and speed of the particle during a cycle of switching and resetting the bit, they could calculate how much energy was dissipated."
Wonder how much heat is dissipated when you mod a post down?
Sheesh, evil *and* a jerk. -- Jade
Yeah, it's kind of like a piece of armor being considered arrow-proof, and then you fire an arrow out of a railgun.
I wonder how you would even measure it, though, and distinguish the heat from a bit changing from the ambient heat from drive operation.
Random Thoughts From A Diseased Mind (Not For Dummies)
To store information, you need the ability to set something into at least two possible states, one of which can be the intrinsic state. No matter what you use for storage, you'll always need energy to reach the non-intrinsic state(s), since the intrinsic state is, essentially by definition, the state achieved with no external energy applied.
If you must add energy to enter a non-intrinsic state, it makes perfect sense that the energy would need to be dissipated to return to the intrinsic state (which equates to erasing the bit). I expect something so obvious wouldn't warrant experiments and articles, so what am I missing that makes this more complicated than it seems to be?
It probably reflects the spirit of Landauer's claims. Claims such as this depend upon an understanding of physics, which was much more common in computing back in the days when innovation depended upon an understanding of physics in order to develop new hardware. You also have to consider that a variety of different techniques were used to make computer memories back then, so his claims had to be based upon the underlying physics rather than a particular memory technology. So it is fair game to apply different physical models to prove his claims.
I mean, this is demanded by Maxwell's demon, right? You need to expend energy to store information in order to not violate the 2nd law of thermodynamics. Awesome that they measured it, for sure.
except for endothermic reactions
None as it contains no information.
Are you sure you weren't trying to post some lameness, but the whitespace filter kept getting in your way?
Millions of bits worth...
For in politics, as in religion, it is equally absurd to aim at making proselytes by fire and sword. - Publius
Yeah but how do you know which bit you measured?
Specifically in the calculation of the Landauer limit, E = kT(ln2), the minimal energy needed to transform a single bit. The interesting thing is that 10^20 bit operations is just a watt. This means that the efficiency of today's computers is just 0.00001%. More details at http://tikalon.com/blog/blog.php?article=2011/Landauer.
Oh it has a law on Wikipedia, must be a waste of time to test or verify it then! Seriously, have a read about how science works before attempting to comment again. A "law" in science is not like a legal law - i.e. it is not a fact merely by self-assertion (a legal law is a law because law makers say so). Scientific "laws" require test and proof; they often require refinement in details. Scientific "laws" do not exist as abstract facts about the universe - they are human attempts to model the universe from the knowledge we currently have. Our limited knowledge means that the detail may be imperfect. A quick survey of the history of science demonstrates that we often get them wrong.
I'm not attempting to challenge the "laws" of thermodynamics - my guess would be that we have the broad picture right (we have a lot of evidence in favour), but again, given the history of science I would be surprised if every detail of taught theory in that area survives the next few hundred years without some modification.
Yes the scientists doing this probably expected some heat to be measured. They were more interested in precisely how much. This is science - an ongoing process.
What? How will we decrease global temperatures by making prime-time television more steamy?
Let 0s be room temperature and let 1s be somewhat below room temperature. Then to erase the memory I expose it to the room. As it erases the memory will absorb some heat from the room instead of releasing heat.
Not really a practical form of computer memory, but seems sufficient to disprove Landauer.
So can we start blaming google for more global warming yet, swiching all those bits?
OMG Ponies!!! with Glitter!!!! I miss Pink
Duh! You divide the result by the HDD capacity!
In 1961, resetting a bit involved passing a huge current through the wires surrounding a toroidal core which represented one memory bit. So to say that it releases heat is ridiculous, it actually consumes orders of magnitude more heat than could possibly be considered in theory or measured in practice.
its not necessarily stupid test.. in terms of science, we can estimate the amount of energy from various sources, suchas nuclear plant, or total earth energy, or our solar system, or galaxy... using that estimate, we can put an upper bound on the maximum amount of computational power we have at our disposal.. such as, a certain problem is shown to require X calculational complexity, and X exceeds or the amount of disposable energy in our solar system, thus, X is uncalculable given current technology.
Now, let X be some sort of encryption complexity. now do u see how it could be useful?
Which require you to put strictly more energy to prepare reagents for the reaction than would be consumed by the reaction.
I'm not attempting to challenge the "laws" of thermodynamics - my guess would be that we have the broad picture right (we have a lot of evidence in favour), but again, given the history of science I would be surprised if every detail of taught theory in that area survives the next few hundred years without some modification.
Having the broad picture right just means you have a working model, though. It doesn't mean you've actually discovered how the universe works, just that you can make accurate predictions. Maybe later it turns out that what happens, happens for a totally different reason than what you thought.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
A little bit of heat
Does this suggest that by saving up erasures to be done more slowly, perhaps by flipping bits to 0 near the time when they are flipped to 1, could energy be saved and the Landauer limit approached? Also, are there architectures in which a flipping a bit in one direction uses less power, or when blocks of bits can be deselected by some pointer instead of actually erased, trading memory hardware space for power usage?
Not really that surprising, a silicon atom is about 0.11nm and the lattice grid in a silicon crystal 0.54nm, which is still way smaller than the 32nm processors he's talking about. I don't know how many electrons flow down each 32nm path but they're between 0.1nm and 0.000006nm in diameter depending on what model you use - quantum mechanics makes a mess of this anyway - so it's way more than one. If you want single electron calculations you'll have single electron signals, one quantum event and your signal is lost. So the limit is likely to remain a very theoretical limit.
The other thing is that this only includes the operation itself, no clock cycle, no instruction pointer, no caching, prefetching, branching, this is the ideal you could get out of a fixed-function ASIC that only does one thing, not even as programmable as a GPU shader. We already know that there's a significant gain to that, but even supercomputers aren't built that specifically to the task. Formulas must be tweaked, models adjusted, parts must be able to be used in many computers. We've already seen that a GPGPU can beat a CPU by far on some tasks, but even they aren't close to such an ideal.
If you think about this in encryption terms it's not that much... it says you can at most improve 23-24 bits, in encryption most have used the Landauer limit to "prove" there's not enough energy to break a 256 bit chipher by brute force. In some places I don't think it's that relevant either, in for example mobile I think the energy involved in bandwidth use will be more significant. Want to stream a HD movie? It's not the decoding that kills the battery, it's the 3/4G data connection. Just like cameras get better but good optics still isn't small, light or cheap.
Live today, because you never know what tomorrow brings
Having the broad picture right just means you have a working model, though. It doesn't mean you've actually discovered how the universe works, just that you can make accurate predictions. Maybe later it turns out that what happens, happens for a totally different reason than what you thought.
Science is all about making predictions, and not about discovering how anything works (formally, anyhow). Or as a physics professor put it: "There are no particles, only clicks in my Geiger counter".
Or, in reverse:
Cool down the disk to a point where you can measure the temperature changes really well. Now start the encryption. How much information does the change in temperature of the disk (or SSD, or RAM) give you? Could be interesting.
Therefore, by the (faulty) logic you're using, you're just a cow with a keyboard - osu-neko (2604)
Landauer's claim was about the relationship between entropy as used in information theory and entropy as used in thermodynamics: specifically, that entropy in information theory is identical to the entropy in thermodynamics. The scientists used this set-up so they could measure a change of exactly one bit (the information-theoretic conception of entropy) while controlling outside heat influences (the thermodynamics conception of entropy), and see if the change in information corresponded to the change in heat as predicted by thermodynamics and information theory.
Without precisely controlling the change in information and precisely measuring the change in heat, the result is much less clear. That's why they used this methodology and equipment. Moreover, as this is empirical evidence for a very general identity between heat and information, the result will hold for computer memory as well.
In that case, I could have used a mechanical switch to represent 0 and 1 and told you that heat was dissapated. There needs to be a little more to draw a parallel between a random experiment and computer memory.
Additionally, the prediction was a great deal more specific than "durrr it will get more hot," it was more: "the heat will change by this particular amount, relative to the ambient temperature, as predicted by these equations,"
By monitoring the position [AND] speed of the particle...
Unpossible! Measure one or the other, but not both...
"If anything can go wrong, it will." - Murphy
Computer memory is a bunch of mechanical switches. The point is that they have a lot of sources of heat aside from reductions in the information content of the physical system. The researchers built a switch that was as efficient as possible so the vast majority of heat dissipation could be attributed to changes in the information content of the switch. Real computer memory will have heat dissipation due to changes in information content along with heat dissipation from such things as moving read/write heads.
additionally, the point isn't just that heat was dissipated, but rather that a specific quantity was dissipated as predicted by thermodynamics and information theory.
>>entropy in information theory is identical to the entropy in thermodynamics
Is there a name for this law?
Also, what does this say about the reality of information itself?
You are thinking of scientific theories or hypotheses. Scientific laws are based on observations, but they are not proven. In fact, they are the assumptions and axioms upon which proofs are built.
I can appreciate that. But I question the actual relevance of the results, given that the "memory technology" used doesn't resemble anything I've ever heard of being used in a production computer in 30+ years.
The fact that energy would be needed to force a state change should have been intuitively obvious to anyone with even a Grade 12 physics education.
I do not fail; I succeed at finding out what does not work.
"IBM Scientists Measure the Heat Emitted From Erasing a Single Bit"
All of this seems like a bunch of hot smoke to me. Can't these scientists find something better to do with their pay?
Wonder how much heat is dissipated when you mod a post down?
Less than the heat that is saved by not displaying the down-modded post in millions of basements all over the world.
Ezekiel 23:20
It says that information is disorder. And thermodynamic entropy is (for some definitions of order) order as well. If you have all of the air molecules in a room compressed into the corner, maybe that's ordered? But that's one small lump of air, and a whole lot of vacuum. Evenly distributed air is more ordered because it is uniform. If you let a system starting in any arbitrary corner-gas configuration (and there are a lot, since each molecule can have any number of different values describing it) progress for X amount of time, you find that almost certainly you have ended up in an even-gas configuration. On the other hand, if you start in an even-gas configuration, and progress for X amount of time, you will almost certainly still be in an even-gas configuration. This may seem at odds with the fact that laws of motion are time reversible (at least if you assume that molecules are like frictionless billiard balls, as physicists are wont to do). But it's not. If you take some specific corner-gas start A , and run it for X time, you will (probably) have an even-gas configuration B. If you take B, reverse the velocity of all molecules, and run it for X time again, you will be at A (again, assuming molecules are frictionless billiard balls). But, with discrete space and velocity, you can count the possible velocity and position vectors. There are a LOT more even-gas configurations than there are corner-gas configurations. So, with a tiny room and only a few molecules, you can establish the chance that after X time starting at even-gas, you end up at corner-gas. And even for very small systems it basically 0. Entropy is the concept of changes to a system that are not reversible, not because of laws of PHYSICS but laws of STATISTICS. The second law is the observation that, by statistics, you will tend to a uniform (ordered) system because there are a lot of ways to go that direction, and very few ways to go the other direction.
Landauer's observation is that any computational device, at the end of the day, stores information mechanically (again, I refer you to the fact that for our purposes, subatomic particles are frictionless billiard balls, so even things like the atom-trap from TFA are mechanical devices). So if you have a 32 bit register, it has 2^32 configurations. If you consider how many possibilities there are for ordered bit flips involving X bit flips total, it's 32^X. And if you start at 0, almost all of those ordered flips will take you to a pretty chaotic state. But if you start from a random state, almost none of those same bit flip orders will get you to 0. So treating the system as a completely mechanical one, thermodynamics applies and puts limits statistical limits on such changes. What Landauer did is establish a maximum circuit temperature T for your memory/CPU, and observe that you won't want Brownian motion breaking your system, so 0/1 need a minimum separation for the system to be useful at temperature T. This puts a lower bound on the state counts, and lets traditional thermodynamics establish a minimum energy dissipation to go from a high entropy state to a low one (like a 0'd out register). What information entropy does is take the same thing and say that therefore the disordered information has intrinsic entropy, since regardless of system design it requires a certain minimum entropy to store that information. It's avoidable if your system is reversible, which is possible if you have more ways to represent a bit pattern the more ordered that bit pattern is. So if you have fewer ways to store 10010101 compared to how many ways you have to store 00000000. It's also beatable if you find a way to store information non-physically. But good luck on that front.
Neat, huh? I took a course on Kolmogorov Complexity, which is somewhat related, and pretty cool.
ASCII stupid question, get a stupid ANSI
I've seen serious claims that "reversable computation" can be done with no energy input at all. What this doesn't cover, of course, is setting up the initial conditions, or extracting the results of the computation. One requirement is that at the end of the computation, the state of the system should be identical to the initial state.
I must admit that I don't understand either the utility, or the feasibility, of such a system. But there have been serious claims that computation does not, itself, require any energy at all.
OTOH, I don't see this experiment as any proof that all writing of a bit requires a minimal amount of energy. It shows that using THIS technology it requires THAT minimal amount of energy. This is a *far* different statement.
I think we've pushed this "anyone can grow up to be president" thing too far.
No kidding? You DO WORK and ENERGY IS RELEASED? Is anybody surprised to see that Landauer was right? Nobody?
What's surprising is that somebody bothered to verify a result that's obvious to everybody with a basic understanding of physics. If the claim weren't true, the machinery that they used to perform the experiment wouldn't have worked either.
Science publishing is not what it used to be.
I'm going out on a limb here, not having had the time to study this stuff enough,
but my intuition says that the unification of information theory and physics will yield a great breakthrough in physics.
I take the view that thermodynamics and Shannon information theory are literally about the same thing exactly, not just by weak analogy.
Related factoids:
1. All information is embodied mutual information.
a. It must be embodied in some local configuration of matter/energy.
b. It must be mutual in that the information in some clump of matter/energy is either about itself (the various parts/bits that comprise itself) or the information must be about some other configuration of matter/energy and spacetime somewhere else. Those things somewhere else also got information about the clump during the interaction.
2. clumps of matter/energy gain information about external parts of the universe only by interacting with them (during which bits of (mutual) information are transferred).
3. Light speed (and planck length) places a limit on the rate of mutual information transfer across a boundary (of a certain area) in spacetime. (Holographic cosmology stuff?) Q: What is that limit, in bits/second/m^2 ?
4. It is not just special things like human/slug brains and computers that have information about their surroundings. Every clump of matter/energy i.e. every non uniform local configuration of spacetime has (embodies) such information, which is some function of the interactions that clump has had.
5. Complexity of sequence of interactions over time as clump evolves through spacetime means that the information a clump has about any particular past interaction (or past encountered other thing) is necessarily always decreasing/dissipated/radiated into a larger space over time.
6. Such information about specific past/far off things is also necessarily intermingled (within the clump's boundary) with more and more noise (information about other things). This may be saying the same thing as the "local information" dissipation statement.
7. The second law of thermodynamics is explained by 5. and 6.
8. Information (the amount of local embodied mutual information) is what fundamentally characterizes configurations of matter/energy, space, and time. Other laws of thermodynamics are implied by this. And 1st law, conservation of energy, is the same exactly as saying conservation of (the amount of embodied, mutual ) information in the universe.
Where are we going and why are we in a handbasket?
It is in the spirit of what Landauer was considering. The larger question is if information entropy and thermodynamic entropy are related.
Fantastic. Thanks for writing this up.
Excellent response, thanks.
Pretty far afield followup question: every time Work is performed, Entropy increases. Using the Landauer Principle, it seems like you could you consider information processing to be a sort of Work being done, leading to a similar increase in entropy. If our conscious minds are a form of information processing engine, could consciousness be a byproduct of the Work being conducted by the information processing, which manifests itself simply as extra heat being radiated by the system?
It's also beatable if you find a way to store information non-physically.
I think this is what throws everyone when they think about the physics of knowledge. The vast majority of people don't realize that the physical embodiment of information must obey the laws of physics, and even many who do seem to believe knowledge ought to have some form of "soul" not shackled by physical constraints.
If I have been able to see further than others, it is because I bought a pair of binoculars.
I must admit that I don't understand either the utility, or the feasibility, of such a system.
Wikipedia gives an answer:
"Although in practice no nonstationary physical process can be exactly physically reversible or isentropic, there is no known limit to the closeness with which we can approach perfect reversibility, in systems that are sufficiently well-isolated from interactions with unknown external environments, when the laws of physics describing the system's evolution are precisely known.
Probably the largest motivation for the study of technologies aimed at actually implementing reversible computing is that they offer what is predicted to be the only potential way to improve the energy efficiency of computers beyond the fundamental von Neumann-Landauer limit [2] of kT ln(2) energy dissipated per irreversible bit operation.
[..]
Although achieving this goal presents a significant challenge for the design, manufacturing, and characterization of ultra-precise new physical mechanisms for computing, there is at present no fundamental reason to think that this goal cannot eventually be accomplished [..]"
There are rational objections to this proposal. Landauer's principle is really an expression of entropy in information systems -- which can be mathematically modeled as though they were thermodynamic systems. It's a bold claim to say this has a physical reality and a loss of information actually does release energy -- and since Landauer's principle expresses this as heat energy, wouldn't it then be detectable (i.e. not dark)?
Well, so much for *that* objection. :)
http://http//www.universetoday.com/85855/astronomy-without-a-telescope-holographic-dark-information-energy/
I've got a bad attitude and karma to burn. Go ahead. Mod me down.
Back in the Uni library, I once had an old ('60's?) book in my hands which stated that for every logical AND circuit, combining two '1' bits would also result in heat. The author suggested designing AND circuits so taht they would have two results: the logical outcome, and the overflow 'exhaust', both connected to the rest of the circuitry. This would be used to keep the processor from generating heat, but might also have more practical, logical uses. (He probably said similar things for other kinds of circuits.)
I thought it was a wonderful book at the time, and wondered if anyone ever tried to work out this man's arguments.
Now I wonder if anyone is familiar with this? Haven't remembered the author or anything.
"We can confirm that Debian does *not* ship the version with the trojan horse. Our version predates it." [CA-2002-28]
You are absolutely right. And that's why we have modern technology and, in fact, physics themselves: because people began verifying obvious "facts".
Forget magic. Any technology distinguishable from divine power is insufficiently advanced.
I remember reading in Bunnie Huang's book on Hacking the Xbox that a computer just enumerating 2**256 (let alone doing anything useful) would require enough power to boil the oceans.
Maybe it wasn't 256, but it was related to cryptography.
I thought the energy to flip a bit was already measured in Quantum Computing devices as it tends to cause de-coherence?
If not, then it should :-)
The thing is, reversible computation doesn't appear to allow the answer to be extracted. So it's not clear what use it is. And it hasn't been actually built, so I'm not convinced that it's feasible.
I think we've pushed this "anyone can grow up to be president" thing too far.