Slashdot Mirror
Claude E. Shannon Dead at 85
Multics writes "It is with considerable sadness I report the death of the parent of Information Theory -- Claude Shannon. Here is his obituary over at Bell Labs. He was 85."
← Back to Stories (view on slashdot.org)
85 isn't a bad innings (ref. cricket score). Sad to see visionairies go but it's not like he had a short life. In that regard I would have felt more affected by Alan Turing's death.
--
Jon - TheSpork
Why the US Patent Office has granted patents that violate his "so-called" theory, and they never make mistakes
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
...check out Shannon's classic 1948 paper "A mathemtical theory of communication". It is available in postscript (460 Kb), gzipped postscript (146 Kb), and PDF (358 Kb) formats, here on the Bell-Labs site. Warning, though: it's 55 pages, including 7 appendices.
Claude Elwood Shannon was born in Petoskey, Michigan on April 30 [1916]. One of his childhood heroes was the inventor, Thomas Edison, whom he later learned was a distant cousin. Both were descendants of Colonial leader John Ogden.
And we wonder where people get the ideas to do things like this.
Obviously this sort of thing has a long standing tradition.
"It is a greater offense to steal men's labor, than their clothes"
One quote from the obituary interested me:
"Another example is Shannon's 1949 paper entitled Communication Theory of Secrecy Systems. This work is now generally credited with transforming cryptography from an art to a science."
By providing a firm mathematical basis Shannon managed to turn cryptography from a messy discipline without any real means of proof into a real science where mathematical methods could be used to ensure the security of cryptographic systems. Today, we wouldn't dream of thinking of cryptography as an art, it is well-established as a branch of applied mathematics.
But despite similar techniques being available for programming, coders still stick to their "tried and tested" methods of programming which owe less to science and more to guesswork. Rather than using formal techniques to ensure that code is 100% correct, they would rather just sit down and knock something out that sort of works. And this has led to the proliferation of buggy and insecure software we see today.
Why is it that programmers feel that they are somehow above the need for these techniques? It seems to me to be pretty arrogant to avoid using tools that would make their code better, almost as if they would rather not have perfect code. They would sneer at a cryptographic system that hadn't been rigourously proven, but be happy with any old hack that another coder knocks out whilst eating cold pizza and drinking Mountain Dew.
Why is there such a reluctance to produce properly proven code, when they demand it for other algorithmic systems?
Jon Erikson, IT guru
The IBM Deep Blue site mentions him ...3 .3a.html
http://www.research.ibm.com/deepblue/meet/html/d.
Back in '91, (Through emails)I spoke to Mr. Shannon a few times while researching Crytography and the applied Mathematics involved for my Thesis. I trully respect him for the fact that he took time out of his schedule to actually respond to me. Through his emails, I found him to be sharp witted, humorous, and charismatic. I'll miss him.
Because these techniques don't exist. This was proven by Alan Turing in his paper Computable Numbers. The only way to "prove" a piece of software is to run it.
Oh, you're probably thinking of things like OO and top-down and all those gimmicks they teach in CS courses. Well, sometimes those techniques help you write better code (at a cost) and sometimes they make you write worse code (because of the costs).
Top-down, for example, makes sense when you have 100 programmers doing the software for a bank. It makes no sense at all and results in inferior code and user interfaces when you have 1 or 2 guys writing code for a PLC.
A line pops into my head from Greg Bear's book Eon. A character from a civilization 1,200 years advanced beyond ours is explaining to a contemporary person why they still have conflict. To paraphrase, there are limited resources and even very wise and educated people will inevitably come to have different ideas about how to solve problems. Each will want to use the limited resources in his way, because he believes it is the best way. Thus there is conflict.
This is why the language, OS, and technique wars will always be with us. What works in a palm does not work on a desktop, neither works in a PLC, and none of those things works for an enterprise server. And human beings are still subtle and complex enough compared to machines that attacking a problem of the right scale as an "art" will produce superior results.
Brackets contain world's first nanosig, highly magnified:[.]
when are we going to establish a computing hall of fame? might be a member of the inagural class?
...they just descend into a state of increasing disorder.
Everyone here knows the programming references in this troll are nonsense, but just a note on the crypto side.
For the most part, mathematical methods do not *ensure* the security of our systems. We have no proof that, say, AES is secure; all we know is that the world's greatest cryptanalysts have attacked it as hard as they can, and come to the conclusion that there won't be a practical break for a long time to come. In theory, it could be heinously broken tomorrow. We don't have a proof that it won't be, just decades of experience in attacking cryptosystems and finding out what makes them weak in practice.
Cipher design is a mixture of a science and an art. There's certainly artistry in the design of AES!
--
Xenu loves you!
Does this mean that research into perpetual motion would be a way of cheating death for the scientists involved?
((((((You(obviously)don't((know)much)about((LISP)o r)))you))wouldn't)make(such((a)naive))remark)
While Shannon's contribution to crypto was large (like his contributions in other fields), this is an overstatement which greatly distort's Shannon's own motivations.
Shannon's article was the capstone on a body of work which was mostly created during World War II by people like Alan Turing and John von Neumann, as well as dozens of equally important but less famous people who worked at places like Bletchey Park.
If you want a "sinister figure" why not pick on John von Neumann, who instead of riding his unicycle in the corridors was working hard to make the H-bomb a reality (alongside the original Dr. Strangelove himself, the truly sinister Edward Teller).
Brackets contain world's first nanosig, highly magnified:[.]
...for Lots of Irritating Single Parentheses.
Brackets contain world's first nanosig, highly magnified:[.]
There is a large field in computer science dedicated to research about proving properties of programs, it's called formal methods. However, in practice this is still only used for hardware and very low level software.
main(i){putchar((1397178701>>(i/2+(i==3))*8)+(i/2
... The NY Times gives his birthdate as April 30, 1916. That'd only make him 84. Not that this is really all that important.
Story
There's always sufficient, but not always at the right place nor for the right folks.
The New York Times Obit can be found here. Of course this will expire way to quickly because the NYT is only the newspaper of record for the current week.
Formal methods allow you to reduce larger problems to "trivial" status than you can hold in your head, or to distribute them across multiple programmers without affecting their triviality. When the scale of the problem and the resources available to solve it are of the right size, this makes such methods appropriate.
Formal methods have two major costs. The first is that you can't easily back up and punt; when you realize half-way through that the formal design which looked good on paper is eating resources or presents a poor user interface in practice, it is very hard to go back and improve things. The second is that they increase resource requirements across the board, because you cannot apply creative solutions which become apparent midstream to streamline the project.
Sometimes the need for a formal method is due to the use of another formal method. For example, if you use top-down (appropriate for a large project) you are forced into all-up testing, which means you need strongly typed languages, etc. or you will be debugging forever. You can program bottom-up and prove each module as a trivial exercise as you build to higher and higher complexity, but your final project (while efficient and bug-free) may not resemble the formal spec much. This may not be a problem for a game; it may be a big problem for a bank.
Strongly OO languages like C++ which dereference everything eat processor cycles. It may not be apparent why this is a problem until you try to implement a 6-level interrupt system on an 80186 (as one company I know of did).
So yes, it is possible to prove some software, but this is not true of most software on a useful scale. When dealing with interrupts or a network or multithreaded or multiuser anything, there are so many sources of potential error that the formal system will bog down. You simply cannot check everything. You end up with either your intuition or no project.
Turing was interested in the problem of duplicating human intelligence. Thus, he aimed his theories at a high level of general abstraction, an "umbrella" which would cover his goal no matter what its eventual form might turn out to be. One result of this is that Computable Numbers is sometimes cited to support the concept of free will. Another is, as computers get more and more complex, Turing's take on the situation becomes more accurate and useful. His point was that even a fully deterministic machine can be beyond deterministic prediction. It's a shame he didn't live to see his vindication by Mandelbrot.
Brackets contain world's first nanosig, highly magnified:[.]
Or compress any two bytes by one byte, for that matter, on average. Or one bit.
Or one petabyte by one bit, on average, for that matter.
I am for the complete Trantorization of Earth.
You not so smart...
True, but irrelevant. Intelligence has limited practical application. Sometimes you have to be ignorant of what cannot be done so that you can accomplish the impossible. I don't know much, but everything I do know I learned from the reading motivational posters put up by my employer.
How do you compress a file of size 1 byte by 1 byte
Ah, that is part of my proprietary, patent pending method. Did I mention that my algorithm is not guratneed to halt? (Note that by the law of the excluded middle, anything that cannot be proved false must be true.)
By the way grasshopper -- a sequence doesn't have to be one bit long to be irreducible (by means other than my patent pending non halting algorithm of infinite size of course). Consider this riddle -- what is the definition of a random sequence?
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
Shannon's work provided the groundwork for programs you use all the time, whether you know it or not!
Data compression
Huffman's compression algorithm reads like an appositive to Shannon's work.
Error Correction
Like your data to arrive in one piece? No corruption?? Thank Shannon for providing the groundwork for efficient insights to error correction.
Encryption
Without Shannon's take on information theory, encryption would be the same hunt-and-peck game that was played during WW2.
Have a little respect for something you couldn't do.
There is another obit (a little longer) that is in the NYTimes (<p>I am really sorry that I never go to meet him, he is certainly one my heros. His picture and Richard Feynman's are above my desk. I hope he makes it onto a postage stamp some day. I don't think a lot of CS people realize that without Shannon, they wouldn't know about Boolean algebra (well this is a bit of an exageration), but he was the first one to really make the link between boolean algebra and it's use in circuits and then computers. He was actually in the MIT electrical engineering department, I believe. Even though he was really a mathematician.</p>
"Politics is for the moment, an equation lasts eternity" -A. Einstein
Digital signals and boolean calculas seems "obvious" to
us now, but not so in the 1930s when a binary
switching tube cost dollars instead of micro-cents
today.
Was digital a "pregnant idea" about to be discovered
by nyone half-way smart,
or would it take a really smart person to accelerate the discovery?
... here.
Stupid job ads, weird spam, occasional insight at
A character from a civilization 1,200 years advanced beyond ours is explaining to a contemporary person why they still have conflict. To paraphrase...
... it's because Greg Bear is a tedious hack who can't think of anything better. Eon has to be the worst SF book of the 80's. I did have some fun reading the worst bits out to friends but that was it.
TWW
"Encyclopedia" is to "Wikipedia" what "Library" is to "Some people at a bus stop"
One thing the Times obit did not mention was Shannon's interest in juggling late in his life. He learned to ride unicycle and juggle balls at 70+. I remember going to an international juggling convention in the '80s and seeing
film of some of his juggling simulator machines -- very clever. Some of my juggler/computer jock friends were astonished that he was still alive and kicking back then, having only seen his seminal work from relatively early in computer history.
Not a web designer.
These activities bear out Shannon's claim that he was more motivated by curiosity than usefulness. In his words "I just wondered how things were put together."
Nowdays his curiosity could easily land him in trouble. Figuring out how things are put together seems to be a big no-no in the US today. Gee, you might discover all sorts of things big companies don't want you to know - like how crappy is their encyption, how stupid their blocking software, or how lame their security.
--- Hot Shot City is particularly good.
The rule is that they will not issue a patent for a perpetual motion machine without a working model .
So everyone get out their hammer and saw and start building a machine that runs forever. You need more than just an "idea"... You actually have to be able to produce one and show it to them.
They shouldn't grant a patent that gaurantees compression by at least 1 bit without a working program which should be run on itself once for each bit it has so that they can prove to us that every piece of data in the world can be reduced to 1 very random bit.
Education is a better safeguard of liberty than a standing army.
Edward Everett (1794 - 1865)
[further rantings munched]
(I know this is a troll, but denigrating the memory of a well-respected scientist is cowardly and inexcusable.)
Your post is unadulterated nonsense. Americans have had a paranoid culture all the way back to before the Founding. Like it or not, a lot of our core national makeup stems from Puritan attitudes. One of those attitudes was paranoia (Salem witch trials, anyone?).
Your assertion that people were open and honest before the 20th century is ludicrous on the face of it. History is chock full of betrayal, double-dealing, and all the other fun stuff that wackos and trolls such as yourself claim sets us apart from the march of ages. Besides, it's not as though Dr. Shannon single-handedly invented cryptography. Or have you never heard of rot13? It's the canonical example of what's known as a Caesar cipher (yes, that Caesar - as in Emperor Julius Caesar, of ancient Rome). People have been trying to hide information for centuries, and as technology and knowledge has expanded, so have methods for information hiding and information discovery.
--Ford Prefect
The work of Turing and Church proved that there does not exist a general procedure to determine if an arbitrary piece of software terminates (and by extension, that many other properties are similarly undecidable). Did you ever study computer science?
They did NOT show that one cannot "prove" software. In addition, the notion of "proving" software is nonsense unless you say what you are proving about the software. Can I prove that every well-formed C program has a main? Of course, it's in the definition. Can I prove that an arbitrary well-typed Standard ML program has a well defined evaluation sequence (that it is "safe")? Yes. Can I prove that an arbitrary program in the simply typed lambda calculus always terminates? Yes. There are many extremely useful properties of programs that actually can be proved mechanically.
Furthermore, just because a procedure can't do it in general doesn't mean that a human can't prove specfic programs, perhaps with the help of a computer. Look at CAR Hoare's logic for proving iterative program correctness, for instance.
Because it links information transmission and retrieval to the noise, which is a result of temperature, you can ultimately prove that a material link is always needed to transmit information. Transmission of information without a material channel, i.e. by "spiritual" means, would violate the second law of thermodynamics. The outcome: we would perceive time as a bi-directional dimension. There are some "Maxwell's daemons" whose impossibility can only be proved by the use of information theory.
Further, his work on information entropy is fundamental. Entropy is like a dual to the Gaussian (normal, bell-curved) distribution, which I hope everyone has at least heard of. Entropy measures are used in many machine learning algorithms, which makes them useful in AI, data mining, etc.
--
Marc A. Lepage (aka SEGV)
--
Marc A. Lepage
Software Developer
He once almost beat top Soviet player Botvinnik (IIRC; it might have been another famous Soviet); he was up the exchange with a win in sight, but then made some silly blunder. The interview I read this from was an old OMNI, with Shannon & his wife.
Of course, I get the impression that at that point, Botvinnik used tricky smoke & mirrors tactics against Shannon, but I still imagine that Shannon was honest enough that he wasn't embellishing his advantage so much.
This mention that people should be judged by their "social way", whatever that may be, always pop up when scientists are mentioned. Also, obituaries in slashdot always have a couple of "we should not be sad" trolls. Funny once, man, not funny twice.
What did Shannon do for humanity?
Shannon worked on crypto during WWII from the American side. He spent time with Alan Turing exchanging ideas. Just from that he made a significant contribution to the world.
He was a great scientist and an interesting man, let people be sad about the loss.
I must put together a brief crypto FAQ for slashdotters one day. Summary: QC is interesting but way overhyped.
--
Xenu loves you!
>Come on, he's just a computer figure. Being a computer engineering student myself,
>and holding several figures in the computer world in very high regard,
>I have to say I will feel sad for the death of people who have advanced
>humanity in a more social way, perhaps in charity or peace work
So you're saying his death shouldn't be a sad event because "he's just a computer figure". Maybe you're just jealous that your own death won't even be 1% as sad as his.
No. The trickle down effects aren't BS. Were it not for his theories the Nazis might have won the WWII. Figure that.
Wow... Talking about far-fetched...
As I understand it (though I could be off on this), the patent office doesn't strictly require that a device or process do what it is claimed to do to give you an exclusive on it. They stopped requiring that long ago - about the time they stopped requiring you to deliver a working model (for lack of storage space).
If you want to pay a bunch of money and effort to get a 17-year exclusive on building something new that doesn't work, fine with them.
So a patent is not an endorsement by the US government of your scientific or engineering claims.
An exception is perpetual motion machines. There were SO many applications wasting SO much of their time that they still require you do deliver a working model. THAT put a big spike in the fad, though people still try.
One fellow had trouble patenting a very efficient still. It was very tall to use gravity to create a decent vacuum at the top (like the original water version of a mercury barometer), and acted as a counter-current heat exchanger, scavenging heat from the condensate and brine to preheat the raw water feed. You still had to add the heat of solution and some more to deal with losses, but that's a LOT less than boiling off the whole output stream at local atmospheric pressure.
They initially refused his application, thinking it was a perpetual motion machine, and he had to argue long and hard to get the patent.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
dude how is it off topic? one article said 84, and the other said 85. i mean, that shouldn't be too hard to relay. apply some thought next time.
slashdot: where everyone yells sarcastic metaphors to themselves to understand the issue
... it is possible to prove some software, but this is not true of most software on a useful scale.
Actually I claim that this isn't true. It is possible to prove that two very different expressions of a problem are EQUIVALENT, but that's a very different thing from proving either of them is the CORRECT solution for a given problem.
This is not as bad as it sounds: The most effective way known to notice bugs is to express the problem in two very different ways and compare them. Primary example: The spec versus the code. (That's why the spec is IMPORTANT. "Self-documenting code" is a myth. When you go to verify it and the only spec is the code itself, the only thing you can check is that the compiler works right.)
Strongly OO languages like C++ which dereference everything eat processor cycles. It may not be apparent why this is a problem until you try to implement a 6-level interrupt system on an 80186 (as one company I know of did)
For most OO languages this is true. But for C++ it is not. You can get away without the dereferencing if you need to. (You COULD even fall back into a style that amounts to compiling essentially ANSI C via a C++ compiler.)
But to get away without the dereferences you also have to abandon the things that need them - like polymorphism. This doesn't mean you have to abandon all the features. (For instance, you can define a class hierarchy so you don't get polymorphc support unless you need it, and specify that you don't use it on particular function calls - at the risk of violating encapsulation checking. (Overloaded operators are very handy and often don't need to dereference anything.) But you're starting to get into the "why bother" area.
The big problem is that to do this efficiently you need somebody who:
A) Knows how to program well in C, including how to get efficient compiler output in the tight spots.
B) Knows how to program well in OOP style.
C) Knows HOW the C++ compiler achieves its effects, so he knows how to get it to do things efficiently.
And now you're talking about a VERY rare and expensive person. A is years of training. Adding B to someone who has A is typically 6 months minimum of learning curve just to start being productive again. (Starting with B makes it hard to get really good at A, while starting with A puts up mental roadblocks when you're relearning how to do things B style.) Most procedural programmers who learn OOP stop there.
But adding C is much like repeating the learning techniques of the pioneering days of computer education. Everybody learned assembly and how pre-optimizer compilers worked, because they were expected to WRITE their own compler some day. In the process they learned how to write source code that would trick a non-optimizing compiler into putting out good assembly. Technique C consists of doing much the same thing to a C++ compiler to get the equivalent of tight C out of it.
So if you happen to have a small team of people who already HAVE all this knowlege, they might make a C++ interrupt handler that is as efficient, or nearly so, as one coded in C or even assembler. But why not just do it in C, assembler, or some hybird in the first place? Then people with only skill set A can understand it and work on it. B-)
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
How about this answer: Every subset of the sequence is independent of every other disjoint subset of the sequence. (Or, knowing any particular subset of the sequence gives you no information about the rest of the sequence, whether from the past, the future, or both).
Excellent. And what can we conclude about the length of a program that is capable of producing a seqence of a given length n?
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
What Shannon did for science helped convince me to stick with a lot of my theoretical work. He was a truly inspiring and committed person and his death greatly saddens me.
Thank you Claude Shannon for all you did.
He said that Shannon had showed up at a conference a just a few years ago. Word quickly spread around the conference floor that Shannon was there. Finally someone was brave enough to break the ice and talk to him, and instantly a whole crowd was surrounding Dr. Shannon, just to meet him or say hi. "Like a rock star," my professor said.
Claude Shannon may well be the most influential mathematician no one has heard of.
i first came across shannon in a basic semiconductor class when designing my first digital circuit. then i came across him in a bioinformatics class when talking about information entropy in genomic DNAs. and then again while reading recreational computer books, and then in statistical field theory, and then again... and again... i always thought he was one of the giants of modern science, whose work, though seemingly confined to the field of information theory, actually reached across disciplines and influenced them all in subtle but notable ways. i pay tribute here.
Since I work in telecom, Shannon's Law is close to my work, and I'm saddned by his death.
-RSH
You could've hired me.
J.W. Cooley and J.W.Tukey and the Fast Fourier Transform, ring a bell yet ?
How is entropy the dual to the Gaussian distribution? Entropy is defined for any probability distribution, not just Gaussian.
L'etat n'a pas besoin des savants.
- Robespierre, refusing clemency for Lavoisier
Given that this is an article about the death of a famous mathmetician, I find it ironic that the headline reads "Shannon dead at 85" but the linked article reads "Shannon dead at 84" We could at least get the number of years in his age correct!!!!
---K2K was here---