Slashdot Mirror
Claude E. Shannon Dead at 85
Multics writes "It is with considerable sadness I report the death of the parent of Information Theory -- Claude Shannon. Here is his obituary over at Bell Labs. He was 85."
← Back to Stories (view on slashdot.org)
Why the US Patent Office has granted patents that violate his "so-called" theory, and they never make mistakes
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
...check out Shannon's classic 1948 paper "A mathemtical theory of communication". It is available in postscript (460 Kb), gzipped postscript (146 Kb), and PDF (358 Kb) formats, here on the Bell-Labs site. Warning, though: it's 55 pages, including 7 appendices.
And we wonder where people get the ideas to do things like this.
Obviously this sort of thing has a long standing tradition.
"It is a greater offense to steal men's labor, than their clothes"
The IBM Deep Blue site mentions him ...3 .3a.html
http://www.research.ibm.com/deepblue/meet/html/d.
Because these techniques don't exist. This was proven by Alan Turing in his paper Computable Numbers. The only way to "prove" a piece of software is to run it.
Oh, you're probably thinking of things like OO and top-down and all those gimmicks they teach in CS courses. Well, sometimes those techniques help you write better code (at a cost) and sometimes they make you write worse code (because of the costs).
Top-down, for example, makes sense when you have 100 programmers doing the software for a bank. It makes no sense at all and results in inferior code and user interfaces when you have 1 or 2 guys writing code for a PLC.
A line pops into my head from Greg Bear's book Eon. A character from a civilization 1,200 years advanced beyond ours is explaining to a contemporary person why they still have conflict. To paraphrase, there are limited resources and even very wise and educated people will inevitably come to have different ideas about how to solve problems. Each will want to use the limited resources in his way, because he believes it is the best way. Thus there is conflict.
This is why the language, OS, and technique wars will always be with us. What works in a palm does not work on a desktop, neither works in a PLC, and none of those things works for an enterprise server. And human beings are still subtle and complex enough compared to machines that attacking a problem of the right scale as an "art" will produce superior results.
Brackets contain world's first nanosig, highly magnified:[.]
...they just descend into a state of increasing disorder.
While Shannon's contribution to crypto was large (like his contributions in other fields), this is an overstatement which greatly distort's Shannon's own motivations.
Shannon's article was the capstone on a body of work which was mostly created during World War II by people like Alan Turing and John von Neumann, as well as dozens of equally important but less famous people who worked at places like Bletchey Park.
If you want a "sinister figure" why not pick on John von Neumann, who instead of riding his unicycle in the corridors was working hard to make the H-bomb a reality (alongside the original Dr. Strangelove himself, the truly sinister Edward Teller).
Brackets contain world's first nanosig, highly magnified:[.]
Formal methods allow you to reduce larger problems to "trivial" status than you can hold in your head, or to distribute them across multiple programmers without affecting their triviality. When the scale of the problem and the resources available to solve it are of the right size, this makes such methods appropriate.
Formal methods have two major costs. The first is that you can't easily back up and punt; when you realize half-way through that the formal design which looked good on paper is eating resources or presents a poor user interface in practice, it is very hard to go back and improve things. The second is that they increase resource requirements across the board, because you cannot apply creative solutions which become apparent midstream to streamline the project.
Sometimes the need for a formal method is due to the use of another formal method. For example, if you use top-down (appropriate for a large project) you are forced into all-up testing, which means you need strongly typed languages, etc. or you will be debugging forever. You can program bottom-up and prove each module as a trivial exercise as you build to higher and higher complexity, but your final project (while efficient and bug-free) may not resemble the formal spec much. This may not be a problem for a game; it may be a big problem for a bank.
Strongly OO languages like C++ which dereference everything eat processor cycles. It may not be apparent why this is a problem until you try to implement a 6-level interrupt system on an 80186 (as one company I know of did).
So yes, it is possible to prove some software, but this is not true of most software on a useful scale. When dealing with interrupts or a network or multithreaded or multiuser anything, there are so many sources of potential error that the formal system will bog down. You simply cannot check everything. You end up with either your intuition or no project.
Turing was interested in the problem of duplicating human intelligence. Thus, he aimed his theories at a high level of general abstraction, an "umbrella" which would cover his goal no matter what its eventual form might turn out to be. One result of this is that Computable Numbers is sometimes cited to support the concept of free will. Another is, as computers get more and more complex, Turing's take on the situation becomes more accurate and useful. His point was that even a fully deterministic machine can be beyond deterministic prediction. It's a shame he didn't live to see his vindication by Mandelbrot.
Brackets contain world's first nanosig, highly magnified:[.]
Actually, LISP's funny interpretation is "Lots of Irritating Superfluous Parentheses".
Check it out at the Jargon File:LISP
Somewhat more on topic, Shannon is mentioned directly in the Jargon file, where we find that
Claude Shannon first used the word "bit" in print in the computer science sense.
Also, I just read the book "Crypto" by Steven Levy and was surprised to find out how critical Shannon's work on information theory was to early non-government crypto research. If it wasn't for his papers, it is likely that the US government would still have the only really strong cryptography in the world.
He will be remembered.
Torrey Hoffman (Azog)
Torrey Hoffman (Azog)
"HTML needs a rant tag" - Alan Cox
... it is possible to prove some software, but this is not true of most software on a useful scale.
Actually I claim that this isn't true. It is possible to prove that two very different expressions of a problem are EQUIVALENT, but that's a very different thing from proving either of them is the CORRECT solution for a given problem.
This is not as bad as it sounds: The most effective way known to notice bugs is to express the problem in two very different ways and compare them. Primary example: The spec versus the code. (That's why the spec is IMPORTANT. "Self-documenting code" is a myth. When you go to verify it and the only spec is the code itself, the only thing you can check is that the compiler works right.)
Strongly OO languages like C++ which dereference everything eat processor cycles. It may not be apparent why this is a problem until you try to implement a 6-level interrupt system on an 80186 (as one company I know of did)
For most OO languages this is true. But for C++ it is not. You can get away without the dereferencing if you need to. (You COULD even fall back into a style that amounts to compiling essentially ANSI C via a C++ compiler.)
But to get away without the dereferences you also have to abandon the things that need them - like polymorphism. This doesn't mean you have to abandon all the features. (For instance, you can define a class hierarchy so you don't get polymorphc support unless you need it, and specify that you don't use it on particular function calls - at the risk of violating encapsulation checking. (Overloaded operators are very handy and often don't need to dereference anything.) But you're starting to get into the "why bother" area.
The big problem is that to do this efficiently you need somebody who:
A) Knows how to program well in C, including how to get efficient compiler output in the tight spots.
B) Knows how to program well in OOP style.
C) Knows HOW the C++ compiler achieves its effects, so he knows how to get it to do things efficiently.
And now you're talking about a VERY rare and expensive person. A is years of training. Adding B to someone who has A is typically 6 months minimum of learning curve just to start being productive again. (Starting with B makes it hard to get really good at A, while starting with A puts up mental roadblocks when you're relearning how to do things B style.) Most procedural programmers who learn OOP stop there.
But adding C is much like repeating the learning techniques of the pioneering days of computer education. Everybody learned assembly and how pre-optimizer compilers worked, because they were expected to WRITE their own compler some day. In the process they learned how to write source code that would trick a non-optimizing compiler into putting out good assembly. Technique C consists of doing much the same thing to a C++ compiler to get the equivalent of tight C out of it.
So if you happen to have a small team of people who already HAVE all this knowlege, they might make a C++ interrupt handler that is as efficient, or nearly so, as one coded in C or even assembler. But why not just do it in C, assembler, or some hybird in the first place? Then people with only skill set A can understand it and work on it. B-)
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way