Studying Intelligence Thru Entropy?

An Anonymous Coward asks: "Given that entropy is the measure of order or disorder. Given that any force that changes the entropy of any system in a predictable way is an 'intelligent' force. Is it true that the study of HOW entropy changes in any given system is the study of intelligence itself, in that given system? I is it true that producing systems whose sole purpose it is to capture and synthesise changes in entropy is the production of intelligent systems?"

"A case in point. Neural networks are weighted switches. They store their 'weights' in the neuron. The storage of these weights determines the networks ability to perform an intellectual task. Therefore studying the 'entropy' of these weights and what and how they change and the effects of these changes is to study the networks 'intelligence' directly?

Another case in point. Genetic algorithms can search a solution landscape and then select the 'best' solution as a seed to the next iteration. This 'best current solution' will have an entropy or measure of order or disorder. So, in these terms, the system is measuring the level of chaos in the system according to some rules and selecting the solution that produces the least chaos (most entropy)

Is this striking any cords with anyone?"

Comments

Re:Information "entropy" is not entropy.

[grammar nazi]

I wouldn't be the grammar nazi if I didn't take point out that: "Is this striking any cords with anyone?" (line from original story) is incorrect. The proper homonym is chord, not cord; "to strike a chord"

The first paragraph makes the following hypothesis: "...the study of HOW entropy changes in any given system is the study of intelligence itself...?" That seems to be the main question in this ask slashdot.

In the second paragraph, he shows us how little he knows about neural networks (or grammar). It is not the storage of the weights in a neural network that determines the networks "intellectual" ability, but rather the value of the weights (I would argue it's the training method, network structure, rather than the actual weights). Furthermore, this sentence about the storage of the weights does not lead to the next sentence: "Therefore, studying the "entropy" of these weights... is to study intelligence". One point does not imply the other point. I'm not sure what he/she is trying to say here. As a side point, there is little entropy to the neural network nodes anyways. Similar networks, performing similar but different functions, have similar node weights (I know this from working developing an OCR system for 2 years at Lockheed Martin).

The author's third paragraph is to study the entropic nature of Genetic Algorithms. However, the entropy derived in each generation of a genetic algorithm is directly derived from a random number generator. All GA entropy is derived from randomness in the selection. To study each generation is equivalent to studying random number generation, nothing more.

Finally, I wouldn't be doing my Mathematics degree justice if I didn't point out that CHAOS IS NOT ENTROPY. Chaos is marked by having a complicated, seemingly random, system described by a simple structure or order (Period 3 implies Chaos, York, Lee, et al). Entropy is a random system having NO simple structure or order.

Does the original author still have a valid question? Probably. Little understanding of intelligence is to be gained from studying randomness and entropy in GAs and Neural Networks. Perhaps, the time would be better spent studying entropy XOR GAs and NNs. Or learning a little bit more about any of these things and then reposing the question.

Re:Information "entropy" is not entropy.

[Anonymous Coward]

I suggest that they read this book to get a generalized understanding of the topic. Then they should suppliment in a particular area to be come an expert in a sub set. I also suggest that one should avoid making giant leaps when working with things scientific... rather check and re-check yourself. Einstein had the famous "E=mc^2" five times in his notes and verified with friends who were skilled mathematicians several times before he actually published it.

Damn. slash is forcing me to post anon.