Slashdot Mirror
Studying Intelligence Thru Entropy?
"A case in point. Neural networks are weighted switches. They store their 'weights' in the neuron. The storage of these weights determines the networks ability to perform an intellectual task. Therefore studying the 'entropy' of these weights and what and how they change and the effects of these changes is to study the networks 'intelligence' directly?
Another case in point. Genetic algorithms can search a solution landscape and then select the 'best' solution as a seed to the next iteration. This 'best current solution' will have an entropy or measure of order or disorder. So, in these terms, the system is measuring the level of chaos in the system according to some rules and selecting the solution that produces the least chaos (most entropy)
Is this striking any cords with anyone?"
Negative changes in entropy are information, not necessarily intelligence. Intelligence has connotations that information does not. Therefore, while several tons of algae are able to produce more information (by converting CO2, H2O and energy into carbohydrates) than a similar mass of humans, I suspect that the humans possess more intelligence. I think in order to have an intelligent discussion on the subject, an effective definition of intelligence would be needed. Not being a cognitive psychologist, I'll leave that to the experts and go on growing tons of algae.
Only studying the predictable change in entropy would follow from your suppositions. ie.
HOW predicatble entropic changes in any given system is the study of intelligence itself...
-- The morphemes of your disquisition are ascertainable, but they have eschewed an ambit of transpicuous exposition.
Here's a cool site where you can take on the role of Maxwell's Demon and use your intellect to create entropy.
"I assumed blithely that there were no elves out there in the darkness"
Entropy is a measurement of a microcosmic physical property. The generalized idea of "disorder" that led to the idea of information entropy is related but seperate.
This is important because it is a pernicious error to conflate the two, an error which often results in false conclusions about thermodynamics and the macrocosmic world.
Well, my first thought is that just because something is changed in a manner that is able to be predicted, *does not* mean that you (or anyone else) will be able to predict it. This is very similar to the halting problem (see also Turing machines), in basic computing theory. How do you know if you can't predict the behavior (ie it's truly random) or just that you haven't found the correct functional description yet?
My second thought is that your first premise, as stated above, can be taken in (at least) two ways, a strong sense and a trivial sense. First, the trivial sense: you're simply labeling anything that can predictably change the entropy of a system as intelligent. simple, and actually setting yourself up for a nice, simple tautology of equivillences. The strong sense: Intelligence is *required* to change the entropy of a system in a predictable way. This then requires a definition of what you mean by intelligence and I somehow don't think that this strong sense is what you mean. So, it's the trival case you're interested in (that is to say that you've defined intelligence for us).
Is it true that the study of HOW entropy changes in any given system is the study of intelligence itself, in that given system?
Inasmuch as the "how" really gets at the "what" (or that they're intimately connected, see Aristotle's 4 causes, covered well at everything2).
The real issue though, is that you seem to be trying to accurately describe/define intelligence but do not do a good enough job accounting for the common usage of the word to be anything more than either putting forth a simple tautological statement or you are failing to accomplish your goal in an effective or substantial way...but that's just my simple opinion.
-inco
Life is an expression of the information carried in genetic material. Information exists independent of entropy. Because information and entropy are specificly not linked, life forms can grow more complex over time. In fact life forms can and therefore must grow more complex over time, because even after life is complex enough to have trancend direct competition with it's enviroment, There is still direct competition with others of the species.
Intelligence and entropy are most emphaticly NOT linked.
If voting were effective, it would be illegal by now.
> Given that any force that changes the entropy of any system in a predictable way is an 'intelligent' force.
The second law of thermodynamics is pretty predictable, but it has nothing to do with intelligence. Unless you consider randomly colliding molecules to be functionally intelligent.
No flame intended, but have you by any chance been listening to the proponents of "intelligent design theory", the latest reincarnation of creation 'science'?
> A case in point. Neural networks are weighted switches. They store their 'weights' in the neuron. The storage of these weights determines the networks ability to perform an intellectual task. Therefore studying the 'entropy' of these weights and what and how they change and the effects of these changes is to study the networks 'intelligence' directly?
You seem to be confusing the training of the network with its operations after it has been trained.
> Another case in point. Genetic algorithms can search a solution landscape and then select the 'best' solution as a seed to the next iteration. This 'best current solution' will have an entropy or measure of order or disorder. So, in these terms, the system is measuring the level of chaos in the system according to some rules and selecting the solution that produces the least chaos (most entropy)
Actually, depending what problem the GA is working working on and what exactly you measure for the entropy calculations, the entropy may either increase or decrease as it progresses. (I know this for a fact, because I've done it.)
> Is this striking any cords with anyone?
Yeah, the same kind Lister strikes when he plays his guitar on Red Dwarf.
There is certainly room for applications of entropy to the study of these things, but you don't seem to be off to a good start. For some basic applications of information theory to neural networks, see Haykin's textbook. There's surely lots more literature out there, if you care to track it down.
Sheesh, evil *and* a jerk. -- Jade
> systems [that] change
> entropy [are] intelligent systems?
Only for a very unabitious definition of
"intelligent". An internal combustion engine
would be one example. An air conditioner or
a food processor might be other examples.
They also take inputs, you will notice...
signals, even.
-I like my women like I like my tea: green-
of "Intelligence" seems to be our definition of Biomass.
If voting were effective, it would be illegal by now.
The difficulty is in this first step. It's not like you can go to the hardware store and buy an "entropymeter" like you can a voltmeter or thermometer. So how do you measure entropy? i.e. how do you look for patterns in data when you might not even understand the process that creates the data? We do have some tools (heck, I like bzipping the original and comparing compressed with uncompressed file sizes) but they do not find all relevant patterns by any means.
I'm not saying that entropy is a useless concept; I'm just saying that it's not an easy thing to measure in complex (AI-type) systems. When your tool fails to detect a pattern it will increase your measured entropy, but the pattern is still there. I am extremely unconfortable if changing my tool changes my measure of the entropy.
The questions and particularly the assumptions are gross over simplifications.
Try this. The power to your neural net fails. The result is a very predictable and massive change in entropy. But, there is absolutely positively no intelligence involved.
My answer is also a ridiculous over simplification but, that's the point.