Slashdot Mirror
Evolution and the 'Wisdom of Crowds'
An anonymous reader writes "An essay by a developer of recommendation systems makes a case for why so many people have trouble grasping Darwin's theory of evolution. Downplaying its conflict with religion, the essay suggests that evolution is in a specific class of "equilibrium seeking" concepts that tend to be extremely counterintuitive to most people. The hypothesis is supported by the observation that so many people reject the notion that evolution-like systems such as Wikipedia, prediction markets, and recommendation systems can actually be effective. Particularly fascinating is the description of his surprisingly simple algorithm for competing in the Netflix prize contest."
>why so many *Americans* have trouble grasping Darwin's theory of evolution
There, fixed it for you.
I want a list of atrocities done in your name - Recoil
The hypothesis is supported by the observation that so many people reject the notion that evolution-like systems such as Wikipedia, prediction markets, and recommendation systems can actually be effective.
While there may be many that reject that these systems can be effective at all, I'd suggest that there's many more that would actual argue that while these systems do work, they aren't necessarily the best or only method that is effective.
The "Wisdom Of Crowds" put George W Bush in power, twice. Had Americans believing Saddam caused 9/11 and was a threat. Then of course there is religion..
Libertarian Leaning Political Discussion Forum.
Is a great theoretical concept, but unfortunately it only makes sense in the context of assuming that everybody really thinks for themselves. As soon as the media enter into the equation the crowd becomes as manipulatable as the most stupid upper limit that can still be sold a bill of goods. If that's > 50% then the equation no longer holds, no matter how much the rest invests in staying educated. You'd almost have to filter out media bias somehow because otherwise anybody with an agenda and some money to burn will come out on top. Witness politics, marketing of unnecssary goods and services and so on.
MP3 Search Engine
Crowds contain individuals, and some of these individuals know what they are on about. Collect together a sufficiently large crowd and you will find a number of experts on many different subject.
Isn't that the obvious conclusion?
threadeds blog
> Comparing it to evolution, an edit of Wikipedia might be considered equivalent to a genetic mutation. A
> mutation, of course, is non-directed...that is, "random." It could be bad or good, but most of the time
> it is bad.
IMNSHO this is simply untrue. If this is true Wikipedia is dead for long: it never keeps a large, visible "pool" of "genes" (different version of the same article) that the "nature" (viewing public) can "select", and the "nature" simply is too busy to "select" them anyway. They have many version of the same article, but there are not many who will go into the version and select to revert to one of those. To me, the success of Wikipedia is that those who don't know much about a subject will normally refrain from editing the subject, so most edits are actually of a rather high quality. It is a social behavior, not an evolution behavior.
Here's to hoping the language in your post evolves into a coherent one.
As the parent mentioned, the "Wisdom of Crowds" put Bush in power. Honestly, it seems to me to be nothing more than overhyped bullshit pushed alongside "Web 2.0" and other over-hyped concepts that are filling the current bubble with hot air. People love to cite Wikipedia as proof of the wisdom of crowds, but let's stop and analyze that for a moment:
Who controls the content of Wikipedia articles? Is it a large crowd of seemingly random contributors each imparting their own bits of wisdom? Or is it a small set of contributors providing the base of an article with a few mostly minor revisions submitted by random people passing by? In my experience, it's the latter. Usually a small set of people, no more than 3 to 5 which make the core of a Wikipedia article.
These same people are also generally the ones that cultivate the article and keep it consistent and well editted. Occasionally these same few people come to disagreements and end up in "edit wars" in which they call in another set of few members interested in judging to judge the issue. There's no "crowd" at work here, it's a lot of small groups of vested individuals who have interest in a particular domain and an efficient way of contributing and collaborating in that domain.
There may be hundreds of such groups, but they typically stick to their domain or they become edit whores and stick to minor revisionary work on a large amount of articles. Either way, I don't see much of a "crowd" once I break it down and look close, much less a wise crowd. Have you ever noticed that different subsections of Wikipedia have their own "feel" or "identity"? Maybe the particular manner of phrasing or the type of consistency shown throughout that sub-section which differs somewhat from another unrelated domain. This is largely a result of edits by the aforementioned small group of vested individuals. Each group leaves their own tint which colors a section and gives it life.
Wisdom of Crowds? No. Small, intelligent groups of people focused on achieving a well defined goal? Yes. If you really want to test this "Wisdom of Crowds" concept, take a look at SomethingAwful.com or any of the various large web forums and learn of the "Wisdom of Crowds". Even there, it's generally a very few amount of people contributing intelligently with the rest just being meaningless drivel. This meme needs to die.
Those who doubt the veracity of 'evolution-like systems' such as "Wikipedia, prediction markets, and recommendation systems" should not be compared to those who argue against evolution. The Theory of Evolution has a great deal of scientific evidence supporting it; indeed, much of the 'theory' is actually considered scientific Fact.
While I support Wikipedia, I don't consider those who doubt its value to be idiots. Those who argue against evolution, on the other hand....
- Demosthenes
cynicsreport.com
1. People -- as a general rule -- process complex ideas granularly. People are also generally lazy thinkers who do not attempt to refine their comprehension with falsifiable methodology. As a result, individual perceptions of value are often biased towards the simplest conclusions at the greatest level of granularity that a person can casually grasp rather than on evidence that intellect and practiced reasoning might produce. In large groups, it is possible to predict behaviors through statistical sampling using assumptions based on this model of granularity and intelligence. The conclusions of such studies are, themselves, subject to individual evaluation under the same model of granular perception. People who don't understand this are stupid religious types. If those same people were smart then they would be compelled to believe in evolution.
:)
2. Most people can't see the forest for the trees. Everybody who is not as smart as the author needs to take remedial education and secular-deprogramming classes.
Now you don't have to read the article.
You're welcome.
Societies may have "invented" the notion of religion because religion led to ethics, which led to less killing of their neighbors. All of the sudden, it's survival of the fittest, as non-ethical tribes tended to be killed off, while religious tribes thrived.
An obvious second example is the notion of being against birth control (or for large families). Tribes that were for large families and passed those beliefs down to their children tended to grow.
So my question is: Even if there is no God, and you are an atheist, is it possible that a world containing religious people is actually a "better" society than a world full of atheists? The Earth's people evolved into a world of mixed beliefs (some religious, some not), which could be argued to be the survival of the fittest idea or world. The mixed-belief world appears to be the "fittest" world, as opposed to such less-fit worlds of all atheists or all Christians, as examples.
If we evolved to be a mixed world of beliefs, as the "fittest", perhaps we should accept that, and quit trying to convert people with arguments for our favorite religious/non-religious belief.
Just a n-dimensional random distribution, with small adjustment steps. The 'n' of the system being chosed by hand, not even automatically computed. It works for Netflix because the domain being modeled is not 'wild' statistically, and have a very simple topology.
The 'presumed' relation with a 'wisdom of the crowds' concept is just coincidence, try to apply such a simple system to a really complex domain (ie: natural language syntax) and it will fail.
On the other hand, it's true that simple statistics can be used for a lot of tasks (ie: language/topic detection), but nothing really new here.
What's in a sig?
I recently had to start a Wiki for 1st year undergraduate students. I found it really hard to make it writable by everybody, since I was sure that it would result in a lot of vandalism. However, if you think about Wikipedia, the vast majority of pages can be edited by anyone and yet you almost never see malicious edits by people just dicking about. In the limit, people who visit Wikipedia prefer order. That's actually quite a comforting idea.
Obviously the more subtle stuff is harder to protect against.
"Conventional wisdom says that the primary reason why so many people do not accept Darwin's theory of evolution is that they find it threatening to their religious beliefs. There is no question that religion is a big part of the reason behind the large number of people who reject evolution. But I am convinced that just as often, the cause and effect is reversed: people hold onto their fundamentalist religious beliefs because evolution by natural selection -- the strongest argument against an Old Testament-type creator -- is so counter-intuitive to so many."
Honestly, I find these kinds of statements to be a bit off-base. I really get the feeling that Creationism and Evolution/Darwinism are artificially pitted against each other as if one or the other has to "win."
The interesting thing is that there is absolutely nothing in either of the standpoints that cannot coexist with the other. I would say that the consistant framing of them being exclusive is what causes resistance (from both sides, most likey) when it isn't even needed.
If one wants to get anyone to believe in a scientific theory they are having difficulties with, framing it as, "you should believe this because what you believe is wrong and you are stupid," is not really going to win anyone over. Especially when one could easily take the stance of, "here's why this theory makes sense, and really it doesn't have anything to do with what you may or may not believe."
I've seen no strong theology that would rule out that evolution did not happen. Creationism is about a supernatural force overseeing things--it says nothing specific about how things actually happened. (And, I think, most theologists will agree that Genesis is highly metaphorical.)
So, bottom line is, if science-minded people want others to "see the light" on this one, stick to the facts and leave the religion-bashing alone. Making people defensive generally is not an effective way of getting an idea across.
What's a sig?
The success of wikipedia has nothing whatsoever to do with evolution.
1. It is not like a 1000 monkeys typing randomly on a type writer came up with the wikipedia.
2. The content of the wikipedia is controlled more so than most people think. There are editors, there is peer review etc.
3. You don't find a million slightly varying copies on a single topic which are then "naturally selected"
A wikipedia has as much value as shouting out a question in a packed stadium to receive the answers from a million people. Most of those who will bother to answer are those who will know something about the subject and most who won't answer are most likely those who don't know enough about the topic to comment.
How is this in any sense similar to evolution?
There is a part of me hoping this article gets discussed by Dilbert creator/evolution denier Scott Adams, and another part dreading it. link
And I don't mean it as an offense, this being /. and all ;-)
According to TFA, the "Wisdom of Crowds" WRT Wikipedia is: "If I can't do better than this, I won't touch it". So, the very definition of the "Wisdom of Crowds" (and I agree with you that "Wisdom" is a very innapropriate term) is that "an active minority drives the inactive majority".
It's better to be the foot on the boot than the face on the pavement. ~~ tkx Kadin2048
I think the main problem with people's understanding of evolution is the fact that it is not taught very well in schools, and people get the strong idea that evolution is a random process. I also think it is a problem with the timescales involved, which are hard for the human mind to grasp.
What is happening today to the common man is that he/she is getting immune to technology, which leads us to the possibly false premise that the lay person understands technology any better than say, evolution. Given this assault of seemingly illogical and complex information (which completely undermines a person's ego, mind you), religion provides a very convenient framework to make life simple, seemingly secure, and less fragile. Religion is hence, more of a survival tool for a society that shields away a person's insecurities. For that matter, that is the reason why societies and families formed in the first place, which is to increase the probability of our survival and proliferation. For the common man, religion and society practically mean the same thing, and hence interchangeably attribute the positive aspects of one with the other. This is also why they are willing to put up with the restrictions and rules of religion, just as we do for society's laws and restrictions!
The only reason people do not accept Evolution is because they cannot believe that apes are our brothers. Partly because of mis-placed egotism, and mostly because of religion. After-all if we were made in "god's image" to "rule over all beasts", then it Evolution is an unacceptable concept.
_
\\/ are accustomed' - First Lensman
Comment removed based on user account deletion
Are you kidding me? Calling Darwin's theory a theory of random mutation show how little you understand of it, because you emphasize the wrong half of his theory. There's basically two parts about Darwin's theory: (1) mutation and (2) selection. Most people consider the first one as most important, but nothing could be further from the truth than that. It's SELECTION that is the keyword here. The mutation part is merely the "fuel" that feeds the selection "engine". In fact, the mutation doesn't even need to be random at all. Let me say that again:
... to have a Darwinian evolution at work. It doesn't have to reflect the biological method of evolution at all. At its core you only need (1) mutation (2) selection. Once you have that, you have Darwinian evolution. I believe it was Richard Dawkins who coined the term MEME to apply Darwin's theory to cultural evolution! Though it is has entirely different mechanism than biological evolution, it still consists of mutation and selection. Not all variants work equally well though: sexual mutation seem to work better than asexual mutations , cultural evolution is much faster than biological evolution, because the latter can only pass information between generations what is very slow. The evolution of mankind in the last few thousands of years are mostly cultural driven.
The mutation in Darwin's theory does NOT have to be random!
Although random mutation is perhaps the most effective way compared to its complexity. It surely is the most simple way for nature to "implement" it. And most of the time it results in very good "fuel". About your example: although the mutations are made by intelligent designers, some designs are rejected and some accepted (to be built further upon). The mutations are not random, but the selection is still in place. That's good enough.
So if you don't want to call Darwin's theory a theory of evolution, call it a THEORY OF NATURAL SELECTION instead.
Keep in mind: you do not need DNA, big gene pools, parallel mutations, sexual mutations,
Pat Robertson said what he said, and was condemned by many Christians. How many prominent Muslims have openly condemned the contract on Salman Rushdie's life?
Yes, the Bible contains a lot of violence, and yes, many Christians say they believe "every word," but in reality they aren't going to kill their kid for reading a book on Wicca. Well, there might be one nutjob out there who would, but every other Christian would consider the thought horrific. You don't see Christians in Colorado City stoning people to death for adultery or for breaking the Sabbath. How many are stoned to death in Muslim nations?
There are Christian nutjobs, and some have been caught with nerve gas, bombs, whatever, but the fringe of the fringe of the fringe of Christian zealotry is not comparable, size-wise, to the support the suicide bombers find in the Muslim world. I'm an atheist and I'll argue all day that faith undermines rational thinking, that Christianity doesn't make you moral, and that we should keep US society secular, but it is just vastly wrong to conflate the scale of violence perpetrated by the faithful of each of these communities qua their faith. Violent rhetoric isn't violence.
I'm not saying Christianity is violence-free. There IS violence (Matthew Sheppard getting stomped to death, etc) but it isn't as prevalent, sanctioned, or, well, normal as it is in Muslim nations. I'd say that religious killing in the Muslim world is probably as frequent, if not worse, as race lynchings were at the ugliest point in US history. No, I don't have the numbers to support that, but I think the scales are similar.
"Wisdom of Crowds" systems produce good results because there is a feedback loop, and elections don't have that feedback.
To sift the wisdom from the noise, there has to be some method of determining which are 'good' inputs and which are 'bad'. With Evolution, the feedback is easy to understand, bad mutations die/fail to breed/whatever, good ones get more food/sex/whatever and are more likely to reproduce.
An election has no such feedback. There is no good method of looking at the individual inputs from the results and pruning out the bad ones or promoting the good ones. Nobody gets to change their mind and alter their vote when they see the results, because for each election, the candidates change. The crowd isn't able to look at Bush v Gore and apply the results to Bush v Kerry, because Gore != Kerry. Only once in the last hundred years have the candidates been rematched (Eisenhower vs Stevenson in 1952 and 1956), so only one has meaningful feedback been applied... and even then it's not very good feedback as the first election was Eisenhower vs Stevenson and the second was President Eisenhower vs Stevenson, which is a different case even if you don't look at increased age, policy shifts, running mate changes, etc.
A pizza of radius z and thickness a has a volume of pi z z a
And in one way or another, we're all atheists. Is the world worse off because people don't believe in Thor anymore?
"Men never do evil so completely and cheerfully as when they do it from religious conviction." Historically, some of the worst atrocities have been carried out in the name of God
That's simply not true. This charge has been parroted by the anti-religious people, and it completely ignores the historical record.
Let's compare the post Roman world to the pre-Roman world. Prior to Christianity, the world believed in conquest without justification. IF someone had more stuff than you, you sent in an army and took it. Then you brought home a bunch of loot, and were rewarded for it. Look at all the Roman celebrations of conquest - called "triumphs." In the ancient world - if the people were not of your country, it was desirable to kill them and take all of their stuff.
Julius Caesar was no bible thumper, but under him, the Romans practiced a particularly vile form of ethnic cleansing in Gaul. Imagine the outcry today if someone wrote a book bragging that they killed over a million people. That is what Caesar did, and it made him MORE popular, not less. And then there all the lesser cultures that have been wiped our destroyed. Read about emperors of various ancient empires having all of the children killed, burning cities to the ground, and so forth. It was the advent of religion and the idea that people had souls which ultimately drove the idea that everyone had some sort of natural rights.
Similarly, Islam spread as quickly as it did in the middle east because of its promises of fairness and lower taxes to the people.
Both religions, carrying with it a divine proposition against killing, act as a natural brake against social forces that otherwise glorify it.
The last time we had an organized group of people that held the ancient view of empire, we called them the most evil people that had ever lived. The NAZIs didn't kill out of a belief in God - rather, they just felt that conquest and ethnic cleansing were part of the natural order of things, and they fused ancient roman values with modern ideas about evolution to back them up. Even today, extreme racists reject christianity (particularly in American prisons), precisely because of its moral condemnation against genocide and other racial killings.
This is my sig.
Comment removed based on user account deletion
I think the reason is simply that many people cannot intuitively comprehend the vastness of time, or at least they don't dedicate the time to really try. My own personal opinion is that for many people the concept for God is simply a placeholder for this line of thinking. Viewed from a typical human's perspective, the time that evolution has taken to produce life as we know it today, is functionally the same as pure infinity. To them infinity is associated with God. Talking to them about "change over time" doesn't make sense, because in their intuitive understanding of time, the time of kings and knights in Europe was extremely long ago. And everyone knows that animals and people were basically the same back then.
To stretch this line of thinking--I sometimes wonder if the United States is not hampered by its relative youth. The entire history of the white culture of this nation fits into about 500 years. Whereas for the rest of the world, 500 years is not that long ago in their history. In places, people are still fighting over things that happened more than 500 years ago.
Build a man a fire, he's warm for one night. Set him on fire, and he's warm for the rest of his life.
"That's pretty much what I tell people when they start preferring religion to science - they are the same thing!"
More precisely, they are both attempts by people to make sense of the world around them.
But they do not work the same way, nor were they meant to.
"Do you really believe God was too stupid to create a universe that didn't have to be held together with magic? Studying science is studying God's work, and thus you are in fact learning about God."
I read a rather eloquent way of saying the same as the above, though I forgot who I'm quoting: "Religion tells us about what God did. Science tells us about how He did it."
The problem with the above is that religion and science should and were intended to offer different answers to different questions. They fulfill different needs and expectations in the way that reason and faith do or the way philosophy and spirituality do. None of the aforementioned pairs are truly opposites any more than science and religion because they too are human endeavors to extend limits of ideas that aren't always so tangible or approachable to everyone.
Unfortunately, there are those on both the faith/religious side and on the reason/rational side who either can't or won't tolerate other world-views that well. These are the folks who have good intentions, but they end up applying answers found in their faith to questions better left to science (think "creation museums").
The other side of this paradigm is more common on Slashdot where we find really good "reasons" not to believe in God and "proof" to back it up.
His netflix is very similar to a standard methodology called Self Organizing Maps (SOM). It's cool to implement because it is automatic. You can use it to map 3 or more dimensions onto 2 dimensions and look at it. The output also changes depending on how you start it. Map the same data set in reverse or some other order and you end up with different clustering. You can use it to solve a problem as the author described. Or, you can use it to simplify complex input before you feed it into a further model, neural net, or whatever.
There is a book written in 1841 by Charles Mackay titled "Extraordinary Popular Delusions and the Madness of Crowds". It describes some "bubble" markets, such as the Dutch Tulip Craze (when people would invest their life savings in a tulip bulb, only to see the market eventually crash) and then goes on to describe other non-market crazes.
The book is frequently referenced in discussions of investment strategy, especially so-called "contrarian investing", which often makes money for its followers. The contrarian investing principle can be summarized as being that when the crowd overwhelmingly agrees on something, go the other way.
The book describes market behavior at least as well as Adam Smith's "unseen hand", and may also well describe other aspects of crowd behavior. I had never heard of the "wisdom of crowds" before this posting, but I have heard of the "madness of crowds" for many years.
" Well, no . Japan, Holland, Canada, and a slew of other nations have a lower instance of religious belief, and a lower rate of crime, lower infant mortality, etc."
...and higher suicide rates. ...Japan and Canada anyway.
"Even within the USA, the Bible Belt states (actually the Red States in general) have higher infant mortality rates, lower productivity rates, higher crime rates, worse education systems, along with being worse-off in a range of other criteria."
Congratulations to NYC, Chicago and Los Angeles! You are now "Red, Bible Belt states". ;-)
"It isn't a stark difference--I'm not saying they're in the dark ages--but the difference is easy to spot if you look at the data."
Just for a moment, imagine that we live in a world where numbers don't tell the whole story.
...and then snap out of it. We're already there.
"And in one way or another, we're all atheists."
...as is each and every tree, rock, and banana slug are all atheists too. One significant difference is that they have no doubts about their atheism. ...even when they near the end of
their respective existences.
People are supposed to be different because of what we believe and the small matter of being capable of believing at all.
...not because of what we do not believe.
"Is the world worse off because people don't believe in Thor anymore?"
Not that many people believed in Thor to begin with. It's a rather smaller number today, but they might even have a website.
Apollo on the other hand... There was a thunder god.
Evolution by natural selecton isn't the strongest argument against an Old Testament-type creator. The story of Creation as given by the Old Testament follows the form of Hebrew poetry. And the author of it clearly could not have been at the point of Creation. Those who say that the author was somehow inspired to write the exact sequence of events of the creation of the world by God such that they would be exact are... well, putting things into the Bible that aren't there. Trying to apply scientific logic to fails even in the absence of evolutionary theory, given that there is day and night as early as the first day, but no sun until later. Only the dim-witted would consider the Creation myth a literal retelling of the story of Creation.
The theory of Evolution, our growing understanding of our universe and how we apply it are, if anything, fulfillment of Genesis 1, verses 27-28: "(27) So God created man in his own image, in the image of God he created him; male and female he created them. (28) God blessed them and said to them, 'Be fruitful and increase in number; fill the earth and subdue it. Rule over the fish of the sea and the birds of the air and over every living creature that moves on the ground.'"
So it's hardly accurate to call Evolution a strong argument against an OT-type Creator. The text of Genesis 1 itself is the strongest argument against Creationism, but hardly any argument at all against the existence of a Creator as the Bible describes.
Not to put too fine a point on it, but Young Earth Creationists can claim that their interpretation is derived from neither a literal nor an educated interpretation of the Bible. Creationism is ultimately anti-Biblical. But it's taking that silliness to an extreme to then say that Evolution somehow is an argument against the existence of a God.
That's one crucial difference between science and religion: everything the Pope says is religiously significant, whereas a scientist's statements only matter to the degree to which they can be tested and supported. Even most scientists are unwilling to consider the possibility that *gasp* if evolution be true, not all races are created equal, and that some might be statistically inferior to others. Scientists are quite willing to believe all sorts of things, provided they can be objectively proven. They're just not willing to take someone's word for it.
Build a man a fire, he's warm for one night. Set him on fire, and he's warm for the rest of his life.
Except teapots don't pull people's arms out of their sockets when they're disbelieved.
Play Command HQ online