Slashdot Mirror
Gamers Outdo Computers At DNA Sequence Alignments
ananyo writes "In another victory for crowdsourcing, gamers playing Phylo have beaten a state-of-the-art program at aligning regions of 521 disease-associated genes form different species. The 'multiple sequence alignment problem' refers to the difficulty of aligning roughly similar sequences of DNA in genes common to many species. DNA sequences that are conserved across species may play an important role in the ultimate function of that particular gene. But with thousands of genomes likely to be sequenced in the next few years, sequence alignment will only become more difficult in future. Researchers now report that players of Phylo have produced roughly 350,000 solutions to various multiple sequence alignment problems, beating the accuracy of alignments from a program in roughly 70% of the sequences they manipulated."
I'm highly skeptical that these gamers are really using some un-automatable human-only deep skills, especially since they aren't exactly extensively trained in this game, not to the level of, say, good Go players. So the interesting question to me is not that they beat current algorithms, but whether data mining these hundreds of thousands of alignments can tell us something about how they're doing it. My guess is that there are some heuristics that can be mined from this data that would massively speed up search.
That's a more general point about how these stories are always pushed, though, sometimes by media, sometimes by the researchers themselves. Imo the most exciting thing about successful uses of "human computation" isn't that we can harness people to do things, but that we can gain some large data sets that will make it so we don't have to get people to do them anymore. Or at least, that should be the baseline, imo: that humans can beat some hand-crafted algorithm is one thing, but can they beat machine-learned algorithms trained on those humans' own gameplay logs?
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Makes the original premise of The Matrix that much better than the "lol we're batteries!"
We processor now!
So can we extract any insights from this, and use them to improve diff?
Those who fail to understand communication protocols, are doomed to repeat them over port 80.
... a beowulf cluster of these!
A fantastic example of why the building blocks of human life should not be patentable and hidden away by pharmaceutical companies.
I just started playing and I am haveing a slight trouble with it. These people must be geniuses!
I wrote this. Please don't hate me.
Cured Lupus! 150G / Platinum Trophy
insensitive clod overlords obligatory xkcd car analogy russian reversals whoosh pedant fanbois ftfy in 3...2...1..PROFIT
This is an interesting finding, but let's not get too carried away. If you read the article, you'll see that: a) The phylo-based alignments are partial solutions. They are simplified for the human user by leaving many orthologous sequences out of the alignment. This means there is another algorithm that finishes these partial solutions before they can be compared to solutions produced solely by algorithms. b) Only 36% of the _best_ phylo-based solutions, once completed, were better than the algorithms' solutions. This is still an improvement, but it DOES NOT suggest that humans are better than computers at multiple sequence alignment. If you were to ever try to solve a real MSA problem by hand, you would quickly understand how completely hopeless it is. In fact, even aligning 2 sequences of any appreciable length by hand is a chore. The problem here is the misguided title: "Gamers outdo computers at matching up disease genes" which should read: "Gamers + computer outdo computers only at matching up very small fragments of disease genes, some of the time"
Link to the English version that actually works:
http://phylo.cs.mcgill.ca/eng/
Not only that, they also seem to be illiterate. Having to watch a video tutorial, narrated by a girl who couldn't read a book for kids lest her live be saved, just to learn how the damn thing is scored? I thought they know how to write, being in the academia and all? Their results are obviously secret, because if you just happened to educate yourself on a puzzle that you can't solve, the par results are inaccessible. Big stinkin' sikret, I tell ya. But no, they must have included the stupid car game countdown, and time-wastin' transitions. And they think that just because it requires a mouse hover and has a fade-in and fade-out, it must be cool. I've been having a lot of genuine fun with the zooniverse project; phylo in comparison seems done by braindamaged web designers who never bothered using their own fine creation, and decided that once all the transitions, music and sfx are done, they'll proclaim it done. I have never been so genuinely disappointed by a project of that kind.
A successful API design takes a mixture of software design and pedagogy.
I haven't played the "game", but I suspect that there are a lot of things like time limits that can serve as a motivation factor that actually increase user output in the aggregate. Having a time limit can give you a sense of urgency that will force you to work faster. The error rate may increase, but overall productivity could still be higher given that higher number of "answers" given per unit time.
Imagine two Magic The Gathering players. One assembles decks painstakingly, spending hours crafting card ratios just right, and researching combos to get the perfect balance of # cards to power of combo. Then he play tests it, goes back and makes adjustments, etc. The other throws decks together quickly and play tests them very quickly. He adjusts the deck without as much deliberate thought, but rather more quickly (perhaps intuitively). He is able to iterate much faster, and it's easy to imagine that if each player were given 1 month to pursue these strategies, the latter could easily come out with more decks that met some minimum standard of success (that was suitably high).
(Obviously, it's easy to see how inane, useless rewards can spur gamers to expend more time and "contribute" more to the game... just look at badges, trophies, etc. But I think it's just as possible that "negative" reinforcement ideas, such as a time limit, can have the same effect.)
Ceci n'est pas un sig.
I can tell you that there is a a time limits and they work really hard to add useless rewards that will hook up the users :).
I couldn't even solve one puzzle, so gave up.
My hypothesis is that humans may have learnt how to find a path to parsimony. We have evolved to use resources efficiently, so finding stepwise approaches that use resources most parsimoniously would have been important. MSA seems like mostly a parsimony problem - what arrangement of bases most parsimoniously explains the likely evolutionary relationships. Typical computational approaches to this involve MCMC and various more or less random moves to try to find the most parsimonious solution. Humans are clearly using many less moves than computers to solve this, so are much better than computers at seeing where the best hill is, and climbing that hill directly, rather than randomly exploring the likelihood landscape. We should find a way of classifying the moves people are making to discover whether they can see the big picture, or whether they are just very efficient at exploring the landscape.
Korma: Good
Humans: Millions of years
Computers: Tens of years.
Not sure there is a story, here...