Slashdot Mirror
Gamers Outdo Computers At DNA Sequence Alignments
ananyo writes "In another victory for crowdsourcing, gamers playing Phylo have beaten a state-of-the-art program at aligning regions of 521 disease-associated genes form different species. The 'multiple sequence alignment problem' refers to the difficulty of aligning roughly similar sequences of DNA in genes common to many species. DNA sequences that are conserved across species may play an important role in the ultimate function of that particular gene. But with thousands of genomes likely to be sequenced in the next few years, sequence alignment will only become more difficult in future. Researchers now report that players of Phylo have produced roughly 350,000 solutions to various multiple sequence alignment problems, beating the accuracy of alignments from a program in roughly 70% of the sequences they manipulated."
I'm highly skeptical that these gamers are really using some un-automatable human-only deep skills, especially since they aren't exactly extensively trained in this game, not to the level of, say, good Go players. So the interesting question to me is not that they beat current algorithms, but whether data mining these hundreds of thousands of alignments can tell us something about how they're doing it. My guess is that there are some heuristics that can be mined from this data that would massively speed up search.
That's a more general point about how these stories are always pushed, though, sometimes by media, sometimes by the researchers themselves. Imo the most exciting thing about successful uses of "human computation" isn't that we can harness people to do things, but that we can gain some large data sets that will make it so we don't have to get people to do them anymore. Or at least, that should be the baseline, imo: that humans can beat some hand-crafted algorithm is one thing, but can they beat machine-learned algorithms trained on those humans' own gameplay logs?
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Makes the original premise of The Matrix that much better than the "lol we're batteries!"
... a beowulf cluster of these!
Cured Lupus! 150G / Platinum Trophy
insensitive clod overlords obligatory xkcd car analogy russian reversals whoosh pedant fanbois ftfy in 3...2...1..PROFIT
This is an interesting finding, but let's not get too carried away. If you read the article, you'll see that: a) The phylo-based alignments are partial solutions. They are simplified for the human user by leaving many orthologous sequences out of the alignment. This means there is another algorithm that finishes these partial solutions before they can be compared to solutions produced solely by algorithms. b) Only 36% of the _best_ phylo-based solutions, once completed, were better than the algorithms' solutions. This is still an improvement, but it DOES NOT suggest that humans are better than computers at multiple sequence alignment. If you were to ever try to solve a real MSA problem by hand, you would quickly understand how completely hopeless it is. In fact, even aligning 2 sequences of any appreciable length by hand is a chore. The problem here is the misguided title: "Gamers outdo computers at matching up disease genes" which should read: "Gamers + computer outdo computers only at matching up very small fragments of disease genes, some of the time"
I haven't played the "game", but I suspect that there are a lot of things like time limits that can serve as a motivation factor that actually increase user output in the aggregate. Having a time limit can give you a sense of urgency that will force you to work faster. The error rate may increase, but overall productivity could still be higher given that higher number of "answers" given per unit time.
Imagine two Magic The Gathering players. One assembles decks painstakingly, spending hours crafting card ratios just right, and researching combos to get the perfect balance of # cards to power of combo. Then he play tests it, goes back and makes adjustments, etc. The other throws decks together quickly and play tests them very quickly. He adjusts the deck without as much deliberate thought, but rather more quickly (perhaps intuitively). He is able to iterate much faster, and it's easy to imagine that if each player were given 1 month to pursue these strategies, the latter could easily come out with more decks that met some minimum standard of success (that was suitably high).
(Obviously, it's easy to see how inane, useless rewards can spur gamers to expend more time and "contribute" more to the game... just look at badges, trophies, etc. But I think it's just as possible that "negative" reinforcement ideas, such as a time limit, can have the same effect.)
Ceci n'est pas un sig.