The study is seriously flawed. There are a large number of possible things that could differ between a group of college-age liberals and college-age conservatives, and any of those things could be the real reason their ERP measure differed between the groups. That ERP measure is larger in people who are depressed or who have obsessive-compulsive disorder. It is smaller in people who are impulsive, aggressive or antisocial. It is also smaller in people who are drunk! The authors didn't attempt to see whether their groups differed in any of these other ways. If they had bothered to do a careful study in which they controlled for other possible influences on the ERP, they might have found no difference or even come to the opposite conclusion. But, it's a lot easier to make a big splash in a press hungry for big splashes than it is to do careful science.
Contrary to some of the comments here, a large body of research shows a link between exposure to media violence and aggressive behavior. There is a consensus among experts on this point (see this joint statement by the American Medical Association, American Psychological Association, and American Psychiatric Association, and American Academy of Pediatrics). Much of the research to date has concerned television and movies; not as much research has looked at video games. But there is growing evidence that the influence of video games is at least as strong and may be stronger. If you want to read some of the research for yourself, the Lion and Lamb advocacy group has a helpful page of links. (I am not affiliated with the group.) The research evidence is high-quality and includes longitudinal surveys showing long-lasting effects of early exposure to media violence on later aggressive behaviors. In showing a relationship between media violence and aggression, the studies rule out other factors that might be correlated with violent-media exposure, such as low socioeconomic status, poor parenting, low intellectual ability, etc. Of course, many of these things can also contribute to a person becoming violent, and no single risk factor is likely to make a person violent. Still, the evidence is strong that exposure to media violence is one risk factor. One study (Johnson and colleagues, 2001) noted that the effect of media violence on aggression is larger than the effects of calcium intake on bone mass and childhood lead exposure on IQ. As for video games per se, an analysis of the results of several published studies (Anderson and Bushman, 2001) showed that the size of the effect of violent video games on aggression is about the same size as the effect of condom use on the risk of HIV infection. (The Johnson and Anderson studies are linked at the LL site.)
My favorite (and one with enough critical praise to make me think he'll be read for a long time) is George Saunders, a writer who publishes frequently in the New Yorker and has two books of short stories, Civil War Land in Bad Decline and Pastoralia. Some of his work has a science fiction edge, being set in the future, but it really defies categorization. One recurring subject is the downtrodden and picked-on. His stories are unbelievably imaginative, and there is both wit and heart in his writing -- in 2 pages one can go from laughing out loud to crying. My favorites are The End of Firpo in the World (in Pastoralia) and the 400-lb. CEO (in Civil War Land). At least one of his stories is published online here.
I'm ecstatic about developments like this, and I think that a Slashdot model for academic publishing would (with some tweaks) be better than the current system. Slashdot-like publication technology presents an opportunity for a welcome change. A few points:
1) Even with all the freedom of the Machine Learning people's new model, the entrenched editorial board still places a biased bottleneck in the the propagation of information. The presence of an editor (or a publisher) is not equivalent to unbiased and effective peer review. In an edited journal, peer review often reinforces the 'old boys' network (I think Kuhn pointed this out in his book on Paradigms), and because journal space and reviewer's time are both limited, lots of good data never see the light of day (not only for this reason, but also for some of the reasons others have mentioned).
2) As for "Correctness", I would suggest that one thing added to the Slashdot model of academic publishing that would help make it work would be the posting of raw data. "Correctness" isn't the issue addressed in the peer review process. Nobody *really* knows whether results are "correct" (meaning that the experimentation was done properly) just by reading a paper, no matter how detailed (unless you work in a field that works on formal proofs rather than experiments). The review process weeds out the grossly incompetent stuff, and shapes up the other work to be more useful to the readership. "Correctness" is usually assumed: journal articles are so brief that the information necessary to establish correctness simply cannot be included. I say, put the whole damn data set on the internet, and anybody who wants to can dig around in the data to see whether what the author says matches up with the truth in the data. The only people who will dig around in the data are are those invested in the particular finding, but the end result is that for any given relevant finding, *somebody* who is skeptical will do a cross-check.
Recently a neuroscience journal (Journal of Cognitive Neuroscience) proposed a requirement authors submit their data to a massive neuroimaging database, see this essay. The debate was *very* heated, and JOCN eventually backed down (Those of you with journal access can see an editorial in the September 2000 issue of
Nature Neuroscience for a rehash of the events). There were many reasons for the controversy, but one of the big ones was that people didn't want others to have access to their data -- authors want to be able to use their data for publishing new results before somebody else does. Really, though, authors don't want to let other people rummage around in their data because much of the research process involves putting a spin on things, and people not sharing the same views will not put the same spin the data.
3) An additional point of resistance to change is that Universities rely heavily on the current system for tenure and promotion evaluations. The currency in these is evaluations how many articles you've published, and how many of these were cited by others (and how frequently). It will take some work to figure out how to do this with a new system, but it should be possible, and at any rate this is a lesser goal than the one of truth-seeking.
4) Finally, the issue of credibility. From another post: "A comment made by a random AC on slashdot, in contrast, is not worth my [professional] time. There are just too many posts by too few knowledgable [sic] people." This is a good point, but it underestimates the power of the Slashdottish process: it would take a conspiracy of crackpots and cowards to cause a large enough wave of review-posts to give bad research any prominence. Slashdot's "threshold" function, even within a few hours of the original post, seems to work pretty well at weeding out the stuff I don't want to read; something developed for the purpose of academic research would be even better.
5) A related comment on the quality of peer review: In a Slashdottish system, the attractions of signed (i.e., not-anonymous) reviews would increase, because criticism of an article does not prevent it from being published. I would suggest optional signed reviews. In a Slashdot-like system, "signing" your review would give it more weight, perhaps (through cross-linking to your own work and to your other reviews) by establishing that you the reviewer are highly "cross-linked" and thus worth paying attention to. Plus, a Slashdottish system would naturally put a lot of pressure on people not to flame (and presumably the anonymous idiot-flames would soon die out).
General comment: The whole point of scientific publishing is to facilitate the search for and dissemination of truth. The rest -- journals protecting their territory, editors protecting their power, authors protecting their spin on their own data, and Universities protecting an easy method of performance evaluation -- is an unnecessary side show that impedes the primary purpose of uncovering truth. Let's have the authors post their papers and their data, submit them to the scrutiny of everyone else in the field (without the intervention of editors) and see what stands the test of time. The flaws in the system are the flaws of natural selection and democracy, of course, but that's just the way reality and people work. The truth will survive the process and it will come out more quickly and accurately with a bottom-up researcher-driven strategy than with any strategy driven by corporate profits.
Slashdot Mirror
The study is seriously flawed. There are a large number of possible things that could differ between a group of college-age liberals and college-age conservatives, and any of those things could be the real reason their ERP measure differed between the groups. That ERP measure is larger in people who are depressed or who have obsessive-compulsive disorder. It is smaller in people who are impulsive, aggressive or antisocial. It is also smaller in people who are drunk! The authors didn't attempt to see whether their groups differed in any of these other ways. If they had bothered to do a careful study in which they controlled for other possible influences on the ERP, they might have found no difference or even come to the opposite conclusion. But, it's a lot easier to make a big splash in a press hungry for big splashes than it is to do careful science.
Contrary to some of the comments here, a large body of research shows a link between exposure to media violence and aggressive behavior. There is a consensus among experts on this point (see this joint statement by the American Medical Association, American Psychological Association, and American Psychiatric Association, and American Academy of Pediatrics). Much of the research to date has concerned television and movies; not as much research has looked at video games. But there is growing evidence that the influence of video games is at least as strong and may be stronger. If you want to read some of the research for yourself, the Lion and Lamb advocacy group has a helpful page of links. (I am not affiliated with the group.) The research evidence is high-quality and includes longitudinal surveys showing long-lasting effects of early exposure to media violence on later aggressive behaviors. In showing a relationship between media violence and aggression, the studies rule out other factors that might be correlated with violent-media exposure, such as low socioeconomic status, poor parenting, low intellectual ability, etc. Of course, many of these things can also contribute to a person becoming violent, and no single risk factor is likely to make a person violent. Still, the evidence is strong that exposure to media violence is one risk factor. One study (Johnson and colleagues, 2001) noted that the effect of media violence on aggression is larger than the effects of calcium intake on bone mass and childhood lead exposure on IQ. As for video games per se, an analysis of the results of several published studies (Anderson and Bushman, 2001) showed that the size of the effect of violent video games on aggression is about the same size as the effect of condom use on the risk of HIV infection. (The Johnson and Anderson studies are linked at the LL site.)
My favorite (and one with enough critical praise to make me think he'll be read for a long time) is George Saunders, a writer who publishes frequently in the New Yorker and has two books of short stories, Civil War Land in Bad Decline and Pastoralia. Some of his work has a science fiction edge, being set in the future, but it really defies categorization. One recurring subject is the downtrodden and picked-on. His stories are unbelievably imaginative, and there is both wit and heart in his writing -- in 2 pages one can go from laughing out loud to crying. My favorites are The End of Firpo in the World (in Pastoralia) and the 400-lb. CEO (in Civil War Land). At least one of his stories is published online here.
1) Even with all the freedom of the Machine Learning people's new model, the entrenched editorial board still places a biased bottleneck in the the propagation of information. The presence of an editor (or a publisher) is not equivalent to unbiased and effective peer review. In an edited journal, peer review often reinforces the 'old boys' network (I think Kuhn pointed this out in his book on Paradigms), and because journal space and reviewer's time are both limited, lots of good data never see the light of day (not only for this reason, but also for some of the reasons others have mentioned).
2) As for "Correctness", I would suggest that one thing added to the Slashdot model of academic publishing that would help make it work would be the posting of raw data. "Correctness" isn't the issue addressed in the peer review process. Nobody *really* knows whether results are "correct" (meaning that the experimentation was done properly) just by reading a paper, no matter how detailed (unless you work in a field that works on formal proofs rather than experiments). The review process weeds out the grossly incompetent stuff, and shapes up the other work to be more useful to the readership. "Correctness" is usually assumed: journal articles are so brief that the information necessary to establish correctness simply cannot be included. I say, put the whole damn data set on the internet, and anybody who wants to can dig around in the data to see whether what the author says matches up with the truth in the data. The only people who will dig around in the data are are those invested in the particular finding, but the end result is that for any given relevant finding, *somebody* who is skeptical will do a cross-check.
Recently a neuroscience journal (Journal of Cognitive Neuroscience) proposed a requirement authors submit their data to a massive neuroimaging database, see this essay. The debate was *very* heated, and JOCN eventually backed down (Those of you with journal access can see an editorial in the September 2000 issue of Nature Neuroscience for a rehash of the events). There were many reasons for the controversy, but one of the big ones was that people didn't want others to have access to their data -- authors want to be able to use their data for publishing new results before somebody else does. Really, though, authors don't want to let other people rummage around in their data because much of the research process involves putting a spin on things, and people not sharing the same views will not put the same spin the data.
3) An additional point of resistance to change is that Universities rely heavily on the current system for tenure and promotion evaluations. The currency in these is evaluations how many articles you've published, and how many of these were cited by others (and how frequently). It will take some work to figure out how to do this with a new system, but it should be possible, and at any rate this is a lesser goal than the one of truth-seeking.
4) Finally, the issue of credibility. From another post: "A comment made by a random AC on slashdot, in contrast, is not worth my [professional] time. There are just too many posts by too few knowledgable [sic] people." This is a good point, but it underestimates the power of the Slashdottish process: it would take a conspiracy of crackpots and cowards to cause a large enough wave of review-posts to give bad research any prominence. Slashdot's "threshold" function, even within a few hours of the original post, seems to work pretty well at weeding out the stuff I don't want to read; something developed for the purpose of academic research would be even better.
5) A related comment on the quality of peer review: In a Slashdottish system, the attractions of signed (i.e., not-anonymous) reviews would increase, because criticism of an article does not prevent it from being published. I would suggest optional signed reviews. In a Slashdot-like system, "signing" your review would give it more weight, perhaps (through cross-linking to your own work and to your other reviews) by establishing that you the reviewer are highly "cross-linked" and thus worth paying attention to. Plus, a Slashdottish system would naturally put a lot of pressure on people not to flame (and presumably the anonymous idiot-flames would soon die out).
General comment: The whole point of scientific publishing is to facilitate the search for and dissemination of truth. The rest -- journals protecting their territory, editors protecting their power, authors protecting their spin on their own data, and Universities protecting an easy method of performance evaluation -- is an unnecessary side show that impedes the primary purpose of uncovering truth. Let's have the authors post their papers and their data, submit them to the scrutiny of everyone else in the field (without the intervention of editors) and see what stands the test of time. The flaws in the system are the flaws of natural selection and democracy, of course, but that's just the way reality and people work. The truth will survive the process and it will come out more quickly and accurately with a bottom-up researcher-driven strategy than with any strategy driven by corporate profits.