Slashdot Mirror
Cutting Out the Middle Men in Scientific Publishing
Black Parrot writes: "Just got a message that was sent to several mailing lists used by machine learning researchers, announcing the mass resignation of the Editorial Board of one prominent ML journal (i.e., the scholars who make a peer reviewed journal work). The reason? 'Times have changed. ... We see little benefit accruing to our community from a mechanism that ensures revenue for a third party by restricting the communication channel between authors and readers.' It's the music industry vs. artists and consumers, writ small. You can see the full text of the message at the UAI archive. This sort of thing has been bubbling for a couple of years. The letter mentions other cases, and I know that several thousand biological researchers have threatened to go on strike against any journal that does not make their articles downloadable for free after a fixed delay from the date of publication. The trend toward more toll booths is not the only force at work in the Internet Age!"
...but who pays for the hosting and the bandwith?
Information may want to be free, but a fat pipe will cost you.
> Good, but the editor at first was the one separating good studies from stupid ones...
:-)
Well, no, not really. The whole idea in a peer-reviewed scientific journal is that the people performing the editing are experts in the field (ie peers of the person/people submitting the article for publication), not employees of the company which publishes the journal.
There's no suggestion of doing away with peer review.
> Will Scientists be able to apply this instead of the usual bickering ?
No, they'll still bicker...
-Andy
http://www.gimbo.org.uk/
The slashdot model is the best one I've ever seen. Think about it. We have peer review. We are prolific (several posts a day) and everybody can have access, even Anonymous Cowards like myself.
However, the way research has evolved in the western civilisation allows for the creation of "star" researchers, the ones that are prolific in publishing in certain journals. Their success is not mesured directly by their scientific contributions, but rather by their exposure to the scientific community through peer-reviewed periodicals.
I hope they create an online journal just like this one and post their research material more often, for the whole world to see and contribute.
Imagine if you could publish your initial findings on some protein's effect on cancerous cells. Since they are just initial findings, you don't need to write a book, just a brief article. Imagine how much the broad peer review would benefit you and help you steer your research where it would have more chances to succeed.
- Published papers are available virtually forever. Go down to the Bodleian library and book out articles published by Michael Faraday or Robert Hook. Whatever they do here has got have the same sort of permanence.
- Remember all those tapes that NASA has that they don't have the drives for anymore, or don't know the format? Paper doesn't have that sort of problem. Again, they need to ensure that whatever they produce can be moved on to whatever the current technology is
Other than that I think it is an excellent idea. I hope it scares the shit out of whatever the journal publishing equivalent of the RIAA is.as far as i can tell, the editorial review will still be in place. the same bickering and backroom politicing of who knows and likes whom will go on as usual.
one problem this actually would address is the very long delay between acceptance of a paper for publication and the actual printing of the paper. for an ieee journal this can be one to three years -- nearly an eternity.
also, since the people writing the paper are doing so for fame (fortune like tenure is a side effect) the results are likely to be better distributed. no middleman enforcing an artificial scarcity is needed much less desired.
This is good. From what I skimmed of the message, they're still advocating a review process--they're only against publishing systems that take all of the rights away from the original author(s) of the paper, and/or those systems that charge extremely high fees to allow others to view the papers.
It's important to move away from systems like that 'cause they're bad (duh). However, it's also important that there still be some form of peer review; without it, it will be impossible to seperate the actual research from the line noise.
Furry cows moo and decompress.
Why limit the peer review process to the opinions of a select group of people?
/. -- very often comments are moderated up as "insightful" or "informative" even though the comments are actually *completely wrong*.
Because there's a lot of idiots in the world.
The purpose of peer review isn't so much to decide if results are interesting enough to be worth publishing; the purpose of peer review is mainly to decide if results are *correct* enough to be worth publishing. For an average paper in a major scientific journal, there might only be a few dozen people in the world who are qualified to make that judgement. Fortunately, the journal editors know who does what research, and thus the journal editors can send papers to the right reviewers.
If you open up peer review to the masses, you'll have people "moderating up" papers because "they look interesting", even if the papers are complete BS -- because most people can't tell the difference between a scientific paper and BS. We see the same thing on
At an absolute minimum, peer review should be restricted to people who have published papers in related fields; ideally, it should be restricted to people who have published several major papers in related fields... which is exactly the status quo of peer-reviewed journals.
Tarsnap: Online backups for the truly paranoid
The new open journal, JMLR that replaced the old one is a great solution to the problem. However, I'm dismayed to find that they publish using proprietary formats. Namely PDF and Postscript. Wouldn't it, thus, cost money to save to those formats? I think they should use open standards *only*. Why not use LaTex or just plain old HTML 4? This would better allow scientists from developing countries to publish their work rather than wasting precious money buying licenses of Adobe Acrobat. It seems they are fighting closed proprietary standards in the first place and should not be supporting them.
Furthermore, I could actually read the articles much easier than saving to a file and launching a bloated Adobe application or, gasp, use that annoying PDF plug-in that usually crashes my browser.
Uh, I think folks are missing some fundamental points. The Scientific Method completely negates any need for Peer Review. That's not to say Peer Review is a bad thing... Ideally published papers that are peer reviewed should require that the reviewer replicate the experiment. I can't count the number of times I've seen graduate students and professors fudge (or completely fabricate) data in order to get past a peer reviewer to finish their PhD work sooner. Remember, peer reviewers have a body of papers and an ego that likes to be stroked just as much as anyone else who might find it expediate to kiss that ass of those who control budgets/carrers/etc.
Hacking academia is not entirely unknown to happen, even *with* peer review...
I suppose it depends on the field, but if you're churning out 50+ papers in the time it takes to do your PhD, you've either hit the motherlode of all research topics, you're Erdos, or more likely:
What I don't understand about all of this is why are there so few people who can see 'the big picture', including our law makers, businessmen and leaders.
.com boom and bust as demonstrating that the internet isn't as important as was generally thought, because they assumed that what would happen is just a variation on what went before. They don't understand that the internet is causing fundamental changes to the way the world works.
Many people stupidly saw the
The world now has a new a new set of rules which over the coming decades are going to completely change the business landscape. Record companies - which currently make money by duplicating, distributing and promoting physical manifestations of music - are going to die, because duplicating, distributing and promoting music can be done at virtually zero cost by anyone now.
Software companies that create and charge high prices for infrastructure and very widely used software are going to die out because their software is so widely used that it will make more sense for their bigger customers (companies and governments) to contribute programming time to open projects than to pay for products from a single company.
General travel agents are going to die out because they are middle men that will serve no useful purpose in the future. Ditto with publishers of scientific journals.
Unfortunately some of our law makers seem to think this natural progression is unjust and are creating laws to restrict or outlaw the technology. There is historic president for this type of response - the Luddites, who smashed machinery during the industrial revolution because it threatened their professions. Hundreds of business types and professions died as a result of the industrial revolution. The same will happen over the coming decades of the Internet revolution.
Unfortunately our leaders and lawmakers, under the influence of the threatened professions, are acting like Luddites in a very literal sense.
ArXiv is great, but the problem is the non-refereed aspect of it. We need properly refereed free (or cheap :) journals... and there is really no reason why not. People don't get paid to referee papers, or submit papers... there is just this man in the middle eating all this money for no net gain to the community.
A journal that had all its articles freely available electronically could easily charge for paper copies as long as it was reputable. All we need is for some big names to get their act together. An example where they *have* got their act together is from my University:
Geometry & Topology
This is a peer reviewed journal on (guess what) geometry and topology. All the articles published in the journal are freely available electronically, or you can order the paper version at a low price.
I don't see any reason for any extra distribution fee.
-- Help Digitise the Public Domain at DP.
There should be some kind of registration process so some 12 year old kiddy can't submit a journel on UFO study and get all his friends to rate it up. The registration won't stop that, but most kiddies won't bother going through a registration to screw with a website.
But a bunch of creationist adults will devote 5 hours a day every day to doing just that, but with papers sufficiently well written that they seem scientific to a casual reader. And all the sudden, you have a bunch of "peer reviewed" highly rated anti evolution creationist BS* sitting on a respected journals lap. NO WAY!
And don't even tell me that the negitive ratings from all the good scientists will ballance it out. Even if they suspect its BS, a decent scientist will not moderate something down until she has taken the time to look it over and check the methodology to make sure she isn't rejecting it out of hand because she has tons of experience that the end result has to be wrong. As for "peer reviewers are biased against some conclusions", sure you get biased after the 20th time the same conclusion turns out to be supported by sloppy work, wishful thinking and all out lies, but you still check the methodology to see which one it was this time.
But the point is that honest work takes longer than lies. Debunking lies takes more time and effort than presenting them. Moderating down conclusions that contridict your holy book takes less time than the propperly designed research it took to come to them. And the people who care the most about spreading lies are often devoting most of their lives to it, while the people most motivated and qualified to correct those lies are doing other possitive research and don't have the luxury of playing wack-a-mole with the latest psuedoscientific voodoo all day.
Nutshell: TRUTH IS HARDER. In an open marketplace of ideas managed by libertarian principles and voted on democratically, the truth will get its ass kicked. I'm sorry that we don't live in that perfect world where "the solution to bad information is good information, not supression" or "the truth will out" or any of those other nice thoughts with no basis in reality. Really sorry.
*and just to not pick only on the biggest target, lets not forget perpetual motion, psychic healing, ESP, alien visitation, racial infer/super-iority, gender work from both sides of the fence, conspiracy theories, power lines cause cancer, soil theory, homeopathy, dowsing, ok just put "Flim Flam" table of contents here....
Kahuna Burger
...will work for Chick tracts...
> It's the music industry vs. artists and consumers, writ small.
If I don't want to buy new CDs (especially copy protected ones) I can still listen to live music, music recorded by my friends, second hand records, the radio, CDs I already own, etc..
Doing science that way doesn't work.
rant
In my university they judge a person's performance by the number of papers they produce. As such some professors would frequent just about any publication at all to get the numbers. Given this context, the quality of a person's research these days co-relates poorly with the number of papers they publish. Hence I have to applaud the responsible attitude of your senior staff.
In general, if a paper do not contibute to new understanding, better let it be unpublished than to burden the world with more garbage. Reviewers' and readers' times are not less precious than yours!
There are huge problems in the current academic research system and pay-per-view online access is just one of them. Here are a few of the problems:
:-( ] which is a lot of fun. Guess what: conference publications are not usually valued much in the academic rat race in comparison to journal publications.
.ps formats) and place them online. I don't see how this is different from Napster (since we usually sign away our copyright to some publishing company).
1. Presentation of current research work in a conference is now the preferred way of rapidly communicating ideas. You get immediate feedback and get to stand up and give a talk in front of your peers [usually in Powerpoint
2. A conference presentation is usually written up as a paper and placed in a collection. Later, when the author writes a journal article, there's naturally a lot of overlap between the conference publication and the journal publication. Technically, this is a copyright violation.
3. Most of us in the academic community freely share our publications (in
4. Technically, for every figure, table and other content which is repeated in different papers, one needs to get and acknowledge permission from the authors and publishing company before the content can be reproduced. How archaic is this?
Anand
Anand Rangarajan anand@cise.ufl.edu
Peer review is strong, valid method of rigourously insuring that research gets aired by the academic community as a whole, ideas a thrown up, explored, demonstrated to be flawed and rejected, or duplicated by others and begin to solidify into the main accepeted knowledge of the field. Parasitically upon this are the journals. You pay to submit your work and you pay to read it, though they do provide a vital initial step in the peer review process, ultimately its only the view, the accepted wisdom? of a couple of experts in that field and a couple of glossy covers with a brand name on the front. Then even more significantly, others have to pay large sums indeed to read about your work, subscribing to the journal in question, or forking out for a individual copy from the journal [£5 a shot in the UK]. Moreover by publishing you've signed over all rights to the journal in question.
An illustration...
Step 1) Come up with nifty idea
Step 2) Decide whether you want to do it yourself, or hive it off to a phd student [see 2i].
Step 2i) Apply to department for a phd student, hoping that your prior publishing record is strong enough to secure the funding for such from the department, dependent on the number of papers and the ammount of research funding you bring in, department infighting and so forth.
Step 3) Write a Proposal to the relevant Research Council/Charity/etc. There a commitee of experts in that field decide on your worth, the idea's worth, whether it conflicts with how they want to help push forward their particular field, whether they got out of bed from the wrongside, whether you've pissed them off in the past.
Step 3i) Get the funding! Yay! Now try to keep your department from getting their fingers on it.
Step 3ii) Fail to secure funding, back to 1 and take step further towards being pushed towards teaching, away from the stars of the department who have a light/nonexistant load and spend all their time researching.
Step 4 Do the work, avoid any fellows ripping it off (and that includes your superrvisor if you're a phd student). Many long repetative and perhaps futile days ahead.
Step 5) Poster/Conference time. Yay! an opportunity to meet up with your peers and consume a variety of toxic to edible food stuffs with them. Incidentally an opportunity to listen to others talk about their research, hopefully being selected yourself to give a talk or a poster.
Step 6) The results look good, hopefully you thought about their potential analysis before you started actual experiments, chat to fellows in department about such, hoiping the head of department will recognise your potential brilliance and renew your short term contract.
Step 7)Write it all up. This may be a combination of work of a couple of your fellows whom you've spent the last three years or so working very closely with: decide who gets the top billing, primary authorship, oh and yeah you'll have to tack on your head of department/supervisor even if they didnt lift a finger to help.
Step 8)Choose which journal you would like to try and get published in. In the world of academia such are much more presdigious than others, on some weird relative scale worth more points if it were to publish in, are the heights of Nature for you? Perhaps a more specialist journal, or something really obscure (oh well)?
Step 9)Send of your paper to the editor of that journal, along with the fee for such. He then flicks briefly with your paper and decides whether its worth his time, whether its suitable for his journal. If so he selects a couple of people whom are friends/experts in the field/owes a favour too and sends of a copy of your paper to them.
Step 10)Your paper is peer reviewed, by at this time by a small number (1-3) hopefully knowledgeable, impartial anonymous experts in the field poor over your work, looking for any flaws or areas which need to be explored further. Hopefully they're not too red pen happy! Now choose either 10i, 10ii, or 10iii
Step10i) Yay! Your paper is excepted without question. This is the buzz the reason your spent months crouched over hazardous/smelly/expensive chemicals/creatures/equipment for! Time to break out the champaign. Recieve a pat on the head from the Head of Department and take a step towards dropping your teaching load! Woot! Procede to 11
Step 10ii)Erk, red pen, flaws in your work! Hopefully it just means you need to reanalyse something, change a graph, though you may have to go back and do some more research. Resubmit to the editor once you are done: goto 9
Step 10iii)Paper rejected, try an less popular journal: goto 9 or break out the shedder.
Step 11)Time to cough up the goods and pay the editor per page, graph and so forth to include your paper in their journal! Order some reprints (at your expense) so if some freeloaders, I mean fellow scientists write to you directly you can send them a copy of your paper directly and something to pin to the board outside your office, a talisman against the damoclean sword of dismissal (we're reorganising the department... yes thankyou for your 40 years working for us... sod off now, clear your office by Monday...)
Step 12) Presuming you've got your paper in a journal your university has payed (a lot!) to subscribe to, bask in the glow of you work being perused by your fellows and students in the library reading room.
Step 13) Other researchers read you paper, muse on it, write letters to the editors pointing out potential flaws, embark upon their own reasearch to either demonstrate the flaws in your hypothesis, or push the field you've opened up a little futher. This is the gist of peer review. Goto 1
A summary of the problem:
Peer review is strong, valid method of rigourously insuring that research gets aired by the academic community as a whole, ideas a thrown up, explored, demonstrated to be flawed and rejected, or duplicated by others and begin to solidify into the main accepeted knowledge of the field. Parasitically upon this are the journals. You pay to submit your work and you pay to read it, though they do provide a vital initial step in the peer review process, ultimately its only the view, the accepted wisdom? of a couple of experts in that field and a couple of glossy covers with a brand name on the front. Then even more significantly, others have to pay large sums indeed to read about your work, subscribing to the journal in question, or forking out for a individual copy from the journal [£5 a shot in the UK]. Moreover by publishing you've signed over all rights to the journal in question.
Fun-fun.
troodon.net
As someone who publishes in traditional peer-reviewed journals, let me add a few comments to this. What journal peer review provides that can never be replaced by an open forum like Slashdot is credibility. The journal editor is responsible for selecting reviewers who are knowledgeable in their field. These editors, at least in the physics journals I've worked with, are responsible professionals committed to publishing a high-quality journal and not just ego-driven hacks. Because the editors are considered trust-worther by the journal readers, articles subjected to a peer-review process can be take seriously (they can still have mistakes, of course, but usually not obvious ones).
Similarly, because article reviewers are selected based on their professional credentials, reviewer comments in a peer-reviewed journal are worth my serious consideration. I may disagree, with the reviewer, but that's part of the process and I'm given the opportunity to respond to the reviewer's criticisms as a normal part of the review process. Finally, because the review is blind (either single-blind or double-blind depending on the journal), the reviewer can safely criticize a more established colleague without fear of retribution. On a peer-reviewd journal, with an editor as the final arbitrator, this procedure works. On the internet, though, either reviews are public which opens the viewer up to retribution, or the reviews are completely anonymous in which case the reviewer lacks all credibility.
A comment made by a random AC on slashdot, in contrast, is not worth my [professional] time. There are just too many posts by too few knowledgable people. Even with the established pre-print servers (which are not peer reviewed but of considerably higher signal-to-noise ratio than most other public forums), it isn't worth the time and effort to read the articles unless I already know one of the authors, either personally or by reputation.
In sort, peer-review is not some arbitrarily imposed requirement from on high; when handled properly it is a valued part of the scientific process and I will not take the time to read any journal that does not maintain peer review. There are only so many hours in a day and I would prefer to use them to get real work done instead of wading through a stack of dubious articles on the off chance that one of the authors will have something worth-while to say.
-JS
P.S. For the record, I read slashdot on my own [personal] time. This is entertainment, not work.
Vanity of vanities, all is vanity...
The whole problem is a simple economic problem. In the old days communicating peer reviewed papers to colleagues involved getting a printed copy to them. It was not more than logical to use the services of publishers to take care of the logistics and naturally they were in it for the money. In addition the publisher needed copyright in order to be allowed to print.
:-).
Nowadays there is an alternative means of getting the scientific content distributed: the internet. Many scientists have already discovered the internet as a means of distributing and obtaining papers. I can't remember the last time I actually had to go to a library to look up a paper. The internet totally eliminates the need for a publisher and could potentially greatly reduce the cost involved with publishing articles without affecting the level of quality of peer review in any way. In addition the internet provides additional possibilities in the form of for example moderated forums, mailinglists etc.
Unfortunately the publishers have built up an enormous power and are very reluctant to release it. Scientific journals are very profitable since they require a minimum of investment on their site and they have a small but loyal audience who are willing to pay relatively large amounts for journals. However over the past few years there has been a pressure to move to an internet based distribution of papers and journals. Despite this publishers still receive enormous amounts of subscription fees. At our university department, the library sucks up obscene amounts of our budget for online subscriptions and regular subscriptions. All this money (mostly coming from our government) flows to the publishers who add little or no value to the scientific product since they completely rely on external input (voluntary peer review, editing and academic authorship).
Most journals of significance in my field are either associated with ACM or IEEE, both of which are non profit organizations, both of which are charging significant amounts of money for membership and access to papers. A step forward would be if these two organization would abandon the publishing process and focus on internet publishing only. If those two organizations set the necessary steps, the rest would have to follow. As an author who has contributed to several IEEE conferences and who has even been a IEEE member (I have resigned because of the outrageous lack of value added by a membership) I would very much like to see this happen. After all they are supposed to represent our interests rather than work against us.
Basically I would like to see two things:
- Authors should retain copyright over their articles. The arrangement proposed for the ML journal seems reasonable, perhaps some GNU like legal document could be created to formalize it.
- Papers should be available and searchable at no cost. Currently I have to pay to be able to be able to download my own articles from the IEEE. I find that offensive (luckily I have copies on my homepage
In addition I would also like to see some innovations:
- XML based structure for papers to enhance searching.
- Addition of the before mentioned forums.
- distributed storage of content ala freenet.
If these requirements are met (even minus the innovations), I'll gladly renew my IEEE subscription and sign up for the ACM as well. I'm fully confident that with the logistics of the physical publishing process eliminated the membership fees can be reduced significantly. Alternatively this money can be used to set up more conferences and finance promising researchers.
However, currently I feel that neither of these organizations are working in my interests.
Jilles
If I send those to a slashdot-style forum, my boss will laugh. They'll count something less than a one-line quote in a small town newspaper.
This is true. Even though I supported the public library of science initiative (reported earlier on slashdot, and also here , I'm about to submit an article to TIBS. An article on GenomeWeb just won't cut it.
However, that is not going to be the case forever. As data mining techniques become more available and sophisticated, that is to say, when real data mining (as opposed to just text matching) becomes a major way in which academics access content, articles in "free" journals are going to be *more* visible to your colleagues, and as important discoveries are coordinated using such techniques, citations will rise, and those journals will rise in prestige. This is going to aggravate what is allready a real schism within the academic community, of int. property versus the pursuit of truth. I think it's going to be a struggle - for the very soul of academia which is really under threat here - but I think we're going to win, because cutting out the middle-man makes for a more efficient way of sharing information, because history isn't over and materialism isn't really the driving force behind human creativity, and because people who love science for it's own sake are better at it.
An earlier poster said that the current system drives the smartest people out of academia and into industry. I couldn't disagree more - I don't think people in (the biotech) industry are very smart at all. They're kind of pathetic, mostly. I'll agree that there are a lot of frankly stupid "scientists" doing terrible work at supposedly public universities on the private dollar, and that the people who jump ship into industry are often a bit smarter than they are; but the smartest, most devoted people are still in the public sector.
The good and new comes from no quarter where it is looked for, and is always something different from what is expected.
You're right- this won't solve all of the major problems of getting published, but it will solve the problem of money-hungry publishers exploiting libraries and subscribers for every last penny.
But if this model does take off, then interest and contributions to the journal will only rise. Which in turn, means that more good work will get out there. Hefty online interest can mean that e-journals will have the ability to dictate to print publishers the terms.
NB: Some journals now send out articles for peer-review with the names of the authors blanked out. Blind peer-review. It's rather surprising who actually gets turned down in these cases.
As a student, how is it affordable for me to pay $100-200 per year per journal for online access? That's $1200-2400 per year I'd have to pay for the journals I'd like to have access to. For a yearly salary of $17,000 (pre-tax), I don't think I'd be able to spend that. Moreover, the library cannot provide full online access for a few journals because, I know for a fact, that the library pays more per person to provide online access than if the person went out himself/herself and subscribed. How is this possible? Wouldn't one think that "buying in bulk" would save the library money?
What I find even more annoying is that because publishers find that this business is so lucrative, each and everyone of them are enacting their own standard of distribution and dissemination online--e.g. Elsevier Science, BioMedNet, CatchWord, Lippincott (just to name a few of the large ones). Instead of agreeing on one good standard of dissemination (i.e. PubMedCentral), they're all broken up and if I wanted access to each publisher's site, that's an entirely different login name and password. Additionally, some publishers are notorious for not providing access to individual journals--access can only be provided via the university library at outrageous costs. Either way I'm SOL, because for certain journals, my library cannot provide access, really just because of the cost.
So from my perspective as a graduate student trying to learn as much as possible, how does the current publishing system benefit science? This is how I see the current University earning and spending cycle: the huge budget that our library carries comes from the university; the university generates a large proportion of its revenue via licenses of its intellectual property (some only comes from the state, even though it's a state university); the intellectual property (IP) of the University comes from it's staff of scientists which do go all the way down the hierarchy to and includes their slaves, the graduate students; IP can only be generated and formulated by the scientist via a fusion of past knowledge and literature provided by the library. Imagine cutting the time in this cycle by provided faster, more consistent, cheaper access to the literature.
The current publishing system is stifling my progress, when it needs not to be. Publishers cannot be dinosaurs of the past, the privatization of their dissemination model is not a good example of how useful they can be, especially when one takes into account the efficiency of training new scientists.
Linux at home