Slashdot Mirror
New Ideas for Scientific Publishing Online
The printing press provided a means to distribute multiple copies of political pamphlets, advertising posters, legal documents, novels -- and Journals containing scientific discoveries.
Scientific Societies use these Journals to disseminate research findings. The expense of Society membership and the additional fees for subscription to multiple Journals has gone out-of-bounds for both individual investigators and institutional libraries. It is just not feasible to expect a scientist/researcher or a member of the public to be able to subscribe to all of them or even to a few specialized Journals.
PubMed provides a powerful search engine for locating biomedical articles. PubMed searches are superior to browsing the specialty Journals. However, once you have used PubMed to find an article of interest you must then locate a library housing the Journal in which it appears, that has a Xerox machine with which to make copies of the sought-after articles.
Harold Varmus proposes that an open access e-repository be established to maintain permanent on-line and downloadable archives of scientific literature. The most obvious advantage of this to the researcher is immediate access to any published report via a hyperlink from the PubMed database. E-reports can also contain more information than print Journals, including larger data sets in various formats, pictures with greater detail, or even movies. Many of the costs associated with the publication of a Journal are avoided. Cited literature [footnotes] can also be hyperlinks, which simplifies in-depth background analysis for serious researchers.
Harold Varmus's proposal describes two methods for submission of a new report that could operate side-by-side. The first is to use the established editorial boards, and the second would be through a publicly available preprint repository.
European backers of Varmus's Proposal tend to favor the first, "closed" method of submission. Their claim is that by sticking to the traditional method there is less chance that the database would be flooded by poor quality reports. A subliminal reason for their desire to maintain editorial control might be that delayed publication gives the group that reviews the data extra time to analyze and extract ideas for future research before it is made available to the world.
With the second submission method, each submitted report would only need to be given a cursory review to eliminate voodoo science (SPAM for health care scams or unhealthy foods, etc.) before it was placed, unedited and unreviewed, into the preprint repository, where any interested party could read it. Each "preprint" report could be given a version number like most Open Source Software projects use. Perhaps the "development" version could show editorial strike-outs and new text in different colors from the original. The next higher, "stable" version would be the reviewed, edited, author-corrected copy. Still higher versions might contain supplementary information. Even after they are published, the lower versions should be archived and accessible for historical use.
The Harold Varmus Proposal would require an article to obtain two favorable reviews, perhaps from members of established editorial boards, before it was transfered from the preprint repository to the general repository. Varmus also touches on the possibility of more open reviewing "in which critiques of the scientific reports are accessible and signed." (Today, most scientific papers are reviewed anonymously.) I suggest that, in addition to solicited reviews, signed, unsolicited reviews should also be considered.
SummaryThe electronic submission, publication, editing, indexing, archiving, retrieval and utilization of scientific reports, abstracts, and data is taking a significant turn, and Varmus's Proposal may help make that a turn for the better.
I, like Varmus, believe that since most scientific research is funded by the public, the public should have free access to it from open E-libraries via the Internet, and that the only person who should be allowed to claim "ownership" of a scholarly article is the person who wrote it.
No Scientific Society or Journal Publisher should be allowed to hold a copyright on scientific knowledge. The researcher is the only one who has the right to claim, "I discovered it and I reported it."
Back when I was in grad school, my research happened to make a notable contribution to a hot topic at the time. I was (usually with other authors) submitting papers to IEEE journals at a rate of about 1 per 3-6 months. I also attended several conferences and got to know a lot of the major contributors in my research area.
Typically, every submission got sent to 3 experts for review. My professor (and one of his collegues) even forwarded to me several papers they were asked to review. I noticed a couple of things regarding peer review:
1. For every submission, there was a 50/50 chance that none of the three reviewers would know what the paper is really about. Part of this problem is that the IEEE journal editors simply can't know about all the topics being researched in their area and would often pass the submission to the wrong experts. These "experts" (in the wrong subject matter) often wouldn't give a shit about trying to learn about the topic at hand and would just give it a bad review. You could tell this was happening when you got your submission back with comments and criticisms that simply made no sense. I was asked to review a couple of papers that were out of my research area, and I did my best to research the prior literature on the subject and give a fair review, but many professors didn't have the time and would slam the paper rather than pass it on to somebody who would understand it.
2. There is an "old boys" network present in every field of research. Most of these operate like an old fashioned closed businessman's club. Once you are accepted into the fold, everyone else kisses your ass and gives great reviews to any paper with your name on it, regardless of whether it is worthy of publication. The people who make it to this stage rarely make useful research contributions anymore, but they get their names on lots and lots of papers. All of the lesser known researchers practically beg the big boys to co-author their paper, thus virtually guaranteeing its publication. Some of the members of the "old boys" network have unbelievable egos that require constant stroking if you plan on ever making a name for yourself in the field.
3. There are some researchers who will go to great lengths to stab you in the back. I remember the case of one little known French researcher who wrote a landmark paper in our field. In the review process, it was sent to one SOB who was a big player in the field but was notorious for being a backstabber and had questionable intelligence (I never knew how he got so well known in the first place). The SOB managed sit on the review and delay the paper's publication for over a year while he attemped to figure out what the French guy had done and duplicate his results. He sat on it so long in fact that he got his own paper published on the subject before the French guy's and stole all the credit. This dickhead, and several others like him, were also notorious for attacking their peers (and especially their peer's grad students) at any conference where one of their sponsors or potential sponsors were present.
I left grad school thoroughly disgusted with the whole research community. Your status was measured by how many papers you published, not how many real contributions you made. There were too many people who capitalized on the original research of others by pumping out lots of papers covering slight variations in the application of the aformentioned original research. To keep up, you had to sell out and produce lots of junk papers with the right names on them. I managed to get published in IEEE journals 6 times in 2 years, but I'm only proud of two.
This proposal is reminiscent of Drexler's suggestions for effective use of hypertext as outlined (for instance) in Engines of Creation (which is web-available, but I'm too lazy to find it right now). In fact, when he was describing hypertext in that context (before it was layered on top of the internet to create the Web as it exists today - which actually only implements about a third of the features of his proposed system) his discussion seemed to think of it primarily as a means of publishing scholarly information, reviewing it, and correcting for the lag-times and lack of back-links in conventional print media.
This proposal doesn't seem to implement backlinks, but the availability of updating to correct bad information makes up for some of that.
Some benefit might be had by looking at integrating some of the features found in the CritSuite (authored by Ka-Ping Yee and to be found at crit.org), which uses a proxy-server method to implement many of the missing features of true hypertext as a layer on top of existing WWW content.
A good versioning system with history information available (analogous to CVS) is also desirable - this is somewhat reminiscent of Daniel Dennett's "Multiple Drafts Model" (which he used as a metaphor/model to describe how the mind handles memory, but which could equally well be taken at face value as a method for handling bona-fide document drafts in an electronic environment), but with better memory.
I'm kind of skeptical here: a poorly implemented system is in some ways worse than no system at all, because it can lead to complacency. As long as careful consideration is given to which features are desirable, this could be a Very Good Thing.
One characteristic to watch for is how much centralized control is given to "editors" over content and filtering. Automated filtering methods, trust networks, reputational ranking, etc. have been fairly well-developed ideas for years. It would be an unfortunate oversight to fall back on print-age social technologies when something better is available.
Academic publishing seems to be mostly based on the reputation or authors and reviewers (in fact, some papers originating from students of influential professors get published even though they are total crap).
The other flaw in the current system is usually hidden from the masses - most professors don't review the papers themselves, but their grad students do. The professors rarely have the time to go in-depth on the paper, check the math, etc.
Sometimes the name of the grad student is attached to the review when it's sent out, and sometimes not. So it's impossible to determine the reputation of the real reviewer of the paper.
--> Any fool can criticize - and many do --
as stated in previously, without an editorial process, preprint servers are largely useless because of the amount of garbage one must wade through to get something useful.
well, there are free, online journals that exist. they use the same peer-review process that print journals use, but eliminate the cost by eliminating the printer! for examples, The Electronic Journal of Combinatorics has been around since 1994. if you look at the list of editors, you're sure to recognize a few names.
i believe that this is the answer. the free availability of this information is what is sought after.
- palspeaking as a biocomputing geek, the NCBI website is a great starting point...
Subscriptions to scientific journals easily cost $3,000-$5,000 a year. For this, university libraries get a bunch of scientific articles written by scientists (who are mostly fully paid by universities and science foundations), and reviewed by the same group of people. All of this costs the publishing company about $0.00. The publisher then has to do a bit of editing and finishing up. It's practically a free lunch. And Elsevier (to name just one) is raising it prices by 10% each year.
On top of that, almost all journals demand the scientist to sign a Transfer of Copyright Agreement. If you're not careful, you could be sued for publishing your paper on your homepage.
But now there's internet. We don't even need a press anymore. Potentially, there's lots of (library) money available that can be used to replace the old-style publishers. All it takes is for scientists to unite.
A subscription to this journal, I might add, costs several hundred dollars per year and of course it does not pay any of its reviewers (like most academic journals, it is considered an 'honour' to be asked to review a manuscript.)
Someone, somewhere is making a lot of money out of the whole journal scam.
Nick
-- "It's a sad day for American capitalism when a man can't fly a midget on a kite over Central Park" - Jim Moran
Thanks
Bruce
Bruce Perens.
I agree that peer-review is vital to filter out crackpots and commercial propaganda. But rather than the traditional editorial board approach, why not slashdot style moderation?
Give each reader and each paper a credibitlty score. The credibility of the reader is based on the average credilibity of each paper they have submitted. Each reader can give a credibility score to each paper they read. The score for the paper will be the submitted scores, weighted by the credibility of each reviewer.
Now it is easy to make foulups in a paper the first time round, so an initial submission could be made to an editorial area. People could add comments slashdot style, and the authors could use these to revise the paper for final submission.
When I started reading this, I hated it. But now I love it! If someone launches a journal in my field this way, I'll gladly submit a paper. Of course, there is no real need to have individual journals, if the database can be searched flexibly.
In the various subfields of Physics, this idea
of a public "preprint server" has been implemented
for some time: check out the Los Alamos
Physics Preprint server.
I've been active in research (astronomy) for
the past ten years or so, and I've had many
conversations with other researchers on the
future of scientific publication. Some of the
main points are:
1. Review/moderation is necessary. There are
a _lot_ of people who have crackpot theories
about the universe, and some of them aren't
shy. Without refereeing of some sort,
the number of scientifically worthless --
see definition below -- papers will grow to
the point that they may swamp the worthwhile
papers. At that point, many users will stop
using the archive.
Note on "scientifically worthless": science
is an enterprise which depends on its
workers to adhere to a set of rules, such
as understanding basic physical principles,
checking the existing literature, creating
falsifiable hypotheses, verifying new
results, repeating experiments, etc. Papers
describing ideas which aren't developed
along these rules are, by definition,
scientifically worthless.
2. Scientists depend on their publication
records to land good jobs, and to advance
in those jobs. At the moment, in astronomy,
at least, the existing
electronic archives are NOT viewed as
"real publications". There's a little bit
of a chicken-and-egg problem: until the
electronic archives are taken seriously,
many people won't publish in them
exclusively. But if everyone publishes
elsewhere, why take electronic archives
seriously?
3. Many people, myself included, worry a great
deal about the use of electronic archives
10 or 20 years hence. I have paged through
bound journals dating back more than 100
years, and used them occasionally in my
research. I can interpret the information
easily. But I don't think it will be an
easy matter to keep electronic media up-to-
date over a century. The librarians to whom
I've talked are _very_ worried about this.
Yes, I know that it may not be difficult
in THEORY to copy old materials to new
formats and new media every N years;
but in practice, it's a royal pain. In an
era of shrinking library budgets, it may
become fiscally impossible.
On the other hand, I do very much support the
idea of "Open Source" publications. It will
enable many more scientists to publish their
ideas. In my field, for example, the authors
have to pay the journals about $125 PER PAGE
for the papers they publish. My last paper cost
over $2000, and I had to pay for some of it
myself (since I work at a small university that
doesn't have a lot of money to support research).
The tricky thing will be to find a mechanism
which keeps the good points of the current
scientific journal system, while avoid the
pitfalls (some of which I've mentioned above).
Michael Richmond "This is the heart that broke my finger."
mwrsps@rit.edu http://stupendous.rit.edu