Slashdot Mirror
Free Online Scientific Repository Hits Milestone
ocean_soul writes "Last week the free and open access repository for scientific (mainly physics but also math, computer sciences...) papers arXiv got past 500,000 different papers, not counting older versions of the same article. Especially for physicists, it is the number-one resource for the latest scientific results. Most researchers publish their papers on arXiv before they are published in a 'normal' journal. A famous example is Grisha Perelman, who published his award-winning paper exclusively on arXiv."
When I was a freshman at the University of Minnesota, a professor instructed us to use Arxiv as a resource (I think Citeseer was another but paled in comparison). A large part of my undergrad and grad school days were spent perusing Arxiv and sometimes implementing ideas I had read in the Computer Science section. My hard drive became strained by the sheer number of PDF/PS files in my user directory. My room was littered with papers printed off to read on the bus or at work. My base knowledge of computer science I owe to my professors, most of the things beyond that came from Arxiv.
...
I owe a lot of my knowledge to that site. Here's to another 50,000 papers, Arxiv. And another and another and another
Also, the Arxiv Physics blog is a regular favorite in my Liferea news feed account.
My work here is dung.
i'll beat all the cynical punch savvy posters to the punch!
that comma is in the wrong place, i see 50,0000. I guess they need another article on properly writing numbers.
is that a typo for 50,000 or 500,000?
Here are some in fields I follow :
In astrophysics, almost all new papers appear first in Arxiv.
In planetary physics, some but by no means all papers appear in Arxiv.
In geophysics, basically no papers appear in Arxiv.
I don't know why there are these differences, but there it is.
If it's a science publication, should it have hit a kilometer-stone instead of a milestone?
Likewise, every particle physicist also puts his paper there before they are published (my three are all there). While it is great as a source of open information, one thing to bear in mind is that it is not peer reviewed, *anybody* can stick *anything* there. This is the major reason why we still unfortunately need paper journals. We need somebody to read it and say yes this follows basic scientific procedures and to the best of his/her knowledge there are no mistakes. Because theres a fairly low signal to noise on arXiv and whats there is not guaranteed at all to be of proper scientific merit and correctness.
But the question we are all asking ourselves is
Who got the first post?
The answer is Exact Black String Solutions in Three Dimensions by James H. Horne and Gary T. Horowitz
Slightly better than the "Fkrst Pist" attempts on Slashdot!
An Eye for an Eye will make the whole world blind - Gandhi
Because quantity == quality...
I'm not going to pretend 50,000 is a lot, but the fact it's 50,000 and growing should make them worry. I hope the celebration of this milestone will help accelerate it's growth so we see 100,000 sooner than later. The quicker pay-for-access science disappears the better for all of us.
Oh honey look... How cute... an angry slashdotter!
It seems that a lot of people follow their field by reading pre-prints posted to arXiv. Isn't this kind of dangerous, considering the lack of peer-review? Or is there no problem because people only actually _use_ the results after they have been published in a proper journal?
I've seen that they've started a system where you need an endorsement from another arXiv author to post a pre-print, but is an endorsement enough, considering the likely fact that endorsers don't really check the paper properly?
Congralculations on that SCIgen benchmark!
Do not trust this signature.
cat /dev/null
one thing to bear in mind is that it is not peer reviewed, *anybody* can stick *anything* there.
This is true. However, they do have a group of moderators which recategorizes what they think are "merely mediocre, speculative, or erroneous articles". See http://front.math.ucdavis.edu/ifaq#nonsense
Of course, this is not the same as peer-review, but at least it's something.
By the way, who moderated the post pointing out that the comma was in the wrong place as "offtopic"?? The proper moderation is "insightful", since at least half the commenters in the discussion following seem to think that the comma was right and the extra zero at the end wrong.
http://www.geoffreylandis.com
Vegeta! What does the arXiv say about their number of articles?
It's fifty ten THOUSAAAAAND!!!
3. Profit!
2. ???
1. On Soviet Slashdot, a Beowulf cluster of alien Natalie Portman overlords welcomes YOU!
In it he writes:
As an experiment, Greg Kuperberg looked at the publication status of the first 100 papers in theoretical high energy physics posted to the arXiv in December 1998. He found that 81 had appeared in journals, 11 were conference proceedings or invited lectures, and 2 were Ph.D. theses. "Thus at least 94 of the 100 have been blessed by some form of peer review," he concludes.
They could implement some sort of karma, moderation, and meta-moderation system, which would be even better than peer review, right?
Wow, that's a lot of ten-thousands of papers!
Coding with assembly is like playing with Legos. Coding an application in assembly is like building a car with Legos.
But note that there is no impediment in order to publish just-online peer reviewed journals... maybe that's the future or arXiv. Paper must die, it just creates silly troubles... we end needing, for example, sites like JSTOR in order to access out of print numbers or foreign non imported titles.
I can't see how that would be better. To properly review a paper you need to take a lot of time to really understand it. It's not like on slashdot, where you just skim through comments looking for funnies or insightful one-liners before you moderate..
I thought it was the third zero that was wrong.
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
Not insightful, but informative.
He's not showing any insight here. Instead, he's presenting information.
You suck. Now THAT's insight.
I just pooped your party.
PubMed Central, the central repository for open access Life Sciences research articles, is pushing on 1.3 million articles. These repositories is a wet dream of text mining researchers.
I thought it was the third zero that was wrong.
Looks like a lot of people thought the same thing.
In fact, though, it was the comma that was wrong, the zeros that were right.
http://www.geoffreylandis.com
You jest, but if what very, very little I understand of Japanese is in order... Well, maybe our great Taco has merely been watching too much anime.
In Japanese, ten-thousand is "man" (pronounced with an "a" somewhat like the "a" in "father": "mahn"). What we would call "five hundred thousand" would instead be called "go-jyu man" ("go" = "five", "jyu" = "tens", and "man" = "ten-thousands": "five tens, ten-thousands"). So, basically, "fifty ten-thousands" would be a fairly accurate English representation of Japanese-style numbering.
Of course, my Japanese is only slightly better than my Lisp, so don't put this in Wikipedia or anything. ;)
it is not peer reviewed, *anybody* can stick *anything* there.
I think they've changed things a little bit over time. It does seem like anyone is able to register an account, which would allow them to start submitting papers. But looking at the help pages, I see this on an endorsement system: "Effective January 17, 2004, arXiv.org began requiring some users to be endorsed by another user before submitting their first paper to a category or subject class." They note that this isn't peer review, but it "will verify that arXiv contributors belong [to] the scientific community". They also moderate submissions, and the help page on this topic says: "arXiv reserves the right to reject or reclassify any submission." While also not real peer-review, it "helps to ensure that arXiv content is relevant to current research".
Perhaps some areas are better than others about self-moderating/reviewing submissions. My experience with the astro-ph archive, which I've read for many years, is that most of it is generally good material, often pre-prints of papers that will appear in peer-reviewed journals or conference proceedings. Not all of it is like that of course, but I think there's a lot more signal than noise in the astro-ph section at least. Just my opinion.
I realize that you were being snarky, but you accidentally hit on a corner of the truth. The real value of the ArXiV is indeed its quantity of results, mixed with the ease of access. The traditional journals typically restrict access to their output -- unless you are at a subscribing institution, it costs $15-$50 to access a single article from a single traditional scientific journal (depending on publisher). At professional institutes and universities, which typically have online subscriptions to journals, it is possible to surf through the Literature (depending on field, back about 10-15 years) and find recent relevant knowledge extremely quickly. If you aren't at an institution that subscribes, you're SOL. ArXiV fixes that - if you publish your article both in a journal and in the ArXiV, most indexing services will notice that it is the same, and suddenly everyone on the planet has unrestricted access. That's a no-brainer for an author.
The way that professional scientists (like me -- I am a solar astrophysicist) access the Literature has changed drastically in the last ten years. My office has about 12 linear feet of Xeroxed journal articles in three-ring binders, but I practically never refer to them. It's far faster and more convenient to access (say) the entire archives of Astrophysical Journal online than to go "grep dead trees" at the library. Citation indices such as ADS (Google for adsabs) hyperlink both references and citations, so that I can search through 50 articles relevant to a topic in less time than it used to take to look up one article and Xerox it for reading outside the library.
Old-style pay-to-read journals get in the way of that rapid access - for example, I have rarely cited articles in Astronomy and Astrophysics, because it's a pain in my ass to download them. Until recently, my institute didn't subscribe, so I had to either pay on a per-article basis (which adds up if you are skimming for the one relevant article in a dozen possibilities), or travel to the local university to get the paper I wanted. This is a very common problem: even large universities generally don't subscribe to all the relevant journals in a given field, because web subscriptions cost thousands to tens of thousands of dollars per year per journal!
For everyone not fortunate enough to have a computer account at a large institute that can actually afford to subscribe to dozens of journals, ArXiV is the best way to access a large volume of the literature. Hence, articles posted to the ArXiV get cited more. That makes authors want to post to the ArXiV as a matter of course. It's a virtuous circle.
So, er, yes, quantity is quality in this case -- ArXiV was canny and/or lucky enough to get a critical mass of good work, and the quantity is the driving force that keeps the whole thing going.
But that was 1998 where a) the general population was just getting online and b) pretty much only scientists knew about arXiv. There is a lot of peer reviewed stuff on there (every paper submitted to a journal tends to be submitted) but as more less mainstream scientists have access, you regretably get more noise. Looking at Oct 2007 for hep-th and assuming that it would be mentioned in the summary is its published or going to be published (and trust me people mention this...), out of the first 25, 12 are published in a journal and or conference proceedings. So less than 50% were blessed by some form of peer review. And its the other 50% tend to be the most sensational :)
Note I still think its very valuable for to have a place where non-peer reviewed material can be uploaded as well as peer reviewed but if its not peer-reviewed its a lot more likely to be incorrect somehow and the reader needs to be aware of that.
ah didnt realise they had moderators, thanks for the correction. So there is atleast one layer of quaility control to keep the crazies out. When I submitted my papers I just clicked upload and didnt realise that somebody reviewed it. Glad nobody objected to mine :)
But that was 1998 where a) the general population was just getting online and b) pretty much only scientists knew about arXiv.
These are valid objections, I agree.
Looking at Oct 2007 for hep-th and assuming that it would be mentioned in the summary is its published or going to be published (and trust me people mention this...), out of the first 25, 12 are published in a journal and or conference proceedings. So less than 50% were blessed by some form of peer review.
As a comparison, I did the same thing in my own field of mathematics. I looked at the first 25 articles uploaded to arXiv in October '07. As for your field, only 12 were either published or were PhD theses.
But, FWIW, from my quick look at them, there were no obious nonsense articles.
It is with great joy that I watch those who feel entitled to withhold knowledge in order to benefit their own avaricious needs by controlling the dissemination of popular art and science (starting with the Church to todays corporate greedy) - lose their hold on said resources faster than you can say Wall Street Meltdown. Kudos to the Internet and all those who espouse the FREE exchange of ideas.
A man spends the first half of his life accumulating stuff, the second trying to get rid of it all.
We really need to begin compiling our scientific knowledge into a hyperlinked wiki/database of sorts.
Wikipedia's great for basic stuff, though there's still gobs of information (much of which is in the public domain) that's inexplicably confined to books and journals.
Hyperlinks (and extended data sets) should be *standard* for all journal articles these days, given that we have the technology to do so. There's no reason that the arXiv needs to remain as a repository for dead-tree PDFs.
-- If you try to fail and succeed, which have you done? - Uli's moose
To clarify, arxiv is a document repository (you submit your papers there). If you want a scientific papers search engine, use citeseer.
Note that citeseer also indexes arxiv documents :)
"This is the major reason why we still unfortunately need paper journals. We need somebody to read it and say yes this follows basic scientific procedures and to the best of his/her knowledge there are no mistakes."
Darwin did not do any of this with the origin of the species and many scientific ideas from the past came out in lay/not overseen books for the reader. The fact that ideas are peer reviewed or not is quite irrelevant to it's truth. In fact peer review is flawed now knowing what we know about human reasoning, and the fact that reasoning is not as the enlightenment had us believe.
Most scientists don't even have a clue what has been discovered in the neurological sciences over the last 30 years and how it undermines the enlightenment's view of reason and enlightenment's view of science and education. Most people still operate under the enlightenment's false view of reason.
The enlightenment fallacy:
(quick version)
http://i35.tinypic.com/10fruxh.jpg
Longer version:
http://www.linktv.org/video/2142
I mod you all, +1 to Ogre Slaying!
8bit theatre, D&D
Go go Gadget Nailgun!
The best nonsense articles are those that require more than a quick look to determine they are nonsense. In any joke, the punchline has to come at the end, not the beginning.
The problem is when someone who is not an expert in the field comes to a site with unreviewed articles. He can't determine in a quick look what is bogus and what isn't. If you are trying to learn about something new, unreviewed papers are a crapshoot.
If I publish a paper to arXiv, is it peer reviewed before being posted, or is it just accepted? Just curious
Currently hooked on AMP
was the original .. with the skull/crossbones icon. Now its all too easy and happy looking.
This isn't a bug, it's a feature.
If people are looking for quality-filtered articles, they should restrict their search to something other than just "everything in arXiv". If they don't, and take everything in there as gospel, then they're fools and deserve what they get.
ArXiv doesn't put itself out there as a peer-reviewed source; it's pretty up-front about not doing that, in fact. There's a place for peer-reviewed, high-quality sources, but there's also a demand for something else: access to information. Sometimes you don't care whether everything in a database has been peer-reviewed, and if the cost is paying $15 per item (which is what it costs to get something from a lot of peer-reviewed journals), then it's not worth it. With traditional journals you don't have a choice; you're stuck paying regardless, even if you know the paper you want and don't care about their "value added." With papers in a repository like arXiv, there's still a place for journals and various kinds of peer review, but all the papers -- even the un-reviewed ones -- are also accessible if you want to just browse the repository.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
You know only terrorists need scientific information.
---- Booth was a patriot ----
What makes it a fallacy? The obvious problem with the claim above is that people aren't trained to reason and even if they are, they need time (which may take years) to reason.
Yes I think I was a little harsh on the low signal to noise comment. I recant that poor phrasing of words now. Its because mostly in recent years I've seen the 'crazy' papers pointed out to me in arXiv than the large body of well regarded scientific works that I have this slightly unfair opinion. I still stand by my main point that on arXiv the papers are not peer reviewed (and this is in my opinion a good thing) so the reader should be aware that the conclusions should not a prior a be held to the same level of scientific fact as a peer reviewed paper.
I completely see your point. I really do. And I really think arXiv fills a valuable role in making non peer reviewed research available for this very reason. Its just that for every Darwin, Einstein or other scientist that fundamentally changes the foundations of science, there are thousands, if not more, of crazies. Seriously go to the poster session of particle physics the APS meeting and you'll see what I mean.
Here's how it works (for me at least):
First you write a paper - this is the hard part. Then you can submit it to Arxiv - usually done at the same time as submission to a journal, though some choose to wait for any initial backlash/corrections before doing this. Arxiv normally publishes it the next working day with no peer review (8pm EST the night before) for all to see online. Meenwhile your journal is still looking for peer reviewers. No journal in physics can now ask to be the sole source for any article - all authors have to sign a contract often stating that no other commercial source will exist, but that the author can have a copy on his/her homepage (or other place) for free distribution, and on the preprint archive.
Double-blind journal review becomes single-blind when you publish on Arxiv - you can't see your reviewer's name but he can easily find yours. This way it's still possible for someone who gets your paper to screw you over if they don't like you, but then again you can appeal to the editors if you think this is happening. Since you don't know your reviewer, you can't exert any pressure on them (in theory) to accept your paper, so the review part keeps most of its integrity. Reviewers are also required to give detailed reasons for rejecting/accepting papers so you really have to justify your reasons not just "I like/dislike this guy".
In many fields, the blind part of peer review isn't all that blind. Often from the suggestions for citations for example you can get an idea of who your reviewer is. In small fields it's pretty hard not to know everyone in it anyway - especially since you're normally familiar with everyone else' work in your area, and because reviewers are chosen as experts in that area.
Might I reccommend the four year college University of Maryland University College. They have a vast collection of electronic texts and one can access any of them while taking a (distance education) course. Just keep up your GPA and for the price of 1/4 time school you can access any journal you want. I have yet to find something (excepting textbooks) that I couldn't get either online or e-mailed to me. I personally intend to keep taking courses after I graduate just for the fact that its worlds cheaper than buying books on stuff I need to know. Disclaimer: I will in no way benefit from you having this information.
If video games influenced behavior the Pac Man generation would be eating pills and running away from their problems.
It's like Wikipedia for people have no lives.
What we need is wiki-like research, where a mass of people collaborate to finish a research project little by little, asynchronously and spontaneously, just like wikis. If you are interested see this project of mine which although still in pre-alpha mode I hope could be useful some day.