Hoax-Detecting Software Spots Fake Papers

sciencehabit writes: In 2005, three computer science Ph.D. students at the Massachusetts Institute of Technology created a program to generate nonsensical computer science research papers. The goal was "to expose the lack of peer review at low-quality conferences that essentially scam researchers with publication and conference fees." The program — dubbed SCIgen — soon found users across the globe, and before long its automatically generated creations were being accepted by scientific conferences and published in purportedly peer-reviewed journals. But SCIgen may have finally met its match. Academic publisher Springer this week is releasing SciDetect, an open-source program to automatically detect automatically generated papers. SCIgen uses a "context-free grammar" to create word salad that looks like reasonable text from a distance but is easily spotted as nonsense by a human reader.

Comments

Interesting Response

[Roger W Moore]

arXiv is not peer reviewed. What I found interesting though was the response of the publisher: write a program to detect fake papers. Even the most simplistic peer review - i.e. reading the paper - would immediately catch these papers. If they need to write a program to catch fake papers then their peer review model is essentially worthless and frankly a journal that poor is no better, and liekly worse, than arXiv: at least arXiv doesn't pretend to have peer review.