Slashdot Mirror
The Scientific Paper Is Obsolete (theatlantic.com)
James Somers, writing for The Atlantic: The scientific paper -- the actual form of it -- was one of the enabling inventions of modernity. Before it was developed in the 1600s, results were communicated privately in letters, ephemerally in lectures, or all at once in books. There was no public forum for incremental advances. By making room for reports of single experiments or minor technical advances, journals made the chaos of science accretive. Scientists from that point forward became like the social insects: They made their progress steadily, as a buzzing mass.
The earliest papers were in some ways more readable than papers are today. They were less specialized, more direct, shorter, and far less formal. Calculus had only just been invented. Entire data sets could fit in a table on a single page. What little "computation" contributed to the results was done by hand and could be verified in the same way.
The more sophisticated science becomes, the harder it is to communicate results. Papers today are longer than ever and full of jargon and symbols. They depend on chains of computer programs that generate data, and clean up data, and plot data, and run statistical models on data. These programs tend to be both so sloppily written and so central to the results that it's contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you've actually discovered, clearly enough that someone else can discover it for themselves.
The earliest papers were in some ways more readable than papers are today. They were less specialized, more direct, shorter, and far less formal. Calculus had only just been invented. Entire data sets could fit in a table on a single page. What little "computation" contributed to the results was done by hand and could be verified in the same way.
The more sophisticated science becomes, the harder it is to communicate results. Papers today are longer than ever and full of jargon and symbols. They depend on chains of computer programs that generate data, and clean up data, and plot data, and run statistical models on data. These programs tend to be both so sloppily written and so central to the results that it's contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you've actually discovered, clearly enough that someone else can discover it for themselves.
We are now in an era where only very few people actually need to know how reality works. The rest of us can become brand managers and youtube content creators.
That you can flood a scientific paper with reams of computer generated data is NOT science. That's technobabble. The point of the scientific paper is to lay down, ON PAPER, the technique you used and what you observed and then, in a separate section, editorializing what you've proven (or refuted).
That so-called scientific papers will merely dump the computer generated data or flood the paper with technical jargon without exposing the underlying algorithm or technique IS the problem.
At one time science was intended for the masses - that the Atlantic attributes this to "a simpler time" is also moronic as it was the intent of the authors, in fact all authors of the time, to write clearly and succinctly so that anyone could understand their work. You see that not only in the scientific papers of the time but also in the laws (the US Constitution). You also see the same problem in laws today where laws are now tends of thousands of pages long. How is any one person (let alone a dedicated group) supposed to understand the law as written?
To wit - it's a societal problem, not a scientific one or a problem with "overcomplex science"
If programs are central to the evaluation of a paper then programs need to be published alongside the paper - in source code form.
It doesn't matter if the source code is published with Apache, BSD, GPL, MIT or no license at all (remains copyright to the authors.)
What matters is that the source code is available to review alongside the paper. In this, it isn't performance that is critical, but bugs that influence results, be they buffer overflows or simply logic errors.
A group of people separate to those that do the peer review of papers then needs to review the source code for correctness as to the results it produces.
As a scientist who has published papers in peer-reviewed journals, I strongly disagree. I don't care for many aspects of the publication process or how academia works, but the scientific paper isn't obsolete.
Regarding the use of software in creating results, it is definitely a problem when that software is difficult to use or isn't available at all. The same goes for data sets, many of which aren't released publicly for a variety of reasons. These are issues that need to be addressed.
In my own work, I try to release the software used to perform my analysis under the GPLv3. I also try to include adequate documentation to allow the work to be reproducible. I also support making data publicly available, but sometimes the volume of data sets makes it prohibitive to redistribute the work. The best option in that case is to provide detailed instructions on how to generate the data set. But sometimes it's even worse, such as when the research requires using a closed source program with a license that prohibits redistributing the output. The license makes it relatively difficult to obtain the software for anyone outside of the US government or academia, unless they pay for a commercial license. There isn't a comparable piece of software, but the university that licenses the software uses restrictive licensing requirements to increase their revenue.
To the extent that it's possible, scientists receiving grant funding ought to release their software as free and open source software. The data sets should be released or detailed instructions should be included to allow others to generate the same data set.
Papers can be difficult to read, but there are at least some steps in the right direction. One of the encouraging changes is the use of more first person in scientific papers and less passive voice. The formal tone is being being phased out in favor of readability. I wholeheartedly support this because the subject matter is difficult enough to understand without awkward sentence structures. Peer-reviewed papers also provide some level of quality control for research, though not to the extent of reproducibility.
Scientific papers still have a place, because they're still the best opportunity for scientists to present the key points of their results and describe their methods in detail. Simply releasing software and data sets is not enough. Those also aren't peer-reviewed. The bigger issue is that there just isn't a lot of funding to ensure that results are reproducible. Due to funding limits, there just isn't a lot of effort to reproduce all but the most surprising results. It would be great if funding agencies like NSF would allocate more funds for ensuring the reproducibility of existing results. Unfortunately, funding is so competitive that researchers have to sell their work as being very novel rather than verifying existing research.
At one time science was intended for the masses - that the Atlantic attributes this to "a simpler time" is also moronic as it was the intent of the authors, in fact all authors of the time, to write clearly and succinctly so that anyone could understand their work.
Science has never been "for the masses". Most concepts of any meaningful complexity are not going to be written at an 8th grade reading level. Issac Newton's Principia is certainly not written "for the masses" nor should it be expected to be dumbed down. As Einstein once put it, things should be made as simple as possible but no simpler.
You see that not only in the scientific papers of the time but also in the laws (the US Constitution).
The Constitution isn't really a law. It is a framework for the laws. It sets the boundaries that are fleshed out by the laws. The actual federal laws are the United States Code, the United States Reports and the Code of Federal Regulations. There also
You also see the same problem in laws today where laws are now tends of thousands of pages long. How is any one person (let alone a dedicated group) supposed to understand the law as written?
You are presupposing that it is a good thing that laws be so simple that a single person can understand and know all them. The reality of our society is that it is so complex that the laws governing it inevitably will be similarly complex. Make a simple law and there are going to be gaps in that law unless you add complexity to deal with the corner cases. To circle back, science has no obligation to be simple enough for "the masses" to comprehend any or all of it.