Slashdot Mirror
The Scientific Paper Is Obsolete (theatlantic.com)
James Somers, writing for The Atlantic: The scientific paper -- the actual form of it -- was one of the enabling inventions of modernity. Before it was developed in the 1600s, results were communicated privately in letters, ephemerally in lectures, or all at once in books. There was no public forum for incremental advances. By making room for reports of single experiments or minor technical advances, journals made the chaos of science accretive. Scientists from that point forward became like the social insects: They made their progress steadily, as a buzzing mass.
The earliest papers were in some ways more readable than papers are today. They were less specialized, more direct, shorter, and far less formal. Calculus had only just been invented. Entire data sets could fit in a table on a single page. What little "computation" contributed to the results was done by hand and could be verified in the same way.
The more sophisticated science becomes, the harder it is to communicate results. Papers today are longer than ever and full of jargon and symbols. They depend on chains of computer programs that generate data, and clean up data, and plot data, and run statistical models on data. These programs tend to be both so sloppily written and so central to the results that it's contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you've actually discovered, clearly enough that someone else can discover it for themselves.
The earliest papers were in some ways more readable than papers are today. They were less specialized, more direct, shorter, and far less formal. Calculus had only just been invented. Entire data sets could fit in a table on a single page. What little "computation" contributed to the results was done by hand and could be verified in the same way.
The more sophisticated science becomes, the harder it is to communicate results. Papers today are longer than ever and full of jargon and symbols. They depend on chains of computer programs that generate data, and clean up data, and plot data, and run statistical models on data. These programs tend to be both so sloppily written and so central to the results that it's contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you've actually discovered, clearly enough that someone else can discover it for themselves.
As a scientist who has published papers in peer-reviewed journals, I strongly disagree. I don't care for many aspects of the publication process or how academia works, but the scientific paper isn't obsolete.
Regarding the use of software in creating results, it is definitely a problem when that software is difficult to use or isn't available at all. The same goes for data sets, many of which aren't released publicly for a variety of reasons. These are issues that need to be addressed.
In my own work, I try to release the software used to perform my analysis under the GPLv3. I also try to include adequate documentation to allow the work to be reproducible. I also support making data publicly available, but sometimes the volume of data sets makes it prohibitive to redistribute the work. The best option in that case is to provide detailed instructions on how to generate the data set. But sometimes it's even worse, such as when the research requires using a closed source program with a license that prohibits redistributing the output. The license makes it relatively difficult to obtain the software for anyone outside of the US government or academia, unless they pay for a commercial license. There isn't a comparable piece of software, but the university that licenses the software uses restrictive licensing requirements to increase their revenue.
To the extent that it's possible, scientists receiving grant funding ought to release their software as free and open source software. The data sets should be released or detailed instructions should be included to allow others to generate the same data set.
Papers can be difficult to read, but there are at least some steps in the right direction. One of the encouraging changes is the use of more first person in scientific papers and less passive voice. The formal tone is being being phased out in favor of readability. I wholeheartedly support this because the subject matter is difficult enough to understand without awkward sentence structures. Peer-reviewed papers also provide some level of quality control for research, though not to the extent of reproducibility.
Scientific papers still have a place, because they're still the best opportunity for scientists to present the key points of their results and describe their methods in detail. Simply releasing software and data sets is not enough. Those also aren't peer-reviewed. The bigger issue is that there just isn't a lot of funding to ensure that results are reproducible. Due to funding limits, there just isn't a lot of effort to reproduce all but the most surprising results. It would be great if funding agencies like NSF would allocate more funds for ensuring the reproducibility of existing results. Unfortunately, funding is so competitive that researchers have to sell their work as being very novel rather than verifying existing research.
More seriously, we are in a world where very few people can understand how reality works.
Anyone with a high school level of scientific background can understand Newtonian mechanics, most people have trouble with special relativity but with the right mindset, it is not that hard. General relativity and quantum mechanics pretty much require years of specialized studies, and these are what form the basis of reality as we know it today. Mastery in these fields are a requirement in order to go further.
As for our understanding of nature, we know the physics of throwing rocks very well, no need to do more research about that. The unsolved problems involve crazy accurate measurements, scales that are well beyond human, or complicated interactions.
How many scientists engaged in climate science research understand even the basics of the Carbon Cycle?
It is more or less generally accepted in the climate science community that the 20th century increase in atmospheric CO2 accounts for half of humanity's emissions of CO2, with the remaining half absorbed by sinks that are only partially understood. There is evidence from high-precision analytical chemistry methods for quantifying atmospheric oxygen available for only the last decades that about half of the net "sunk" CO2 is taken up by net photosynthesis over respiration whereas the remaining sink must be inorganic where free oxygen is not released in exchange for sequestered CO2.
It is broadly reasoned that emissions of CO2 are changing the carbon-isotope profile of the atmosphere -- what is called the Suess Effect after R. Revelle and H. E. Suess (1957) Carbon Dioxide Exchange Between Atmosphere and Ocean and the Question of an Increase of Atmospheric CO, during the Past Decades, Tellus IX (1) pp. 18-27. This dilution of carbon isotopes in the atmosphere from combustion of fuel with a "fossil" isotope profile differing from the atmospheric baseline was throwing off C-14 carbon dating until it was recognized data methods were accordingly corrected.
The Suess Effect dilution of carbon isotopes, however, is considerable less than expected from a simple addition of combusted carbon with the fossil isotope profile. The carbon capacity of the ocean is 50 times that of the atmosphere, so why doesn't nearly all of the emitted CO2 isn't absorbed into the ocean. The rapid extinction of radioactive C-14 after the 60's Nuclear Atmospheric Test Ban Treaty suggests this should be the case.
The linear Henry's Law for solubility of gas in liquid would require that the dilution of carbon isotopes should be in direct measure with the emitted CO2, but Revelle's earlier work posited a chemical buffer system, where the mineral stew that is ocean water binds absorbed CO2 into what are called "soluble inorganic carbonates." Equilibrium in chemical reactions is non-linear and follows product law in the concentrations of the reagents. Physical chemists regard Revelle's buffer system to follow a 10th power law in the concentration of atmospheric CO2. This means that it takes a 10-fold increase in atmospheric CO2 to effect a 1-fold increase in CO2 in the ocean buffer system.
The ocean is vast, and even with the Revelle Factor of 10, most of the CO2 emitted by humanity should have disappeared into the ocean. Revelle and Suess in 1957 speculated on possible large natural sources of CO2 emission to balance this out. Since then, it is a scientific consensus that it takes a long time for the deep ocean to "turn over" by natural circulation and the combination of the Revelle buffer with a "compartmental" ocean model accounts for what is observed without that large, unknown natural source. But the deep ocean is difficult to measure, and the modelling is hand-wavy, at least in comparison to the multiple sigmas required to discover a new subatomic particle, although the atmospheric oxygen measures suggest a bound on how much CO2 is absorbed in the ocean.
But not just in the annual seasonal fluctuation but also year-to-year and over longer time scales, the variation in net CO2 increase in the atmosphere is large in comparison to the human contribution, suggesting variation in the exchanges of CO2 with the biosphere, the flows of which are known to be large compared to human-caused emissions. This variation also correlates with atmospheric temperature data. NOAA Carbon Cycle maven Pieter Tams argues this variation to result from changes in the rapid rotting of leaf litter in the tropical rain forests and is only short term. There is a body of literature emerging that rising global temperature are having a multi-decadal effect on increased CO2 driven out of temperate region soils, a potentially dangerous positive feedback leading to a runaway greenhouse.
Actually, you cannot have it