Slashdot Mirror
Is Good Scientific Journalism Possible?
scida sends in a link to his blog post exploring the question of whether, roughly speaking, science journalism is an impossible task. From the post: "I have spent the better half of the past six months trying to understand one thing: how can you effectively present primary scientific literature to the general public? Is this even possible? ... During the past few months, I have spent entire days locked up in my office, writing my first manuscript to be submitted to a peer reviewed scientific journal. While doing so, I have come to realize the following: details can change everything. There are a number of assumptions I have been forced to make while analyzing my data, many of which are critical for both my methodology and the development of few of my arguments. Why? Often, the information I require simply isn't available (the studies haven't been done, or the studies that exist are based on assumptions of their own). Now, can someone unfamiliar with a particular field, nay, a sub-discipline of that field, recognize these assumptions for what they are?"
There are a number of assumptions I have been forced to make while analyzing my data, many of which are critical for both my methodology and the development of few of my arguments. Why? Often, the information I require simply isn't available (the studies haven't been done, or the studies that exist are based on assumptions of their own).
Or worse yet for your readers, even the studies that do exist are locked behind a pay-to-read model of electronic publishing- so they can't tell assumption from fact. My suggestion: Make everything explicit. If you're forced to make an assumption, admit that it is an assumption up front and explain why you're making that assumption. If you are referencing a study, don't just link to the study or reference it in a bibliography, also copy the relevant portion of the data and explain the assumptions of that study AND it's relevance to your study.
Until the peer review system stops being broken by pay-to-read studies, I see no other option. And remember- to anybody outside of your special field of study, any assumptions at all will look like sloppy science based more on emotion than data.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
It's just rarely done. Most journalists would prefer to write fluffy hype-pieces, exaggerate claims (or allow exaggerated claims to be published), and otherwise print a lot of BS. Regular, honest science pieces just don't sell as well.
From the ark-tickle: I recently attended an interesting seminar, titlted, "The Informed Science Journalist: How Much Science Do You Need to Know?" led by UBC journalism Professor and Director of the School of Journalism, Stephen Ward. During the discussion, one theme in particular caught my attention: you don't have to have any background in science to write about science. Anyone with a keen interest for a field and sharp mind can write about anything, from philosophy to advanced string theory to climate modeling.
Is this true? Is a keen interest sufficient?
Well, it's a good starting place, but I think that "sharp mind" bit is more important... and judging by the quality of most science journalism I read, there's not a lot of 'em in the trade. I imagine deadline pressures aren't helping the quality of science reporting, either.
dear elitist:
within the mind of your average joe blow, you will find two shocking things:
1. amazing depths of stupidity
2. amazing heights of intellect
therefore, you sell sophisticated information to joe blow in the only way possible: straightforward. no watering down, no soft pedaling. then watch as what you deem ungraspable (that's the elitism in you) getting grasped notheless
dear insular academic:
not everything has to be explained. communication is not about impressing upon someone else's mind every little delicate detail. nor is it necessary to do that for joe blow to grasp important pieces of information
in fact, there is no value in science that cannot be communicated and explained. in the mind of the most advanced intellect can be the understanding and insight of the most amazing things. but if said great intellect can't open his or her mouth and explain it to someone else, in his head this great insight stays, and it dies with him, and becomes dust. in other words, dear insulated academic, i am saying your ability to communicate your research is actually more important than your ability to grasp every nuance of your own research
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
Can anyone not in the field explain what these are to the general public.
So as the author notes details are everything. And, at the same time, details such as assumptions mean the difference between science and bunk. I think that a good, scientifically trained, journalist could point these out if they were looking for them but whether that is what their bosses will let them do is another thing.
I have interacted with a number of journalists and have noted that, for many of them, there are two things that they need to do a good story: time to do research and an absence of biased assumptions. Ironically these are the same basic requirements to do good science. Even more ironically both are often denied them under pressure of tight deadlines and preexisting editorial biases. Not "liberal" or "conservative" biases but more the, if the elected officials say it it must be true or "there must be two equaly sides to every story" which trips up those comparing science to pseudoscience. Such gaps negatively affect reporting on all issues from science to war.
In many ways I think the question is really, can journalists do good journalism, and that is something I used to believe was true. Now, I'm not so sure.
Does anyone actually DO it anymore?
Not as far as I can tell.
Most of the stuff I've seen in the last 15 years or so shuffles between two categories.
1: Highly technical and scientifically accurate as far as information is available, but written in such a way that stereo instructions (in Japanese no less) are more intelligible to the common man.
2: Written to the understanding level of the common man (or slightly above if they don't use crayon), but woefully inaccurate and filled with assumptions and self-fulfilled hypotheses. Stuff that a generation ago, would have been laughed out of most scientific journals. An "in depth" study that winds up within a 15 percent confidence? Sorry, but 5% used to be considered shaky, but publishable. Lax standards and sensationalism now rule the roost.
Chas - The one, the only.
THANK GOD!!!
The scientists themselves know what the results are. But they have wildly exaggerated ideas about the practical implications (the principle of "anything I don't know how to do must be easy") and the stories are filtered through university PR offices who love to exaggerate even more.
What I'm listening to now on Pandora...
Publish more, think less. It's what everyone else does.
I'm not a Scientist, but here's my take.
The only things that can be responsibly reported are things that are well established. But they aren't news. And the irresponsible are then left to report on the news. So we need responsible journalists to report on Science. Which they can't do properly.
The outcome is what we have. Science news that is at best inaccurate. More often it's sensationalized and misleading.
If there's a solution to this I certainly don't see it.
-Peter
Really I think that the greater problem is with mass media grabbing a few points and stating them as 'truths'. Alot of the bad press that scientific studies have nowadays is from some major new organization blowing something out of proportion, having everyone accept that as 'truth', and then discovering a few years later that it was wrong/inaccurate/whatever. On a side note I agree with the previous poster that as far as getting the information across most people are not concerned with the details so much as the general idea. If then, in your article, you provide a sense of how 'certain' the information is most people will pick up on that as well. You can then get the idea across, as well as an impression of how many assumptions have been made or how fragile the results are. To most people that is better than a lot of detail that is out of context.
Think about it - journalism is generally the result of a person in possession of what amounts to half of an English degree with a few communications and maybe an ethics class thrown in trying to explain complex issues from geographical and topical areas with which, odds are, the journalist has no training whatsoever.
Ask any professional - legal, medical, IT, academic, business, etc. - if journalists adequately explain the complexities of their work. I guarantee you that the answer will be a resounding "no" on almost all counts. Part of this is the nature of the media - the need to get stories that sound interesting (and life is often boring and must be juiced up) out very quickly - more quickly than the other guy. The other is simply the result of the fact that the person writing the article simply has no idea what he's talking about - they taught him about dangling participles and conflict of interest, not what a summary judgment, kidney, floating point or profit margin is.
Truly great journalism is written by experts in the field they are writing about. Many often complain that these experts are "biased", but I think it is plain that most journalists are as well - and most media outside of the U.S. recognizes this. While these experts may have biases, they also have one thing journalists lack entirely - intimate knowledge of the subject at hand. While one cannot necessarily accept what these experts says at face value, one can also be sure that the writer has done more in-depth research than browse the Wikipedia article for a few minutes or ask some other, often unnamed "expert" whose bias bleeds into the article but often without attribution, leaving the reader to assume that the idea of one expert is a consensus opinion.
well, there are a few issues with your suggestion - well at least things I think would be issues
1) copyright - how do you copy relevant portions of a publication without getting caught up in this nightmare? could you imagine the price of journals if this were required? There are now plenty of journals that allow you to read content for free.
2) not everything can be made explicit. There are many aspects of any scientific field that are "fundamental" and would be tedious to have to re-explain everytime
3) putting that much data into an article may make it too large and unwieldy to read. If people have issues with something, they can pay or do whatever else it takes.
4) to state that any assumption will look sloppy may be true; however, unless you are willing to conduct many more experiments prior to leading up to whatever your studying, wouldn't you be forced to make some assumptions. sometimes - esp for a small study - you are willing to leave certain things unanswered so you can publish and get the money that you may need to prove your assumptions were true to begin with. As long as disclaimers are made in your original paper stating further study needs to be done, this may not be an issue
When all else fails, try.
Here's another vote for Science News. A little dry perhaps for the average person, but perfect for the enthusiastic amateur.
From the original post:
Now, can someone unfamiliar with a particular field, nay, a sub-discipline of that field, recognize these assumptions for what they are?"
One of the things that a good scientific journalist does is get opinions from the other big players in the field that the new paper is being published in. It's often the most interesting part of the article. When they say, "It's an interesting paper, but I won't be convinced until I see more data on..." we as the general public can get a better idea of just how far along the research really is.
# (/.);;
- : float -> float -> float =
Or worse yet for your readers, even the studies that do exist are locked behind a pay-to-read model of electronic publishing- so they can't tell assumption from fact. My suggestion: Make everything explicit.
Peer reviewed is one thing, journalism is another. In peer review literature you have to make everything explicit and checkable. Journalism comes from judgment and is based on opinion. If you discover something and it has far reaching implications, you need to present your findings to the general public as an opinion. People need that opinion, because they only have an average of 15 minutes a day for news. Ideally, all of the information will be accessible so that anyone, including your peers, can dig further. Electronic publication promises to reduce barriers to knowledge but it's not a time machine.
Recently, Slashdot reported that the German government was paying scientists to edit Wikipedia. That is nearly ideal because it can link to sources directly and the contents can be edited to eliminate confusion in a way that traditional news never could.
DMCA, Hollings, Palladium. What might have sounded like paranoia is now common sense.
What you're proposing about assumptions is unproductive and impossible. In my field (semiconductor lasers and photonics), if every assumption was expounded upon, each paper would be thousands of pages long. I suspect this is true for all but the newest of fields. Journal articles need to assume that the reader already has a general familiarity with the material, because the target audience has this knowledge. For example, if I am writing a paper, I am not going to explain undergraduate quantum mechanics, solid state physics, or electromagnetics. Not only would that reflect poorly on me (as it would seem patronizing to the typical reader), but it would also be a poor use of my time.
Fortunately, there are ways that you can get the background required to understand journal papers. The most obvious way is to attend a university and study the material. Yes, it costs money and time, but that's the price you pay. If that doesn't work for you, there is plenty of reading material available. You can start by looking at undergraduate textbooks at your local college library. If those don't help you, move on to graduate textbooks. If you need more, then you can look at course graduate course lecture notes (many of which you can find for free online). If you want the most direct background, review articles are the way to go. If even those don't help, look up the author's past papers. If a person outside a field wants to understand a paper, then it is that person's responsibility to read the background material.
Regarding "pay-to-read studies," the system is not as broken as you make it out to be. Practically everyone who wants to have access to a journal can get it. Universities and research-oriented companies subscribe, so all you have to do is walk into a library and peruse them yourself.
Having lived through the period in question, and having been a voracious reader since about 1955 (with access to literature from before that), a regular listener to radio since before transistor sets, and an intermittent viewer of television since the McCarthy era, I can safely make assertions about the quality and quantity of science writing during that period.
It started going downhill about the "summer of love", with the demonization of technology and the popularization of the Malthusian dystopia, and was on a continuous slide until the advent of computers and networking (in the form of netnews, conferencing systems on university timesharing services, and BBSes on the early CP/M and Apple machines) created a new venue for the technophile culture.
Yes there was always some good stuff to be had. (For instance: in Scientific American - until they were bought out, dumbed down, and rendered PC by their current publisher.) But in the mainstream media it has been progressively fewer and farther between, even into the current decade.
That's why we spend our time ON the net instead of in front of the Boob Tube, isn't it? B-)
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
But unless you do, how do you know that maybe biology couldn't make your discovery better, or lead your research into new directions? You don't- because you've specialized too much- and so have everybody else, the guy writing the paper on the transformation chemistry of DNA isn't stooping to link to papers on undergraduate biology either. But that can be rectified with the web- EVERYTHING can be available to EVERYBODY without worrying about cost. We don't need to limit science to specialty anymore.
The problem isn't one of physical resources, it's one of temporal and mental resources. Just as you say, practically everything I would need to know about biology, I can find on the web. Textbooks are easily found on P2P networks, course notes are readily available, and I can access any journal I want through my university. (Of course, this is ignoring all laboratory experience, but that is a whole other issue.) Why, then, am I not a biology expert? Because every person has a finite capacity for information, and because information takes a finite amount of time to absorb, and I have a finite time on this planet. I choose not to be proficient in biology so that I can be more proficient in my area, and because I'm not all that interested in biology.
Even if I was interested in learning about transformation chemistry of DNA, I wouldn't expect to find the basics of biology in the paper. You seem to be arguing that journal articles should contain nicely-packaged summaries of the field, but I think you're missing my point. That's not the goal of journal articles. Nicely-packaged summaries of mature fields already exist: they're called textbooks! Likewise, nicely-packaged summaries of younger fields also exist in the form of review articles. The purpose of journal articles is to inform and enlighten people who are already experts (or, in the case of grad students such as myself, are on their way to becoming experts).
You wouldn't expect a high school algebra textbook to cover the basics of counting, would you? Why should you expect someone with a Ph.D., who has studied their field for at least eight years, to explain the basics of their field? It's the same difference in number of years in school!
It's not insane. Just because you develop software for a living, do you think you could write better magazine articles about software development than somebody who does that for a living? Or better novels about software development? Or better plays etc.? If so, then you should probably get out there and do it, because it would be a really easy way for you to make some extra cash.
People always want to make this assumption: I have a keyboard, I can write. I send emails; I write every day! But they never seem to make this assumption about race car driving (I own a car!) or telecommunications engineering (I own a phone!). Why make it about writing? Other people have computers. Does that mean they can program?
Writing is a skill that a lot of people work very hard at. When what you want to do is write something, it's more important to have that skill than to be the master of the thing you're writing about. You don't have to be a mafia hitman to write about the mob. You don't have to be a surgeon to write about medicine. You don't have to be a football player to write for Sports Illustrated. You don't have to be a programmer to write for a computer magazine. If you think you can do it better than those who do, then by all means give it a shot. Editors are often quite desperate for good writers.
William Zinsser has a great anecdote near the beginning of his book, "On Writing." In it, he talks about how he was sitting on a panel with a surgeon named Dr. Brock who had written a couple of magazine articles, and how Dr. Brock had waxed on and on about how fun and "spiritual" and whatever else it was to be "a writer." Zinsser was taken aback, because it had never once occurred to him -- in all the long, solitary years of writing and rewriting and refining his work -- that writing could be easy. "Maybe I should take up surgery on the side," he said.
Breakfast served all day!
Is there a *market* for good scientific journalism? I don't think so, based on a few decades of observing the journalism market and the public's interest in the topic.
I wish it weren't so.
"Oh, poor scientific writer, needing to actually explain "fundamentals" because it's TEDIOUS. I suggest that a free-to-read model could replace such explanations with mere hyperlinks"
But then you are WRONG! Do you think this kind of hyperlinks were invented in the Internet days? Look at *any* scientific paper: they are FULL of hyperlinks. Each time you see "this happens to be A[1]" or "we already know that to be true[2]", that's an hyperlink. At the end of the paper you will find quite some references (usually a *lot* of references) that links the current paper to the immediate antecedents. Those in turn provide new citations to other references and only very few seminal papers happen to be more referenced that the references they link to.
So all the information is already there, but do you know what? Even then, unless you are already an expert on the matter it still seems to be archaic Chinese to you (unless you are an archaic Chinese expert yourself in which case it will seem to be Quantic Chromodynamics no less). It is not that the information is not already "gettable", but how many information we can grasp in just one bit. I usually offer this example: I'm absolutly negated about dancing, so I admire those dancers from TV programs: each day the team offers three/four dancing numbers on their program "how the heck they manage to learn all those movents without failure?". Till I remember they are professional dancers and that means that they do not learn their movents like I'd do: "the left foot goes 45 degrees to the right then the left hand follows, two steps to the right, then I find the girl coming to me, I move my arms towards them, but don't forget to gracily elevate my hips..." they just need to memorize higher level abstractions: we start in first position, then we go for an "eigth lace" then take her in third, then rondó... Because they are professionals they already have a basis that allow them to grasp complex concepts by just looking for the "big landscape": the details are already known and taken for granted. Well, scientific papers are just the same and without all the "taken for granted" any ten pages papers would become a 1000 pages book and no one that already knows the 1000 pages book would understand the 10 pages paper anyway.
You just try to understand Einstein's paper first published on "Annalen der physiks" titled "on the electrodynamics of moving bodies" without a firm understanding on both newtonian theory of movement and maxwellian ecuations: you will see it doesn't matter it was published by 1905, when your "copyright overlords" were not so strong, everything was published and proper citations were both accesible and properly in place. And please remember it's not even a very hard paper; currently any minimally cute 16 year old boy should understand its maths without many problems. But still, you either already have the maths and the underlying theories already grasped or no matter how many citations or how free, the article will still seem Chinese to you (unless you are Chinese, in which case it will seem archaic Saxon to you).
"The only place this is an issue for is for those who believe that science leads to a definition of reality"
I must say "bullshit". Science *is* our definition of reality. It can be controversial how much our definition of reality pairs the "real reality" or if there's in fact a "real reality", but there's no doubt science *is* our definition of reality. Only this assumption allows even you to not think that the seven lane bridge you cross to go to job is not suspended over the river by any magic force.
In my opinion, Scientific American took a nose dive a few years ago when it decided to dumb itself down, maybe to compete with Popular Science and Discover magazines. It seems to be very easy to read now, but short on information. I used to read it all the time, and learned a lot, but it seems more of a waste of time now. Sure, some of the articles used to be hard to read, but that's the price you sometimes need to pay in order to convey novel (to the reader) information.
Scientific journalism isn't a special case. In any writing endevour, but especially in journalism, you have to make assumptions about what your readers know, make choices about how much detail to provide, and generally be able to summarize complex stories. The choices writers make when determining how to address those issues affect what readers understand.
But it's in no way restricted to science. Political journalist, auto writers, even the folks writing for the 10:00 news--they all gloss over details. Sometimesin ways that are (intentionally or not) misleading. Ask anyone about news reports that cover their area of expertise and they'll tell you how often the writers get it wrong.
Guess I'm saying you're not alone
Interested in a Flash-based MAME front end? Visit mame.danzbb.com
Don't use inaccessible language if your target audience is the general public, for starters. The language used in scientific papers is needlessly complex. Fortunately, it's also very formulaic, so once you've read a few papers, you can immediately figure out what the author(s) are really doing.
.9, which supports our hypothesis at alpha=.05."
.9 is so great, or what alpha=.05 is supposed to mean.
The common response when I present a paper to a member of my family is "well, I almost understand the title", even if nothing particularly tricky is going on in the paper. Maybe they'd get it if they read a few papers, but the language ensures that they won't even make the attempt. And my family tends to be more educated and more open to new ideas than the general public.
So the first step is to eliminate the jargon unless it's actually necessary. I know that writing that way is more precise, but it is also harder to read.
Some of the discussion of background is interesting to other scientists but not to a lay audience, as well. The way to write an accessible article is to start from an accessible overview, going into details as necessary after clearly presenting the main idea. That is what abstracts are supposed to do. Also, laypeople do not need to understand all of the methodologies underlying the analysis; they're not performing work in the field and it's unlikely that they will be capable of critiquing the research, so they simply need to know the impact of the results.
Here's an example:
"We analyzed the texture of mammograms and found that certain patterns correlate with an x% increased risk of breast cancer".
is accessible. Your mammogram looks like this, you have a higher risk of cancer. Simple. People get it. If I had to summarize this research results in one sentence, I'd do it that way.
In a scientific paper, it would sound like this:
"We performed Principal Component Analysis (PCA) to extract statistically uncorrelated discriminative texture features from the biomedical images. PCA can be performed in the following manner: Let X be a collection of feature vectors... (etc.)
We then performed k-nearest neighbor classification on the extracted feature vectors. Classification accuracy is given by the following ROC curve: (ROC curve that no layperson would have a hope of understanding). The area under the curve was
Etc.
Scientists can understand that. Laypeople cannot. I essentially just gave the reader the conclusion in that last sentence (plus associated figure of ROC curve), but it would fly over the head of anyone who didn't understand what an ROC curve is, why the area being
If you're talking about publishing in, say, Scientific American, you're talking about a step or two above the general public ("scientifically aware" is how I'd describe this group), so this may not necessarily apply. But you probably can't discuss any highly specialized knowledge in such things and expect the majority of readers to get it.
> I'm a scientist too (well, a mathematician). Let me tell you: if you can explain what you
> did in three sentences then either you work in an extremely new field (analytical chemistry
> in the 18th century; discrete mathematics in the 1930s), or you are lying to your
> audience.
I work in agricultural science, not exactly a new field.
"I'm currently working on a physically based model of how pesticides can move with water through the soil and reach the drain pipes. The idea behind this research is that once we have this model, we can use measurements taken from the drain pipes to help estimate how many pesticides might reach the depths where we extract our drinking water."
That's not a lie, and can be used as a basis to fill on details in the unlikely case the listener is interested.
Not all scientists work on number theory or quantum physics, most of us work on stuff that is quite more down to earth.
It's by now pretty clear that you've never written a paper...
Lots of people do put their papers on one or another free electronic resource (arxiv, homepage, whatever) as well as submitting to a journal. So, you can avoid most of your precious 'economic problems' that way, and if you ever bother to be minimally polite you can probably find some friendly university professor to give you access to his library anyway.
It's a nice idea that a scientist could just add a few hyperlinks and magically his paper would become intelligible to everyone, but it's just not true. I, for example, work in combinatorics, which is a fairly basic area of mathematics, where you don't really need to know too much to get started.
Here's a paper of mine:
http://www.cdam.lse.ac.uk/Reports/Files/cdam-2006-10.pdf
which in turn is a very simple paper; it doesn't reference all that much because it doesn't need to, and those things it does reference are mainly just to mention people who've done similar things. I'd guess that if you spent enough time you probably could understand that: but you'd need to start by understanding what I mean by a graph and getting a bit of basic graph theory. Now, I could've provided a hyperlink to Diestel's book (which is available online for free) to explain that - but then I'd have to keep updating it whenever the link changed, et cetera. And when I write a more complicated paper, I do not want to write an undergraduate course to go with it - so I am not going to try to explain all the stuff that everyone in the field already knows. If I mention the blow-up lemma in passing, and you want to know about it, read the paper describing it. When you discover you need to read a bunch of other papers to understand, go read those. Since they won't be all that helpful, you'll probably need to find a book on probabilistic graph theory. Which you can probably get hold of for free if you want to: but it will take you a few months to understand enough to know what the blow-up lemma is. That's nothing to do with economics, it's to do with how fast you can absorb information.
Put it another way: if you really want a helpful link to let anyone understand a scientific paper, then probably the most helpful one is a link to a nearby university's lecture list. They don't generally check that the people in the audience are registered students. And yes, it will take you a few years.