A New Kind of Science Collaboration

Scientific American is running a major article on Science 2.0, or the use of Web 2.0 applications and techniques by scientists to collaborate and publish in new ways. "Under [the] radically transparent 'open notebook' approach, everything goes online: experimental protocols, successful outcomes, failed attempts, even discussions of papers being prepared for publication... The time stamps on every entry not only establish priority but allow anyone to track the contributions of every person, even in a large collaboration." One project profiled is MIT's OpenWetWare, launched in 2005. The wiki-based project now encompasses more than 6,100 Web pages edited by 3,000 registered users. Last year the NSF awarded OpenWetWare a 5-year grant to "transform the platform into a self-sustaining community independent of its current base at MIT... the grant will also support creation of a generic version of OpenWetWare that other research communities can use." The article also gives air time to Science 2.0 skeptics. "It's so antithetical to the way scientists are trained," one Duke University geneticist said, though he eventually became a convert.

Comments

Isn't it just....

[Fluffeh]

Like what the internet was originally developed for by those physics chaps - before all the advertisers found out they could make money off it?

It's almost like going back in time to the future to go back in time.

The Cathedral And The Bazaar, anyone?

[iris-n]

As a scientist, I have to say that this model is utterly beneficial. One of the greatest problems we run when trying to replicate experiments is that the dirty lab details are (intentionally or not) omitted from the fine print articles, making us lose quite a time figuring them out. Obviously it would disappear if such openness became the standard.

Although the idea of making science collaboratively is as old as science itself, it merits having a working model (just don't patent it!) and standing the principle quite out.

Oh and I *hate* this marketing way of naming everything like software versions.

Re:The Cathedral And The Bazaar, anyone?

[moosesocks]

That's intentional for a variety of reasons.

1) Journal articles are generally supposed to be concise explanations of your research findings. Not a thorough documentation of your lab procedures. I agree that most papers should go into more depth than they actually do, although omitting a detailed explanation of your experimental procedures is perfectly acceptable because:

2) Eliminating the nitty-gritty details forces you to create your own experimental procedure to verify the results. This greatly helps when attempting to find/isolate flaws in the original researcher's findings that may be due to faults in his procedures. A decent paper should at the very least provide a "roadmap" for repeating the experiment.

3) If this all fails, phone up the original author of the paper, and tell him that you're having trouble repeating 'X' part of his experiment. More likely than not, he'll be flattered that you've taken an interest in his research, and will be happy to assist you, as external verification will greatly increase his reputation and validity of the paper. Conversely, he really doesn't want somebody to publish a paper stating that his findings were not reproducible.

Tradeoffs

[LwPhD]

As a professional academic scientist who does both experimental and computational projects, I think there is a very good argument that for many types of science, this sort of approach will fail miserably, even after the technology to take care of it is completely mature. For example, take a genomics project of moderate complexity and moderately broad interest. Such a project may not be SO important or SO interesting or SO difficult as to require an entire consortium of scientists to complete. However, it may be sufficiently complex that it will require coordinated experiments that will cost into the 10s of thousands of dollars and require more than a man year of work to complete. In such cases, it is almost always best for a single lab to do all experiments (for quality control reasons). If a lab were to complete all experiments at great expensive (for a regular lab), why would they then give up that data immediately for others to work on? Sure, it would be quicker, and more insights would come faster. But to be perfectly honest, this would probably decrease the ability for that lab to promote its members by getting priority with good publications. Currently (at least in genomics) there is no way to reward a scientist through contribution to the community in this way. Now, if a way to award credit for this type of work were to be created that allowed:

• students to apply such work to graduation requirements;
• postdocs to apply the work to faculty job applications;
• junior faculty to apply their contributions to tenure review;

then I think this could be a viable system. However, in academia, this is very unlikely for a very long time. It is amazing and wonderful that journals like PLoS are trending in that direction. And it is even better that MIT is pushing from the University side of the equation. But until Science 2.0 methods are explicitly taken into the incentive system of academic review, this type of approach is a non-starter for expensive, time consuming, experimental science. On the other hand, I could see this sort of approach being very useful for computational science. With much data already freely available, it is usually super quick to get certain types of data analyses done, though quality is frequently questionable. (Go to a journal club on a bioinformatics paper if you want hear academic work seriously shredded.) However, this kind of work responds rapidly to the sort of peer review described in TFA. So, perhaps science could start with the bioinformatics model and figure out how to meaningful track credit in that arena before applying the model to experimental work?

Re:Tradeoffs

[JKelly555]

One fact of life in "open science" discussions is that there are going to be people who think it doesn't make sense in their (very competitive) field to be open early about their work. However, there are many, many scientists whose principle problem isn't having their work stolen - it's that no one notices their work. This is especially true among younger scientists still making a name for themselves or folks in smaller fields.

I think there is already significant incentive for young scientists to publicize what they are doing as openly and early as possible. This open group will either be 'scooped' out of existence, or will be more successful thanks to all the unintended benefits of making your work accessible early. We really won't know which it is until we run the experiment, but you can probably guess where I lie on this one.

I think your points about changing the way we award credit are correct on some time scale, but the first group of people who open up will do it totally unprotected. If we need elaborate infrastructure and a change in the scientific reward structure before people open up then we're dead in the water. At OpenWetWare we're trying to create a community that values openness as early as possible in the research process. We support this community by providing simple web tools to make the process of sharing info as easy as possible.

I think it is early days for a lot of this stuff, but from the perspective of a PhD student being open early is a major win. A very small fraction of PhD projects are even "scoop-worthy", and a very large fraction spend time down dead-ends that could have been avoided by the right person noticing a mistake / making a suggestion. Openness wins in this case, IMO.

You should consider joining the site and sharing what you're working on -- you might be surprised that it pays to be more open.

Re:Tradeoffs

[LwPhD]

I have a great deal of respect for this approach to science, but interestingly not for any of the reasons you cite.

I believe that broadening collaboration (expanding collective knowledge) and rapid development (increasing the "effective population size" of a meme pool to use a popgen analogy) are much more compelling arguments for adopting this type of science than are problems publicizing work. If the "Science 2.0" becomes the norm, it is anybody's guess if the signal (work of students wanting their science to be heard) will increase faster than the noise (work of competing students). For early adopters in appropriate fields, they may very well catapult themselves to fame through avenues previously unavailable. However, this seems to be only possible during the transition to "Science 2.0". After attaining equilibrium, I suspect the new system will provide analogous challenges to being heard above everyone else. The advantage of the new system won't be promotion of science of an individual. I think instead (in an ideal world), it will be a more efficient mechanism of promoting good work, faster, and in a more egalitarian way.

I also think that any work worth doing is worth rewarding and/or scooping. Just because a young scientist may not have a thesis or project "worthy" of publishing in Science or Nature, doesn't mean other scientists in similar situations wouldn't be interested in reading that work or getting credit for it. A publication in a medium to low impact factor journal can count towards graduation in most places, and therefore is VERY valuable to an individual student. And most of these journals have ambiguous policies with regard to novelty and its intersection with "Science 2.0". As a result, even with "unworthy" publications, there is significant risk to going open source if there is ambiguity as to how the work will eventually be disseminated.

I think it is essential that the incentive and reward systems for science should definitely change to incorporate this framework. Science can and has changed in the very recent past, and I don't think requiring this sort of organizational change to promote open source science is a deal-breaker at all.

This is another train of thought that doesn't feed directly into my arguments, but here goes. I disagree that going completely open-source science for most projects at all stages of development is ultimately healthy for science. I would tend to think that this would create one huge echo-chamber that is extremely efficient at amplifying its own dogma. I very much think that the interplay between multiple schools of thought is very useful, as they tend to have different biases, and explore different issues frequently ignored by other schools. This is no argument for not using open-source collaboration. However, I do think that this sort of collaboration does break down so many barriers to migration of ideas that it has the potential to create one large "panmictic" population of ideas, leading to the echo-chamber trap. I suspect that a model where everyone instantly accesses everyone else's ideas isn't optimal for good science, as anyone who works in science knows that successful ideas frequently aren't the best ideas. (I mean successful in the popularity sense.) They frequently are the "coolest" fads or the incremental advances that flatter "poobahs" of the field without challenging dogma. Amplifying these ideas will impose a cost. I also think that, in such an environment, it might even be more difficult to promote science that challenges the conventional wisdom. If your new idea is immediately challenged by the community before you've had time to develop it fully, would you not be less likely to pursue it?

The benefits of "Science 2.0" are legion and probably outnumber the costs. But I think adopting and promoting it can be done cautiously. After all, even if it takes 10-20 years to fully integrate these advances, we've really only lost the blink of an eye. And when I'm advising students, I'll certainl

A new model

[stox]

I wonder if this could lead to a new model in science, a split, those who produce the data and those who digest it. To a small extent, this is already true in the HEP community. It could lead to an an exciting new era in research.

Re:Share and share alike

[Ihmhi]

Well, wouldn't the information on a scientific wiki/collaboration be covered under a GPL? That would prevent someone from using your contributions for profit.

Thorough analysis of a page can clearly show who did what. A scientist may have made only one edit, but that edit may have been the missing component of a crucial piece of research. The records would clearly show this (as anyone who has ever checked through the backlogs of a wiki article can attest to.)

I concur with you the hard sell: scientists would, in effect, be giving up many things they love: credit, funding (which, for many scientists, is their livelihood), awards nominations, etc.

After all, only three people can be nominated for a Nobel Prize, not three hundred.

Re:Share and share alike

[NetSettler]

Well, wouldn't the information on a scientific wiki/collaboration be covered under a GPL? That would prevent someone from using your contributions for profit.

Copyright protects the form of an expression, not the content of the expression. You can't copyright an idea or a fact. You can only copyright the words you used to express it.

If you have a brilliant idea, patenting can get you some rights, but patenting rewards the first to submit an application. You could propose that the global science/wiki-thing should be the patent office, and propose that the first to edit an idea in would always get the money. But the next day the wiki would be full of random junk put there by speculators, and you'd be sued for removing a single word of it. So since it wouldn't work for this to be the patent office, you'd either still have the patent office (and someone watching for edits would be submitting patent applications) or else you'd have to get rid of patents as an obsolete thing--eliminating another source of funding.

By the way, I absolutely don't believe the goal should be to keep people from profiting. I'm totally for the idea of profit. I just think that the people who contribute the work must be among those who profit!

Even if copyright would work for this, the GPL is a terrible model. In practice, you're forbidden from charging if you built your work on anyone else's--and it's just plain too administratively complicated to actually pay all those underlying people. So everyone throws up their hands and just gives it all away and that's that. I'm not saying it's impossible to make money under GPL, I'm just saying I doubt any claim that the contributors will be routinely well taken care of.

I don't want Scientists to have to have jobs as cooks, janitors, etc. just to earn a living wage. I want them to spend as much of their time doing what they do best, and I want us to reward them for it directly, not make them have to spend their free time (or even their full time) chasing money so they can squeeze in a little time doing Science if there's any time left at the end of chasing money.

Nor do I think it would be good for them to resort to "applied science" for their money. Science and its applications are different things. Basic research is not the same as product development, and the two should not be confused.

I don't even think it would be bad to have a few millionaire scientists. Money runs the world, and no amount of giving stuff away will fix that. The people who are a threat to Science have plenty of money; if Science doesn't find ways to enrich some of its own, it won't have the power to hold the forces of anti-Science at bay.

Thorough analysis of a page can clearly show who did what. A scientist may have made only one edit, but that edit may have been the missing component of a crucial piece of research. The records would clearly show this (as anyone who has ever checked through the backlogs of a wiki article can attest to).

True. But no one would care. Once the information was out, people would argue it wasn't valuable, or that it was obvious. Or that they were about to come out with the same thing. People pay for what is scarce, and the moment you publish something world-wide, it is not scarce.

I'm not saying I want scientific research to be scarce. I'm saying I want scientists not to be scarce. And asking them to give up any financial incentive for doing their work doesn't sound like a recipe for motivating people to contribute to science.

If you're imagining a promise up-front that you'd be paid if you just contributed something, even something "important", I'd like to see the wording of that promise before I'd bother to discuss it, because I doubt any such promise is forthcoming.

Re:Credit

[Mathinker]

Reality is a little more complex than that. Even though the following story doesn't match the scenario which you are talking about, where someone steals the current work of an active academic, I think it brings up other issues which you ignore.

I know of a case where a Russian mathematician published an original result in Russian but then left academia and his result got little publicity, except in Russia. Many years later, a German mathematician (who is known to be able to read Russian) "rediscovered" and republished the first mathematician's work without giving him credit (obviously, since the second mathematician really did not add anything of significance, and in fact, didn't even change the original notation much). The mathematical discovery in question has therefore become much more well-known in the mathematics world (since the second mathematician is in academia, so he is constantly lecturing about it in conferences, and such).

The first mathematician (disclaimer: I know him personally and heard the story from him) is of course very upset about all of this, but claims to actually have very little recourse, because he is no longer an academic, and therefore has practically zero political power in the academic circles involved. He still has a few friends here and there, and found out about the story from one of them.

Now from the point of view of kiddie good/evil, it's clear that the second mathematician has sided with the "dark side" (if we believe the first mathematician's opinion, that the second one is merely stealing his results). But from a different point of view, by stealing the first mathematician's work and publicizing it (as his own) he may be doing society a favor by enabling a possibly significant result to gain more recognition (i.e., that might be worth more to society than the damage caused to society by the second mathematician getting more grant money, etc., than he actually deserves).

Re:Credit

[LingNoi]

What nonsense, they'd have to describe how they got to B in the first place at which point you could see that they've taken your work.

The "old" system is just as bad for theft anyway so it can't get any worse. You make it sound like the end of the world.

Re:Share and share alike

[Grishnakh]

If you have a brilliant idea, patenting can get you some rights, but patenting rewards the first to submit an application. You could propose that the global science/wiki-thing should be the patent office, and propose that the first to edit an idea in would always get the money. But the next day the wiki would be full of random junk put there by speculators, and you'd be sued for removing a single word of it. So since it wouldn't work for this to be the patent office, you'd either still have the patent office (and someone watching for edits would be submitting patent applications) or else you'd have to get rid of patents as an obsolete thing--eliminating another source of funding.

I'm all for eliminating patents, but it's wrong to claim this would eliminate a source of funding for scientists. Scientists don't invent anything; they discover things. If any scientists have patents on anything, that's a failure of the patent system. Invention is the domain of engineers and other inventors. This doesn't mean that a scientist can't invent something, but it's tangential to his role as a scientist. As a engineer, I might stumble upon some new discovery (not likely in my particular field), but it's not my primary job. I might also write a great song, but that's certainly not part of my role as engineer either.

Even if copyright would work for this, the GPL is a terrible model. In practice, you're forbidden from charging if you built your work on anyone else's--and it's just plain too administratively complicated to actually pay all those underlying people. So everyone throws up their hands and just gives it all away and that's that. I'm not saying it's impossible to make money under GPL, I'm just saying I doubt any claim that the contributors will be routinely well taken care of.

I don't want Scientists to have to have jobs as cooks, janitors, etc. just to earn a living wage. I want them to spend as much of their time doing what they do best, and I want us to reward them for it directly, not make them have to spend their free time (or even their full time) chasing money so they can squeeze in a little time doing Science if there's any time left at the end of chasing money.

Nor do I think it would be good for them to resort to "applied science" for their money. Science and its applications are different things. Basic research is not the same as product development, and the two should not be confused.

I don't even think it would be bad to have a few millionaire scientists. Money runs the world, and no amount of giving stuff away will fix that. The people who are a threat to Science have plenty of money; if Science doesn't find ways to enrich some of its own, it won't have the power to hold the forces of anti-Science at bay.

This all sounds great, but it just doesn't seem to work that well in practice due to our capitalist profit-driven economic system.

Unfortunately, human behavior I think keeps science from advancing as far and as fast as it could under ideal circumstances. The way I see it, there's only three ways for science work to get done: 1) a scientist does it on his own for free (obviously this doesn't happen that much), 2) it's done by private industry, or 3) it's done by the government.

#2 works well for applied science, where the company has a certain goal in mind, but it doesn't work very well for basic research because it takes so long to realize any returns on it. Companies used to do more basic research, decades ago, but they don't any more, and I'm not a good enough historian to tell you why things have changed this way, though I think it has to do with the post-WWII economic boom in the USA and its consequential dominant status in the world economically and politically: big companies could afford the luxury of funding basic research because they didn't have much competition here, and absolutely none abroad. These days, there's a lot less resources, and a lot more people and countries fighting over them.

#3 works better for basic