Ask Slashdot: Successful Software From Academia?

An anonymous reader writes "A lot of masters and PhD theses are about development of software targeting the solution or the automation of a specific problem. Bioinformatics, for example, has a lot of journals about software tools that are coded in academic environments; some of this software is the final result of a four-year PhD. But my question is, how much of this software will see the light outside the universities? I know of some examples, like BSD, but they are an exception, right? Is there any list of successful software created entirely inside universities' labs that became widely used?"

Comments

Under sufficiently large definitions of "widely"

[betterunixthanunix]

I am pretty sure that, for example, Condor started as an academic project, but now it is Red Hat's grid computing platform:

http://www.cs.wisc.edu/condor/
http://www.redhat.com/mrg/grid/condor/

Re:latex ?

[Shikaku]

Blackboard.

At my university...

[AdamJS]

Grad projects and student-assisted research projects (most recently some OCR work and some prime factoring studies) are always released at the end of each study term under some sort of open-source license. I'm not sure _where_, though. I know my friend used the project he contributed to in his own business software, but don't have any examples beyond such simple anecdotes.

Defining success as outside of the university...?

[ByOhTek]

That seems silly. When I worked in a bioinformatics group as an undergrad, we use a *LOT* of software that was only used inside of a university, partially because the kind of research it targeted wasn't necessarily popular in commercial areas yet, and some because what we used was OSS and many commercial organizations preferred closed sourced alternatives (sometimes for speed optimizations, sometimes for support reasons).

Maybe you should define your criteria as widespread use in the context of the target field, rather than outside of a university?

That being said, I think a lot of it either directly or indirectly (through a third party reimplementation), does make it out.

PostgreSQL?

[0racle]

That work for you?

PostgreSQL

Re:PostgreSQL?

[Anonymous Coward]

And Berkeley DB

Re:PostgreSQL?

[Svartalf]

Beat me to the punch there...

Re:PostgreSQL?

[he-sk]

A lot of database systems start out as academic research systems that are later commercialized. Examples include H-Store (commercialized as VoltDB), C-Store (commercialized as Vertica), Monet (commercialized many times, the latest incarnation is VectorWise).

Actually, the entire database field traces its roots to academic systems, starting with INGRES which was published in the 1970s by UC Berkeley.

Re:PostgreSQL?

[Anonymous Coward]

Guido van Rossum who created the Python programming language, at CWI(atleast while employed there).

Re:PostgreSQL?

[brokeninside]

Not just PostgreSQL.

Postgres (in its SQL and non-SQL forms), Sybase, Microsoft SQL Server, Nonstop SQL, and Ingres all lay claim to the same heritage that began with the Ingres project.

Re:PostgreSQL?

[Jonner]

I develop web applications every day with PostgreSQL and Python, both very popular projects which originated in universities. I also depend on the ubiquitous Apache HTTP server which was originally a derivative of a university project. Both my development and production environments are GNU/Linux. GNU and Linux were not projects at universities, but they were non-commercial and inspired by experiences in universities.

Though Unix originated at AT&T, the additions from BSD have profound and lasting effects on all modern operating systems, especially Unix-like ones. The Internet was developed at universities and TCP/IP was originally implemented on BSD Unix.

kerberos

[Anonymous Coward]

kerberos, ganglia, folding?

"Widely used" isn't the norm

[93 Escort Wagon]

In this day and age, most good software developed in acadamia tends to get spun into a business venture that makes its academic developers very, very rich. See Google, for example.

Re:"Widely used" isn't the norm

[multimediavt]

In this day and age, most good software developed in acadamia tends to get spun into a business venture that makes its academic developers very, very rich. See Google, for example.

And Sun, and SGI, and Oracle, 3Com, that list is almost endless. There are many commercial software packages for fluid dynamics, molecular dynamics, quantum mechanics and a host of other engineering and scientific fields. MathCAD and Mathematica are some examples along with PATRAN and NASTRAN, CHARMM, VASP, etc. There are TONS of companies that are software spinoffs from universities. TONS of them die every year, too, and end up sold off to patent trolls if they are lucky, err, cursed?

Re:"Widely used" isn't the norm

[martyros]

More examples:

Coverity

VMWare

Xen

Then again, the work I did for my thesis never made it past "research prototype". Papers are the "coin of the realm" in academia. There's a very long way between "proof of concept that runs well enough to take measurements and publish" and "something I can sell to someone", and there are no papers in between. That leaves entrepreneurs, or departments / people who do it just out of the goodness of their hearts.

I've got a good heart I think, but nowhere near good enough to spend all day at work coding, and then come home and code some more. :-)

Re:"Widely used" isn't the norm

[BrokenHalo]

Many resourceful students or academics make some attempt to cash in on some aspect of their work, within the IP restrictions imposed by their institutions. After all, most never succeed in scoring full-time or fully-funded work, so are usually left to scratch a living in any way they can. While there are a few big-name success stories, most have to make do with modest success in a specialist niche. Here's an example whom I know personally...

Possible example:

[BlaKnail]

There was this company called Google that came out of some phD students' work. I think it's still around and doing business.

Well,

[gcnaddict]

Successful software created entirely inside universities' labs... I wouldn't know about that, but Facebook and Google are products of students at universities at the time said applications came into existence. Both addressed specific problems (Google -> search, Facebook -> social contact/updates/etc.).

Re:Well,

[optimism]

Facebook is not software; it is a "special-purpose investment vehicle" designed to extract a few $B from some high-net-worth shmucks before it vanishes.

If Facebook was ever forced to publicly disclose their real numbers, I think you would also have to remove them from the "successful" category.

Unless you work for Goldman Sachs, of course. ;o)

Re:Well,

[Darinbob]

I wouldn't list Facebook as "from academia". It wasn't a research project or even a class assignment. It was something a person created while a student, similar to "Hello World" or Hack.

Re:Well,

[BrokenHalo]

why in the world did Zuckerberg not take the $7B or whatever the latest FB offer was for and run with it?

Another equally pertinent question: why run with it? Why not simply sit down and spend the money, give (or gamble) it away, or whatever rocks your boat?

I will never have such funds at my disposal, but if I did, I would probably give most of it away to some suitably worthy cause, while keeping a portion to set up a few projects of interest. I know one or two people who have managed to amass very large fortunes, and I struggle to understand their mindset where most of their waking moments are spent on preventing taxation offices from getting their claws into those funds. Seems to me that there are better ways to spend one's life.

Logical Reason for the Dearth

[eldavojohn]

The problem with software in academia is that it is often devoted to a sole purpose. It is not a generalized solution -- conversely -- it's often a demonstration of a solution so specific that it's never been done. Hence the awarding of a title to the creator. On top of that the teams are usually small and time is usually tight. It's also usually a side effect of the greater thing, the thesis. It will always take a backseat to the theory.

When software is widely adopted, it is because it has been widely supported and is a more generalized solution to a problem. If it uses hardware, it supports all kinds. If it reads or writes files, it covers all formats. This leads to widespread adoption but also takes a lot of time and a lot of contributions. If you're also working on your thesis, this is a daunting task to work on the side.

Nobody gets their PhD by making a predecessor's implementation support more file formats or hardware. So this is left to the licensing of the originator and the community -- who are often recognized as the real workhorses that go from prototype to actual usable software. That's why you don't find many PhD projects turned instant open source hit.

In bioinformatics , a relatively young field, most consumers of the software work in a lab and the input is fairly simple. But even with simple input they first had to agree on a format (those are just a few of what used to be many). BLAST and FASTA go back to the 1990s and 1980s respectively ... if it had depended on hardware or the constant change of text files like PDF and DOC, I think you can understand how hard it would be for academia -- let alone the originating researcher(s) -- to maintain and support for the community. An open source effort could pick up that slack but then who deserves credit for that work?

Re:Logical Reason for the Dearth

[RecoveredMarketroid]

The problem with software in academia is that it is often devoted to a sole purpose. It is not a generalized solution -- conversely -- it's often a demonstration of a solution so specific that it's never been done.

Absolutely true. And much of the software is nearly unusable by anyone else-- it was built by the researchers to validate their own work, not to be used by others. If you've ever tried to use any code generated by grad students, it is often buggy, brittle, inflexible, indecipherable, etc... (I'm a late-stage PhD student, so I've run into this MANY times...) And that's the code that the researchers saw fit to release to the public-- imagine what the stuff that wasn't released looks like.

Re:Logical Reason for the Dearth

[boristhespider]

I used some stuff in my thesis that was written in Fortran 66, adapted from Fortran IV of all places. It took years more to clean out the rest of that and get everything at least into F950/95. It's still buggy, brittle, inflexible and probably indecipherable, but it's slowly getting better and more adaptable.

Re:Logical Reason for the Dearth

[boristhespider]

Actually I probably should have mentioned that I think it's a further reason for the problems - the code is often patched together from inherited libraries and routines passed on by the PhD supervisor, who themselves inherited quite a lot of it from their own supervisor. Some code used in large academic projects honestly dates back to about 1970 or before, and hasn't been touched since other than to hack into double precision and hope that that doesn't break something subtle. Since some of these archaic routines are random number generators, it actually can break things quite badly.

There's no way I'd release my codes to the wild even if anyone else found them useful - their brittleness is a mixture of the way they were programmed, aiming directly at a very specific problem with no error trapping if the inputs went slightly outside an assumed region, and the routines that went into them. It would take significant work to clear out all the junk and reprogram it in an even vaguely modern language.

Even then, that language would be Fortran, which would put off quite a lot of developers. In my field, at least, academia is still stuck in Fortran and it's often F77. People are slowly shifting to a mixture of C++ and Python but it's taking a very long time.

Re:Logical Reason for the Dearth

[tixxit]

I think there is a bit of a sea change though. At the lab I work at and others, it is becoming much more common place to have a mix of full-time software developers with scientists and grad students. This is good, since, for the programmers, originality is given a back seat to usability and better support for other software. This is probably a reaction to how developed the field has become; people expect certain features now and support for file formats from popular tools is a must. Bioinformatics is also special because our users, while very smart, are often people with no proper technical background and so usability is very important.

Re:Logical Reason for the Dearth

[gilleain]

The problem with software in academia is that it is often devoted to a sole purpose. It is not a generalized solution -- conversely -- it's often a demonstration of a solution so specific that it's never been done.

... it was built by the researchers to validate their own work, not to be used by others.

The main purpose of academic software is to produce papers ... except for text mining software, which does the opposite :)

Software engineers would be horrified at most academic software, which tend to have just one 'release'/version, no source control, no formal testing framework, no design documents, no possibility of bug reports, etc. I know because I used to write stuff like that...

Nowadays, I contribute to projects that have these things (CDK, bioclipse, my stuff on github) but I've seen some bad examples. Even code where the README file said "there is little commenting in the code as I don't believe in doing that" !!

Re:Logical Reason for the Dearth

[gilleain]

I used some stuff in my thesis that was written in Fortran 66, adapted from Fortran IV of all places. It took years more to clean out the rest of that and get everything at least into F950/95. It's still buggy, brittle, inflexible and probably indecipherable, but it's slowly getting better and more adaptable.

Bah! I've tried to read code that was auto-translated from fortran to c. (Also, I was walking uphill to work both ways! :)

Re:Logical Reason for the Dearth

[gilleain]

In bioinformatics , a relatively young field, most consumers of the software work in a lab and the input is fairly simple. An open source effort could pick up that slack but then who deserves credit for that work?

Well one good example of generally used academic software is RasMol, and it's (spiritual) successor, Jmol. RasMol started as a project by Roger Sayle as part of a PhD on graphics software, but became the main free viewer for macromolecular structures.

Jmol has taken the idea of a viewer much, much further and is even used by mathematicians to show surfaces. It now supports translucent surfaces for orbitals, a complex scripting language like a subset of javascript, and reads a large number of file formats. It has been used in 3D caves, may come to Android in the near future, is used on wikipedia, and tons more.

The strange thing is that the project has gone through a number of 'owners' from the original author. the current one - Robert Hanson - is only the latest 'Doctor', in the metaphor of 'Dr Who' of Peter Murray-Rust. He publishes papers on his work with Jmol, just as the other Dr.s did : Dan Gezelter, Bradley A. Smith, Egon Willighagen, etc. So it is both open source and publishable, which are both important

Re:Logical Reason for the Dearth

[Mr. Slippery]

If you've ever tried to use any code generated by grad students, it is often buggy, brittle, inflexible, indecipherable, etc...

And this differs from code produced by industrial software developers how, exactly?

99% of code sucks. Open source, closed source, academic, government, private industry.

Re:Logical Reason for the Dearth

[boristhespider]

Haha, I'm in cosmology - one of the "big" CMB codes is CMBEasy, which for almost a decade was the only CMB code written in C++. The author originally "wrote" it by taking an F77 code (CMBFast), about half of which is inlined for speed and which is replete with common blocks with variables arbitrarily renamed for reuse in different routines (the original variable name obviously *also* being used in the routine) and running it through f2c, and then goggling at the result and trying to make sense of it while he rewrote and refactored the lot. It made him a hell of a good cosmologist, and quite a good programmer, too.

Re:Logical Reason for the Dearth

[Darinbob]

The best university software is often created by a large team and it's used as a base to do research with. BSD, Macsyma, Minix, TCP/IP, Ingres, etc. Because the software needs to be used and modified by many students it is necessarily more maintainable and better documented.

Re:Logical Reason for the Dearth

[jellomizer]

You hit the nail on the head.
However they tend to suck in different ways.

Open Source: We did the fun coding portions and we got it to work. But we will leave out making a clean UI. It is open source someone may come in and do it at some point... Oh by the way we will not bring in any of your UI changes into our project because we got use to using the program the way it is.

Closed Source: It works... It looks good... Lets hope you never see how we got it there... And we will never touch that function again, it works as best as we can tell.

Academic: We solve this problem, it is good enough for me to get my PHD or keep my funding. I got what I needed lets put it on the shelf if someone wants it they can have it.

Government: Here are a bunch of discrete use cases to be coded as individual modules. The fact that 80% of them are redundant doesn't mean you shouldn't do them or break away from coding each use case as a separate module.

Private Industry: We got the program working mostly feature X doesn't work. However only 10% of the customers are affected by this and 4% will get a refund and 6% will complain but wait for the upgrade. But the time save we can get it to market and gain 20% increase in sales over the life cycle of this version.

Re:Logical Reason for the Dearth

[cashman73]

As a computational biologist myself, I agree with this wholeheartedly! And to top that off, I think that a lot of the software that's out there is poorly documented. The code has variables in it and other functions with no comments telling someone else what those variables mean. And it was posted on a website with very poor documentation or instructions as to how to run it. Sure, they published a paper in Nature or PLoS Computational Biology. But a scientific paper is not an instruction manual, and writing both requires different forms of writing.

Re:Logical Reason for the Dearth

[BrokenHalo]

Heh. I wrote lots of Fortran IV code back in the day, and I wouldn't be too surprised if a lot of it was still in use. As the famous article says, "If you can't do it in Fortran, do it in assembly language. If you can't do it in assembly language, it isn't worth doing."

I sort of missed the boat with F77. By the time that came around, I was already playing with C. However, I always found FTN a lot more concise.

Re:Logical Reason for the Dearth

[BrokenHalo]

99% of code sucks.

Wrong: a 10^3x wrong. 49% of code sucks.

Another 49% blows.

Google, RSA Encryption

[Anonymous Coward]

both ended up commercial, but both were academic projects. I think as a general principle, academia does not have the capacity to maintain commercial products - academic institutions don't have sales, customer support, marketing etc. So...genesis in academia, but significant "light of day outside of academia" is almost always going to involve a commercial entity.

Re:Google, RSA Encryption

[Sir_Sri]

Universities also can't fund large scale testing, and, generally, the university doesn't own the work, the author does (professor or grad student), as soon as something can make money you leave the university, or at least farm it out to a separate corporation.

MSc's and PhD's aren't really about writing a big full blown program though. They're about finding novel solutions to problems and demonstrating that with a program. So they're different goals. You may have a big enough team that you produce a full on program, but more often that not, even if you're working directly on a commercial product, your part of it is relatively small.

Tons

[aaaaaaargh!]

I don't know of any list but I'm pretty sure that tons of successful software has come from academia.

Racket is a particularly nice example. I'm too lazy to Google so perhaps others can provide a few hundred more.

Re:Tons

[140Mandak262Jamuna]

Buddy, Google page rank algorithm itself was a PhD thesis. Basically.

X Windows, Ingress / Postgress

[angel'o'sphere]

Subject says it, X was mainly developed at MIT. I guess Ingress and Postgress where originally also university projects.

Re:X Windows, Ingress / Postgress

[Alomex]

X windows as mainly developed at MIT

but he said successful as opposed to "kludge used in the absence of anything else".

Re:X Windows, Ingress / Postgress

[dkf]

X windows as mainly developed at MIT

but he said successful as opposed to "kludge used in the absence of anything else".

Then it's a very successful kludge. I wish all mine were even 1% as successful!

A few...

[sl3xd]

* Kerberos (Widely used, part of Active Directory)
* X11
* AFS (Andrew File System)
* MACH (Used by GNU HURD and OS X)

And that's just a starting sample.

Re:A few...

[PybusJ]

A couple of more recent examples, now in wide use:

* LLVM (University of Illinois)
* Xen hypervisor (University of Cambridge)

In general, if a University project becomes widely used it will either have been spun off into a commercial operation or become an open source project which gains outside contributors.

Re:A few...

[yup2000]

rsync

Re:A few...

[bcrowell]

Unison: http://www.cis.upenn.edu/~bcpierce/unison/

postgres RDBMS

[Anonymous Coward]

Postgres (now Postgresql) was started at UC Berkeley. There was a long evolution, but it is an outstanding and widely-used relational database.

Re:How about...

[Ruie]

And valgrind

nobody to blame?

[Anonymous Coward]

The problem is that a lot of useful academic code disappears when the degree is awarded to the candidate. It's a big problem that's only exacerbated by the publish-or-perish culture of the scientific community at large--this code is often written as an afterthought to acquiring a degree and the development practices exercised by the authors often reflect that.

I think a major contributing factor is that the academic culture breeds advisors who are less interested in sound software development and more interested in graduating their students/getting papers published. It's difficult to place much blame on anyone in particular, since this behavior appears congruent with our expectations. It's really too bad that society loses out on interesting software this way.

Web Browsers? TCP/IP?

[cwgmpls]

web browsers
TCP/IP
I'm pretty sure these are used outside of universities.

Several

[PiMuNu]

I think most of the finite element/multiphysics packages started as research projects, either in university or government labs (some military, some conventional). For studying e.g. electromagnet design, heat deposition by currents /EM radiation e.g. microwave studio. Most of the radioactivation and nuclear shielding simulations used by the nuclear industry for designing radiation shielding are or were academic projects (e.g. MARS, FLUKA, MCNPX).

Windows?

[fartrader]

Oh wait that was Pre-K

Yes

[Hatta]

There are quite a few open source projects on bioinformatics.org. Some of these are little more than quick command line tools. Others are entire frameworks. Personally, I use the following tools on a regular basis. Bioconductor (with R), EMBOSS, Primer3, and ImageJ.

Re:How about...

[staalmannen]

and LLVM

TeX

[WillAdams]

Subject of several theses:

http://www.tug.org/docs/liang/

http://www.pragma-ade.com/pdftex/thesis.pdf

https://www.tug.org/docs/plass/plass-thesis.pdf

(John Hobby's on METAPOST http://ect.bell-labs.com/who/hobby/thesis.pdf )

Probably others. More information at

http://www.tug.org/

and

http://www.latex-project.org/

and

http://wiki.contextgarden.net/Main_Page

William

Wrong approach?

[Superken7]

I think this is the wrong question to ask/wrong approach to take. Providing a final, production-ready product is not usually the goal of scientific research (my area). Especially when it's not done in collaboration with some engineering firm.

Usually the goal of scientific research is to provide new knowledge about some very specific domain. For example, some of my coworkers are developing a "cloud simulator" which models EC2 and allows you - among other things - to do stuff like predicting how many machines you will need and what the cost will be.

My point is that research does not have a goal to deliver production-ready software. That should not be the goal of a PhD thesis. The scientific method is about something different: Science, that is. :)
I'd say it is an engineer's task to take those new advances in Science to produce something which is "working" for them.

TL;DR;
So to provide my final answer in a nutshell: I would say yes, that is the exception. Science is about providing advances in knowledge that others can then use for producing better.. "stuff".

Advanced Aircraft Analysis

[CompMD]

It started out as someone's graduate research project in the late 80s/early 90s, and today it is the #1 aircraft design software tool in the world. Its installed in universities, aircraft manufacturers, aerospace consulting firms, and government and military institutions across the planet.

Disclaimer: I worked on the software after it went commercial.

Re:Advanced Aircraft Analysis

[LWATCDR]

Is there a non commercial version?

Re:Advanced Aircraft Analysis

[CompMD]

There is only one version, but different pricing tiers for educational and commercial users. This way, students and professors can get the full functionality at a huge discount off what businesses pay. There's node-locked and floating license options also. A student nodelocked license for a year was only $150, with $100 annual renewal including phone and email support. A single commercial floating perpetual license was something like $9000, but I don't remember the annual support renewals cost.

Re:Advanced Aircraft Analysis

[LWATCDR]

Out of range for hobby users, modelers, and homebuilders.
Really sad.

Re:Advanced Aircraft Analysis

[CompMD]

No, sorry, that's the full range, I should have been more specific. The price really depends on what you want to do with it. If its for a non-commercial use such as for modelers and homebuilders, the price will fall on the lower end of the spectrum. If you're Boeing, it will fall on the higher end. I remember writing licenses up for homebuilders.

Re:Linux

[hedwards]

If it doesn't *BSD definitely does.

Sometimes..

[Dop]

Being formerly from a research institution I can say it happens... sometimes. Usually the people in charge of the project realize they can make money at it and spin off a company.

Other times, really excellent software, that would be great for the community, goes absolutely nowhere because there isn't an easy path to profit. Once the grant money ends, the project dies. Then other groups write more proposals to solve the same problem over and over because there's nothing in the market.

Few off the top of my head

[jfp51]

Rocks clusters (http://www.google.ca/search?gcx=w&ix=c1&sourceid=chrome&ie=UTF-8&q=rocks+clusters) CHARMM (http://www.charmm.org/) Gaussian as an example of how academic-inspired software should NOT be commercialised (http://www.gaussian.com/)

Silly question

[abigor]

Everything from GNU and Symbolics to (as others have noted) PostgreSQL, Google and beyond.

Let's not forget programming languages like Python and Haskell. Even Facebook was conceived in a university environment, if not as a part of any specific research.

Re:Silly question

[clyde_cadiddlehopper]

Even Facebook was conceived in a university environment, if not as a part of any specific research.

Careful now, fella. You are coming darn close to saying something nice about Bill Gates. This *is* Slashdot, ya know.

LLVM

[Lally Singh]

The backend for quite a few compilers, and a few shader compilers...

FalconView

[Philosa]

I'm not sure if "Research Institutes" count in your criteria, but as far as software for dealing with maps, http://www.falconview.org/ comes immediately to mind.

Other Successful Projects

[Anonymous Coward]

o The LLVM Compiler Infrastructure (University of Illinois, Chris Lattner's Master's Thesis)

o The Mosaic Web Browser (NCSA)

o The Apache Web Server (Based on a web server from NCSA)

o BSD Unix (which encompasses *a lot* of programs) (University of California Berkeley)

o The Pine mail reader (Washington University). Other mail readers were probably in academic institutions as well.

o Eudora (originally developed from NCSA, I think)

o Washington University IMAPd

o I think one or more versions of popd

o Potentially Valgrind (Julian Seward has a few papers on it)

o SESC (hardware simulator from University of Illinois, I think)

o I think VMWare was born out of a research project, but I can't find a definitive source.

Matchmoving software Icarus

[Anonymous Coward]

The University of Manchester developed a free matchmoving piece of software called Icarus (http://aig.cs.man.ac.uk/research/reveal/icarus/). You can still use the old versions of this software for free.

It was later turned into comercial software that is a major player in the comercial matchmoving software area:
http://www.pfhoe.com/products.php

Students are short term

[pavon]

Is there any list of successful software created entirely inside universities' labs that became widely used?

That is an odd restriction to make. Students are only at university for a short time. If their work during that time turns into something useful then they naturally continue it after they leave, either as a an open source project or as a business venture. This is how it is meant to work, and there are tons of examples of such software.

MATLAB and Maple were both created at universities and later commercialized. Same for SPICE. On the open source side there is Apache, Sendmail, PostgreSQL, and the original implementations of nearly every RFC protocol on the internet.

Re:Students are short term

[Svartalf]

Squid...don't forget that one...or Macsyma... :-D

Re:Students are short term

[Herve5]

+1 for Maple that really changed the world (of both mathematicians and engineers) and set the standard of formal maths years and years before its pale private copy Mathematica...

How about these?

[videohead]

Three more example: LLVM -- http://en.wikipedia.org/wiki/LLVM
RT Linux -- http://en.wikipedia.org/wiki/RTLinux
KSplice -- http://en.wikipedia.org/wiki/Ksplice#History

UCD/NET SNMP, IMAP

[Metiu]

the SNMP server
http://www.net-snmp.org/

A good IMAP server
http://www.washington.edu/imap/

Mosiac

[Registered Coward v2]

From Univ of Illinois - it arguably changed the internet from a tool for techies to a new way to do business. One of the problems is if something is really good commercial companies may morph it into products that eclipse the original; but their contribution, when though of as basic research, was invaluable. So the definition of success should not be limited to widely used, popular, or well know; but also include defined a new industry or way of approaching a problem.

Re:Mosiac

[Ped Xing]

Not only was Mosaic the foundation code for both Netscape Navigator and Internet Explorer (as well as a variety of other, now forgotten browsers), but the NCSA project at the University of Illinois also produced the NCSA HTTPd web server. This eventually morphed into the Apache web server. See http://www.apache.org/history/timeline.html.

Answered in the post

[ArhcAngel]

Posted by timothy on Tuesday September 27, @10:41AM from the how-about-ncsa-mosaic dept.

autostitch

[FunkyELF]

I think autostitch is used in real life....

http://www.cs.bath.ac.uk/brown/autostitch/autostitch.html

LINPACK/LAPACK/Netlib

[Arathon]

right up front: I know about this only because I work for these guys, but...

there's a whole host of Linear Algebra-related software written for high performance computing environments that is attributable largely to various teams of academics throughout the past 30 or so years. It is my understanding that these libraries get used by most anyone doing high-performance computing.

http://www.netlib.org/lapack/ http://en.wikipedia.org/wiki/LAPACK

Re:Linux

[ByOhTek]

Linux itself wasn't used for a thesis or for grade was it?

Parts of the kernel might have started that way, but as a whole, wasn't it just a project for self-education, fun, and production of a more useful too than what was otherwise available?

The Web

[gzipped_tar]

Others have given pretty many examples. It may be worthwhile to mention the Web, from CERN http://public.web.cern.ch/public/en/About/webstory-en.html

Re:The Web

[PPH]

The jury is still out on that success thing. FaceBook might just push the verdict the other way.

FFTW

[JambisJubilee]

Here's a list of my (important) favorites:

meep (finite difference time domain electromagnetics)

FFTW (fourier transform)

MPB (photonic bandgap solver

Elmer (finite element solver)

Re:Linux

[Kristian T.]

The title of Linus' thesis is: "Linux: a Portable Operating System" - so yes, it counts.

  The real question is, if it is enough that a project can trace it's roots back to a academia - even if >90% was added later and or by developers outside academia. I bet many products considered purely commercial started out started out in the back of the head of students during their studies. Many of those dropped out to build a company rather than stay and write a thesis about it. If you include those, and even consider some studying other majors than CS - your probably looking at the bulk of all software in existence.

Re:How about...

[pnewhook]

And QNX http://www.qnx.com/

Depends on who you ask...

[PSandusky]

Frequently the software doesn't start in a given academic lab, so much as it starts somewhere in a given research community and propagates to the academic labs as research needs dictate. ImageJ, for example, started at NIH, but now it's available to all and in use all over the place (including my lab).

Other software is developed cooperatively, and then academic contributions are added as they're needed to enable someone's research. If you run R (the statistical program) and start looking through all the extensions available in CRAN, you'll see tons of additions that have been generated in academic labs and released for use by the wider research community.

I work in biomechanics, and I've seen a few programs come out in that field through largely academic development. AnimatLab began (I think) at Georgia Tech, and I think Cofer et al. are still developing it within the university. OpenSim started at Stanford as an open source musculoskeletal simulation program, and is vastly preferable to the godawfully expensive SIMM, which does pretty much the same kinds of things. OpenSim is still alive and well at Stanford, although the developer network spans multiple institutions, academic and otherwise.

Much as I might wish that I could spend more of my time developing programs and playing with software within the academic sandbox, more often it's simply more practical to cast the nets for software from someone, somewhere doing somehow similar research, and then using the software you find if it's useful to your work, rather than reinventing the wheel in favor of advancing academic software development.

rsync

[Short Circuit]

IIRC, rsync was the culmination of its original author's thesis.

Re:rsync

[Anthony]

Tridge started Samba.

HPC Applications

[rockmuelle]

A number of HPC applications funded by NSF/DARPA/DOE grants are able to provide a continued source of new research while maintaining and improving the applications.

One example is OpenMPI. BLAS/LINPACK/LAPACK are also examples. Some of the C++/Boost libraries also are maintained in academic, such as the Boost Graph Library.

-Chris

Some examples

[vpaul]

Hugin (well, at least parts of it), Postgres (at least its origins),
Emacs and other GNU software (RMS worked for an university then,
afaik), LISP from MIT, Pascal from ETH Zürich,
Python (at least originated) from CWI, ...

Ho ho ho

[anom]

FWIW, I'm a PhD student at a reasonably large institution in the US.

Very little of this stuff sees the light of day. The vast majority of software is written simply as a proof of concept for some particular method/system/algorithm in order to get published. Good conferences/journals will typically want not only a well thought out idea, but an idea that you can and have implemented it to some extent, and that it works. That having been said, most of what gets produced is complete and total garbage -- typically just enough code to be able to prove that something runs correctly and in a given amount of time.

Personally, I have written a bunch of junk code during my time here. I'd like to think I know more or less how to write good code after all these years, but writing good, well documented, well tested code takes time we don't have -- writing code is simply a means to an end (publication) -- and so most of the code I write is hasty and ugly. This even applies to code that people say is for "wide distribution".

Before you go hounding on academia however, I'd warn that writing "good code" isn't really the point of what we're doing -- the point is to produce a reasonable method of solving some particular problem or type of problem. Going into bioinformatics for example, there are a whole bunch of problems that involve performing more efficient analysis of certain types of graphs. If a researcher discovers something along these lines, he/she will likely write some junk code to prove that the bare algorithm works, perform some analysis of it, publish it and move on. This may or may not end up actually being a useful improvement -- if it is however, then some implementer whose actual job it is to code whatever medical software might be using this algorithm then has a basic blueprint of how to proceed.

As for some examples of software from academia that have made it out, let me think...

Coverity - static code analysis tool, started at Stanford then moved into being a startup and is now quite successful
PostgreSQL - Originally from Berkeley
Bro (Intrusion Detection System) -- written by a researcher from Berkeley/ICSI -- is still somewhat "in academia", but I have heard of several production deployments

That's all I feel like coming up with right now, but I think the general pattern here is that if/when some piece of software produced in academia is seen to have value in its own right (e.g., away from the original research/publication that spawned it), it typically gets spun off in a start-up or a more concerted effort is given to its development, at which point one can actually spend the time to write good code.

Re:Ho ho ho

[ideonexus]

Thank you for posting this. It explains why so many of the Academic solutions to problems I'm researching are so incredibly lacking. I've seen 200+ page theses claiming to solve a problem, but after getting through them I find that the paper actually only describes how one would go about solving the problem and the author hasn't produced one shred of real code, but lots of pseudocode, to make their point.

Honestly, this is why managers avoid hiring Ph.D.s like the black plague in the software development world. I shared a cubicle with a Ph.D. who had just come out of spending the last 10 years crafting some incredibly complex formula for accurately predicting something to do with radio waves. We gave him a report to write. Six months later he gave us a white paper on how he was going to go about writing the report. Six months after that he got a job writing white papers for another software company. We gave the report to one of our average programmers, who got it done in a month.

For the amount of effort and verbosity that goes into Academic research, it sure seems like very little comes out of it. No wonder only 40% of research papers get cited in the first five years of their being published.

Re:Ho ho ho

[lorinc]

And to be fully honest, I prefer Ph.D. students to have novel, nice and efficient ideas and algorithms than to produce well written and well documented code. If I needed someone to write good code, I would have hired an engineer, not a Ph.D. student. Just saying...

Re:Ho ho ho

[Rostin]

I'm a PhD student, and I never heard that 40% stat before. It's eye-opening. I don't have mod points so I thought I'd just reply to thank you for posting it.

Re:Ho ho ho

[HuguesT]

In Academia you get the absolute best and worst at the same time, like in any other field.

Honestly, this is why managers avoid hiring Ph.D.s like the black plague in the software development world.

I think you should tell Google that, they should stop their practice to hire the best PhDs. While some research work is not good, some is totally excellent, and more often than not, game changing. Think that the Web initiated at the CERN, the research environment par excellence, to cite only one example.

While only 40% of research gets cited, this is not necessarily an indicator of the quality of the research itself or the researcher. An individual researcher can have simultaneously papers that are not cited at all and others that are cited hundreds or thousands of times.

Re:Ho ho ho

[Anthony]

Thanks for the links, especially the SIAM one. Some of those ideas I touched on in my Honours thesis as my project was directly affected by the issues of lost programs; old media storage where was were parts of programs found; a paper with an idealised function only described with a graph; incomplete mathematical description of a model component. I have begun planning future research along with a methodology to avoid a lot of the above issues.

Wuala

[Lazy Jones]

Wuala is a recent example, developed at the ETH Zürich, then spun off and bought by LaCie.

A list

[Sangui5]

Firefox came from mozilla, which came from netscape, which came from NCSA Mosiac; done by Marc Andreessen at UIUC.

The LLVM compiler & runtime are both university projects from Vikram Adve @ UIUC.

VMWare came from SimOS, via Mendel Rosenblum of Stanford.

Coverity come from the work of Dawson Engler's students at Stanford.

BerkeleyDB started from work by Margo Seltzer at Berkeley.

Kerberos was done by Steve Miller and Clifford Neuman at MIT.

The Lustre filesystem is due to Peter Braam at CMU.

For a long time a lot of OSes used TCP/IP implementations out of academia (either Stanford, Berkeley, or University College London, depending).

Apache started as a series of patches against NCSA's HTTPd code (UIUC again).

PostgreSQL started from Ingres, which is from Micheal Stonebraker's group at Berkeley.

And now I'm bored looking these up. Let's just say there is a LOT of software that came out of academia.

Loaded question

[stephanruby]

This is a loaded question:

But my question is, how much of this software will see the light outside the universities?

The truth is that most software projects (even the ones developed outside of Universities) never get widely adopted. Just take a look at Sourceforge, how many of those projects become widely adopted? or even get anywhere? 1 percent? 0.00001 percent?

That being said, I believe Universities have a disproportionately strong influence on the software industry (which is by design of course). Take for instance, Intel, Linux, Google, InfoSeek, SendMail, Internet Explorer/Netscape (which both used the initial code base from Mosaic), Cisco, Yahoo, Sun, etc. They were all started initially as some kind of school projects.

Re:latex ?

[somersault]

Backrub (Google)

Why is everyone forgetting...

[j2kun]

PageRank! Google was just one big master's thesis. And quite academic, if you're a stranger to linear algebra.

Re:Linux

[Kristian T.]

It started out that way - but by the time Linus graduated in 1997, linux had become a huge thing, and I bet that if he hadn't made it the topic of his masters - he wouldn't have finished at all.

Definitions of success.

[leastsquares]

Does a software package need to be "widely used" to be classed as "successful"?

My company, for example, was built around an academic software package. We are nowhere near the league of the Googles or Oracles out there, but we provide a fair number of employees with a good salary. I'd never say our software was widely used as I can count our customer base on the digits of my hands and feet. Our kind of niche market will often use software from academia - because that's the main source of innovation - and the purely commercial argument for developing and validating the software in the first place would be weak.

To answer your question directly, in my field a vanishingly small fraction of academically developed software is ever used outside the research group that produced it. Even in the cases that the software would be more widely applicable, it just isn't shared/sold/licensed more widely. A couple of times, we have tried obtaining commercial rights to software that we thought could be valuable outside academia but we've never managed to negotiate realistic terms with the universities. Either the researchers aren't interested in pursuing this option as there is no personal reward for them, or the I.P. departments get greedy and the royalties they demand just makes the whole idea unviable.

Most projects spin off

[macwhizkid]

If you're looking for examples of "successful software created entirely inside universities' labs", you're never going to find anything. University research is fundamentally different from product development. While a grad student or faculty member might do research on a hard problem and write some software to solve it and publish a paper, that software is going to be enormously buggy, perpetually incomplete, and probably require constant support in order to work at all.

But there's another side to it, which is that nobody's going to pay for that team of developers either. Even an R01 grant (the classic "run a research lab" grant) is around $250k/year in direct costs. You can't hire a full time .NET or Java developer (let alone a team of them) and pay your own salary and tuition for a couple grad students on that budget. Even if you could, you'd have a hard time justifying it as a research expense to the federal government.

So how does stuff end up on the market? Spin-offs, collaborations, and cross-licensing. Universities have smartly become much more willing to license out technology developed on campus, even when done 100% on University time & money. Having that flexibility encourages your smart people to think big, and discourages the best people from leaving to do it.

Re:apache's mod_backhand

[PPH]

Apache got its start when NCSA (U of Illinois) decided to turn support of their httpd server over to an outside group.

Rumor has it that, upon looking at NCSA's code, they remarked, "This certainly is a patchy web server" (true?).

In my experiance mostly no

[inhuman_4]

While there are some academic software projects that make it big, that vast majority don't. This is because PhD's are trying to do research, publish, and get their doctorate, not write quality code. I know I would never release any of the code I have written so far, not because I'm anti-FOSS, but because the code is crap and I know it.

The software is written to see if something was possible, or to scratch some very specific itch. The result is that there is no documentation, very little abstraction, and a lot of cowboy coding. The problem is you write code to follow the results, so you cannot plan ahead. You can write beautiful code for a week, then get the results and discover the whole approach needs to be changed. You also tend to code very specific to the task because you don't want your results to be messed up by coding shortcuts or needless abstractions. But most important, for me at least, is time. Most PhD just don't have enough of it to spare starting and maintaing a project. Sometimes Profs do (Zeus bless them) but the students rarely.

What you do see happen is for research to prove something novel, then get rewritten to become a proper project sometimes FOSS, sometimes as the poduct of a new company. I have seen a few Profs. start open source projects and have their students work on it in partnership with other researchers, and that maybe more common in the future. But right now, most code never gets out and you wouldn't use it even if it did.

Re:Spice, Magic, TCL, (Al)Pine etc.

[Smallpond]

Hspice and Pspice are popular commercial, closed source versions of Spice. I wonder if Spice would be used more or less if it had been GPL? It certainly resulted in a lot of money that did not go to the original developers.

NetDot (Network Documentation Tool project)

[modestgeek]

I've been using a Network Documentation Tool (Netdot) for a while now. It is still in active development but certainly useful. It can easily be adapted to networks outside of academia. They implement feature requests pretty often too. https://osl.uoregon.edu/redmine/projects/netdot/wiki

listing successes

[Onymous Coward]

• Perspectives
• VMD
• Chimera
• Foldit

Etc.

But my question is, how much of this software will see the light outside the universities?

Impossible to answer. What defines a serious project versus someone's pet project or proof of concept? Then of those, how do you measure success? How many Sourceforge projects "see the light" outside Sourceforge?

Is there any list of successful software created entirely inside universities' labs that became widely used?

This is the question you seem to be getting an answer to in the forum here. Hopefully it helps.

Re:latex ?

[tautog]

Blackboard.

*shudders*

Someone tell me their thesis was rejected...

spice

[Fnord666]

SPICE is a general-purpose circuit simulation program for nonlinear dc, nonlinear transient, and linear ac analyses. Circuits may contain resistors, capacitors, inductors, mutual inductors, independent voltage and current sources, four types of dependent sources, lossless and lossy transmission lines (two separate implementations), switches, uniform distributed RC lines, and the five most common semiconductor devices: diodes, BJTs, JFETs, MESFETs, and MOSFETs. SPICE originates from the EECS Department of the University of California at Berkeley.

Universities aren't/didn't-used-to-be Corporations

[smoothnorman]

The critical part of "see the light outside" is marketing. Universities were never supposed to be corporations with a marketing department. now, i'm sadly aware, that many universities are changing in that regard ("office of technology transfer" by any other name) but historically there's your explanation. That being mumbled, if you were to start with every major software package "outside" and faithfully trace it back to its origins, i'd say you'd nearly always discover yourself in a university setting. ("why that's mere sophistry, sir! because computers themselves can all be tied back to university professors and the like!" ...yep)

SAS and R

[kj_kabaje]

Both SAS and R were originally developed inside academic environments.  I'd say they both enjoy a rather wide audience (one FOSS, the other rather on the expensive side).

Re:Under sufficiently large definitions of "widely

[i.r.id10t]

WebCT was developed at the University of British Columbia, became widely used enough to become a company and then it was big enough to be bought (and put out to pasture to die) by Blackboard...

BIND DNS

[egamma]

I can't believe nobody's said this yet...

BIND

BIND was written by Douglas Terry, Mark Painter, David Riggle and Songnian Zhou in the early 1980s at the University of California, Berkeley as a result of a DARPA grant. Versions of BIND through 4.8.3 were maintained by the Computer Systems Research Group (CSRG) at UC Berkeley.

SAS

[ckblackm]

The basis of SAS came from while a student at NCSU. From Wikipedia: SAS was conceived by Anthony J. Barr in 1966.[2] As a North Carolina State University graduate student from 1962 to 1964, Barr had created an analysis of variance modeling language inspired by the notation of statistician Maurice Kendall, followed by a multiple regression program that generated machine code for performing algebraic transformations of the raw data. Drawing on those programs and his experience with structured data files,[3] he created SAS, placing statistical procedures into a formatted file framework. From 1966 to 1968, Barr developed the fundamental structure and language of SAS.

Multi-Touch

[JasterBobaMereel]

In relation to another post - Multi-Touch as implemented in the iPhone came from Academia ....

Several Numerical Mathematics Packages

[Anonymous Coward]

I'm not sure if you're only considering popular non-commercial software or work specifically done by grad students, but there are several examples of numerical packages that started out in some sort of academic environment.

* SAS - used for a variety of statistics-related purposes by businesses, originated from some academics at North Carolina State University. They eventually spun off and created the company that now develops the software (and donated a bunch of money to build a new math / statistics building here)

* R - programming language for statistical computing. This is apparently an implementation of the "S" language, invented at Bell Labs, but R was created by some academics at the University of Auckland.

* MATLAB, LAPACK, LINPACK - a library for numerical linear algebra. This is the forerunner to LAPACK, which is used by MATLAB and lots of other numerical-computing related software. This was developed by a number of people, including Cleve Moler, who went on to develop MATLAB for use by his students at University of New Mexico, which he later commercialized with some of his collaborators. Also Jack Dongarra, who was one of Cleve Moler's students, is credited.

* Octave - this is a mostly-MATLAB compatible numerical computing language. It originated as a project by John Eaton, to accompany his nuclear reactor design course (I guess at University of Wisconsin). I don't actually know how widely used this is, but I personally use it as an alternative to MATLAB in my research (I'm a graduate student in applied mathematics).

I'd imagine most of these things are exceptions to the rule, though. Generally, as other posters have said, a lot of academic software (at least in applied mathematics and some related fields) is written as a proof-of-concept to validate whatever the researcher is working on, rather than a practical set of tools for others to pick up and start using.

ASCEND

[Colven]

I use a GNU GPL licensed modelling program created at Carnegie Mellon. http://www.ascend4.org/

Porrf of concept

[wfstanle]

I think the original poster is looking at it from the wrong perspective. Often academic research papers are about a new technique and any accompanying software is rather rough from a user standpoint. Such software in not intended to be marketed to the masses but rather to demonstrate that the idea is sound. Business then takes ( licenses ) these ideas, improves on them and produces something that they can sell. Just because academia doesn't directly market software doesn't mean that their contribution to development is small. How many of the products we use today are based upon some academic research?

Written for advisor(s)/committee

[perpenso]

And a thesis and accompanying software tend to be written for your advisor(s) and the committee that will review your work (mostly the written thesis). Spending time tailoring software for the public is often considered a waste of time, something to be done after graduation.

Re:Written for advisor(s)/committee

[Darinbob]

Agreed. The "markets" are entirely different. If you write a commercially successful or popular program as a PhD student that most likely will have nothing whatsoever to do with your degree. You're not getting a PhD in writing software, you're getting one in Computer Science or Mathematics or Physics or whatever. What the university wants to see is a certain level of research from the students, and that's not necessarily the sort of stuff that makes for popular programs.

In addition even if the novel idea is a big hit outside of academia, you need to add a lot of stuff to make it a research project that wouldn't be there in a non academic version - measurement metrics, having multiple algorithms doing the same thing so you can compare them, etc.

What comes out of academia usually are big projects built as foundations for research by several students. Ie, no one got a PhD writing BSD but many students got PhDs based on work they did using BSD as the base and that research later gets used to improve other academic or commercial operating systems.

ERP Systems (From Universities)

[Kookus]

www.kuali.org

Universities are coming together to create financial, research administration, student, human resources and many other enterprise level applications. These are targeted more towards non-profit organizations, but they'll be all over the world sooner or later.

R - statistics (sort of)

[Anonymous Coward]

R was created by John Chambers, who worked at Bell Labs initially. He's now at Stanford, and most of the development for R is done in academia.

Re:R - statistics (sort of)

[gumbi west]

You are getting R confused with S. S is a language created at Bell Labs. R is a FOSS implementation, written in the academy. R is the mozilla of stats software where, "there is an add-in for that." An approach that has not worked/happened for other stats software which focus on the cathedral model.

One or Two

[UrbanaMan]

the University at Champaign-Urbana lays claim to one or two projects that have some popularity ..

the Mosaic browser and its offshoots Netscape, Internet Explorer and Oracle Screens began there.

Javascript (as part of Netscape??)

Apache web server

Project Gutenburg

and, if 'travelling' across the universe fictionally counts as 'widely used outside of the university' then there is HAL in 2001, that (who?) claims to have been activated at the Urbana campus.

WordPerfect

[bfwebster]

The original version of WordPerfect was developed by Bruce Bastian while a grad student at BYU (with Alan Ashton as his faculty advisor). At that time, it was a screen-oriented editor that ran on Data General minicomputers. I know because I shared an office with Bruce during my senior year at BYU (1977-78) and used his existing version of the editor to write several papers for my classes. :-) Bruce & Alan went on to sell a (DG) version to a local city government (Orem, UT) and then founded Satellite Systems Int'l to commercialize the product and ended up owning the MS-DOS word processing market. Word Perfect still might be dominant were it not for Microsoft's brilliant head-fake with OS/2 and Windows 3.0, but that's another story. ..bruce..

Re:WordPerfect

[BJ_Covert_Action]

Ah, I miss the days of WordPerfect .... such a powerful word processing program, and pleasant to use as well. For the life of me I can't understand why MS Word got so widely adopted....

search and ye shall find

[Anonymous Coward]

a couple of small businesses come to mind, there's one called google - the pagerank algorithm was developed by Page and Brinn at Stanford, and another called Autonomy which initially started out as a company called Cambridge Neurodynamics who's main product was the CEO's PhD thesis

Re:How about...

[hazydave]

And Mach (kernel developed at CMU, used in NeXT and MacOS).

Spice bred a large family tree

[kral]

There were two very different versions of SPICE - SPICE2 was a fortran program, and is the basis for the PC version PSPICE (Microsim>OrCAD>Cadence) and minicomputer version HSPICE, though many newer simulators are based on the code for spice3 re-written by a subsequent Berkeley effort in c. Its legacy in electronics engineering is such that even independently generated simulators (Eldo, spectre) rely on the conventions and methods from SPICE, though incorporating incremental improvements (a new algorithm here or there, and distinguishable mainly by how it differs from SPICE).

Eclipse Mylyn

[caniszczyk]

The Eclipse Mylyn (http://eclipse.org/mylyn) project started out as a university research project before it came under the eclipse.org umbrella. Furthermore, It became so successful that a company was launched around its work, Tasktop (http://www.tasktop.com).

Re:Under sufficiently large definitions of "widely

[Gerzel]

You really need to define what you mean by "created" and "entirely within." Very few software development projects are purely academic and a lot of development is done with both academic and non-academic contributors.

GIMP

[Sheik Yerbouti]

Gimp... Well at least the core image processing part was an 11th hour switch from a failed LISP compiler thesis project.

Re:GIMP

[jellomizer]

Successful... Not abominations...

A few I used in my masters

[Georules]

UCODE
PEST
Dakota
Visit
PETSc

There are a good number of small packages like these from academics with one intended purpose that are successful within the field they were developed for. Many times, people not in the field won't have any use for them.

AMANDA

[rgviza]

Advanced Maryland Automatic Network Disk Archiver

Written at University of Maryland College Park.

Solves the problem of backing up zillions of servers and workstation to a single massive storage medium (tape, SAN, whatever)

http://www.amanda.org

Lua (programming)

[Teckla]

It seems no one has mentioned it yet, so I will.

I believe Lua (the programming language and virtual machine) is from academia -- the Pontifical Catholic University of Rio de Janeiro in Brazil.

Lua is used in a lot of games and embedded in a lot of software. I think even World of Warcraft makes use of it.

For more information and links, see About Lua.

Emacs for Windows

[Dolemite_the_Wiz]

NT Emacs (now GNU Emacs for Windows) - University of Washington Computer Science and Engineering

A lot of good software gets its start in academia

[Anonymous Coward]

The original winsock code for Windows was taken directly from BSD source.
The apache web server and the first graphical web browser (Mosaic) were originally developed at NCSA.
And as you said BSD gets plenty of use in 'the real world'

But more importantly, many concepts that are now industry standard get their start in academic prototypes. A lot of standard operating system concepts were developed specifically from BSD version of Unix. The first relational database system (ingres?) was an academic (berkeley?) effort. One could also argue that cloud computing really owes its beginnings to the grid computing effort, which was driven by academic work.

OCaml

[kiwix]

Just one more project that I haven't seen in the previous posts: OCaml is a nice programming language that is used for teaching in France, and also used in a few real-world projects.

Add R to the list

[Musically_ut]

Most new additions to R project are highly academic works, many coming from BioInformatics research as well.

However, some of the modules which people find really useful are rewritten by the core team, so one could say that they were not an output of the PhD/Masters.

In the larger scheme of things, the solutions by academics remain solutions for academic only until they are widely adopted. Then they permeated textbooks, and become the standard solutions of a useful problem. For this, there will exist a software (probably a rewrite) which has optimized it to within an inch of its life.

So the ideas behind the software live on, while the actual lines of code might not.

Re:Under sufficiently large definitions of "widely

[knuthin]

Scrach project? Developed by MIT guys.

That's a bit flamebait-y

[damn_registrars]

A lot of software that is written for graduate school is by nature specialized. So if you are looking for "widely used software" you probably won't find it there, unless you modify the qualifier to "widely used within a field".

That, and very few people complete a bioinformatics PhD in 4 years as asserted in the summary - unless they enter the PhD program with a master's degree already in hand - most take more like 6-7 years. It can often be one of the most difficult PhDs at any institution in part to the fact that it often involves negotiating a minefield of conflicting departmental requirements between departments of Biology, CSci, Math, Statistics, and others.

Re:That's a bit flamebait-y

[multimediavt]

A lot of software that is written for graduate school is by nature specialized. So if you are looking for "widely used software" you probably won't find it there, unless you modify the qualifier to "widely used within a field". That, and very few people complete a bioinformatics PhD in 4 years as asserted in the summary - unless they enter the PhD program with a master's degree already in hand - most take more like 6-7 years. It can often be one of the most difficult PhDs at any institution in part to the fact that it often involves negotiating a minefield of conflicting departmental requirements between departments of Biology, CSci, Math, Statistics, and others.

With a Biology under grad and a CS master's (or vice-versa) coupled with a competent advisor and a quality research topic, there's little reason why a bioinformatics Ph.D. could not be completed in two to three years like I have watched happen outside my office for 5 or 6 years. If a Ph.D. takes longer than that there's either something really big that's come up in the research and you're building on it or triple checking things, OR you're spinning your wheels because the problem domain is too large or the student (or advisor) is a slacker. YMMV but I've watched some very competent students do some amazing things with the right background and focus on their part and their advisors'.

BASIC

[kgeiger]

There. I said it.

moodle

[ezh]

http://moodle.org/

Re:moodle

[SteveFoerster]

This. For those who aren't familiar with it, Moodle is a learning management system that was started by Martin Dougiamas as part of his PhD research into how open source software could support a particular type of instructional design. It's become the main open source alternative to commercial behemoths like Blackboard, and a number of prominent universities have adopted it.

Re:moodle

[SteveFoerster]

Not this.

Re:moodle

[BrokenHalo]

This. :-D

VLC

[jackercr]

VLC (VideoLAN) player.

Java Settlers of Catan

[magic_user]

It may just be a boardgame to you, but I thought his thesis involved the AI. It WAS widely used.

SPICE?

[berlindx]

SPICE started out as somebody's research at UCLA Berkley and has since "inspired and served as a basis for many other circuit simulation programs, in academia, in industry, and in commercial products." http://en.wikipedia.org/wiki/SPICE

How about machine learning tools?

[QaDN]

If we are talking about my subject (machine learning / NLP) a lot of the successful tools are open source and developed in academia.
Now I admit there aren't many large NLP companies out there (Nuance springs to mind, but they have their own software), but there are quite a few medium-sized companies that do quite well using open source Machine Learning tools. (After all, the machine learning algorithm is not what is unique about a specific solution anyway).

For instance, TiMBL ( http://ilk.uvt.nl/timbl/ ) is widely used in several small companies (mainly for development of solutions as it is extremely easy to use).

SAGE

[Jerry]

The math tool. Fantastic piece of software.

Tripwire

[bpfinn]

I believe Tripwire grew out of an academic environment. (Purdue? I think.)

Bioinformatics

[Vornzog]

The 'problem' with bioinformatics is that the field is extremely broad. Unless you write BLAST or one of the big sequence assemblers, your software is only going to appeal to a tiny fragment of an already small bioinformatics community.

I wrote software as part of my Ph.D. that is now distributed world wide. I guarantee you've never heard of it - it sets the standard for how to do certain types of phylogenetic analysis, but almost no one does that analysis.

During my time as a postdoc, I wrote a very simple curve fitting routine and put a minimal GUI on top of it. I am now getting requests from multiple countries to modify it to read in files from their instrumentation. Once again, only the tiniest handful of people care, but for those people, this is revolutionary stuff.

The question here is, how do you define success? Like a lot of the responses to this thread, I wrote a small script here or there to solve my own problem. Turns out, it solved a problem for someone else, too. My best known piece of software was a hack, a one-off script, written in an afternoon, that I got yelled at for even bothering to spend time on, and was only ever intended for my own use. It turned out to be the lynchpin for our project, got published in a peer reviewed journal, and has since gone global. I found out later that one of my undergrad computer science profs had solved the same problem 20 years before I did, in a more elegant way, and published it in a good, but non-science, journal - no one has ever heard of it.

Neither of us had the expectation that our software would amount to much. I would define the prof's work as 'successful' - he published a paper on an interesting academic topic. I would define my software as 'wildly successful' - I got an unexpected publication and a global (if small) user base, along with a reputation for fixing problems that would later get me a good postdoc position.

This isn't really an academia question. The most common advice in the open source community is 'scratch an itch'. Write something to fix a problem you see. If you write good stuff, maybe your code will become 'successful'. Or, maybe your afternoon worth of hacking will just turn into an afternoon worth of experience you can apply to the next problem.

Re:Bioinformatics

[Michael Woodhams]

I wrote software as part of my Ph.D. that is now distributed world wide. I guarantee you've never heard of it - it sets the standard for how to do certain types of phylogenetic analysis, but almost no one does that analysis.

Given that right now I'm writing* proof-of-concept code for phylogenetic analysis, I might have. Would you care to be more specific?

I've got a paper nearing submission on how to best phylogenetically analyse data generated by a method which nearly no one uses.

* or more accurately, procrastinating from writing

Re:Bioinformatics

[Vornzog]

Most of my phylogenetics stuff is unpublished, or mentioned only briefly in other papers. You pretty much have to use the software to know it exists. The 'phylogenetics' itself is mostly DNA neighbor-joining, but we're doing some ancestral state reconstructions to look at what amino acid changes define certain clades (using Fitch parsimony, mostly), and how those chances correlate with observed antigenicity and activity measurements. Turns out that in my particular application, neighbor-joining is the only thing fast enough, and it produces trees that are nearly identical to any of the statistically rigorous methods. All the interesting analysis happens after you build a tree.

What sort of analysis are you working on? Just because a tiny handful of people use it doesn't mean it isn't a cool technique. Good luck procrastinating on your paper.

Re:Bioinformatics

[Michael Woodhams]

Two things: The one nearly ready is a new distance formula suitable for binary characters which evolve by a Dollo process, which (among other things) is suitable for DArT (diversity array technology) data. The other one, I'm more of a junior author, but it involves group theory, models of DNA evolution, and finding models such that if the model parameters vary with time, the overall process is still within the model. To give a counter example: generate two Markov matrices for the GTR model, with different parameters. Multiply them together (as would happen if your sequence evolves for a while under one model, and then evolves under the other.) The resulting composite Markov matrix will not (in general) be a GTR Markov matrix. We're looking at models which do not suffer this 'defect'. The senior authors have developed the models, but I'm implementing them to see how they go on real world data. The opening salvo of this project is http://arxiv.org/pdf/1105.4680.

ROOT

[startwearingpurple]

FWIW. ROOT is a data-analysis framework/toolbox that was originally developed by a few guys at CERN and is now used all over the work in research. Mostly in fundamental physics, but I believe there are a few companies that use it as well (see the website). The papers application is another interesting example...

I guess gnuplot and xmgrace fall in a similar category. There must be specialised utilties/toolboxes in other fields as well, but I would not know where to look.

Apache Mahout

[transonic_shock]

Apache Mahout is based on a Machine Learning using MapReduce paper by Andrew Ng (Stanford)

Molecular dynamics

[SoftwareArtist]

In my field (molecular dynamics), nearly all of the major codes are from academic groups: Gromacs, Amber, Charmm, NAMD... About the only one I can think of that isn't is Desmond, which is from a private (but non-profit) research institute.

Re:Molecular dynamics

[snoop.daub]

Also LAMMPS and DLPOLY, but they are a bit more niche. The ones you mention are used a lot in big pharma these days, for example.

Staying on the chemistry/chemical physics front, quantum chemistry codes like Gaussian all came from academia.

Havok

[johnwbyrd]

Havok is a physics engine commonly used in video games. It was originally developed by Steven Collins while at Trinity College in Dublin.

Tons

[toppavak]

ImageJ
CellProfiler
Open Microscopy Environment
Hugin
Micro-Manager
R

Oregon Trail (MECC)

[mattack2]

OK, it doesn't actually fit the question, since it asked about "universities' labs", but it was the first thing that popped to mind.

I won't go into the details here, since you can read it at Wikipedia, but Oregon Trail started at a school, then went to MECC, which was state-funded.

http://en.wikipedia.org/wiki/The_Oregon_Trail_(video_game)
http://en.wikipedia.org/wiki/Minnesota_Educational_Computing_Consortium

Java thread pool

[SirGarlon]

I seem to remember the Java thread pool libraries came from the State University of New York at Oswego; now I see that were written by prog. Doug Lea and it looks like they have been absorbed into the Java standard library (java.util.concurrent).

Tcl/Tk

[dskoll]

Tcl/Tk began at UC Berkeley.

Spice

[mark_reh]

I don't know if anyone has mentioned it yet- too lazy to read through the hundreds of responses.
Spice was developed at UC Berkeley. It is the basis of dozens of commercial time domain circuit simulator programs.
You can get a great one for free from Linear Technology- it used to be called switcherCAD, they now call it LTSpice.
Get it here: http://www.linear.com/designtools/software/

Re:Under sufficiently large definitions of "widely

[Weezul]

Isn't the first one that comes to mind the world wide web? CERN is definitely academia. I'd imagine many other protocols originate in academia. Any idea about SMTP, Usenet, etc.?

BSD, X11, Mach, PostgreSQL, and SSH were all explicitly academic projects.

There is also a question about what qualifies as academia beyond simply universities and government labs. Linus Torvalds started Linux while a PhD student but later landed in industry. Bjarne Stroustrup worked at AT&T Research when he started C++ but he landed at Texas A&M shortly after.

Virtually all programming languages originate in or near academia : Lisp was MIT. Python was started at CWI. Haskell. OCaml. etc. Among the non-academic languages most originate within huge organizations who's research departments start to resemble academia : Smalltalk was PARC. Fortran and Cobal were IBM. C was AT&T. Erlang was Sony. etc. Java and Perl were seemingly further from academia, but academia's influences upon them abound.

Afaik, all computational libraries used for serious numerical programming, like stock trading, computational fluid dynamics, etc., were developed in academia.

Re:Linux

[TemporalBeing]

It started out that way - but by the time Linus graduated in 1997, linux had become a huge thing, and I bet that if he hadn't made it the topic of his masters - he wouldn't have finished at all.

It started out as a method for Linus to access his work on the school Minix computers (source: Just for Fun). He later did use it as part of a Masters project for doing multi-architecture Operating systems, but that's it. It is mostly a development as a personal (prior to 1995, part-time/full-time without pay while he pursued academic degrees) and commercial (since 1995 when he's been paid to work full-time on it) project.

Clever question generates its own answer

[AmElder]

Is there any list of successful software created entirely inside universities' labs that became widely used?

There is now.

Pine, Pico and Sphinx

[Gregory Arenius]

Pico and Pine are from some Washington University if I recall correctly. And the Sphinx voice recognition software is from CMU.

XWindows was a Stanford masters project

[peter303]

By Paul Asente of Adobe. MIT decided to use it as its GUI platform and productionized it.

GIMP

[Lemmy Caution]

GIMP was the project of a couple of Berkeley CS students.

I think they may even have been undergraduates.

What helps acceptance of Academic Software?

[__aapopf3474]

A somewhat different question is: What helps acceptance of Academic Software. Off the top of my head:

• An open source license
• High quality, readable code
• An active community
• Test cases and nightly builds
• Regular releases
• A faculty member who is a programmer, or at least was a programmer.

There are many other factors, does anyone have favorites? Note that not all academic software is destined to be used outside of academia or to even survive past the end of the semester. That's ok.

Re:Under sufficiently large definitions of "widely

[Friggo]

Just to nitpick a bit, Erlang was developed by Ericsson, not Sony.

SAS and SPSS

[jeremymiles]

SPSS grew out of Stanford University in the 1960s, bought by IBM for $1.2 billion in 2009. SAS, which grew out of NC State, is the world's largest privately held software company.

More things

[Sheik Yerbouti]

SUN actually got it's name from Stanford University Network and SunOS started there as did Cisco. The first Cisco prototype router was built there. And did not Google start there as well? And wasn't Yahoo's first page hosted on Stanford's hardware and started there as well.

Comment removed

[account_deleted]

Comment removed based on user account deletion

How do we trust the results?

[Latent Heat]

I am not saying there is an easy answer to this, but if "most of what gets produced is complete and total garbage", how to we trust any result generated by said PhD student that appears in a journal paper with their advisor's name on it?

In other words, the code may be ugly, and the code may be brittle, but do academics run validation tests?

A few (or perhaps, more than a few)

[Arrogant-Bastard]

Andrew File System - CMU
archie -- Princeton?
CAP (appletalk for Unix) -- Columbia
cops/tripwire -- Purdue
GNU everything -- MIT
Gopher -- Minnesota
Kerberos -- MIT
Khoros -- New Mexico
Mach -- CMU
NNTP -- UC San Diego
Mosaic -- Illinois
sendmail -- UC Berkeley
BSD -- UC Berkeley
RCS -- Purdue
Usenet -- Duke/UNC
tcl/tk -- UC Berkeley
multi-CPU Unix -- Purdue
cu-seeme -- Cornell

I'm sure I'm forgetting quite a few. And of course not all of these are STILL successful, but in their day they made their mark, and often paved the way for other projects.

Re:A few (or perhaps, more than a few)

[Turmio]

LDAP - UMich

I can think of two

[DERoss]

In the early 1960s, the Biostatistics Unit of the School of Medicine at UCLA developed BiMed (or was it BiMd). This was a package of statistical analysis applications that ran on a main-frame computer before the advent of desktop computers, the Internet, or client-server systems. It was widely requested by medical and non-medical researchers, not only at universities but also at various corporations. BiMed went through several versions. No, I don't know whether BiMed still exists today.

MetaCrawler was originally developed in 1994 at the University of Washington by then graduate student Erik Selberg and Associate Professor Oren Etzioni. This is a meta-search engine that sends queries to other seach engines. If you want to search Google, Yahoo!, Bing, and a few others all at once, your query at MetaCrawler uses all those. As with other software developed in universities, MetaCrawler is now owned by a for-profit company. Nevertheless, I still use it. It's at http://www.metacrawler.com./

Re:VMware

[rubycodez]

nope, but two of the founders met at school and were married before starting it.

More: Kermit, MUSIC, Logo, PLATO, etc.

[BBCWatcher]

Kermit, from Columbia University, was at one point the most popular file transfer program (probably). McGill University's MUSIC enjoyed success as a time sharing system. The Logo programming language originated at BBN in Cambridge, so its academic roots are a bit fuzzy, but it was at one time the most popular instructional tool in computing for children. PLATO, from the University of Illinois, became very successful and invented (or at least popularized) numerous computer-based services we now take for granted, such as online forums and instant messaging. Model 204 originated in academia and runs Australia's social security system, among other things.

Re:apache's mod_backhand

[cloudmaster]

Sort of. It was built from patches a bunch of webmasters had made against NCSA httpd.
https://httpd.apache.org/ABOUT_APACHE.html

MATLAB has it's origins in academia

[enjar]

From Cleve Moler's "The Origins of MATLAB" --

"In the late 1970s, following Wirth’s methodology, I used Fortran and portions of LINPACK and EISPACK to develop the first version of MATLAB."

http://www.mathworks.com/company/newsletters/news_notes/clevescorner/dec04.html

Cfengine

[cloudmaster]

Well, the genere of automated config management based on convergence pretty much owes its existence to Mark Burgess's thesis, and Cfengine is his implementation of promise theory. Now you've also got Puppet - which basically exists because Luke didn't like the way Cfengine was going (my opinion may vary), and Chef because that guy didn't quite like how Puppet worked. And several others which are probably listed in a table on Wikipedia or something.

And if you're a sysadmin completely unfamiliar with those, you should rectify that situation now. You're doing too much work.

Yes & no

[Weezul]

As I note upthread, virtually all important programming languages originated in academic-like environments, even if they are officially corporate.

There are I think two revolutionary non-academic programming languages :

- Smalltalk was developed by Xerox PARC, but ultimately created object oriented programming, which certainly used academia to gain traction.

- C was developed by AT&T, but completely revolutionized our world. It's almost surely the most important language ever written. There had been structured languages before. I think Fortran and Cobal were developed by IBM. And academia had all it's research and teaching languages. Yet, it was C that brought structured programming and type-safty to system level programming, previously dominated by assembler. Imho, const is pure genius. C could not help but succeed with or without academia, but AT&T was still a fairly academic environment at that time.

In other words, your classification of generalized academic project doesn't include either afaik, but clearly both can fall under some generalized academia. You could not design C, and maybe Smalltalk too, without thinking deeply about languages from a hybrid academic and industrial perspective. If you pursue a blind industry perspective, you create garbage like PHP or VB.

Re:Under sufficiently large definitions of "widely

[Tellarin]

Erlang is Ericsson.

Re:Under sufficiently large definitions of "widely

[Anonymous Coward]

Erlang was Sony.

Erlang is and always have been developed at Ericsson.

Soon on Wikipedia ?

[advid.net]

I guess this thread is a good start for writing a wikipedia page of "Successful Software From Academia" .
(Likely to be linked with some OSS pages)

Any volunteer ?

Re:Under sufficiently large definitions of "widely

[psmears]

Yes - the two companies have a joint venture that produces mobile phones, but they haven't merged.

Feynman said it best

[HuguesT]

Science is like sex: sometimes something useful comes out of it, but this is not the reason we are doing it.

The role of academia is not to produce successful widgets, it is to produce successful ideas. As it turns out, in computer science, ideas are embodied in software, some of which eventually is picked up somehow and becomes successful. However academia shouldn't be judged by its capacity to produce widgets, this is entirely the wrong metric. It is engineering's role to turn ideas into products and services. This doesn't mean PhD graduates can't go on to become successful engineers, on the contrary, they might better understand the whole lifecycle from great idea to great product.

Cheers to all.

Re:Under sufficiently large definitions of "widely

[jmcvetta]

Some of my undergrad courses used WebCT... god that was awful software.

Re:Under sufficiently large definitions of "widely

[BrokenHalo]

and then it was big enough to be bought (and put out to pasture to die) by Blackboard...

I wish. WebCT is alive and well, and still in use at many universities and other colleges. I have seen it in use over 12 years of academic life, since it seems to have become a fixture at my uni. That doesn't stop me hating it, though.

Re:Under sufficiently large definitions of "widely

[Anonymous Coward]

Erlang was Sony.

Huh!

Erlang = Ericson Language

Ericsson developed Erlang in the 1980's, about 15 years before they had a joint venture with Sony. Sony, Ericsson and Sony-Ericsson are different companies. Sony (an consumer appliances maker) and Sony-Ericsson (a mobile phone maker) have nothing to do with Erlang, as far as I know these companies don't even use it in any product or production, Ericsson (a maker of communications system, the worlds largest, the first maker of automatic telephone systems in the 1870's, the ONLY maker of automatic telephone systems for about 40 years) on the other hand, use it in most of its products.

Even before Sony-Ericsson, the mobile cell phone manufacturing that Ericsson started in 1954 (yes, nineteenfiftyfour) was never more then a side project, something to show off the communications systems with and something to keep in touch with the customers customers. In 2001 they started the joint venture with Sony and ceased production of their own mobile cell phones, because the cost of making mobile phones had increased as they had become increasingly popular outside Northern Europe since the late 1980's. The only large manufacturers of mobile cell phones at that time was Ericsson and Nokia (wow, it is only a decade ago, how the world have changed) and Ericsson wanted more mobile phone users in the world to sell more of their mobile phone systems. It was believed that Sony's expertise in making consumer products would improve the mobile phones user interfaces and their expertise in making large quantities of consumer appliances would reduce manufacturing cost.

Re:Under sufficiently large definitions of "widely

[CertGen]

While there are former U.Wisc. Condor staffers at Redhat now, and Redhat is a major contributor to the Condor project and has much invested in it, it is most definitely owned and operated by Miron Livny and the U.Wisc. computer science department. There's a large group of staff and students there that run the project and add the bulk of the new features to Condor. Anyhow, Condor is definitely an academia project with huge success that comes to my mind.

Exim MTA

[NoseyNick]

Exim MTA - http://exim.org/ - may still be the default sendmail-replacement in quite a few ditros.