The Future of NetBSD

ErisCalmsme writes "In this email Charles Hannum (one of the founders of NetBSD) tells us that 'The NetBSD Project has stagnated to the point of irrelevance. It has gotten to the point that being associated with the project is often more of a liability than an asset. I will attempt to explain how this happened, what the current state of affairs is, and what needs to be done to attempt to fix the situation.' What will happen to NetBSD?"

Comments

This is too important NOT to RTFA

[RLiegh]

So, for your convience, I'm posting it here:

The NetBSD Project has stagnated to the point of irrelevance. It has
gotten to the point that being associated with the project is often
more of a liability than an asset. I will attempt to explain how this
happened, what the current state of affairs is, and what needs to be
done to attempt to fix the situation.

As one of the 4 originators of NetBSD, I am in a fairly unique position.
I am the only one who has continuously participated and/or watched the
project over its entire history. Many changes have taken place, and at
the same time many things have remained the same -- including some of
our early mistakes.

I'd like to say that I'm some great visionary, who foresaw the whole OSS
market, but the fact is that's BS. When we started the project, Linux
and 386BSD were both little hobbyist systems, both pretty buggy, and
both lacking a lot of important hardware support. Mostly we were
scratching an itch: there was no complete package of 386BSD plus the
necessary patches to make it run on more systems and fix bugs, and there
was no sign that Bill Jolitz was going to resurface and do anything.

Much of the project structure evolved because of problems we had early
on. Probably our best choice was to start using central version control
right off; this has enabled a very wide view of the code history and
(eventually) made remote collaboration with a large number of developers
much easier. Some other things we fudged; e.g. Chris got tired of being
the point man for everything, and was trying to graduate college, so we
created an internal "cabal" for managing the project, which became known
as the "core group". Although the web was very new, we set up a web
site fairly early, to disseminate information about the project and our
releases.

Much of this early structure (CVS, web site, cabal, etc.) was copied
verbatim by other open source (this term not being in wide use yet)
projects -- even the form of the project name and the term "core". This
later became a kind of standard template for starting up an open source
project.

Unfortunately, we made some mistakes here. As we've seen over the
years, one of the great successes of Linux was that it had a strong
leader, who set goals and directions, and was able to get people to do
what he wanted -- or find someone else to do it. This latter part is
also a key element; there was no sense that anyone else "owned" a piece
of Linux (although de facto "ownership" has happened in some parts); if
you didn't produce, Linus would use someone else's code. If you wanted
people to use your stuff, you had to keep moving.

NetBSD did not have this. Partly due to lack of people, and partly due
to a more corporate mentality, projects were often "locked". One person
would say they were working on a project, and everyone else would be
told to refer to them. Often these projects stagnated, or never
progressed at all. If they did, the motivators were often very slow.
As a result, many important projects have moved at a glacial pace, or
never materialized at all.

I'm sorry to say that I helped create this problem, and that most of the
projects which modeled themselves after NetBSD (probably due to its high
popularity in 1993 and 1994) have suffered similar problems. FreeBSD
and XFree86, for example, have both forked successor projects (Dragonfly
and X.org) for very similar reasons.

Unfortunately, these problems still exist in the NetBSD project today,
and nothing is being done to fix them.

--

I won't attempt to pin blame on any specific people for this, except to
say that some of it is definitely my fault. It's only in retrospect
that I see so clearly the need for a very strong leader. Had I pursued
it 10 years ago, things might be very different. Such is life. But
let's talk about the situation today.

Today,

Sounds bleak

[BadAnalogyGuy]

Almost as if NetBSD is dying.

I don't buy it though. It's free.

Re:Sounds bleak

[rsidd]

(If you think there's the slightest chance this was meant to be funny, it was.)

1. Take a perfectly good joke.
2. Emphasise it, highlight it, add a laugh track.
3. Et voila! American humor.

Re:Sounds bleak

[hey!]

Almost as if NetBSD is dying.

If the NetBSD project dies, it will be an interesting to watch from a mad scientists/vivisectionist viewpoint.

One of the important things about free/open source software is that it's not tied to an organization. This is very important in the survival of software.

I'd like to put for the following conjecture:

A piece of open source sofware will continue to be maintained in perpetuity, surviving the demise of its main sponsoring organizations, provided that the following three conditions hold:
(1) It has a modest but reasonable number of users relative to its complexity
(2) There are no other open source projects that users can switch to with very little effot; the threhold level of effort is reduced as the total number of users increases.
(3) There are no patents that cover fundamental aspects of the software's operation.

For example, provided that nothing fundamental to the Linux kernel violates patents, I'd suggest that the Linux kernel is immortal. (1) It is complex, but has a huge number of users; (2) While BSD would be the most logical move (possibly a BSD distribution using the BSD kernel with GNU tools?), it would require a modest amount of retraining for things like networking and system administration. (3) So far as we know there are no credible assertions of IP violations in the Linux kernel.

NetBSD, I'd suggest, is a candidate for extinction under this conjecture.

(1) It is complex relative to the number of users: see the article's discussion of problems with threading and multiple processors. Of the three "big" BSD distros, it has by far the fewest numbers of users.

(2) It is probable that mostof its users can switch to a different BSD with very little trouble. NetBSD's reputation is that it is the most portable of the BSDs, not the most featureful. Therefore if you can switch, it should be easy. The only group that would drive further maintenance would be people who run NetBSD on very old computers not supported by other operating systems.

(3) Patent problems: none known at this time.

Re:Sounds bleak

[spauldo]

There's a lot of reasons to have it around.

The BSDs do stuff differently, and there's a lot of cross-pollination among them (and to a lesser extent linux). Someone might have an idea they implement in NetBSD that ends up getting ported to FreeBSD and OpenBSD, and vice-versa.

You also have the fact that the focus of the three major BSDs are different - FreeBSD is a general system, OpenBSD is focused on security, and NetBSD is focused on portability between different architectures.

This also gives more people the chance to contribute to the system in general. If you've got an idea for a new scheduler, you can try to get it implemented on one of the systems. If it works, other systems may copy it for themselves. If there's only one system, though, it's a lot harder to get into development because there's fifty other people with scheduler ideas you have to compete with.

Then, of course, the real reason why there's multiple BSDs around - developers want to work on them. Let them have their fun - just 'cause they make it doesn't mean you have to use it.

Re:Sounds bleak

[bradkittenbrink]

I thought it wasn't referring to anything in particular, but I think it should have been referring to Debian kFreeBSD. That's the clear migration path for linux users if linux goes belly up.

Very well put...

[chaoskitty]

While there will be those who see this as flamebait, it's high time someone puts into words what many of us are thinking - namely, that something's not quite right, and we should look to those with more experience to give us some clues...

Netcraft?

[DavidpFitz]

Has Netcraft weighed in on this yet?

</troll>

A Sad Day? Or maybe a new start?

[Anonymous Coward]

Okay... I'm just gonna sound off here, as AC.. mostly because this could start a holy war and i don't feel like karma whoring. Which is agaisnt my intentions.

This is simple, I've been a long time linux supporter, user, and contributor. Not one of these slashdot citizens that everytime a new "feature" of Windows version "X" is leaked, go and bitch about installing "Linux distro flavor of the month" on there machine and never use windows again. Then turn around and get the new version of Windows "X".

With that said, this news is both sad, and slightly hopeful for me. As much as I love Linux. I've had a soft spot for NetBSD. Mostly because it can run on anything, really portable and good for embed, applications were Linux is just to heavy. Also for securiy, its one of the best.

I'm also hopeful. NetBSD is a niche' OS, and one hell of a good one. Maybe the light of this could help get people to turn the project around. I for one and downloading the entire source tree as I type. For one, so i havee a virgin copy of release 4.0 and the latest CVS, and for two... to see if maybe i could help out with something. If only in a small way.

Even if I don't plan on using NetBSD on my desktop, which is SuSE 10.1 btw, I beleive it still as much to do in the niche applications, because oif we let niche OSes fail. And one OS expands to do everything, we all lose, and end up in another Microsoft Windows style mess.

Thats my 2 cents for the night.

Obligatory quotes

[zensonic]

"I sense a great disturbance in the Force, as if millions of toasters cried out in terror and were suddenly silenced..."

Doesn't seem right

[bobintetley]

I love NetSBD. It's stable, it's fast, the package management is great (and upto date), NetBSD folks don't seem to feel the need to evangalise and beat people over the head with their OS choice. A lot of interesting development is also done in NetBSD (like integrating Xen into NetBSD 3.0, the CCD driver, RAIDframe, etc).

I don't understand what this guy's on about - I use it and love it, so do lots of other people, we have upto date software and a great base system. How exactly is NetBSD irrelevant again? Is he bitching because of a lack of marketshare compared to other BSD/Linux distros? In a world of free software, why exactly does that matter?

It's disingenuous to bitch about the things he does as if they were important - flash file system? So what? Journaled file system? There's a very good reason for the omission of journalling and you can't tell me this guy doesn't know about softdeps.

Just sounds to me like this guy is pissed off with not getting some kind of glory for his work and it's all sour grapes.

Re:Doesn't seem right

[Guybrush_T]

I used to love NetBSD. The package management is indeed great. BUT :

• About stability, it worked well on a 486, but I never managed to run it stable on a brand-new machine with an Athlon (the kernel was always falling in vm_page_fault traps) whereas linux worked with no issue (hence not a hardware issue, more something like an AMD issue?).
• About speed, again, unfortunately, linux performed better.

I really wanted to stick to NetBSD, but after 1 year trying to have it functionnal, I installed linux, and this day, all my problems were gone.

I'm afraid that the great thing in NetBSD - which is multiple platforms support - will soon be irrelevant, since linux already supports all the currently-used architectures.

In all case, I hope NetBSD will survive and become more usable. But as said, it needs a lot of work.

How many BSDs do we need?

[QuantumFTL]

In a world with Linux, Solaris, OS X (a BSD, sorta), a zillion types of Windows, QNX, etc... why do we need so many BSDs? I mean, if someone wants to make their own version of an OSS project, that's up to them, but if you want to be relevant, you have to offer something new and relevant. If I'm going to run BSD on a server, I'm going to run something security oriented like OpenBSD so I can spend more time developing my applications.

I think the NetBSD folks have done some great work in the past, and it deserves to be remembered, but maybe it's best that they apply their efforts to some more relevant projects, such as another BSD, or better yet, Linux, which has been constantly lagging behind OpenBSD in security and the like.

Diversity is a powerful part of the FOSS model, however it can also dilute things by spreading resources to thin. Thanks for your hard work guys -- lets move on to the next challenge!

Re:How many BSDs do we need?

[despisethesun]

I think it's ridiculous that people complain about how many BSDs there are when there are so many redundant Linux distros out there. Each of the BSDs has a different development focus, making them all more relevant and important than any of the few dozen Debian-based distros, for example, which all really don't do much to build on what Debian has already done. When people quit starting up go-nowhere Linux distros and contribute their efforts to "more relevant projects", guys like you can go ahead and tell these guys to quit wasting their time on NetBSD.

Re:How many BSDs do we need?

[cyclop]

If we all would just settle on Slackware, the ONE TRUE DISTRO, everything would be perfect.

(rumours start in the plaza)

- Slackware? Hah! No package management!The one true distro is Debian and its mighty apt-get!
- What are you saying? It's clearly Gentoo! You compile everything from source!
- Gentoo is for ricers! People that want their work done use Ubuntu!
- Ubuntu? I'm more comfortable with Suse and Novell support...
- What? Bear that RPM hell? Go use Knoppix!
- What about RPMs? On my Fedora work so well...
...

(everything in flames)

Mergers and Acquisition

[delirium of disorder]

OpenBSD was a fork() of netbsd. Is there any chance they could reunite to make a single stronger OS? How difficult would reconciling the politics and the codebase be?

Leadership

[wysiwia]

It's seldom that a founder of a OSS project acknowledges his mistakes but Charles M. Hannum does it. Just for this simple action I value his reasoning very much.

IMO leadership of a project is very important because leaders always have a vision and the drive to force this vision become true. There's no guaranty that a leader will be successful with his vision but definitely comities always will fail they never have a single vision and never can agree to force a single vision become true. So whenever a project is lead by a comity stagnation is not far off.

Yet leadership does not mean dictatorship as often is done by many OSS project leaders. Dictators will equally bring a project down as do comities. There's unfortunately no clear distinction when a leader becomes a dictator as many times good leaders are just lucky avoiding the path to dictatorship by sheer luck.

O. Wyss

respect

[tezbobobo]

I must say, it is an interesting read but I am struck by the humility and honesty of this guy.

Not surprized

[PrayingWolf]

I've been waiting for this to happen ever since I read how Theo De Raadt was treated in there and how he eventually left the group to work on his own branch. I think you can find an archive of his emails with the NetBSD dev team somewhere...

Now the problem is admitted: FTA:

Partly due to lack of people, and partly due to a more corporate mentality, projects were often "locked". One person would say they were working on a project, and everyone else would be told to refer to them. Often these projects stagnated, or never progressed at all. If they did, the motivators were often very slow. As a result, many important projects have moved at a glacial pace, or never materialized at all.

This is basically what drove Theo out (as far as I understand his great ideas were ignored by the boureaucratic system and he felt frustrated) and now the basic reason why NetBSD is dying.

But NetBSD still lives: in its decendants, like OpenBSD. So let us treat NetBSD with the same respect we would give to a dying grandfather :)

Re:Not surprized

[Renegade88]

What I'm surprised about is that you read the email chain but came to the conclusion that Theo's ideas were "ignored". That's not what happened. They desperated wanted his ideas and his code, but they told him he could not COMMIT the code himself, but rather work through an intermediary, one that had no technical skill. It's like telling the former CEO to report to the janitor. You got it half right, but either you didn't read the whole chain, or your memory is failing you.

What goes around comes around

[Renegade88]

I'm a big BSD guy, mainly a FreeBSD user, but I intently follow DragonFlyBSD and OpenBSD. Unless I'm mistaken, this is the same Charles Hannum that was directly responsible for kicking fellow NetBSD founder, Theo de Raadt, out of the core group, removed his CVS priviledges, and made Theo twist in the wind for 7 months until he was forced to leave to found OpenBSD. (reading the log I don't see how Theo lasted 7 weeks, he really made an effort to continue with NetBSD despite all of that). So now the evil cabal takes over and kicks Charles out of the core and removes his commit priviledges. It's sad, and I think Charles' points are spot on, but it's a bitter pill to swallow coming from this messenger. You have to shake your head when you think of what NetBSD could have been had they been able to avoid childish political antics in their "cabal".

Re:What goes around comes around

[LizardKing]

If you'd read the email log more closely, you'd notice that Charles Hannum one of those who was involved in the removal of de Raadt's commit privileges, but then tried to come up with a workable way for Theo to continue working on the project. The whole story has never come out, as the NetBSD core group kept very quiet about what the motivation for removing de Raadt's commit privileges were. However what is not in doubt is that Theo's attitude on the NetBSD mailing lists was abusive towards anyone who he felt was not as technically competent or as well informed as him. This was annoying fellow developers and alienating potential users. Theo was asked to tone down his attitude, or at least ignore postings that he would otherwise have posted inflammatory replies to. He didn't, and my assumption is that the core group removed the commit privileges to distant the "official" project from Theo's shitty attitude. Theo obviously resented this, but continued to badmouth people until he finally forked NetBSD to create OpenBSD - a sandpit where he could fuck people off to his hearts content.

Now it seems Charles Hannum is pissed at someone, and has decided to belittle the work of many current NetBSD developers by cross posting his flame to the Free, Net and Open mailing lists. My opinion for what it's worth? The NetBSD Foundation appears to be dominated by Wasabi personnel, and as a result the decisions it takes may be in the interests of Wasabi commercial interests rather than Charles Hannums. However, there is good work going on in the NetBSD project, and all Hannum's post will do is make the Linux/anti-BSD zealots shriller.

Re:What goes around comes around

[JanneM]

Unless I'm mistaken, this is the same Charles Hannum that was directly responsible for kicking fellow NetBSD founder, Theo de Raadt, out of the core group, removed his CVS priviledges, and made Theo twist in the wind for 7 months until he was forced to leave to found OpenBSD.

Theo "voice of reason" de Raadt? Imagine that, someone not getting along with him. What are the odds, really?

Re:What goes around comes around

[Renegade88]

I read the email log extremely closely. Charles was in the process of creating a "special" set of rules for Theo, that only Theo had to agree to. While he was being jerked around, five additional people earned commit priviledges, but where not made to agree to these "new" criteria. This set of rules was never completed, it was dragged out intentionally, basically "you have to agree to these rules first, but you can't do that until we write them, and we can't give you a date when we will write them even though it's already been weeks".

I would love to post the link to the email log but it would crash the server it's on.

Even though the developer put in charge of Theo's sparc port wanted Theo to have his commit priviledges restored, and asked for it a couple of times, the core refused. The only "workable" solution that was offered was that Theo could pass his diffs on to the port developer and let him merge them. Basically it was a set of conditions that nobody would agree to. The email chain is quite clear that Charles was instrumental in Theo losing the commit priviledges and never intended to restore them. It is also obvious they were jerking him around until he just quit on his own.

My take on Theo:
I think his "utter asshole" reputation is not accurate. He's said some things he probably wants to have back, and likely hurt some feelings. I also think he was cordial during this 7 month jerk-around session, enduring it FAR longer than most people would, and he said all the right things to earn the commit priviledges back. He was willing to "play ball".

Charles might be a good guy, but he wasn't well like during this time in 1995 and forcing Theo out is a black mark on his record. You can't tell me NetBSD is better off now (dying) without Theo then they would have been with him on their team.

The real story here...

[Anonymous Coward]

... is that there are far fewer novel toaster designs being produced every year. It is a Well Known Fact that NetBSD has put forth millions of man-hours into porting their OS to every new toaster design released by manufacturers across the globe. With the recent sharp decline in toaster research, development, and production it was only inevitable that NetBSD development should come to a standstill. Modern convection ovens run Linux, and it's just not the same. What does a penguin know about heat anyway?

Interesting read

[porkThreeWays]

It is a pretty interesting read. I can give you my experience with NetBSD over the past couple of years...

Outside of my regular job we were developing an embedded system. The first thing I thought of was NetBSD. Downloaded it, tested it, critiqued it, and couldn't find enough benefit to use it. The big gotcha was there was no filesystem at the time for running on flash devices. Well, almost every embedded project is going to run on a flash device. Mind you this was a couple of years ago, but according to the post not much has changed. There were a couple of other small gotchas, but in comparing it to Linux, there just wasn't enough reason to use NetBSD.

And therein lies much of the problem. I don't think NetBSD is bad. It's not. However, a lot more people are using Linux for advanced embedded devices than NetBSD and are solving real world problems so you don't have to. NetBSD may run on a plethora of hardware pretty well. But 90% of the embedded world really needs it to run on is i386, arm, and mips. So there is really good linux support for those arches because so many people are developing systems with the linux/uclibc/arm combo. It's the new lamp. NetBSD may have the shock factor of running on things like toasters, but Linux is running on real world things like my phone.

On top of that, the term "embedded" is becoming looser and looser. There was a time when "embedded" meant a 12mhz processor and everything was in assembly and C. Today, I can get a 400mhz gumstix and do all my development in python. I would consider it embedded by today's standards, but in reality that was a normal desktop development machine 5 years ago.

Again, NetBSD isn't bad. If I had to really run something on a 12mhz CPU I doubt I'd be able to use linux/uclibc/arm and NetBSD might be my answer. However, in a world where embedded hardware is the desktop hardware of 5 years ago, there just isn't any benefit to trying to use the same embedded tools of 5 years ago.

Freudian Slip??!?

[sh0dan]

To: None <netbsd-users@netbsd.org>

:)

Re:one more brick in the wall

[Alcoholic Synonymous]

Pay more attention next time. That is outdated news. OpenBSD made it through by picking up some major sponsors, including other F/OSS projects that use part of OpenBSD's code in thier projects.

Re:Nature doing what it does best...

[hclyff]

Pandas are still around because of "survival of the cutest". Much like OSX.

Why I don't use NetBSD

[Danathar]

It's not because I have anything against it, it's just that everytime I think about NetBSD I can't come up with a REASON for running it (other than for pure nerd exploration purposes).

If I want to be secure I run OpenBSD, if I like the "UNIX" model over the "LINUX" way of grokking things I'll run FreeBSD. In the past NetBSD's mantra was portability. I don't think that's a big enough selling point.

Differentiation is what sells (it seems). NetBSD needs to be something the others are not doing.

I hope it survives and hope that the people involved are mature enough not to let their EGO's get in the way.

In some ways they have a GOLDEN opportunity. NetBSD is far enough along that they don't have to start from scratch, but small enough (organization wise) to allow them to possibly do something that LINUX and FreeBSD are too big to handle.

I don't know what that is..but I hope it's something cool!