Bob Bruce on the BSDI/Walnut Creek Merger

Earlier we announced that Walnut Creek CDROM and BSDI are merging. Obviously, this will shake up the BSD playing field considerably, and will doubtless have knock on effects with the Linux and other open source communities too, to say nothing of the various commercial organisations that already rely on FreeBSD or BSDI on their servers. Here, to answer questions about the merger, is Bob Bruce, president of Walnut Creek. Jordan Hubbard, FreeBSD core team member and release co-ordinator, and Gary Johnson, CEO of the new company, also contribute.

But first, a little bit of background.

Walnut Creek sells CDs full of freely available software, and run the world's busiest FTP server (ftp.freesoftware.com, formerly known as ftp.cdrom.com). Walnut Creek has been involved with FreeBSD since the early days, producing the first FreeBSD CD distribution, and providing gainful employment for some members of the FreeBSD development community. This article from FreeBSDzine explains some of the Walnut Creek/FreeBSD relationship.

BSDI was formed by members of the Computer Systems Research Group (CSRG) at UC Berkeley (i.e., the same group responsible for the Unix BSD in the first place). BSDI produce, market, and provide support and training for a commercial BSD Unix (or Unix-like) OS for the Intel platform, and as such, are competing in the same space that FreeBSD, NetBSD, OpenBSD, and Linux (not to mention Microsoft) all play in.

While there have been several IPOs, mergers, and acquisitions by various Linux-driven companies in the relatively recent past, this is the first in BSD space. Here Bob Bruce answers questions about the merger, some of which came from yourselves, via the earlier story.

[Disclaimer: in another xterm I'm nik@freebsd.org, which obviously gives me a greater vested interest in this event than most. Unless otherwise indicated, the answers are from Bob.]

What, exactly, is happening? The reports from DaemonNews are that the two companies are merging, the Wall Street Journal says that BSDI is "acquiring" Walnut Creek. Is this a merger of equals, or will one company be the dominant partner?

Walnut Creek CDROM and BSDI have merged into a single company. I don't think either partner will be dominant. Walnut Creek CDROM and BSDI had very similar company cultures, and both companies had a long history of involvement in the BSD community. Several BSDI people were members of UC Berkeley's Computer Systems Research Group (CSRG), and were key early contributors to the free software movement. Several of the top people in the FreeBSD project worked for Walnut Creek CDROM. So together we have an extemely talented group of people.

The two companies mesh well at the sales and marketing level, too. Walnut Creek CDROM brings online sales and retail channel expertise. Our shrink-wrap products are sold in nearly all major chains, including CompUSA, MicroCenter, Fry's, Border's, etc. BSDI brings expertise in VAR/OEM sales, embedded systems, corporate sales and infrastructure markets. For instance, UUNet runs on BSD/OS.

Why merge at all? FreeBSD and BSDI have coexisted for some years now. BSDI uses code from FreeBSD (I believe) and they have also contributed code back to FreeBSD. What benefits will merging bring?

The obvious advantage is that our development efforts will be sharing codebases. So we can make improvements faster and at lower cost. Our goal will be a single ABI for third party vendors to port to, which will mean more applications available on BSD.

People who are trying to decide whether to rely on Linux or BSD should note that the two commercial BSD-based companies are unifying at the same time that the Linux market is being divided up into smaller and smaller fragments. It seems like every few weeks another company announces a Linux distro. I just read that Motorola is producing their own. Now don't get me wrong, I really wish that Linux wasn't so fragmented. In fact, I would like to go back to the good old days when Slackware had a 90% market share ;-)

Gary adds: A merger makes a lot of sense from a promotion and funding perspective. You'll be seeing much more promotion about BSD in the future. It's often been said that we've got the superior technology, but the marketing has been nearly non-existent. That's going to change.

More money means more development funds as well. For example, an Itanium project is definitely in the pipeline. No firm details yet, but it will happen.

What's the new company going to be called?

Currently BSDI is the name of the company, but we are looking at alternatives.

Where is BSD's focus going to be? The traditional role has been servers, with some developers expressing a disdain for the desktop.

Gary says: We're committed to growing our position in the marketplace. Traditionally, the Internet Infrastructure market has been where we're strongest, and that's where we're going to focus. But not to the exclusion of promoting BSD for the desktop. FreeBSD and BSDI both have major customers in the embedded market, and among infrastructure suppliers.

What will happen to the FreeBSD codebase? Is it going to merge with the BSDI code, or vice-versa? Will there be two code trees?

There are plans for the two codebases to merge, but not immediately. FreeBSD 4.0 is scheduled for release very soon [4.0 should go gold on March 13th -- Nik] and will not contain any code from BSD/OS. Once 4.0 becomes the "stable" branch, the merging work will begin on the "current" branch. There will also be continuing improvements and enhancements to the proprietry BSD/OS.

Jordan adds: I also see us best doing this by gradual convergence, not by simply attempting to ram one group or code-base into another. It will take time for the FreeBSD developers to come up to speed on the various features in BSD/OS they may be merging and the converse is true for the BSDI developers; they've been working on BSD/OS for the last decade and will need time to familiarize themselves with the FreeBSD code base, its development methodologies and its culture.

There are a number of FreeBSD features which existing BSD/OS customers have requested just as there are a number of BSD/OS features which the FreeBSD project will be looking into merging. By doing this work in parallel, we can deal with the familiarization issues on both sides while making the eventual code merger progressively easier.

David Greenman and Mike Karels will be working together as co-architects for the new system. As features are merged in, they will be available for download at www.freebsd.org, and on "snapshot" CDROMs. The completely merged system will be released as FreeBSD 5.0.

Who 'owns' FreeBSD now? Jordan's been talking about a FreeBSD Foundation, is that going to see the light of day?

FreeBSD is "owned" by the FreeBSD Project, which is made up of the core team, the committers, and all the other people who write the code. No company can ever own FreeBSD, anymore than a company can own Linux. The FreeBSD Foundation will be an independent non-profit organization. It will be controlled by a board of directors, which will contain some members of the core team. The Foundation and the core team are still separate entities going forward.

What about the name "FreeBSD", and related tems, like the "PicoBSD" distribution? If someone wants to build the own CD-ROM release, can they call it "FreeBSD"?

The trademark "FreeBSD" was owned by Walnut Creek CDROM, so it is now a registered trademark of the merged company. But, by written agreement, the use of the trademark is controlled by the FreeBSD core team.

Roughly speaking, how many new, full time, developers is this going to bring to FreeBSD?

About twenty. But that will grow. We are hiring, so if anyone out there in Slashdot land needs a job and knows how to code, send your resume to jobs@cdrom.com.

Are there any BSDI only features that are slated for rapid integration into FreeBSD?

No. FreeBSD and BSD/OS have well deserved reputations for robustness and stability. We don't want to jeopardize that by rushing the integration process.

Are any features of the BSDI codebase going to stay proprietary?

At first much of the BSDI codebase will remain proprietary. It will only be freed as it is integrated with FreeBSD. There are some parts of the codebase that cannot be freed because the code was written under a contract that does not permit disclosure.

Jordan adds: This pertains only to the BSD/OS kernel. There is intention to merge the "userland" code as soon as is practical, since there are no issues with any of that code, according to Mike Karels.

Prompted by a question from "dcs" earlier; most of the FreeBSD developers meet through the mailing lists, and no one is "parachuted" in to a committer position without first submitting PRs, having them accepted, being proposed for committership, having a mentor, and so on. Everyone's got where they are by working with one another, and going through the peer review process.

Will the BSDI developers (those of them who will be working on the FreeBSD code) have to go through the same (or possibly accelerated) process? Or will they be dropped in as committers almost immediately?

Jordan: I think it's too early to say how each and every case will be handled, but that it'd also be reasonable to assume that a hybrid approach will be used. In each case we'll be first estabishing just what each potential new committer will be doing and what the priority for that work is, then we'll look at their track record and take prior experience into account just as we've done with many other committers.

Some committers have approached us as complete unknowns and have had to enter the project "the slow way", by submitting PRs and essentially proving themselves to us over time. In other cases, a committer has approached us (or vice-versa) and the core decision to add him has gone in a matter of hours from concept to edits to the access file ("Kirk wants a commit bit? stamp Done! Somebody go grab him before he changes his mind!"). It all really depends on just how much of a known-quantity the person is and I expect that to work in everyone's favor in a good deal of the cases for BSDI developers.

It's been suggested that some of the BSDI people would be coming in as FreeBSD core team members. Is that right?

Jordan: This really isn't clear yet and all we've talked about is the fact that we'd be willing to take some on if they expressed a direct interest. This has yet to happen as I'm sure most of the folks over at BSDI are still too busy staggering around and trying to cope with this Brave New World to think about things like joining FreeBSD-core. I would also expect to be able to spend a little time explaining just what this means in reality to any prospective candidate in advance, such being only fair.

Are there any changes planned for the BSD license?

That is up to the University of California. We have no control over the BSD license. UC removed the advertising clause last year, which was the right thing to do. I can't think of anything else that should be changed. The BSD license is about as free as you can get short of public domain, and its "business friendly" nature is one of the reasons that BSD is so widely used in commercial applications which require customized kernels, such as the IBM Interjet, the Intel StorageServer, and Inktomi's network products.

Will the code still be released under the BSD license? If so, which one? There are three BSD licenses available; the original BSD license with four clauses, the new BSD license, without the advertising clause, and the FreeBSD license, which also omits the fourth clause as well.

It will be released under the newest license, without the advertising clause. Contrary to other reports, once the BSDI code is released under the three clause license it will be usable by anyone, not just other open source projects.

How will this impact on other organisations use of the codebase? For example, Apple use FreeBSD extensively in MacOS X.

It will have a positive impact. We have been unable to work more closely with partners like Apple because of limited resources, especially not enough people. That will change. We will be strengthening existing partnerships and building new ones.

Jordan adds: It should also be noted that people who are doing their own FreeBSD-based solutions can continue to do so, we're not changing the terms under which FreeBSD is being released or can be used in other products. Nothing changes for FreeBSD here and it's in the areas where a customer wants more than FreeBSD can currently deliver for, say, an embedded systems product that I see the commercial possibilities. Anyong buying a commercial RTOS today gets a lot more than a CDROM containing some bits, for example, they also get about four feet of printed manuals, a support contract, tools for doing cross-compilation on other platforms (like NT), etc. Providing those kind of value-adds for FreeBSD on a commercial basis would be a win-win scenario for everyone, I think, and that's just one possible avenue of exploration.

Living and working in the UK, BSD's presence here is limited. I can get various Linux flavours from the bigger PC stores, but BSD seems to be relegated to a few, more specialist, suppliers. Red Hat has announced plans to expand in to Europe, and Suse is based in Germany. Is this a market you're going for, or are you contentrating on the U.S. at the moment?

Gary: In the UK, you may find FreeBSD on the shelves of Dixon's soon [Big consumer electronics chain, computers, cameras, hi-fi systems, that sort of thing -- Nik]. You can expect to see big changes in our market presence in Europe, particularly over the next six to 12 months. I can't say more than that at the moment.

Walnut Creek sell 'competing' products, including Slackware and Redhat Linux. Is this going to change?

Our Slackware division will be spun off as an independent company: Slackware Linux, Inc. But our Linux and BSD developers will continue to work closely together. Patrick Volkerding has moved out here from Minnesota and is now managing Slackware development on a day-to-day basis. We will be releasing Slackware 7.1 by summer.

What about other distributions (or, indeed, other OSs). Would the new company release OpenBSD or NetBSD CDs, for example?

We have no plans to do that.

But we are trying to get the entire BSD community to work together more. For instance, our annual conference, FreeBSD'Con, is being renamed BSD'Con, and will now include all BSD software, not just FreeBSD.

There is frequent cross-fertilization of ideas and code between the Linux and BSD communities. Probably a lot more than most people realize. This is what "Open Source" is all about: both communities benefit from fresh ideas as well as healthy competition. We are at the focal point where these two communities come together, and this is a very exciting place to be.

Comments

Re:FUD alert!

[Guy Harris]

I suspect many claims of fragmentation of Linux may be overstated.

However, I think the claim:

The Linux world is not fragmented at all--one kernel source tree and one user-land source tree goes into every single Linux distribution out there.

is also overstated.

First of all, there's the issue of versions of software. Linux distributions are built from many pieces maintained by different people, so it is possible that a given release one distribution might have a 2.2.x kernel and a 2.1.y glibc and so on, whilst a given release of another distribution that has the same 2.2.x kernel might have a different version of glibc, or a different version of utility XXX, or whatever - meaning that different user-land source trees are going into those distributions. Perhaps the next release of the second distribution uses the same version of glibc as the release in question of the first distribution, but it might then have a different kernel, or a different version of utility XXX or YYY, or whatever.

In addition, some distributions may well add their own changes to some or all of the components they bundle into their distributions. For example, it is not the case that all Linux distributions have "standard" versions of libpcap - Red Hat 6.1 has a patched version that, whilst it has some improvements, changes the format of capture files written by applications that use the libpcap code to write capture files (e.g., tcpdump) and doesn't change the magic number on those files. This obliged Ethereal, which uses its own library to read capture files in order to handle capture files that are not libpcap-format capture files, to go through some pain in order to be able to read

1. standard libpcap capture files;
2. libpcap capture files with the changed format and unchanged magic number;
3. libpcap capture files with the changed format and a different magic number, as produced by a later version of the patch than was picked up by Red Hat 6.1;

transparently (which the patched libpcaps don't do - the later patch can read the original format with the standard magic number, and can read the changed format with the new magic number, but can't read the changed format from files that use the standard magic number and that format; I can sympathize with the patch's developer for not doing so, as the hack I put into Ethereal's capture-file-reading library is really a bit gross, but, for Ethereal, I wanted it to be able to Just Work - hand it a capture file, and it consumes it, without having to be told what it is, regardless of whether the libpcap file that comes with the system on which it's running is capable of doing so). (I filed a bug on this, as did several other people; apparently a future Red Hat release will pick up a later patch, so that the libpcap files with a different file format will have a different magic number, and files from unpatched versions of libpcap can be read by tcpdump, for example.)

And, of course, not all Linux distributions have the same configuration file setup - for example, not all of them use a System V-style init and rc files.

I suspect there are people out there who can cite other examples of being bitten by differences between distributions, so I consider it an error to assert that "one kernel source tree and one user-land source tree goes into every single Linux distribution out there", or even to assert that the differences between the source that goes into those distributions don't cause any problems.

None of this, however, indicates that

1. Linux is necessarily any more badly fragmented than the BSDs (I suspect that, on the whole, it's easier to move software between Linux distributions than it is to move them from one BSD system to another, although I suspect most software probably moves with relatively little pain between Linux distributions, between BSD systems, from Linux distribution XXX to BSD system YYY, from BSD system YYY to Solaris, from Solaris to HP-UX, etc. - although there are presumably examples of pain caused by all of those transitions);
2. there is no fragmentation in BSD.

Market Fragmentation

[panda]

People who are trying to decide whether to rely on Linux or BSD should note that the two commercial BSD-based companies are unifying at the same time that the Linux market is being divided up into smaller and smaller fragments. It seems like every few weeks another company announces a Linux distro. I just read that Motorola is producing their own. Now don't get me wrong, I really wish that Linux wasn't so fragmented. In fact, I would like to go back to the good old days when Slackware had a 90% market share ;-)

We're not getting you wrong, Mr. Bruce. It's painfully obvious that you don't like the vitality of the competition to BSD from GNU/Linux.

Why is it that everyone just assumes "market fragmentation" is bad? Why do they assume that GNU/Linux is going to follow the same old rules that the UNIX marketplace followed in the late '80s?

Yeah, the UNIX market is fragmented with *BSD, SysV, AIX, IRIX, HP-UX, GNU/Linux, etc. ad infinitum.

And yeah, the GNU/Linux market is fragmented, but so what? When 99% of your software will compile and run on your system regardless of the distro that you use, what does it matter? So PHBs get confused about which distro is "better," so what? PHBs shouldn't be deciding which distro to buy/download, the techies ought to be making those decisions. (Yeah, I know, most corporations don't work that way, neither does the one that currently has me enslaved.) Where is the evidence that a fragmented market is bad? I always thought the mantra was "choice [in the marketplace] is good for consumers?" If not, then why is everyone beating up on Microsoft?

So, when you're consumer/choice oriented you call it "competition." When you're a big company whose competing in that marketplace of choice, you call it "market fragmentation."

Of course, I can't fault Mr. Bruce for wishing Slackware had a 90% market share.

Long experience with BSDI and FreeBSD

[speedbump]

I started up my ISP business in 1993 using BSDI 1.1. We still have the original server we started with, and it is running BSDI 2.0. It is our primary domain name server, as well as our primary web server, and handles 10,000 emails a day, over 2 million web hits a month, 500 FTP sessions a day, and still runs mostly idle.

Here's the specs on this server: Pentium 133 (yes, really!) with 64megs RAM. The SCSI-II card is an EISA bus dinosaur, but it works. Although we have PCI slots on the server, the operating system is old enough that they are not supported. This system is definately getting clunky enough that it is time for an upgrade. Even so, this server has stood the test of time, and delivered incredible value for that initial $495 that we paid for the OS.

So I am working on bringing a new system online to replace the old warhorse. The new one is a Micron Netframe 3100, which sports dual Pentium III 450mhz CPUs, with 128MB RAM. The SCSI drives are alledgedly hot-swappable, but I haven't tested this capability yet. The operating system we have chosen is FreeBSD 3.3R, and here are the reasons:

o BSDI's policy of licensing per-user has automatically eliminated their OS from consideration. Too bad, because I'd pay the cash for commercial support in a heartbeat, otherwise.

o FreeBSD looks so much like BSDI that it is difficult to tell the difference. Our shell users (we've got a lot of those) need that compatibility.

o Most of the applications we run are supported on both BSDI and FreeBSD. Stronghold, several shopping cart programs, and the like, are easy to port over.

o We've been using FreeBSD for quite some time as well. In fact, we were using it before we started using Linux, and have never been sorry for that decision. FreeBSD really is one of the best server operating systems I've ever encountered, in terms of sheer bang for the buck performance, reliability, and ease of use.

o Our customers can run Linux apps on the new server if they want to. We like that. We want to promote cross-pollination of the various UNIX camps, as much as possible.

o FreeBSD isn't NT. 'Nuff said.

The Walnut Creek merger with BSDI is the biggest news in FreeBSD land that has happened in quite some time. Since the people involved all assure us that FreeBSD remains free from corporate ownership, then the user community has nothing to fear. And this pre-emptive move by BSDI to pool resources bodes well on the public awareness front for this very worthy operating system.

I look forward to the spread of FreeBSD, and if BSDI can get over this per-user licensing temporary insanity, then they will have my support, both verbal and monetary. The folks from Colorado Springs should be proud of their work, and I expect that they will produce more of the same.

Re:merger and marketshare

[Admiral Burrito]

According to IDC surveys of server operating systems, Linux is the only OS gaining marketshare, accounting for one in every four servers. According to the survey, NT is holding steady at 38%. All other operating systems have shown a steady decline in marketshare over the past 3 years of IDC's survey.
When we look at the IDC marketshare numbers, the rationale behind the merger becomes clearer. Both FreeBSD and BSD/OS are competing for the same niche in that shrinking remaining segment. There just isn't enough of the pie to go around.

Your "not enough pie to go around" statement implies that FreeBSD/BSDI usage is shrinking. It may be that FreeBSD/BSDI's growth is not proportional to market growth, or it may be that FreeBSD/BSDI is simply not (yet) a large enough percentage to get out of a shrinking "other" category (sorry, don't have the IDC survey handy to check that).

I don't know about BSDI but I'm quite certain the FreeBSD usage is growing. FreeBSD's "piece of pie" is getting bigger regardless of whether or not it's getting bigger at the same rate as the whole pie. And FreeBSD's piece of pie is already large enough that nobody is going hungry.

Not "pushing" on the "pull" door

[348]

Interesting that he phrased the answer to who "Owns" BSD in this manner.

FreeBSD is "owned" by the FreeBSD Project, which is made up of the core team, the committers, and all the other people who write the code. No company can ever own FreeBSD, anymore than a company can own Linux. The FreeBSD Foundation will be an independent non-profit organization. It will be controlled by a board of directors, which will contain some members of the core team. The Foundation and the core team are still separate entities going forward.

Non-profit, not for profit's and foundations essentially mean that the company CAN be for profit, the not for profit part comes from the stipulation that the board members are not allowed to profit. With his answer he clearly and firmly syas the corporate structure surrounding FreeBSD is structured in such a way that it will keep FreeBSD free. The foundation of keeping it in the spirit of open source is firmly in place. Got to hand it to Walnut Creek et all, they seem to be doing everything right. Makes me wonder with models like this that obviously work, and work well, why do fly by nights' like LinuxOne keep driving down the wrong path. Reminds me of that far side cartoon with the Midville School for the Gifted. Has a guy trying to go in the front door, pushing real hard. Right above his head there is a big sign that says "Pull". Good to see that the folks behind FreeBSD are not pushing on the pull door.