Why OpenSolaris Failed To Build a Community

xtaski writes "Ted Ts'o, one of the earliest Linux developers, points out some serious flaws in OpenSolaris. There is a severe lack of developers, for one. Apparently, after 3 years, the OpenSolaris 'developer community' is still struggling to get the proper tools for developers to develop! Ted also points out some other flaws which make it clear just how disconnected the executives at Sun are from what's really going on in their 'open source communities.' He notes, 'It was never ... Sun's intention to try to promote a kernel engineering community, or at least, it was certainly not a high priority for them to do so.'"

Comments

For those too lazy too read the article:

[SatanicPuppy]

The answer is: "They acted like a bunch of dicks."

OSS is a labor of love. You've got to want to work on the project, and you've got to be able to work on the project.

If you put a big chunk of your time into something and get rudely dismissed, then its hardly likely that you'll continue to contribute.

Re:For those too lazy too read the article:

[JerkBoB]

Speaking as someone on the inside -- you're right. There are a lot of big egos here.

I didn't come to Sun because I like the Kool-Aid, I came by acquisition. I haven't decided yet whether or not this whole "we love Open Source" thing Jonathan keeps plugging is real or a charade. I'm optimistic, but we'll see.

On better days, I like to think that the people way up at the helm really "get it" and are just waiting for the rest of the ship to slowly (slowly!) turn. On not-so-good days, I start to wonder if maybe someone's trying to pull a fast one.

There are lots and lots of people here who really and truly believe that Linux is just an upgrade path to Solaris. In other words... Once people start running Linux on Sun hardware, they'll "want more", and "step into the big league" with Solaris. It's kind of sad, when it's not irritating.

Anyhow... I could bitch for a while, but I won't.

Re:For those too lazy too read the article:

[JerkBoB]

The thing is, Andy's back, and as far as I can tell, he doesn't give a rat's ass about Solaris. He just wants to make interesting hardware. That's where the money is, after all. Software is a pathetic fraction of corporate revenues here. All the more reason to be mystified about the internal hostility toward Linux.

So, for example, the Thumper is one of Andy's creations. It's pretty hard to beat the storage density you get for the price. Put a mess of those under a Lustre filesystem, and people start to take notice of Sun as a player in HPC. The recent TACC Ranger system is all Sun gear: storage, compute, and network (with sun-built Magnum switches). The OS? Linux.

There's more interesting stuff coming down the pike, and from my perspective, it seems that there's a shift toward making money on volume rather than margins. In other words, somewhat less awesome, but more of it.

I dunno. I don't profess to have much more special knowledge than anyone outside of the upper echelons. I'm hopeful, though. I read somewhere that many of the big Solaris egos were hired away by teh google. Hopefully they keep going. They can have our kool-aid-drunk sales and marketing people too. :P

Re:For those too lazy too read the article:

[Kent Recal]

There are lots and lots of people here who really and truly believe that Linux is just an upgrade path to Solaris. In other words... Once people start running Linux on Sun hardware, they'll "want more", and "step into the big league" with Solaris. It's kind of sad, when it's not irritating.

Funny. It has been exactly the opposite for us.
We're running a bunch of xfires (14 boxes total, 4100, 4200, 4150) here
and initially started out with solaris because the wise guys said it's faster,
more stable, oh and no least you get that shiny "platinum support" badge...

Yea it was all that and the zfs hype, what could possibly go wrong?

Nothing much to be honest. We fell in love with the hardware immediately
and the machines hummed along without too much trouble. Postgres performs
well, java performs well, and ZFS snapshots are a blessing.

Despite all that superficial happyness we switched most of the hosts to linux
(and aim for 100% linux) after a few months. We still love ZFS (and can't wait
for a linux equivalent) but that alone couldn't justify sticking to solaris for us.

What broke it for us is the userland with all its subtle differences
to linux, or in other words: the learning curve. This may sound strange when
talking about a UNIX OS but as a linux shop we're spoiled by the GNU toolchain,
by dead-simple package management and all the little everyday things that just
work a tiny little bit different under solaris.

I'm not saying the linux-UI is better (actually, it is in many
places, but that's not the point here), it's just that we all grew
up with linux, so the solaris CLI "felt like a really old version of linux"
(to paraphrase a coworker) from the start.

We didn't slack, mind you. We tried hard to make that feeling stop. We read the
books and collected bigadmin bookmarks like trophies. We changed the default-shell
to bash and installed the GNU tools to keep our sanity but otherwise did our best
to treat solaris with respect and resisted the urge to dress it up to look more
like linux.

It didn't work out.

I could rant for days about the many little things that drove us away but
I'll try to focus on a few of the most significant points here:

1. Package Management (the lack thereof)
Pkg-add is bad joke when you're used to apt-get and emerge. JumpStart feels
like an insult when you're spoiled by FAI. I can only guess how Sun expects
us to keep our multiple solaris boxes in sync. Maybe they sell that as
one of their many enterprise service?

2. Google doesn't work well for solaris
Not really something we can blame on solaris or Sun but time after time we
were astonished as to how hard it is to find useful help for specific solaris
problems via google. Howto's and Tutorials about all things solaris are generally
very sparse. Due to this "learning by doing" doesn't work as well for solaris
as it does for linux.

3. The sun website SUCKS
Sure there is a lot of documentation, if you can find it in the pile
of rubble that sun calls a website. But even the stuff we found was
not always helpful. Sun documentation tends to be very verbose while
still often glossing over important details. Sun docs often feel like
they expect you to print them out and put them under your pillow.
We don't work that way. We're spoiled by straighforward howtos,
examples, stuff that gets us going fast. We're the impatient
youngsters.

Well, this got longer than I intended. I'll close with saying that
we'll keep buying sun hardware. The xfire series is the best piece of kit
(at a very competitive price) that I have ever seen and it runs linux
like a champ.

But linux as an "upgrade path to solaris"? Ha. Good joke.
In my world solaris has it's place on big iron and in areas where
the last bit of performance really matters. For everyone else the
natural choice is what they're familar with. And who grows up with
solaris these days?

Bureaucracy

[99BottlesOfBeerInMyF]

I think Sun underestimated the importance of casual users. A lot of times the people choosing an OS for a project (be it enterprise deployment, inclusion with hardware, or just use within IT) go with what they are familiar with and also what their current interests are. When Sun open sourced Solaris, there was a lot of interest from the Linux and BSD communities. A lot of those people decided to download a copy and give it a try. The difficulty these casual users had in grabbing an installable copy and getting it running easily were significant. A lot of people just said, "meh" and moved on. The last time I grabbed a developer preview I still had to fill out a bunch of forms with my personal data then deal with Sun's "download manager" and then spend significant time getting it to install, even within a VM customized to run OpenSolaris in particular. That is still better than it used to be. I only have a success rate of about 50% in getting Solaris to install to date.

For most people I think it is just too much of a hassle and all the developer momentum is on Linux. I guess when Sun thinks about open sourcing Solaris, they see it as a way to try to stop their hardware customers from moving away from Sun, which is fine, but does little to leverage the real benefits of an OSS community such as Linux has been doing for a long time.

OpenSolaris fails to build community b/c it sucks

[Jeffrey Baker]

You have to have a good product before you can have a community. Linux built its early community based on tinkerers and hackers who found it easy to play with. Early Linux distributions, you may recall, were all inclined to integrate well with DOS. Some of them could even be installed _in_ DOS. You could install Slackware and be up and running with an editor and compiler in half an hour. OpenSolaris doesn't follow this example. Using it is a tremendous pain in the ass. Its installer runs for 2-4 hours on the midrange PCs I've tried to install it upon. Once it's "installed" you still have to grope around trying to find familiar tools, which are maybe under UCB or perhaps under GNU subdirectories. It's hard to download software from the 'net and ./configure it. Hardware support is very thin.

To get a hacker community, you have to offer fun. OpenSolaris is simply not fun. It reminds me of work.

Installation problems as well

[gearloos]

Another issue with opensolaris for me was the installation. Being a fairly experienced *nix user, years of sunos, aix, linux, bsd, etc.. under my belt and a fairly competent programmer. I tried quite a few times to install OpenSolaris and there was always some major problem. I never did get a stable system working and finally gave up. That said, this all comes as no surprise to me whatsoever.

Re:mirror

[tytso]

Yeah, sorry about that. Thunk.org is a rather ancient machine (> 5 years old) living in a colo facility, and this is how I figured out I had been slashdotted. (The two uptime commands were about two minutes apart):

    14:21:06 up 121 days, 16:47, 2 users, load average: 40.47, 12.41, 4.55
    14:23:05 up 121 days, 16:49, 2 users, load average: 81.43, 36.97, 14.52

  Fortuantely I'm still mirroring my blog onto my old Livejournal account; please read it there for now! The two articles that you want are this one: What Sun was trying to do with Open Solaris and this one: Organic vs. Non-organic Open Source, if you can't get through to thunk.org.

Hmmm - Linux Fan Boys speak out....

[GuyverDH]

The OpenSolaris development community is alive and well, vibrant and resourcefull.
There have been a lot of great development work on OpenSolaris in both the x86/x64 and SPARC worlds.

OpenSolaris (much like it's big brother Solaris) does have a list of valid / tested hardware platforms that work out of the box without issue.

If your specific hardware isn't listed and it's fairly well mainstream, document what didn't work, submit it, and it will more than likely get fixed.

I've used OpenSolaris on IBM/Lenovo thinkpads, IBM xServer hardware, SuperMicro / Intel hardware, homebrew systems with rarely an issue.

I've enjoyed the support of the OpenSolaris community as a whole, and found them to be as resourceful as any *inux / bsd community.

It all depends on what you like / want.

For me, gaining the ability to work with Solaris during development cycles to help in some small way guide / assist with the efforts is worthwhile.

Instant success

[mooreti1]

Yes, because Linux was such an instant success! Wait...no, it wasn't. Everyone forgets that any community, either real or virtual, takes time to build. I believe that counting OpenSolaris as a failed community is premature, at the least.

Re:Not welcoming your Scott McNealy overlord?

[SatanicPuppy]

Actually, no.

The lesson here is: If you're going to try to court people active in OSS development, then you're going to have to be nice to them, and you're going to have to let them take some ownership.

IBM is being smart; they're reaping rewards far in excess of their investment. Effectively they've outsourced their development, and while the terms of the "outsourcing" say that they have to share everything that comes out of the project, they're still in a position to steer, and support the product.

I'm not sure how you equate that with "control"; sounds just like more FUD to me.

of course not

[nguy]

OpenSolaris was an attempt by Sun to throw some sand in the gears of Linux, not to build an open source project. They are doing the same thing with OpenJava.

I mean, who is going to contribute to such a project if (1) Sun engineers keep calling the shots, and (2) anything you contribute needs to be given to Sun so that they can sell it to paying customers?

If Sun were serious about making Solaris and Java open source projects, they'd release them under a single, open source license only. That would probably have to be BSD.

And why not? Solaris was BSD licensed to begin with; it was Sun that made it proprietary.

Download barriers

[SgtChaireBourne]

Downloading is a royal PITA. The registration is usually a deal-breaker. Almost nothing I've ever run across that's worth anything requires registration for download. However, as a (former) long-time Solaris / SunOS user and major FOSS user, I felt compelled several times to try to circumvent that. But then there's no real way do a network install and othewise week download choice.

That gripe aside, the article is a bit premature. Though time is running out and it could become true if Sun decides to keep downloads off of anonymous FTP, AFS and Bittorrent.

As a member of the community...

[saleenS281]

I guess I disagree. I'm on several of the opensolaris mailing lists, and they're ALWAYS busy. And not just with people from Sun, people from all walks of life. To claim that opensolaris has failed is preposterous to me. I guess I don't quite understand what this mans idea of *success* is, but apparently having users and contributers from both sun and the public abroad isn't *success*.

Is his complaint that the majority of code comes directly from Sun? If so... let me just say *DUH*. If you have thousands of PAID programmers writing code, nobody is going to waste their free time re-writing from scratch. On the flip side, there's TONS of public side-projects, I can think of several around zfs like the automatic snapshots. Or maybe that little side project called nexenta.

I think I understand what his issue is... he doesn't even know what the opensolaris community is. By his definition, one distribution of linux is a measure if its success or failure. Last I checked, when we talk about linux, we're encompassing ubuntu, redhat, suse, slackware, etc, etc, etc... Guess what, the same holds true for Opensolaris.

So... basically, it sounds like a linux zealot casting a stone because he's most likely upset that Sun wont' release solaris under the GPL so that linux devs can start ripping code.

Re:OpenSolaris fails to build community b/c it suc

[dotwaffle]

Nexenta works fantastically - I love it. I would definitely use it for any storage servers, or high availability servers that do your normal Apache/SQL/P* stack.

However, for desktop and non-standard services, it still sucks. If it's not a web server, and it's not a storage server, don't use Nexenta, use Ubuntu Server. Or Debian if you know what you're doing :)

Linux Partisan Disparages Non-Linux OS

[planetralph]

This is news?

Re:OpenSolaris fails to build community b/c it suc

[thsths]

> So where does OpenSolaris fit in? It seems to be an OS lacking a niche.

The niche for OpenSolaris is the 64way mission-critical server. Unfortunately, even ultimate kernel hacking enthusiasts rarely have one of those at home.

Re:I didnt bother.

[segfaultcoredump]

Linux just works? Yeah, maybe for a small system running a simple app stack.

I had to setup an oracle cluster: Thanks to Oracle's support policies, we could not use Solaris x86. Nor could we use RHEL5 (no Oracle 9i support), so RHEL4.6 it was. Should be easy, right? Well tested "enterprise" class linux that can do everything the big boys can do.... We took the hardware we were going to use for solaris and switched it to linux. A pair of Sun x4600M2's, 128G of ram, 4 Dual core AMD's. Sun fully supports linux on this box and RedHat lists it on their HCL. Should be easy.

The basic OS install was more or less easy, once we battled through the serial port redirection setup (guess most linux users never used a serial port before. After all, why bother when the box sits under your desk). I stil like serial ports over video for one major reason: issue resolution (when bad things happen, having that panic string saved by a console server can really save the day)

Ok, so the system was kickstarted and now it is time to set it up for use as an oracle DB. This is a production system, and we need lots of space (4TB) and High Availability. This means redundant connections to everything, mirroring and clustering.

Issue #1: multipathing drivers for the SAN. With solaris, you just plug the thing into the san and all of the storage that the host has access to just showed up. Multipathing was instant and I didnt have to do jack. I could see what devices mapped to which physical array with a simple command. I didnt have to guess which array /dev/sde came from vs /dev/sdf. When you have 20 luns mapped to the same host from two different arrays, its kinda important to know which drives come from which array and what the corresponding lun numbers are. That said, most linux admins I've talked to didnt have a clue about what I was talking about since they never had a san.

Issue #2: dynamically add luns: With solaris, you just change the mapping on the array and the host picks it up and auto creates the dev links. That was easy. On Linux? you've got to be kidding me... You get to echo some crazy strings into several spots in /proc and watch the fun start.

issue #3: IP Multipathing. With solaris, dladmin is used to create a bond (if it is going to the same switch and the switch supports bonding) or use the built in ip multipathing to do an active/failover setup if you are going to multiple switches. Very well documented and very easy to do. With linux... yeah, bonding is a fun task. Need to go to multiple switches? no such luck, you are screwed. I eventually used VCS to take over the systems main IP and uses its IPMultipathing agent to do the job for me. VCS on solaris just hands the task off to mpathd since the OS already does it for you.

Issue #4: zones: dont get me started. I dont want to run another entire OS, I just want name space isolation and chroot is so primative it is not even funny. Zones gives me everything I want with minimal overhead. It would have been nice to have since there are a few oracle products that dont play nicely with clusters (*Warehouse Builder*) because they imbed the host name everywhere. We could put it under Xen, but this is an app that moves huge amounts of data around, not exactly a good candidate for virtualization. Zones let us get around Oracle's brain dead use of the hostname, no such luck with linux.

Issue #5: 3rd party drivers vs the new kernel patch. If I install a 3rd party device driver in solaris and upgrade the kernel, I dont have to rebuild/reinstall the driver. Linux (even redhat 4.x with their "back port") forces me to rebuild/reinstall every damn time. Its great if the driver is standard with the kernel, but if you need something outside of that (lsi multipathing drivers to get around #1 and 10G NIC drivers in my case) and you are screwed. No wonder up2date ignores all of the kernel* rpm's by default.

Issue #6: Whats the system doing? Solaris: `mdb -k` and dtrace. Linux: still trying