Slashdot Mirror

As I understand it, a DTrace user has experimented with the program, determined it to be specifically crippled, and given an educated guess about why it is crippled in that way.

"Adam Leventhal is a Staff Engineer in Solaris Kernel Development. He is one of the three authors of DTrace for which Adam has received Sun's chairman's award for technical excellence in 2004, was named one of InfoWorld's Innovators of 2005, and won top honors from the 2006 Wall Street Journal's Innovation Awards. Adam has developed various debugging and post-mortem analysis facilities, and continues his work on user-land tracing to expand the breadth and depth of DTrace's view of the system. Adam joined the Sun after graduating cum laude from Brown University in 2001 with a degree in Math and Computer Science."
(from http://opensolaris.org/viewProfile.jspa?id=21 )

I'd listen to this user.

A while back I started a new job where I'd be maintaining a lot of old code, so I spent a bit of time trying to answer that same question. The first thing I looked at was OpenGrok. Sun uses it for online browsing of the OpenSolaris code. You can see it in action here. The cross referencing is nice, but it takes a while to generate, can use a lot of disk space, and is only marginally more useful than grep with a good editor.

Source Navigator is a bit more useful.

Generating call graphs can be helpful. I know kprof will generate them, but it requires generating profiling information, and more or less requires Linux, so it might not be possible in your environment. I think Doxygen can do it, and I'm sure there are other tools that can.

Other than that, I agree with the people who said to set breakpoints and start playing. Obviously it helps if you have some kind of goal in mind, or a specific area of the code to look at.

A while back I started a new job where I'd be maintaining a lot of old code, so I spent a bit of time trying to answer that same question. The first thing I looked at was OpenGrok. Sun uses it for online browsing of the OpenSolaris code. You can see it in action here. The cross referencing is nice, but it takes a while to generate, can use a lot of disk space, and is only marginally more useful than grep with a good editor.

Source Navigator is a bit more useful.

Generating call graphs can be helpful. I know kprof will generate them, but it requires generating profiling information, and more or less requires Linux, so it might not be possible in your environment. I think Doxygen can do it, and I'm sure there are other tools that can.

Other than that, I agree with the people who said to set breakpoints and start playing. Obviously it helps if you have some kind of goal in mind, or a specific area of the code to look at.

http://opensolaris.org/os/project/opengrok/ and http://www.ece.iastate.edu/~zola/glow/ . The latter requires addr2line which is available for linux, but not OSX :(

you seem to be looking for commandline tools, but their never really going to offer a great way to visualize a new complex program, although they can be quite useful in development.

ide's with class browsers, like eclipse (w/ cdt for non java) or openkomodo are pretty good aids.

for source search, cross linking, and highlighting, the best tool i've come across is opengrok - http://opensolaris.org/os/project/opengrok/

if your more apt to build your own tool, there are a couple of nice libraries out there, scintilla has cross platform language parsers. silvercity builds a python api for looking examing language constructs. for ruby the recently released ohloh contains parsing capabilities, http://labs.ohloh.net/ohcount

also the venerable exuberant tags, is a must for non ide development environments. its a great tool in conjuction with flexible environments like emacs, or textmate.

of course, nothing beats a good debugger, and stepping through the runtime execution of the code paths.

then you need to mkfs, and if you run out of space you're screwed because you can't easily grow.

You can't really expand a raidz either. The best you can do is make a new raidz and add it to your pool. But then you're stuck with extra parity disks. I'd love to use ZFS but until I can throw a disk on to an existing array and expect it to work it's just not good enough.

SUN has been making good progress on the SPARC ZFS boot front as well. It's not as easy as it was under the x86 process as they have to modify the openboot firmware to handle the key pieces for allowing zfs boot.

I believe it's due (at least tentatively) for integration into Nevada build 82 or 83 at this time.

http://www.opensolaris.org/jive/thread.jspa?messageID=182541&#182541

I think the best introduction is the screencast where the guy overwrites his disk from /dev/random and zfs keeps on trucking.

The sources has already been available under an open source license since ZFS came out.

I have been using ZFS (on Solaris) for more than a year, both at work and at home, and I am following closely the latest developments. IMHO the best intro on ZFS is the official ZFS slides (36 pages): http://opensolaris.org/os/community/zfs/docs/zfs_last.pdf

No, Nexenta does not include any of the closed source binaries
http://www.gnusolaris.org/: "NexentaOS is completely open source and free of any charge.".

Here is the list of what is missing from OpenSolaris: http://www.opensolaris.org/os/about/no_source/. Missing is e.g. some device drivers, luckily some of them have OSS counterparts in the net (e.g. http://homepage2.nifty.com/mrym3/taiyodo/eng/).

However, the way Nexenta is built it can use binary only Solaris drivers - a very nice feature IMHO.

I was supposed to give those links in my previous post but somehow managed to screw up ... sorry.

The OpenSolaris project does convey code under a free software
license, but there are core components which are only available as
binary blobs, under non-free licenses. They are necessary to
successfully run a Solaris system.

This is the OpenSolaris binary license under which most of these
non-free components are distributed:

http://opensolaris.org/os/licensing/opensolaris_binary_license/

While it may save you from a catastrophic drive failure, despite popular belief RAID generally cannot detect or repair corruption*... you really need ZFS (part of Solaris 10 and being integrated into OS X).

* - the short answer: because it reads only one side of a mirror and does not checksum. Drives and controllers do not reliably report errors and nor can RAID do anything about hardware issues such as RAM, cable, controller, or firmware bugs. (ZFS has other features further improving integrity, such as COW.)

"Oracle is going after its piece of the hot virtualization market by introducing an open source Xen-based hypervisor to compete against those from Novell, Red Hat, and VMware.

"Oracle is going after its piece of the hot virtualization market by introducing an open source Xen-based hypervisor to compete against those from Novell, Red Hat, and VMware.

"Oracle is going after its piece of the hot virtualization market by introducing an open source Xen-based hypervisor to compete against those from Novell, Red Hat, and VMware.

"you could still manage the performance stuff with your big box of discs, but snapshots and cloning in near realtime without using up any space would not be that easy..."

Which is precisely why NetApp is scared of ZFS. Because it lets you do snapshots & cloning in near realtime without using any space.

http://opensolaris.org/os/community/zfs/

Scenarios that involve physical disturbance to the hardware are not easily testable by automated regression-test systems. I have no idea what Apple uses internally, but "bump the hard drive" is not an option I've seen in most products.

You're not a programmer, I take it.

Sun has far more extensive failure-mode tests than this for ZFS.

I'm not even a systems programmer -- I'm a web programmer right now -- and I write tests for weird crazy failures every day. If it's not tested, then it quite probably has a bug. We have things like mocks, fakes, and stubs, which make it really easy to simulate hardware failure in software tests.

If you can't (or don't) write a test for hardware failures, you have no business being a systems programmer.

The editors seem to have a fascination with stupid pundits. Shall we tag this and similar stories stupidpundit?

Here's one big collective stupidpundit story: when Leopard came out, a bunch of pundits crowed that it was a big leap forward, filesystemwise. Why? Because Leopard has gone over to ZFS as the main file system, and ZFS is the first really new file system in decades.

Except that Leopard hasn't gone over to ZFS. It doesn't even support read-write access to ZFS. Why did so many pundits get it so wrong? Because Leopard introduces Time Machine, an automatic file versioning system, which is "obviously" built on top of ZFS's file versioning feature. Of course, if that were true, you wouldn't have to plug in an external disk to use Time Machine. But you do.

Darn those stupid pundits. They had me all excited because ZFS really is very cool. If Mac OS had gone over to it, I would have been terribly tempted to buy my first Mac.

You're actually exactly right on data integrity issue. RAID controllers are iffy and will not protect you in most of the scenarios you listed above.

What you're looking for is ZFS that guarantees to cover those scenarios and can protect from faulty cables, controllers, bit rot and more. Linus isn't excited about ZFS for nothing!

Read about ZFS more here

They might switch to OpenSolaris, which is not half bad.

"but you'll need a copy of all the source code. Got an AIX lying around?" Please see http://src.opensolaris.org/source/ for UNIX source.

NetApp's WAFL is good, but I expect Sun's ZFS to an equally good job a significantly lower cost.

Hard to say for certain, as comparing WAFL and ZFS ignores the often overlooked but additional integration that you get with a Filer as opposed to a more general purpose system running ZFS, which is still pretty green when it comes to this sort of stuff:

http://mail.opensolaris.org/pipermail/zfs-discuss/2006-November/036124.html.

It's been my experience that data corruption typically occurs in RAM (ECC), at HBA, cabling or drive level itself. The difference in firmware behavior with integrated system vs. some drive you bought at Fry's is where true "End to End" data protection comes into play.

Adding up that level of integration makes the file system merely a component in a very very expensive system ;)

Second sentence on the page is: This driver is based on the source code from OpenBSD, and is provided under the same BSD-type License.

I understand the desire to see your code under the license of your preference but realistically if you're getting a huge leg up (as is the case with Reyk's code for this wireless driver) why would you release your minor modifications under an incompatible license? If you believe in share and share alike give back to the original author the same way you received. It's just common courtesy when dealing with open source.

"Sun picked the CDDL because it's incompatible with the Linux kernel license (something that the Linux kernel developers simply cannot change)."

That's borderline cynical to say. You assume with this statement that Sun deliberately picked a license to be incompatible with the GPL, for the purpose of disallowing works published under it to be integrated with the Linux Kernel tarball:

http://www.opensolaris.org/os/about/faq/licensing_ faq/

If you wanted a copyleft license, why didn't you just use the GPL or LGPL?

We needed an open source license that allowed files released under the license to be linked with files released under other licenses. While a license like LGPL would allow this for dynamically-linked code, we also needed to be able to release software that statically links source files available under different licenses. In addition, we wanted to allow others to add externsions to OpenSolaris with different license terms. This was only possible under a license like the MPL; however, we could not use the MPL because it is not a "template" license allowing reuse by others. Consequently, we crafted a variant of the MPL, taking the opportunity to make it a template license as a step towards reducing license proliferation for others finding themselves in a similar position.

yes you did.

yes you did.

yes you did.

Incorrect.

Solaris now has DRI support for chips such as the Intel 915 since October of last year at least:
http://mail.opensolaris.org/pipermail/xwin-discuss /2006-October/000356.html

Have a look at the method on line 22330
http://cvs.opensolaris.org/source/xref/onnv/onnv-g ate/usr/src/uts/common/inet/tcp/tcp.c

What shared/global filesystem and clustering solution will Sun be providing to compete with Linux' free and relatively-mature clustering?

Clustering: "Sun Microsystems will contribute to the community the source code for Solaris Cluster, Sun's commercial HA Cluster product group, under the name "Open High Availability Cluster.""

Shared file system: "Sun also plans to open source additional storage code over the next several months, including: ... QFS Sun's shared file system software delivers significant scalability, data management, and throughput for the most data-intensive applications. Well known today in the traditional high performance computing (HPC) arena, QFS is increasingly being used in commercial environments that require multiple host, high speed access to large data repositories."

Both have been commercial products for a long time and are quite mature.

I'll bite points 2 and 3.

Also is for Linux (it's right in the name). OSS is into solaris express. Almost any sound device now works on solaris.

Have you ever been to opensolaris lists? Check out the discuss one.

Next time do a little research.

I recently had to try to read the spagetty which is the OpenSolaris TCP implementation and frankly it felt exactly like this "--". Great documentation--; for very line, through the entire monolythic single multimegabyte .c file.

What? I'm not exactly sure what you mean, but it seems pretty good code to me. It's big, and there are some gotos, but it's all well explained. It definitely doesn't seem as bad as you make out.

In that case, you may want to bookmark the following and check in on it regularly: Xen at OpenSolaris.org

see the Polaris site.

Gotta love the Solaris fanatics. Next I suppose you'll be telling me that Linux isn't "real Unix".

Solaris is a fine OS, and it's got some features that nobody else has. But in some areas it's about 10-15 years behind Linux and BSD. Don't take my word for it - take a look at what Sun itself is saying. Here's a few excerpts:

Solaris installation is ugly, slow, and difficult.
...
We use outdated networking technology (RARP and Bootparams) by default, rather than contemporary network protocols, and thus are often unable to automatically determine configuration attributes that are easily discovered by our competition.
...
We don't include the right set of initial configuration tasks, such as an initial user account, that are commonly provided by competitors. This results in an installed system which boots, and can be logged into as root, but it's then up to the user to hunt around and find a tool (or, more likely, edit the configuration files directly due to our paucity of tools and poor integration of those that exist into the desktop) to create a usable account.
...
One of the significant deficiencies in Solaris compared to our Linux competitors is our ability to easily install additional software after the initial installation.

Well, the good news is that Sun is actually working hard to fix these problems.

Add in VMWare, Xen, or whatever, and you have a loaded gun with several rounds in it. That's where servers are going right now: virtual.... and Solaris containers aren't so wonderful.

``As many are already aware, we embarked upon a journey a couple years ago to formally separate the Solaris operating system from Sun's hardware business - as well as bring Solaris to the free and open source software world via a community effort named OpenSolaris. None of these changes were easy, but I'd like to believe both were successful. What's my proof?`` Read the rest in Sun CEO's blog.

Route 1: http://opensolaris.org/os/community/on/devref_toc/ devref_1/
Jump to chapter 1.3.3. Follow instructions.

Route 2: http://www.gnusolaris.org/gswiki
Follow instructions.

HTH. HAND.

No, DTrace is most certainly *not* vulnerable. See:
http://www.opensolaris.org/jive/thread.jspa?thread ID=36929&tstart=0 ...for why this is.

I'd much rather prefer:

SGPJ - Solaris, Glassfish, PostgreSQL, Java

Productivity improvements might be determined by how badly you need Dtrace functionality: http://www.opensolaris.org/os/community/dtrace/
I'm not confident that a clone will make it into Linux any time soon.

In audience terms, I'm thinking that the limiter is still hardware support. I don't get much time to look at OpenSolaris, so I could be in left field.

+4 insightful? Thats flamebait at the very least, and the parent simply has no idea why solaris went the CDDL way..

http://www.opensolaris.org/os/about/faq/licensing_ faq/#why-not-GPL

Of course, that fact that CDDL is more free than GPL (BSDish, even) wont go through the linux trolls.

CDDL is incompatible with GPL? True.

CDDL designed to be incompatible with GPL? False and Flamebait!

Sun has no problem with ZFS and Dtrace going into OSX / FreeBSD.. and OSX has a larger market share than Linux.

The mods need to be more careful before modding one line statements that go with the /. group think

Regardless of the fact that they now have an organisation to handle large-scale donations, the OpenBSD project is still dependent on sale of cds to fund its' development (IE to pay its' programmers).

You might want to persue other alternatives which are not only more featureful, but are financially stable enough that not only are they solvent enough to provide you with downloadable iso images, but they can even mail you dvds free of charge!

Mind you, if you still insist on using code written by people incapable of holding a real job, there's always yubuntu linux; I understand they also provide free ISO images as well...

Regardless of the fact that they now have an organisation to handle large-scale donations, the OpenBSD project is still dependent on sale of cds to fund its' development (IE to pay its' programmers).

You might want to persue other alternatives which are not only more featureful, but are financially stable enough that not only are they solvent enough to provide you with downloadable iso images, but they can even mail you dvds free of charge!

Mind you, if you still insist on using code written by people incapable of holding a real job, there's always yubuntu linux; I understand they also provide free ISO images as well...

I second PostgreSQL. It's great code, especially in the lower level parts of the architecture, and the comments are very helpful.

I have mixed feelings about OpenSolaris (admittedly I don't know much about it). I browsed through the ZFS code a little, and it seemed like good code. However, when I looked here: http://cvs.opensolaris.org/source/xref/onnv/onnv-g ate/usr/src/uts/common/fs/zfs/vdev_cache.c I was very confused about the comment beginning "All i/os smaller...", and it got me more confused when I tried to look for the places those variables were used. The comment at the top says the readahead is 128k, but the code looks like it's 64k readahead (look at the VCBS macro).

I asked on #opensolaris before I posted this, and there wasn't any response. I could be wrong and I'd be happy to hear an explanation.

As large and old as it is, OpenSolaris has fairly readable code. Plus, most of it has comments explaining why it's done the way it is.

If Sun frees up Solaris using the Java timeline, I have a feeling you're going to be using GNU/ Hurd way before then. Sun is going to pussyfoot around for years.

I know. They are dumb nuts. They could have taken a much larger market share long time ago.
Though some, especially Jonathan, are changing:
http://www.zdnet.com.au/news/software/soa/Sun-look s-to-GPL-v3-for-Java-Solaris/0,130061733,339273561 ,00.htm
SUN has effectively tried to go GPLv3 for SunOS for quite a time. But now we have the 3l33t OpenSolaris developpers who torpedo any such effort tooth and nail.
http://www.opensolaris.org/jive/thread.jspa?messag eID=84380&
is one example. Of many.
It is a pity that SUN provided this comfort zone for them; so that instead of doing the dirty work to achieve world domination, they cherish and please themselves in an ivory tower. A place that witnesses the conception of fantastic software (ZFS, Zones, DTrace), but seemingly rather fosters egomaniac arrogance than the notion of sharing and teamwork.

Also, people interested in ZFS should read these slides: http://www.opensolaris.org/os/community/zfs/docs/z fs_last.pdf