Slashdot Mirror
Solaris DTrace To Be Ported to FreeBSD
daria42 writes "It looks like Sun's famous Dynamic Tracing tool - one of the best features in Solaris 10 - is getting ported to FreeBSD. Sun open-sourced the code back in January and it has been picked up by FreeBSD developer Devon O'Dell. The tool provides insanely great advanced performance analysis and debugging features for server software. Good to see some result come out of the Sun open-sourcing process." From the article: "O'Dell told ZDNet Australia the aim of the project -- which commenced a month ago -- was that all scripts and applications that utilised DTrace under its native Solaris environment should be able to run in FreeBSD with no changes. While FreeBSD's existing ktrace function was similar to DTrace, it was limited in scope, according to O'Dell. 'FreeBSD implements a somewhat similar facility for dynamically instrumenting syscalls for any given application,' he said."
Slashdot : All vaporware, all the time
The article doesn't say whether the program will be released under the BSD license (unlikely) or whether it will remain under the CDDL. The latter seems most likely.
I'm not sure how this benefits Sun, but something as awesome as this, I'm willing to assume it's altruism, and I appreciate it.
Nothing great was ever achieved without enthusiasm
I have seen the use of this tool, and seriously, it rocks. There is no other tracing tool to compare with this. So, I am very eager to hear any news about this being ported to Linux, as not many people use FreeBSD ;-)
"In questions of science the authority of a thousand is not worth the humble reasoning of a single individual."
It looks like a really useful tool. I wonder what the performance penalty is when the tool is turned off.
Do you need to instrument the calls you expect to profile? If so, how can you avoid taking that performance hit when deciding whether to perform the profiling or not, even when the profiler is off? It's still got to check the profiler level each time, doesn't it?
Jesus saved me from my past. He can save you as well.
This has been working on Linux sometime in 2004
The official reason is that it wasn't release was because Linus didn't want the BSD folks using it, but the real reason is the Department of Homeland security didn't want the BSD folk to find the last bug in their release.
Thats what I just head right now. (Thanks, voices)
first?
People in the computing field like to spur the use of spurious jargons. The less educated they are, the more they like extraneous jargons, such as in the Unix & Perl community. Unlike mathematicians, where in mathematics there are no fewer jargons but each and every one are absolutely necessary. For example, polytope, manifold, injection/bijection/surjection, group/ring/field.., homological, projective, pencil, bundle, lattice, affine, topology, isomorphism, isometry, homeomorphism, aleph-0, fractal, supremum/infimum, simplex, matrix, quaternions, derivative/integral, ... and so on. Each and every one of these captures a concept, for which practical and theoretical considerations made the terms a necessity. Often there are synonyms for them because of historical developments, but never "jargons for jargon's sake" because mathematicians hate bloats and irrelevance.
The jargon-soaked stupidity in computing field can be grouped into classes. First of all, there are jargons for marketing purposes. Thus you have Mac OS "X", Windows "XP", Sun OS to Solaris and the versioning confusion of 4.x to 7 to 8 and also the so called "Platform" instead of OS. One flagrant example is Sun Microsystem's Java stuff. Oak, Java, JDK, JSDK, J2EE, J2SE enterprise edition or no, from java 1.x to 1.2 == Java 2 now 1.3, JavaOne, JFC, Jini, JavaBeans, entity Beans, Awk, Swing... fucking stupid Java and fuck Sun Microsystems. This is just one example of Jargon hodgepodge of one single commercial entity. Marketing jargons cannot be avoided in modern society. They abound outside computing field too. The Jargons of marketing came from business practice, and they can be excusable because they are kinda a necessity or can be considered as a naturally evolved strategy for attracting attention in a laissez-faire economy system.
The other class of jargon stupidity is from computing practitioners, of which the Unix/Perl community is exemplary. For example, the name Unix & Perl themselves are good examples of buzzing jargons. Unix is supposed to be opposed of Multics and hints on the offensive and tasteless term eunuchs. PERL is cooked up to be "Practical Extraction & Reporting Language" and for the precise marketing drama of being also "Pathologically Eclectic Rubbish Lister". These types of jargons exude juvenile humor. Cheesiness and low-taste is their hall-mark. If you are familiar with unixism and perl programing, you'll find tons and tons of such jargons embraced and verbalized by unix & perl lovers. e.g. grep, glob, shell, pipe, man, regex, more, less, tarball, shebang, Schwartzian Transform, croak, bless, interpolation, TIMTOWTDI, DWIM, RFC, RTFM, I-ANAL, YMMV and so on.
There is another class of jargon moronicity, which i find them most damaging to society, are jargons or spurious and vague terms used and brandished about by programers that we see and hear daily among design meetings, online tech group postings, or even in lots of computing textbooks or tutorials. I think the reason for these, is that these massive body of average programers usually don't have much knowledge of significant mathematics, yet they are capable of technical thinking that is not too abstract, thus you ends up with these people defining or hatching terms a-dime-a-dozen that's vague, context dependent, vacuous, and their commonality is often a result of sopho-morons trying to sound big.
Here are some examples of the terms in question:
anonymous functions or lambda or lamba function
closure
exceptions (as in Java)
list, array, vector, aggregate
hash (or hash table) ? fantastically stupid
rehash (as in csh or tcsh)
regular expression (as in regex, grep, egrep, fgrep)
name space (as in Scheme vs Common Lisp debates)
depth first/breadth first (as in tree traversing.)
operator
operator overloading
polym
Well then you'll be happy to see the adventures of the iGuy!!
Doing battle with the evil ZenMAster!!
For we that don't have a clue what DTrace is, here's what the has to say: DTrace allows to do performance tuning with applications and troubleshoot production systems--all with little or no performance impact. DTrace provides improved visibility into kernel and application activity, giving the user operational insights with which they can make performance gains..
The no performance penalty sounds really cool to me.
--
Superb hosting 4800MB Storage, 120GB bandwidth, $7,95.
Picaday! Soon to be open "Picture of the day web".
Hosting 20G hd, 1Tb bw! ssh $7.95
[ed. note: in the following text, former FreeBSD developer Mike Smith gives his reasons for abandoning FreeBSD]
When I stood for election to the FreeBSD core team nearly two years ago, many of you will recall that it was after a long series of debates during which I maintained that too much organisation, too many rules and too much formality would be a bad thing for the project.
Today, as I read the latest discussions on the future of the FreeBSD project, I see the same problem; a few new faces and many of the old going over the same tired arguments and suggesting variations on the same worthless schemes. Frankly I'm sick of it.
FreeBSD used to be fun. It used to be about doing things the right way. It used to be something that you could sink your teeth into when the mundane chores of programming for a living got you down. It was something cool and exciting; a way to spend your spare time on an endeavour you loved that was at the same time wholesome and worthwhile.
It's not anymore. It's about bylaws and committees and reports and milestones, telling others what to do and doing what you're told. It's about who can rant the longest or shout the loudest or mislead the most people into a bloc in order to legitimise doing what they think is best. Individuals notwithstanding, the project as a whole has lost track of where it's going, and has instead become obsessed with process and mechanics.
So I'm leaving core. I don't want to feel like I should be "doing something" about a project that has lost interest in having something done for it. I don't have the energy to fight what has clearly become a losing battle; I have a life to live and a job to keep, and I won't achieve any of the goals I personally consider worthwhile if I remain obligated to care for the project.
Discussion
I'm sure that I've offended some people already; I'm sure that by the time I'm done here, I'll have offended more. If you feel a need to play to the crowd in your replies rather than make a sincere effort to address the problems I'm discussing here, please do us the courtesy of playing your politics openly.
From a technical perspective, the project faces a set of challenges that significantly outstrips our ability to deliver. Some of the resources that we need to address these challenges are tied up in the fruitless metadiscussions that have raged since we made the mistake of electing officers. Others have left in disgust, or been driven out by the culture of abuse and distraction that has grown up since then. More may well remain available to recruitment, but while the project is busy infighting our chances for successful outreach are sorely diminished.
There's no simple solution to this. For the project to move forward, one or the other of the warring philosophies must win out; either the project returns to its laid-back roots and gets on with the work, or it transforms into a super-organised engineering project and executes a brilliant plan to deliver what, ultimately, we all know we want.
Whatever path is chosen, whatever balance is struck, the choosing and the striking are the important parts. The current indecision and endless conflict are incompatible with any sort of progress.
Trying to dissect the above is far beyond the scope of any parting shot, no matter how distended. All I can really ask of you all is to let go of the minutiae for a moment and take a look at the big picture. What is the ultimate goal here? How can we get there with as little overhead as possible? How would you like to be treated by your fellow travellers?
Shouts
To the Slashdot "BSD is dying" crowd - big deal. Death is part of the cycle; take a look at your soft, pallid bodies and consider that right this very moment, parts of you are dying. See? It's not so bad.
To the bulk of the FreeBSD committerbase and the developer community at large - keep your eyes on the real goals. I
A tool like this could really aid in finding all the bottlenecks. Benchmarks have become an embarrassment for FreeBSD as of late, and it is really sad to see that FreeBSD has fallen so far behind. Hopefully this could start to turn things around.
the best damn piece on slashdot and some moron marked it flamebait. I've modded it up as much as I can (a big +1 whoo!)
It's sad to see slashdotters trod on brilliant posts like this one...
Sure, they may help out the open source community from time to time, but tell that to the scores of US workers they deliberately fired to replace them with cheaper foreignors who now sit in the very same desks; now those US programmers are suing Sun for its greed, since what it did is against labor laws.
More info:
http://malfeasance.50megs.com/
Yet another crippling bombshell hit the beleaguered Linux community when recently Slashdot confirmed that DTrace would only be ported to FreeBSD . Coming on the heels of the latest Microsoft >survey which plainly states that Linux has lost more market share, this news serves to reinforce what we've known all along. Linux is collapsing in complete disarray.
I bet that Sun PHBs and the prima-donna set amongst the engineers are incredulous with anger and disbelief at this, since they behave that those unwashed, long-haired hippy developers are too stupid to port something like this to another OS.
Oh dear. Yet another dastardly plan foiled!
start sarcasm
But it is not BSD! It can't be better than anything BSD has created.
We all know that Solaris is just a crappy system that has no use in the enterprise.
end sarcasm
HGTTG: "I knew that there was something fundementally wrong with the Universe."
OMFG!!! IT'S ALIVE !!!!!!
Linux can be build with "bsd-style process accounting" and as such, can this be made to work in Linux?
It seems like everywhere I look I've heard comments about how great DTrace is, so to see it ported to FreeBSD really makes me happy. I do have a couple of questions about it though, simply going in line with the announcements over the last couple days.
1) Considering the fact that we are currently going through the Beta's for FreeBSD 6, I am curious how, if at all, a fully implemented DTrace would help the devs with tracking down and solving the current beta problems. From my current understanding, it seems that it could be a great help with tracking down and solving the current show-stoppers. Can someone clarify this for me?
2) I have also read an article somewhere where a DTrace dev showed how easy it was to track down a memory leak in a small program. With Gnome currently going on a memory reduction kick, would a fully featured DTrace be able to help with finding these memory problems? I realize that comparing Gnome with a small application is ridiculous so I can't expect it to magically find these problems in just a few minutes, but could it help? Also, if DTrace helped to find these problems on versions ported to FreeBSD, would they easily be ported back into the main linux-based version of Gnome?
Any feedback would be appreciated because from what (admittedly little) I've read, it seems that DTrace could help on these fronts, but I'm really not 100% sure that it would.
"You look so different now, but looks can be deceiving." -- Snuff
What with ZFS and Linux partitions being put off at least until 2006 it might be the *only* feature of Solaris 10 for now. Not to be confused with the "pains" that were added, like insipid way java management console plugins are added/admined, new hiding places for common admin/config files or how general installation is just a pain in the keister. Save yourself some trouble, GNU/Linux passed up Solaris about 2 years ago.
As the guy porting DTrace, I want to clear up a few questions that appear frequently in the comments here. First, though, please be kind to the blog -- it's hosted on our (OffMyServer's) network, which is on a T1. I dunno how bad it got when the story was posted, but just for reference, it'd be nice to not have our network connection die.
:)
FAQ #1 seems to be about the license. Obviously, the CDDL is `viral' in the sense that changes in the code need to be provided under the same terms of the CDDL. In my understanding, this applies only to files in which modifications take place. Extension of something CDDL by adding extra files seems to not require those files to be released under the CDDL. That said, this is a porting effort, and most of the changes I will make will be inside CDDL-licensed files. Thus, anything imported will be under the CDDL. This does not require anything external files to be under the CDDL and thus it can be shipped with FreeBSD without `infecting' other files.
FAQ #2 seems to be whether Sun is happy about this or not. If you have read the article, you would have seen that I've been encouraged to work on this by Sun kernel engineers. Whether Sun as a whole is happy about this is not known to me, but everybody involved in getting it this far has been, so I'm not terribly worried about the rest.
FAQ #3 is about performance incurrences. Certain aspects of DTrace incur performance penalties, but only when DTrace is running. DTrace by itself is a library and a userland tool. All instrumentation is done dynamically and when DTrace is not instrumenting something, no performance hits happen whatsoever. When it is running, we have several advantages to other tools because (unlike e.g. truss) we are instrumenting single processes. Processes which are not being instrumented will not take any performance hits other than the fact that they have a bit less CPU usage since DTrace is instrumenting something.
How do you not take a performance penalty when the profiler is off? You must be root to run DTrace. When you instrument functions inside an application, this is done on-the-fly by rewriting the code that is being executed. When it is not being executed, nothing is being rewritten and it's not even looking to rewrite something. It's just some code resident in memory (in fact, modules are even loaded as needed). It sounds like it might be prone to security flaws, but keep in mind that this has been working in production for a while now.
When will this be in Linux? I don't know. I won't be working on it. I challenge _you_ to do this
Is this vaporware? No. I'm continuing development from about a week off (since I lost my development machine) this evening.
Feel free to ask more questions, I'll try to address them as I see them. But please refrain from bad-mouthing Sun or myself out of spite, jealousy, or whatever. I know it's fun to troll (if you're a troll), but the rest of us really don't appreciate it.
--Devon
www.sitetronics.com/wordpress
numbers continue 'firs7 post' the 'coomunity'
Notwithstandi8g,
One more crippling bombshell hit the already beleaguered *BSD community when IDC confirmed that *BSD market share has dropped yet again, now down to less than a fraction of 1 percent of all servers. Coming on the heels of a recent Netcraft survey which plainly states that *BSD has lost more market share, this news serves to reinforce what we've known all along. *BSD is collapsing in complete disarray, as fittingly exemplified by failing dead last in the recent Sys Admin comprehensive networking test.
You don't need to be a Kreskin to predict *BSD's future. The hand writing is on the wall: *BSD faces a bleak future. In fact there won't be any future at all for *BSD because *BSD is dying. Things are looking very bad for *BSD. As many of us are already aware, *BSD continues to lose market share. Red ink flows like a river of blood.
FreeBSD is the most endangered of them all, having lost 93% of its core developers. The sudden and unpleasant departures of long time FreeBSD developers Jordan Hubbard and Mike Smith only serve to underscore the point more clearly. There can no longer be any doubt: FreeBSD is dying.
Let's keep to the facts and look at the numbers.
OpenBSD leader Theo states that there are 7000 users of OpenBSD. How many users of NetBSD are there? Let's see. The number of OpenBSD versus NetBSD posts on Usenet is roughly in ratio of 5 to 1. Therefore there are about 7000/5 = 1400 NetBSD users. BSD/OS posts on Usenet are about half of the volume of NetBSD posts. Therefore there are about 700 users of BSD/OS. A recent article put FreeBSD at about 80 percent of the *BSD market. Therefore there are (7000+1400+700)*4 = 36400 FreeBSD users. This is consistent with the number of FreeBSD Usenet posts.
Due to the troubles of Walnut Creek, abysmal sales and so on, FreeBSD went out of business and was taken over by BSDI who sell another troubled OS. Now BSDI is also dead, its corpse turned over to yet another charnel house.
All major surveys show that *BSD has steadily declined in market share. *BSD is very sick and its long term survival prospects are very dim. If *BSD is to survive at all it will be among OS dilettante dabblers. *BSD continues to decay. Nothing short of a miracle could reanimate the corpse at this point in time. For all practical purposes, *BSD is dead.
Fact: *BSD is dying
I'm looking for someone clueful to port DTrace to DragonFlyBSD instead. FreeBSD has too many projects where they announce they're going to do something and then never finish. Just look at any of their past status reports. They read like tombstones for unfinished projects. Or consider FreeBSD 5-SMP --- another unfinished project they're sweeping under the rug. At least in DragonFlyBSD, they do what they say they're going to do. More in fact. They do it first then say they've done it.
...for the slashbots to tell us why this is evil - everything Sun does is evil, after all.
How does it work? That's some seriously obfuscated code!
Zonk,
Why do you think all of your worthless posts are worth of the front page?
Why don't you start posting every new release on Freshmeat to the front page as a full article?
Also, you don't ever check your E-Mail do you. Probably at quota with flames.
... facts are facts. ;)
FreeBSD:
FreeBSD, Stealth-Growth Open Source Project (Jun 2004)
"FreeBSD has dramatically increased its market penetration over the last year."
Nearly 2.5 Million Active Sites running FreeBSD (Jun 2004)
"[FreeBSD] has secured a strong foothold with the hosting community and continues to grow, gaining over a million hostnames and half a million active sites since July 2003."
What's New in the FreeBSD Network Stack (Sep 2004)
"FreeBSD can now route 1Mpps on a 2.8GHz Xeon whilst Linux can't do much more than 100kpps."
NetBSD:
NetBSD, for When Portability and Stability Matter (Oct 2004)
NetBSD sets Internet2 Land Speed World Record (May 2004)
NetBSD again sets Internet2 Land Speed World Record (Sep 2004)
OpenBSD:
OpenBSD Widens Its Scope (Nov 2004)
Review: OpenBSD 3.6 shows steady improvement (Nov 2004)
OpenSSH (OpenBSD subproject) has become a de facto Internet standard.
*BSD in general:
..and last but not least, we have the cutest mascot as well - undisputedly. ;)
Deep study: The world's safest computing environment (Nov 2004)
"The world's safest and most secure 24/7 online computing environment - operating system plus applications - is proving to be the Open Source platform of BSD (Berkeley Software Distribution) and the Mac OS X based on Darwin."
BSD Success Stories (O'Reilly, 2004) (pdf) ~ from Onlamp BSD DevCenter
"The BSDs - FreeBSD, OpenBSD, NetBSD, Darwin, and others - have earned a reputation for stability, security, performance, and ease of administration."
--
Being able to read *other people's* source code is a nice thing, not a 'fundamental freedom'.
What We Can Learn From BSD
By Chinese Karma Whore, Version 1.0
Everyone knows about BSD's failure and imminent demise. As we pore over the history of BSD, we'll uncover a story of fatal mistakes, poor priorities, and personal rivalry, and we'll learn what mistakes to avoid so as to save Linux from a similarly grisly fate.
Let's not be overly morbid and give BSD credit for its early successes. In the 1970s, Ken Thompson and Bill Joy both made significant contributions to the computing world on the BSD platform. In the 80s, DARPA saw BSD as the premiere open platform, and, after initial successes with the 4.1BSD product, gave the BSD company a 2 year contract.
These early triumphs would soon be forgotten in a series of internal conflicts that would mar BSD's progress. In 1992, AT&T filed suit against Berkeley Software, claiming that proprietary code agreements had been haphazardly violated. In the same year, BSD filed countersuit, reciprocating bad intentions and fueling internal rivalry. While AT&T and Berkeley Software lawyers battled in court, lead developers of various BSD distributions quarreled on Usenet. In 1995, Theo de Raadt, one of the founders of the NetBSD project, formed his own rival distribution, OpenBSD, as the result of a quarrel that he documents [theos.com] on his website. Mr. de Raadt's stubborn arrogance was later seen in his clash with Darren Reed, which resulted in the expulsion of IPF from the OpenBSD distribution.
As personal rivalries took precedence over a quality product, BSD's codebase became worse and worse. As we all know, incompatibilities between each BSD distribution make code sharing an arduous task. Research conducted at MIT found BSD's filesystem implementation to be "very poorly performing." Even BSD's acclaimed TCP/IP stack has lagged behind, according to this study.
Problems with BSD's codebase were compounded by fundamental flaws in the BSD design approach. As argued by Eric Raymond in his watershed essay, The Cathedral and the Bazaar, rapid, decentralized development models are inherently superior to slow, centralized ones in software development. BSD developers never heeded Mr. Raymond's lesson and insisted that centralized models lead to 'cleaner code.' Don't believe their hype - BSD's development model has significantly impaired its progress. Any achievements that BSD managed to make were nullified by the BSD license, which allows corporations and coders alike to reap profits without reciprocating the goodwill of open-source. Fortunately, Linux is not prone to this exploitation, as it is licensed under the GPL.
The failure of BSD culminated in the resignation of Jordan Hubbard and Michael Smith from the FreeBSD core team. They both believed that FreeBSD had long lost its earlier vitality. Like an empire in decline, BSD had become bureaucratic and stagnant. As Linux gains market share and as BSD sinks deeper into the mire of decay, their parting addresses will resound as fitting eulogies to BSD's demise.
Same old GNU/Linux FUD, that has been disproved countless times..
:)
In short: the MIT research is *11 years old*, and that Rice study on the TCP/IP stack uses FreeBSD *2.2.6*.
(And btw, Eric Raymond advocates BSD license over GPL.)