Slashdot Mirror
Solaris' Dtrace in Detail
paulkoan writes "The Register has a further details about the new Dtrace systems utility bundled with Solaris 10, along with pictures of the authors, and user testimonials. It also highlights Suns vague assurances that this (if it lives up to the hype) amazing utility may or may not end up in the public domain."
It's a mechanism for adding trace calls to pretty-nearly-arbitrary locations.
davecb@spamcop.net
An excellent article at sys admin mag.
It is much, much more than that. The problem I find with a "well-sorted collection of individual tools" in this case is the complexity/difficulty/impossibility to do what dtrace can do. Check out this usenet posting for some examples.
I've been using it for a little while, it's a very interesting tool.
It's a hugely complicated problem and it doesn't have a simple solution except in the minds of markedroids.
I think you do not have a good understanding of the meaning of the word quantum.
Quantum means 'a level, or fixed amount'. In Quantum mechanics this refers to the smallest observable physical effects. At which point changes in energy are in fixed 'Levels' (ohh like Qunatums) rather than smooth functions.
So in this article he is saying that this has reached the next level. And that you are a dimwit.
Thank you very much,
QED
----- 70% of all statistics are completely made up.
Dynamic Instrumentation of Production Systems - this paper was presented at Usenix, and describes how Dtrace is actually implemented.
Dtrace user guide.
A collection of Dtrace scripts
Grr. No-one understands what I meant.
Which is what - you don't have an opportunity to play with it at home? But you do - you can get a Solaris 10 preview through their Sun Express program (linked in the article) and install it on another box, in VMWare, etc.
That's kind of like saying perl is an all round text processing tool, then asking why using perl is better than using cut, sort, and tr.
You can do a lot with cut sort and tr. Often they're all you need, but perl lets you solve problems those three tools can't even address.
I'd recommend grabbing a Solaris 10 beta and having a play with this thing. It's pretty amazing.
DTrace does much, much more than what perfmon does. They aren't even close to the same level in terms of their capabilities. And I say this, while being Windows developer myself..
Not sure what all the hype is about, DTrace sounds alot like HP's Measureware and PerfView. Bundling it with the OS for no addition cost, that sounds good to me.
Slashdot - Where the slash is most definitely to the left.
Well you CAN get Solaris 10 preview from the Express program. It's free for non-commercial use and available for x86. Downloads available here. Check out the hardware compatibility list for x86 here (don't see one specifically for Solaris 10 Express, probably the Solaris 9 list applies).
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
The Kernel Instrumentation Process has already started creating the foundation for a kind of dtrace functionality. The profiling mechanisms appear to echo some of what Sun has done to Solaris to enable this kind of process-less profiling. Hope someone is still pushing this along...
DaGoodBoy
My God! It's full of Voids!
- The execution path is different between the profiled and non-profiled library, so you can not be 100% assured you are getting the same result.
- To engage a profiled library you have to stop and start the target process, and often play all sorts of pre-load and path games.
- Looking into the kernel requires even more invasion, and can mean rebooting the server.
Basically, the idea here is that you don't have to use a possibly "different" library, and you can instrument code on the fly (without stopping the process or the box). This is really nice for production boxes where any interruption or change of software is a really big deal.The article says what this D-Trace does, but yet, doesn't..
The article gives sample uses, but it isn't a howto. I don't usually go to the register for technical howto's, do you? The article does link to more info on how to use the utility.
Not to mention it's neither objective,
Nor is it wholy pro-sun, it jabs the rep for not directly saying whether or not it will be open sourced and repeatedly makes comments about the hype ("It slices, it dices, it spins, it whirls").
and even fails to mention alternatives or relative tools
It mentions that various system utilities could be used to do somewhat the same thing, on a much different level, one of the testimonials mentions that a debugger could have been used too, but would have been slower and another user mentions using prex to get similar information.
Dtrace Usenix Paper
More examples in answerbook from the BigAdmin Dtrace page
Aside from the fact that this tool works with non-profiled binaries, I fail to see the revolutionary aspect.
try reading some of the related links (like the bigadmin dtrace site, the usenix paper, the com.unix.solaris post, or the sys admin magazine article) and you'll quickly realize why this is a revolutionary tool.
the profiling facility is only one of the information providers that dtrace has.
other providers include:
- a function provider (allows you to monitor calls into any kernel function entry or exit point )
- a system call provider
- io provider
- process scheduler provider
- process provider (allows you to place probes in userland processes.)
- the list goes on
these providers taken together allow you to place probe points all over the kernel and userland processes where each probe point can return whatever information you want. (ie, you can get function argument/return values, you can access globals, you can troll kernel data structures, you can collect you own statistics, etc.)
that's a lot of potential information, so dtrace provides you with an easy programmatic way to drop data your not interested in. how about only collecting data when a particular function returns an error after being called from a particular process with a certain parameter set. easily done in dtrace.
then your simple dtrace script can correlate information from all these providers and display it clearly. (with a language which was designed to make simple statistical analysis easy via built in functions for aggregations, averages, etc.)
and of course the best part is that you can do all of this on a production system, running production software, and under normal load.
obvious disclaimer: i work for sun.
I wrote OProfile, and I currently use DTrace daily, and I can assure you that you are wrong when you claim they do the same thing.
OProfile is useful for measuring system-wide resource consumers (for example, you can see what pieces of code are causing cache misses in the kernel when your apache process is in the kernel etc, or which user processes take up the most CPU time).
DTrace can also do something similar (though it needs a little more work yet). But DTrace does a LOT more than this. Imagine a system-wide (kernel, binaries, libraries) 'strace', where you can trivially choose what to print out, and what parts to strace, and under what circumstances. DTrace does even more than that.
OProfile can't tell you exactly why your system call is returning EINVAL. OProfile can't tell you why your application is causing cross-calls. OProfile can't tell you what processes are writing to what files, in real time. OProfile can't debug race conditions.
OProfile is a profiler: it does its job and nothing more. DTrace is, essentially, an instrumentation suite; one of its abilities is to function as a simple profiler.
You won't really get a notion of why DTrace is so useful until you try it.
-- Remove the trailing '\0' to email me.
DTrace is more of a language than an actual app...
Check out http://www.sun.com/bigadmin/content/dtrace - has more info about it.
> No, linux does not already do this.
Yes, it does (well, 85% of it at least), in the form of DProbes. Dynamic Probes was ported by IBM from OS/2 to Linux way back in 2000, is distributed by SuSE and some parts are in 2.6 kernel out-of-box.
This paragraph is copied from the Related Work section.
Kerninst is a dynamic instrumentation framework that is designed for use on commodity operating system kernels[13]. Kerninst achieves zero probe effect when disabled, and allows instrumentation of virtually any text in the kernel. However, Kerninst is highly aggressive in its instrumentation; users can erroneously induce a fatal error by accidentally instrumenting routines that are not actually safe to instrument.3 Kerninst allows for some coalesence of data, but data may not be aggregated based on arbitrary tuples. Kerninst has some predicate support, but it does not allow for arbitrary predicates and has no support for arbitrary actions.
Sounds just like Shark for Mac OS X.
. ht ml
If DTrace is better than Shark, it must be an awesome tool. Shark does everything this article describes and far, far more:
http://developer.apple.com/tools/shark_optimize
The 4.0 version of Shark shown at WWDC was even better. In many cases, Shark can actually give you specific directions on how to optimize your code: both at the source and instruction level.
DTrace is not the SE toolkit. The SE toolkit used a similar method (little langugae scripts) to collect data that was made public by the kernel.
DTrace takes advantage of monitoring hooks placed throughout the kernel and in every system call, and it does things that SE never could.
Furthermore, one look at the DTrace manual and your boss will shit his pants. This is not PHB material by any means.
Chapter 1: The D prograsmming Language...
The major difference: a huge amount of polish. DProbes can crash your system if used incorrectly; DTrace is guaranteed safe. DProbes reports just about everything; DTrace supports speculative tracing and predicates so you can narrow down the fire-hose of profiling data into the useful information. DProbes doesn't scale well across multiple CPUs; DTrace does. (This last one is just like Linux versus Solaris: one scales, the other doesn't. Sorry!)
So, yes. If IBM had spent the past four years tuning DProbes and turning it into a useful tool, they might have something. As it is, Sun spent quite a bit of time and money, and ACTUALLY CREATED a very powerful tool.
[rant]What is increasingly bugging me about open source advocates on Slashdot is how they'll look at any (ANY!) closed-source tool and say "oh, open-source tool XYZ does this / did this a few years ago / will make you walk on water". These advocates will cheerfully ignore that the closed-source tool has months or years of additional polish that have brought it from an internal company toy project into a commercially viable tool - that the open source tool of now is no more advanced than the closed source tool of years ago, except that most open source tools are stagnant and will never leave development and never obtain the degree of polish wide-scale use demands.
A message to open-source advocates: if you don't spend the time turning your hobbyist code into solid, professional code, the closed-source companies that DO spend the time will leave you in the dust - freedom of code (both free as in beer and free as in GNU) notwithstanding.[/rant]
A witty [sig] proves nothing. --Voltaire
Let me rephrase what you are responding to in the hope that this will make things a little clearer:
The point here is that, yes, you could go inserting arbitrary code into any random point of a kernel on your mission critical big server whose downtime causes real dollar losses every minute it's down, but is this a smart thing to be doing? If you read the Usenix paper on it, you'll see that one of the design requirements from the beginning for dtrace is for it to be totally safe to run this tool on a machine. In theory, at least, you cannot do anything with dtrace that will crash your machine.