Slashdot Mirror
Interview with Matthew Dillon of DragonFly BSD
JigSaw writes "Well-known FreeBSD/DragonFly/Linux/Amiga system hacker Matthew Dillon discusses a number of interesting points regarding where the BSDs are going, the status and goals of his latest project DragonFly BSD, the status of his innovative Backplane distributed database, his exciting plans to develop DragonFly into a transparently cluster-capable system implementing native SSI (Single System Image) which is something that no other operating system can do today, and more."
Dragonfly BSD seems to be chugging along quite nicely.
The further away they get from their 4.x FreeBSD roots, though, the more I wish they'd release an ISO. Particularly since the last ISOs for the 4 series of FreeBSD are probably going to be totally gone in a few months.
No BSD secrets for you, Darl!
No this is to do with kernel threads. The userland threading is the same as in FreeBSD 4.x atm, AFAIK. The idea is to keep the model simple, unlike in FreeBSD 5.x where they are having trouble keeping it all sane with their fine-grained mutex model. Have a look at the dragonfly.kernel newsgroup, in nntp.dragonflybsd.org for more details on the SMP model, Matt talks about it regularly earlier on.
* Several monkeys are here, playing banjos and wearing small hats.
~Darl
Yeah yeah forking is always sweet and this sure sounds like a lot of fun already, but what I'm really waiting for is for someone to put together a BSD-from-scratch distribution! I mean, I know I could just build one with Linux.. BUT only having a single kernel to choose means my grimy little subculture won't be as obscure as it could be. Just think how exclusive I'd be if I could pick one of the NetBSD, OpenBSD, either of the active branches of FreeBSD, and PicoBSD, Dragonfly BSD or Darwin kernels..
"The reason for this excitement is that it is becoming clear to us that we can develop very clean-looking, elegant, debuggable, SMP scaleable software using this model whereas using the mutex model generally results in much less elegant (even ugly), difficult-to-debug code. Code complexity and code quality is a very important issue in any large piece of software and we believe we have hit on a model that directly addresses the issue in an SMP environment without compromising performance."
I don't really know what he's talking about, but:
If he's right, everybody wins.
Even if he's wrong and we find out why, everybody wins.
It sounds like Linux isn't hurting BSD any, and methinks for a number of reasons, Linux wouldn't be what it is today without the BSD's.
There's actually something on the front page about BSD. And it says nothing about SCO or linux.
It's simply not true that "a transparently cluster-capable system implementing native SSI" is "something that no other operating system can do today." We were doing it at Locus in 1994 with SVR4 then with Tandem in 1996 with NonStop Clusters for Unixware. Now some of the same folks at HP have introduced OpenSSI, which is essentially the same code, less all the Unixware-related bits, ported to Linux and placed under the GPL. They are coming up hard on their 1.0 release, which is not bad for five people and such a large task.
OpenSSI is the real thing, it has processes that migrate from node to node, distributed file systems, the works. And it's running now on clusters literally all over the world. (Not many clusters, true, but maybe that will change if the Slashdot crowd finds out about it.)
I'm happy to say that there's a lot of my code in that system, as well.
I know a little about what Matt wants to do with his SSI in Dragonfly, but he should certainly take a look at OpenSSI; we had to solve a lot of the problems you run into when you build such a beast.
(And a beast it is. As complex as a kernel can be, when you have what is essentially a distributed kernel across several nodes, the complexity goes up by orders of magnitude. Makes tracking down those weird hangs pretty exciting, in a painful, time-consuming kind of way.)
If you read the article, Matt says (about SSI): "It is something that no non-commercial system today can do"...
* Several monkeys are here, playing banjos and wearing small hats.
This is Slashdot, where any sufficiently advanced opinion is indistinguishable from fact.
The higher the technology, the sharper that two-edged sword.
Kernel threads almost universally stay on the cpu they were originally assigned to. High performance threaded subsystems, such as the network stack, are replicated. That is, the network stack creates multiple threads (one per cpu) and those threads do not migrate because, obviously, they do not need to.
Generally speaking, the purpose of making thread migration explicit instead of automatic is to partition a larger data set across available cpu caches rather then cause the same data to be shared amoungst all cpu caches. The processors operate a lot more efficiently and SMP scales a lot better. Most people do not realize the horrendous cost of moving threads between cpus because the cache mastership change is invisibly handled by hardware, but the cost is still there and still very real.
-Matt
I think of the various Linux distros as "forks" of whatever Linus himself runs. There are literally dozens of Linux forks. Too bad Linus doesn't release a distro, so we'd know what Linux is supposed to look like. If you sit down at a Linux system you have no idea what you're going to find. From a Systems Administration standpoint alone that makes *BSD a better choice for corporations with a large number of hosts, but Linux gets all the press.
If all this should have a reason, we would be the last to know.
There once was a Master of Screws
Who thought it most wonderful news
That the AC's post
was more funny than most
But not all mods agreed with his views.
KFG
Hmm.. yeah, since a recent update I can no longer run a.out binaries from the 2.x era... but for as far as external packages and ports are concerned, thats about the first case where you can't get software for older releases to work with a current version using one of the compatxx packages.
That said, some tools (esp those using kmem) should be kept in sync with the kernel, and when at it, why not just build a new userland, its easier then figuring out what you have to update.
The concurrently developing BSD variatiens allow trying out a variety of low level solutions to problems while sharing a lot of their experiences.
Such diversity doesn't really exist in Linux despite its zillion distributions (which provide a lot of variation in user experience tho)
There once was a fellow named Dillon
Whom everyone thought was a villain,
He cried, "That's not me!"
"I use BSD!"
"Because I find it fulfillin'."
W
-------------------
This is my SIG. There are many like it, but this one is mine.
If your application is licensed under the GPL or compatible OSI license (learn more at opensource.org) approved by Backplane, Inc., you are free and welcome to ship the Backplane open source database with your application.
followed by:
If you power an application using the Backplane database that you market or sell, or use that application to conduct any form of online commerce (selling/buying products or services over a website) you need to purchase the Backplane Commercial License.
The example given is if you run an email service from which you sell access to other companies, you must buy the commerical license.
My question is, what if the program that provides the email service is GPL. Do I have to buy a commercial license or not? One of the great things about GPL software is that if it's an internal piece of software, you can mix proprietary and GPL code as much as you want, as long as you never redistribute the program to anyone.
Also, how does dual licensing work with this? Can I license it under the GPL to myself, and then sell copies under another license to other people? Obviously THEY would have to buy a commercial license, but do I?
Just trying to point out some holes in the licensing..
Oops, just noticed the part at the end saying:
NOTE: In any of these examples, if the entire application or service is 100% GPL compatible, you may use the Backplane Free License.
But that still leaves open the question about dual licensing..
Um, linux is a kernel, not a distro. the linux kernel is what "linux is supposed to look like" to linus.
FreeBSD Addicts
Which is unfortunate in many ways. For example, Matt has introduced variant symlinks into DragonFly, and has major plans involving vfs namespaces etc which will really solve a lot of problems in package management, like allowing two different conflicting versions of a package to exist at the same time. He can do all this because he's looking at the whole picture, and so are the others: the entire source tree for the base system is there on my machine, in one nicely-arranged subdirectory. I don't foresee major changes happening in the linux kernel driven by distributors. To this day, breakages with binary-incompatible glibc etc are constant annoyances with linux unless you choose a stable distributed version from a branded linux distro and stick to it. the linux kernel is what "linux is supposed to look like" to linus.
What is "the linux kernel"? There's a Red Hat kernel, a Mandrake kernel, a SuSE kernel, and you can't really drop a generic Linus kernel into any of the commercial distros and expect it to work properly. (Debian and Gentoo are better.)
I'm not dissing linux, it's better than the mainstream alternatives and has far better hardware support and graphical system administration tools than the BSDs. In fact after 2 years with FreeBSD I myself had switched to Linux on my new machine because of hardware issues (I've now mostly switched to DragonFly and the hardware issues are mostly gone). And I use Linux at work and have no desire to change that. But there are reasons why a lot of technically aware people find the BSDs nicer systems to play with.
Merely my brief experience with Gentoo, when they first upgraded glibc (from 2.2 to 2.3 iirc) and broke half the packages, then downgraded it again and broke everything else. This is really a pet peeve: aren't minor versions supposed to be compatible? And a zillion similar but smaller-scale annoyances, well expressed by Bill Paul many years ago and the years haven't eased the pain all that much.
And BSDs are more likely to introduce binary incompatibilities
Clearly you haven't used the BSDs. You may have library incompatibilities between major versions, but just install the earlier "compat libraries" and you're set. I upgraded from FreeBSD 4 to FreeBSD 5 -- a huge upgrade, over 2 years in the making -- and all my software just worked, even complex stuff like KDE and Mozilla that had been compiled under 4.x.
Granted, if you ran an all RedHat shop or an all Mandrake shop things would be easier than simply an all Linux shop, but the same would be true for an all OpenBSD shop vs an all FreeBSD or NetBSD shop. But if each department is free to buy what they want I'd rather find who-knows-which-BSD on the box than who-knows-which-Linux.
If all this should have a reason, we would be the last to know.
And to a great extent also things like MOSIX, which go back to V7 unix as well as Linux. Also for that matter OpenSSI (openssi.org)
The commercial world is full of SSI systems although its never been clear if transparent SSI is the right answer to any problem except "I need it to work now", because coding good apps for SSI setups is hard.
Dragonfly looks a good project, and looks like it has old BSD folks who actually knew what they were doing working on it.
The LWKT scheduler on any given cpu is only allowed to operate on threads owned by that cpu. If you attempt to wakeup a thread owned by a different cpu, an asynchronous IPI message is sent to the target cpu's LWKT subsystem requesting that the specified thread be woken up. It's really that simple. Same goes for cross-cpu scheduling.
IPI messages themselves are lockless and require no mutexes to operate because the cpucpu messaging uses a software crossbar (array of FIFOs) approach.
In regards to cache issues, lets say you have a quad opteron system. Each cpu has a 1MB L2 cache. If you migrate threads willy nilly you basically wind up in a situation where each of the four cpu's L2 caches contain the same data. In effect, you wind up with a system that globally has only a tad more then 1MB of L2 cache. If you partition data (such as TCP protocol data) across distinct threads, and place those threads on different cpus, then you are in effect partioning your system's memory across all four cpu caches and you wind up with a system that globally has 3-4MB of L2 cache instead of 1-2MB.
There are two costs being saved here. (1) the cost of having to go to main memory when a piece of data is not in the L1/L2 cache, which can run into the hundreds of cpu cycles, and (2) the cost of cache mastership changes for all the data associated with the thread that was migrated (repeated each time the thread migrates).
-Matt
I don't think it's anyone's design in particular, but I tend to sit down and write things from scratch rather then copy other people's ideas. In the case of the thread replication used by the network stack, it is primarily Jeffrey Hsu's work and since he is big on reading papers I'm sure it's a combination of his own design and ideas gleaned from various published papers.