Slashdot Mirror
New Scheduler Available for FreeBSD
flynn_nrg writes "Luigi Rizzo, one of the FreeBSD developers, has just finished the code for a new scheduler. From the announcement: '...as promised, a first version of the Proportional Share scheduler that we developed is available here. These are for a recent -STABLE (i think any version from 4.4 should work; the only 3 files modified are kern_synch.c, kern_switch.c and
proc.h, plus a one-line change to kern_exit.c).
I have tested it a little bit on a diskless system, and it seems to survive running a full X session with the usual set of xterm,
netscape etc. while i do a "renice" of the processes and even switch back and forth between schedulers. But do not trust this yet for a
production system!'
Read the full post here."
I need to schedule some time to install and play with this new scheduler.
This is neat, I need to get a copy.
We're Doomed
It's a good thing that multiplexing FIFO locks were added to discombobulate the BSD thrust manifolds.
It's amazing that it held up this long!
This is what Freebsd needs, wonder what it will look like when its ported to Darwin?
I have been using FreeBSD (as well as NetBSD) for a little while (about 2 years) and I have to say that anyone who would care about this would have seen it three days ago on kerneltrap.org or daily.daemonnews.org.
With this new scheduler, *BSD should be able to calculate exactly when *BSD will finally be DEAD.
Rejoice!!1
FreeBSD is great for security! Glad to hear there's an updated version, but if I see any exploits for it soon I;m gonna be upset.
Everyone is entitled to their own opinion. It's just that yours is stupid.
[ed. note: in the following text, former FreeBSD developer Mike Smith gives his reasons for abandoning FreeBSD]
When I stood for election to the FreeBSD core team nearly two years ago, many of you will recall that it was after a long series of debates during which I maintained that too much organisation, too many rules and too much formality would be a bad thing for the project.
Today, as I read the latest discussions on the future of the FreeBSD project, I see the same problem; a few new faces and many of the old going over the same tired arguments and suggesting variations on the same worthless schemes. Frankly I'm sick of it.
FreeBSD used to be fun. It used to be about doing things the right way. It used to be something that you could sink your teeth into when the mundane chores of programming for a living got you down. It was something cool and exciting; a way to spend your spare time on an endeavour you loved that was at the same time wholesome and worthwhile.
It's not anymore. It's about bylaws and committees and reports and milestones, telling others what to do and doing what you're told. It's about who can rant the longest or shout the loudest or mislead the most people into a bloc in order to legitimise doing what they think is best. Individuals notwithstanding, the project as a whole has lost track of where it's going, and has instead become obsessed with process and mechanics.
So I'm leaving core. I don't want to feel like I should be "doing something" about a project that has lost interest in having something done for it. I don't have the energy to fight what has clearly become a losing battle; I have a life to live and a job to keep, and I won't achieve any of the goals I personally consider worthwhile if I remain obligated to care for the project.
Discussion
I'm sure that I've offended some people already; I'm sure that by the time I'm done here, I'll have offended more. If you feel a need to play to the crowd in your replies rather than make a sincere effort to address the problems I'm discussing here, please do us the courtesy of playing your politics openly.
From a technical perspective, the project faces a set of challenges that significantly outstrips our ability to deliver. Some of the resources that we need to address these challenges are tied up in the fruitless metadiscussions that have raged since we made the mistake of electing officers. Others have left in disgust, or been driven out by the culture of abuse and distraction that has grown up since then. More may well remain available to recruitment, but while the project is busy infighting our chances for successful outreach are sorely diminished.
There's no simple solution to this. For the project to move forward, one or the other of the warring philosophies must win out; either the project returns to its laid-back roots and gets on with the work, or it transforms into a super-organised engineering project and executes a brilliant plan to deliver what, ultimately, we all know we want.
Whatever path is chosen, whatever balance is struck, the choosing and the striking are the important parts. The current indecision and endless conflict are incompatible with any sort of progress.
Trying to dissect the above is far beyond the scope of any parting shot, no matter how distended. All I can really ask of you all is to let go of the minutiae for a moment and take a look at the big picture. What is the ultimate goal here? How can we get there with as little overhead as possible? How would you like to be treated by your fellow travellers?
Shouts
To the Slashdot "BSD is dying" crowd - big deal. Death is part of the cycle; take a look at your soft, pallid bodies and consider that right this very moment, parts of you are dying. See? It's not so bad.
To the bulk of the FreeBSD committerbase and the developer community at large - keep your eyes on the real goals. It's when you get distracted by the politickers that they sideline you. The tireless work that you perform keeping the system clean and building is what provides the platform for the obsessives and the prima donnas to have their moments in the sun. In the end, we need you all; in order to go forwards we must first avoid going backwards.
To the paranoid conspiracy theorists - yes, I work for Apple too. No, my resignation wasn't on Steve's direct orders, or in any way related to work I'm doing, may do, may not do, or indeed what was in the tea I had at lunchtime today. It's about real problems that the project faces, real problems that the project has brought upon itself. You can't escape them by inventing excuses about outside influence, the problem stems from within.
To the politically obsessed - give it a break, if you can. No, the project isn't a lemonade stand anymore, but it's not a world-spanning corporate juggernaut either and some of the more grandiose visions going around are in need of a solid dose of reality. Keep it simple, stupid.
To the grandstanders, the prima donnas, and anyone that thinks that they can hold the project to ransom for their own agenda - give it a break, if you can. When the current core were elected, we took a conscious stand against vigorous sanctions, and some of you have exploited that. A new core is going to have to decide whether to repeat this mistake or get tough. I hope they learn from our errors.
Future
I started work on FreeBSD because it was fun. If I'm going to continue, it has to be fun again. There are things I still feel obligated to do, and with any luck I'll find the time to meet those obligations.
However I don't feel an obligation to get involved in the political mess the project is in right now. I tried, I burnt out. I don't feel that my efforts were worthwhile. So I won't be standing for election, I won't be shouting from the sidelines, and I probably won't vote in the next round of ballots.
You could say I'm packing up my toys. I'm not going home just yet, but I'm not going to play unless you can work out how to make the project somewhere fun to be again.
= Mike
--
What is the purported advantage of the new scheduler?
Sigmund
what the hell happenned to developing SMP and UP capable schedulers ? or does BSD expect to run well on only UP systems ?
Why does this get on the main page, and why does the new Core Team does not?
:)
Either way, FreeBSD does fit the new trend now... more VM's means more freedom
It has to be a miracle. Something actually makes the front page on slashdot that isn't part of the Linux-nazi empire. I can't believe it.
Now, I know for a fact that if I was logged in as myself, I'd get a "Bad Karma", Slashdot really needs to get into the flow; get their act together; and post better stuff about FreeBSD. Why doesn't a FreeBSD release make front page, but a FreeBSD Scheduler does.. doesn't make much sense does it..
Well, for all you Linux fokes who think *BSD is dead, your far from the correct answer. *BSD has been gaining market share as users that use Linux are fed up with the crashing & exploits, and considering a code that wipes out the filesystem in the "STABLE" branch. It can't be that stable if it wipes out your FS. FreeBSD has *NEVER* had anything like that happen.
On top of that, Slashdot should be renamed to Linux Nazi News, Stuff that people don't care about.
--Anonymous "Non-Linux" Coward. aka Bad Karma
There's more info here.
Excerpt:
"There are compelling reasons to use proportional share scheduling techniques to support multimedia and other soft real-time applications on general-purpose operating systems. First, proportional share (PS) schedulers are a good match for existing infrastructure such as a periodic timer interrupt and mechanisms for assigning priorities to applications -- priorities can be mapped to shares in a proportional-share environment. Second, PS schedulers provide stronger guarantees to applications than do traditional time-sharing schedulers: they allocate a specific fraction of the CPU to each thread, and some schedulers provide error bounds on the allocation rate. Third, PS schedulers have clear semantics during underload: excess CPU time is allocated fairly, in contrast with some reservation-based schedulers that must idle or back off to a secondary scheduling policy once all application budgets are exhausted."
Im currently using linux with the low latency patch and pre-emptive multitasking. Does this help X seem a little smoother on BSD also?
It's very sad (not really). It seems this "Less moderating will kill trolling" idea is working. By not giving trolls extra attention (by modding them down), they seem to get frustrated and many have quit. Seems some people do just want attention, even if it is negative.
Annoying troll.
But at least do some research.
"the Girl Guides " indeed.
So you're a European troll, is that it?
qmail.
TeX!
You're so cute when you do that.
As my father lik@(munch munch)...
Is FreeBSD's new one a 0(1) scheduler?
0(1) is a "term" from computer science. When applied to schedulers, it basically means that no matter how many processes there are to schedule, a 0(1) scheduler's overhead will not significantly increase.
Of course, with a small number of threads/processes to schedule, the Linux 0(1) scheduler will have greater initial overhead. It isn't until there are quite a few processes that it starts to show its power, and the more processes there are, the more useful it is.
On a busy server with 4+ processors and thousands of processes, a standard scheduler's overhead is so great that it often exceeds the overhead of most of the individual server processes.
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra
Anybody have any idea when/if Apple will integrate improvements from this scheduler in to Darwin/OSX?
Copying over scheduler code from one kernel to a completely different one is not trivial. I would expect OS X to just go along with whatever people are doing with Mach. All the BSD stuff in Mac OS X is just user-space stuff.
I am not going to claim that FreeBSD is perfect, but FreeBSD is more secure than the vast majority of Linux-based OSes. It has supported features like the new "GR Security" patch for years, and because it shares a great deal of code with OpenBSD which is audited frequently, it benefits from their work as well.
:)
Of note is that FreeBSD's libc is just over half the size of Linux's Glibc (not that has a thing to do with security)
With FreeBSD, for years, admins have been able to set certain files as "append only" (so even root can only add to, not remove from, log files) and "immutable" (so even root cannot modify or delete the file) and has been able to set firewall rules to the same (immutable) so that creative crackers can't add their personal favorites if they root the system.
This can of course be bypassed by restarting the machine in single-user mode and redusing the kernel security level, but that isn't going to be very easy for your average remote hacker.
Furthermore, since 4.0 you can multiple run complete but separate entire copies of FreeBSD on the same system, each with their own FreeBSD system files and such. You can have a single server run an instance of FreeBSD for Apache, one for Postfix, one for BIND, etc. and if any one of them does get compromised (say, BIND since that happens entirely too often) the cracker can not only not effect any of the other instances--he/she cannot even see that they exist! Very interesting stuff.
Of course, IMHO Linux is worlds ahead of FreeBSD on the desktop front, and the new GRsecurity and ACL features will be a real competitor for the *BSD family. It will be most fascinating to see how things turn out. I wish the best to both of them, and I use both of them every day.
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra
Linux has had 3 or 4 different schedulers in the past year alone! Beat that!
the only 3 files modified are kern_synch.c, kern_switch.c and proc.h, plus a one-line change to kern_exit.c
Well that's 4 files modified, now isn't it?
Why aren't more people aware of this? It seems like there are still lots of people out there who consider BSD a viable OS. This information needs to be made more widely available.
Woops!
You're a suburbanite.
You wouldn't think that the losers who use FreeBSD would have enough extracurricular activities to require the use of a scheduler! Will wonders ever cease?
VI ?
gaymo
This is great news, I can't wait for 5.0 STABLE. My experience with FreeBSD has been much better than Linux. It is far more robust and actually more stable.
I'm waiting for Linux to die.
the only 3 files modified are kern_synch.c, kern_switch.c and proc.h, plus a one-line change to kern_exit.c
.. but that's 4 files modified. One-line or a thousand, it's still been modified.
I hate to be picky
</END-RANT>
I've been poking my newbie nose around in scheduling for a little while, and while I still know very little I've found the field very interesting. It's always neat to see new features and techniques being tried, but there's a feature that exists in the windows nt scheduler that (as far as I can tell) is absent in *nix operating systems. Winnt maintains (I think) four process queues (realtime, high, normal, and idle) into which all processes fall. Every time the scheduler is run, it checks to see if any "realtime" processes can be run, then "high", then "normal", and finally "idle". Processes in "less important" queues are only run if all processes in "more important" queues cannot be run (i.e. they're blocking on input or whatever), or those queues are empty. I find this very useful because I can set a long-running cpu dependent process to "idle" priority and it will be run at nearly 100% cpu usage when the machine is idle, but will instantly get out of the way and not be run at all if I choose to run something else (e.g. a game), no matter how high it's "goodness" value gets from not using any cpu time.
Is there any reason why something like this isn't implemented in Linux or FreeBSD? Low on the developers' feature priority list (har har)? Too difficult? Unnecessary?
Thanks. I'd appreciate any feedback.
The scheduler patch is largely the work of Paolo Valente, a student at Pisa, under the supervision of Luigi Rizzo. Luigi would not want to claim credit unduly.
Imagine a Beowolf Cluster of THESE!!!
ls (while escaping on-printable chars)
How is this related to the so-called lottery-scheduler promised in 5.x? Do they have the same goals, such as preventing a single user from dominating the CPU? S.
The Girl Guides are Canadian.
People still use Usenet?? wow!
Hey, people, what are you doing with operating systems that they are crashing like a domino?
I use Linux as well as FreeBSD for awhile with Apache, Sendmail, X11 and lots of other software competing for memory, CPU, exotic devices, network bandwidth and just for disk I/O. No crashes. On linux I use also video, Oracle, Tomcat, JBoss - all works fine, besides own bugs (it's ok for userland, isn't it?).
For my experience, which doesn't meet crashes, it is more important what hardware, filesystems and protocols are supported. And of course I compare systems with only stable manually configured and re-compiled kernels passed "on-the-field" regression tests before I upgrade the production mode with that new kernel.
So, one more time, why should I prefer BSD?
Less is more !
Mac OS X is neither truly monolithic nor truly micro in its kernal architecture.
While both BSD, and Mach sit in the same memory space, this making it monolithic, they are separate pieces of software with a true microkernal in many respects.
It is monolothic in that BSD can take down the whole machine, but Apple argues that you wouldn't care if the kernal was still running when BSD crashed. Furthermore, it is monolithic in that messaging is reduced between the Mach and BSD because they reside in the same memory space as if BSD were part of the microkernal.
However, it is a microkernal in abstraction. Only Mach talks directly to hardware. It is a more portable BSD derivative than many. In addition, the microkernal allows certain online driver loading and unloading techniques that are not possible in monolithic kernals.
When they get some of the threading issues of the OS worked out it should show very good performance. Maybe this experiment will eventually prove to be the best of both worlds for a general purpose OS. Monolithic performance with microkernal maintenance.
The running time can vary for given N, just not for abtrirary N. As demonstrated by the following code. It runs fastest when n = 1, followed by n = 2, but is still O(1) when n is not 1 or 2, because the running time is constant for large n.
int orderOneButVarying(int n) {
if (n == 1) {
return n;
} else if (n == 2) {
for (int i = 0; i 3; i++) {
n*=n;
}
return n;
}
for (int i = 0; i 1000; i++) {
n+= n / 2;
}
return n;
}
::shrugs::
Doesn't even come with it.
Black holes are where the Matrix raised SIGFPE
http://uptime.netcraft.com/up/today/top.avg.html
You'll notice that 45 of those top 50 are BSD
machines. Of those 45, 19 are FreeBSD boxes.
You'll notice 1 Linux box. It's nice to see that
leading industry sites like bongload.com and
twobigirls.com have benefited so much from the
stability of BSD.
The most important thing any republican needs to know.
Right now I've got four systems running Linux (RH-7.3 and 7.2) and one system running FreeBSD 4.6. At times in the past I've run OpenBSD and NetBSD as well.
I can tell you firsthand that in terms of system stability that Linux and FreeBSD are comparable if not indistinguishable. FreeBSD does seem to be more efficient however. The pentium 200 that I have FreeBSD on loads up KDE 3.0 noticably faster than Redhat 7.2 did, and once loaded it is more responsive. On older hardware FreeBSD definitely seems to have an advantage. I consider FreeBSD to be a very fast and well designed operating system. I keep trying to find places where using it instead of Linux would be an advantage.
Not everything about it is all that rosey however. The features and abilities that Linux provides but FreeBSD lacks such as SMP, kernel pre-emption, fast journaling filesystems, certain commerical software packages, 3D acclerated X servers, and generally better device support, make actually using FreeBSD as anything but an interesting toy kind of difficult to justify in many situations.
I worry about FreeBSD. I'd love to see it grow and progress not as a competitor to Linux, but as something of a companion to it. So many people just don't seem to realize that open source isn't about operating systems alone. What Linux and FreeBSD do is provide a foundation, they aren't the whole house. Both provide a powerful and stable platform for running the actual programs that people want to use in the first place. The future of open source development is going to be 90% apps and userland and 10% OS. To have religious and political wars over the OS portion is immature and counterproductive. Linux and FreeBSD aren't genuine competitors from an economic standpoint because it is the applications that both run that make either compelling in the first place.
I want BOTH Linux and FreeBSD to do well, to grow and expand and be the best operating systems anyone has ever seen. I detest the infantile immaturity of those who seek to create division and conflict between FreeBSD and Linux that simply shouldn't be there. I've gotten flames from FreeBSD "advocates" in particular filled with such hatred and obvious zealousness that you'd think they were Mac freaks, all because I described FreeBSD in terms that weren't favorable enough for their religious views. The Linux crowd is full of just as many jackasses, if not more.
Computer enthusiasts are known for generally having high IQ's. Unfortunately our reputation for having low EQ's is equally well earned. There are far too many borderline autistics and asperger's sufferers among us with severely retarded social skills. That is really the only explanation I can come up with when grown men with extensive vocabularies use them to throw a fit on par with that of an eight year old.
Anyway I'm drifting way off from what I wanted to write about. The point that I really want to make is that BOTH Linux and FreeBSD are absolutely fabulous operating systems (save the linux is just a kernel messages for church). The goals and vision behind each are so similar that any ill will between them is manufactured by immature, short sighted assholes. Microsoft is the enemy, not those who prefer another free Unix derivative that runs Mozilla, gnome, kde, etc just as well if not better than what someone else is using.
Lee
Muslim community leaders warn of backlash from tomorrow morning's terrorist attack.
(root@oracle)(3/ttyp0)(08:19P:07/23/02)-/ i386/conf)- grep SMP LINT
(#:/sys
# SMP OPTIONS:
# SMP enables building of a Symmetric MultiProcessor Kernel.
# An SMP kernel will ONLY run on an Intel MP spec. qualified motherboard.
# Be sure to disable 'cpu I386_CPU' && 'cpu I486_CPU' for SMP kernels.
# Check the 'Rogue SMP hardware' section to see if additional options
options SMP # Symmetric MultiProcessor Kernel
Indeed. If I had only known that BSD was dead I would have avoided having so many happy customers who paid me money.
Nail on the head bro' ! Ill probably get my ass kicked by penguin fans but my religion still tells me that FreeBSD kicks the penguin in the nuts.... The deamon gets just in front of the penguin in my book... Solaris and others are way below....
True ravers don't need drugs
That's a weak point. It still doesn't explain
why BSD dominates this "table" and some other OS
doesn't. Why isn't it dominated with Linux or
some other OS that isn't being what you perceive as
"rapidly developed". So Linux can't handle the stress
of a non developed environment? You can FUD
around with your philosophies on the psychology
of sysadmins all you want, but this table shows
something very clear. As much as your love your
pet OS, BSD is the clear stability king.
The most important thing any republican needs to know.