Slashdot Mirror
New Scheduler Available for FreeBSD
flynn_nrg writes "Luigi Rizzo, one of the FreeBSD developers, has just finished the code for a new scheduler. From the announcement: '...as promised, a first version of the Proportional Share scheduler that we developed is available here. These are for a recent -STABLE (i think any version from 4.4 should work; the only 3 files modified are kern_synch.c, kern_switch.c and
proc.h, plus a one-line change to kern_exit.c).
I have tested it a little bit on a diskless system, and it seems to survive running a full X session with the usual set of xterm,
netscape etc. while i do a "renice" of the processes and even switch back and forth between schedulers. But do not trust this yet for a
production system!'
Read the full post here."
There's more info here.
Excerpt:
"There are compelling reasons to use proportional share scheduling techniques to support multimedia and other soft real-time applications on general-purpose operating systems. First, proportional share (PS) schedulers are a good match for existing infrastructure such as a periodic timer interrupt and mechanisms for assigning priorities to applications -- priorities can be mapped to shares in a proportional-share environment. Second, PS schedulers provide stronger guarantees to applications than do traditional time-sharing schedulers: they allocate a specific fraction of the CPU to each thread, and some schedulers provide error bounds on the allocation rate. Third, PS schedulers have clear semantics during underload: excess CPU time is allocated fairly, in contrast with some reservation-based schedulers that must idle or back off to a secondary scheduling policy once all application budgets are exhausted."
"There are compelling reasons to use proportional share scheduling techniques to support multimedia and other soft real-time applications on general-purpose operating systems. First, proportional share (PS) schedulers are a good match for existing infrastructure such as a periodic timer interrupt and mechanisms for assigning priorities to applications -- priorities can be mapped to shares in a proportional-share environment. Second, PS schedulers provide stronger guarantees to applications than do traditional time-sharing schedulers: they allocate a specific fraction of the CPU to each thread, and some schedulers provide error bounds on the allocation rate. Third, PS schedulers have clear semantics during underload: excess CPU time is allocated fairly, in contrast with some reservation-based schedulers that must idle or back off to a secondary scheduling policy once all application budgets are exhausted."
Don't forget Linux will soon have a new scheduler, too. The O(1) scheduler by Ingo Molnar kicks some serious ass, especially with SMP.
"I have opinions of my own, strong opinions, but I don't always agree with them." -- George H. W. Bush
Enabled by default? I don't think so!
/etc/defaults/rc.conf
matt@xena$ uname -sr
FreeBSD 4.6-RELEASE
matt@xena$ grep sshd_enable
sshd_enable="NO" # Enable sshd
matt@xena$
Is FreeBSD's new one a 0(1) scheduler?
0(1) is a "term" from computer science. When applied to schedulers, it basically means that no matter how many processes there are to schedule, a 0(1) scheduler's overhead will not significantly increase.
Of course, with a small number of threads/processes to schedule, the Linux 0(1) scheduler will have greater initial overhead. It isn't until there are quite a few processes that it starts to show its power, and the more processes there are, the more useful it is.
On a busy server with 4+ processors and thousands of processes, a standard scheduler's overhead is so great that it often exceeds the overhead of most of the individual server processes.
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra
I don't know about that, considering in the last month we had 2 big exploits (openssh, and libc resolve bug). The advice for the libc bug was to cvsup the whole system, cause lots of stuff depended on that.
The openssh bug had a one line workaround.
The libc resolver bug has not been successfully exploited yet (so it's not really an exploit). It SEEMS POSSIBLE to exploit it, yes, but it's not trivial (it involves messing up dns replies, so you'd have to have control over an ip block, force the resolver to try to resolve an ip in that block, send the bad response, and then hope it worked). If you know anything about the bsd source code, you know that you can cd
None of the FreeBSD releases, or the -STABLE branch were vulnerable to the openssh bug.
ftp://ftp.FreeBSD.org/pub/FreeBSD/CERT/advisories/ FreeBSD-SA-02:31.openssh.asc
Note the absence of any released version of FreeBSD.
I am not going to claim that FreeBSD is perfect, but FreeBSD is more secure than the vast majority of Linux-based OSes. It has supported features like the new "GR Security" patch for years, and because it shares a great deal of code with OpenBSD which is audited frequently, it benefits from their work as well.
:)
Of note is that FreeBSD's libc is just over half the size of Linux's Glibc (not that has a thing to do with security)
With FreeBSD, for years, admins have been able to set certain files as "append only" (so even root can only add to, not remove from, log files) and "immutable" (so even root cannot modify or delete the file) and has been able to set firewall rules to the same (immutable) so that creative crackers can't add their personal favorites if they root the system.
This can of course be bypassed by restarting the machine in single-user mode and redusing the kernel security level, but that isn't going to be very easy for your average remote hacker.
Furthermore, since 4.0 you can multiple run complete but separate entire copies of FreeBSD on the same system, each with their own FreeBSD system files and such. You can have a single server run an instance of FreeBSD for Apache, one for Postfix, one for BIND, etc. and if any one of them does get compromised (say, BIND since that happens entirely too often) the cracker can not only not effect any of the other instances--he/she cannot even see that they exist! Very interesting stuff.
Of course, IMHO Linux is worlds ahead of FreeBSD on the desktop front, and the new GRsecurity and ACL features will be a real competitor for the *BSD family. It will be most fascinating to see how things turn out. I wish the best to both of them, and I use both of them every day.
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra
The scheduler is far from finished, and would likely need a lot more work. Integrating it into 5.0-CURRENT (and bringing in the SMP support along with it) is already under way, but could probably take some time.
The scheduler is closely tied with the kernel, and MacOSX does not use the FreeBSD kernel at all. It uses the Mach kernel, which is not only a different kernel entirely but a different core kernel philosophy. Mach is a microkernel whereas FreeBSD's is a monolithic kernel. Both types have their advantages and disadvantages, but microkernels are vastly superior for a commercial OS and for driver installations. Monolithic kernels are theoretically faster and easier to implement.
MacOSX gets its BSD label by using the BSD userland utilities. It is great that Mac's OS is no longer junk. In three months I went from "Macs are toy computers for kiddies and Photoshop pros" to "Wow--I can replace every PC and OS in my house with a single Mac! Great desktop, good server, and all the power of Unix."
I have never been happier with the state of Apple Inc.
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra
The Mac OS X kernel (also known as XNU) is a monolithic kernel (unlike Mach, but like Linux and xBSD) with Mach and BSD sitting side-by-side.
Some earlier Apple Unix efforts were true micro kernel implementations. This was also driven by the attraction of a pure hardware abstraction layer. With Darwin this seems to have moved to a more pragmatic recognition that performance matters.
In Darwin, the Mach bits handle memory management, IPC and device drivers. BSD handles users and permissions, the network stack, the virtual file system and POSIX.
So, this won't directly benefit Darwin, though if it is generally useful then someone/anyone can try and put it into Darwin---long live open source! I confess I don't know how the Mach part of Darwin handles scheduling, though I had heard that the Mach VM and scheduling was pretty good.
Protoplasm. Quiet Protoplasm. I like quiet protoplasm.
Not needed. Linux and FreeBSD both handle different priority levels quite nicely, and in fact can handle them in a much more fine-grained fashion. NT actually has additional priority levels in-between each that you described above, but Linux and BSD have a total of 41 possible priority values (from -20 to 20, including zero)
If you set an application to a priority of 20, it isn't going to be bothering any other processes, and if you set an application to -20, it is going to be worshipped like a god by the scheduler.
As far as I know, neither have a real-time scheduling mode like NT, which is actually a good thing in many cases. If a program running at real-time priority goes into an infinite loop, or for any reason uses 100% of hte CPU (SETI@Home, for example) than the system is locked the hell up. Even the mouse will not get any time for cursor movement, and you have to reset the machine.
Read "man nice", "man renice", and probably "man top" (which I use to change priorities of running processes as root)
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra
Is there any reason why something like this isn't implemented in Linux or FreeBSD?
FreeBSD does have something like this: idle, normal, and real time. By default it is normal, but you can change it to idle (idprio(1)) or real time (rtprio(1)).
In the man page for rtprio(1) is one relevant bug:
"Under FreeBSD system calls are currently never preempted, therefore non-realtime processes can starve realtime processes, or idletime processes can starve normal priority processes."
Maybe the new scheduler will fix this?
There can be problems if your "real time" process misbehaves but then don't do that.
NOTE: The term "real time" is used to mean best effort at real time, for true real time you don't use a general purpose OS (Ie. NT doesn't count either).
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
Not everything about it is all that rosey however. The features and abilities that Linux provides but FreeBSD lacks such as SMP, kernel pre-emption, fast journaling filesystems, certain commerical software packages, 3D acclerated X servers, and generally better device support, make actually using FreeBSD as anything but an interesting toy kind of difficult to justify in many situations.
1. FreeBSD does support SMP, albeit not as good as 2.4; but way better than 2.2.
2. Kernel pre-emption is a hack, I'd like it but many developers think it's the wrong path.
3. SoftUpdates are in most ways alot better than journaling, the only time that it isn't is in the event of a crash; as you have to whipe the dangling blocks in the background. But as performance is much better during up-time, it's all worth it.
4. Can't argue with that, but then, I'd run Windows if it was all that important.
5. Accelerated X, the X Server, is available for FreeBSD.
6. BSD was way ahead of Linux when it comes to FireWire, USB and SCSI. In fact, the USB system in linux comes from NetBSD( 2 years later! ); and the SCSI system is way better on FreeBSD than on Linux.
(root@oracle)(3/ttyp0)(08:19P:07/23/02)-/ i386/conf)- grep SMP LINT
(#:/sys
# SMP OPTIONS:
# SMP enables building of a Symmetric MultiProcessor Kernel.
# An SMP kernel will ONLY run on an Intel MP spec. qualified motherboard.
# Be sure to disable 'cpu I386_CPU' && 'cpu I486_CPU' for SMP kernels.
# Check the 'Rogue SMP hardware' section to see if additional options
options SMP # Symmetric MultiProcessor Kernel