Slashdot Mirror


Bossa, a Framework for Scheduler Development

Eugenia writes "The recent activity in Linux kernel development caused by the introduction of a new scheduler by Ingo Molnar has emphasized for ordinary Linux users the importance of schedulers in modern operating systems. This article gives you a glimpse of what scheduling development is like by letting you implement your own Linux scheduler thanks to Bossa, a framework for scheduler development."

33 of 152 comments (clear)

  1. Always behind the times... by grub · · Score: 4, Funny


    Most commodity computers can only run one process at a time

    Ha, with Longhorn 2010XP+++ and Office 2012 I'll be able to have two Clippys simultaneously! Take that, hippy!

    --
    Trolling is a art,
  2. Let the scheduler algorithm flamewar begin.. by GillBates0 · · Score: 4, Funny

    I see your FIFO scheduler and raise you my Elevator algorithm!

    --
    An Indian-American Hindu committed to non-violent thought/speech/action alarmed by the global explosion of radical Islam
  3. Ordinary users? by MisterFancypants · · Score: 4, Insightful
    "The recent activity in Linux kernel development caused by the introduction of a new scheduler by Ingo Molnar has emphasized for ordinary Linux users the importance of schedulers in modern operating systems"

    I know some people will take this as flamebait, but I honestly don't mean it to be. However, as long as Linux is in a state where developers think that "ordinary Linux users" have to even care what a scheduler is, Linux will be a failure for mainstream desktop usage.

    Users don't care about OS internals. Don't send them to a page explaining OS scheduling, just tell them "All new Linux makes your applications more responsive!". That's all they want to hear.

    Seriously.

    1. Re:Ordinary users? by ErichTheWebGuy · · Score: 4, Funny

      parent poster is right. one of my wannabe-geek coworkers saw that and asked me, "doesn't evolution take care of your scheduling needs?" I am not joking. True story.

      --
      bash: rtfm: command not found
    2. Re:Ordinary users? by Short+Circuit · · Score: 2, Informative

      What the poster meant was that the introduction of the O(1) scheduler, and the benefits to user experience it provided, caused a lot of "ordinary users" to take notice. That doesn't necessarily mean most "ordinary users" know, or even care.

      However, for those that do, you'll Soon be able to drop in the latest IO, CPU or 'whatever' scheduler, now being developed independantly from the kernel.

    3. Re:Ordinary users? by GammaTau · · Score: 2, Insightful

      Users don't care about OS internals. Don't send them to a page explaining OS scheduling, just tell them "All new Linux makes your applications more responsive!". That's all they want to hear.

      I don't think the ordinary users (at least the ones you talk about) are readers of Slashdot or OSNews. This is not marketing material you are reading. It would be silly to judge it as such.

    4. Re:Ordinary users? by johnnyb · · Score: 2, Funny

      In that case, it seems to be pretty easy to do so:

      you: Honey, do you want to come home and see my new scheduler?

      wife: Can you show it to me while I eat the leftover pizza, I'm hungry!

      you: Sure, but please pay attention this time, it's really really cool and neat!

      wife: That's what you said about your new Perl program, but it just looked like you banged on the keyboard for a couple of hours and held down the shift key.

    5. Re:Ordinary users? by djcapelis · · Score: 2, Funny

      Evolutionary algorithms are a wonderful new field of...

      Oh.

      --
      I touch computers in naughty places
  4. nanokernel: scheduler by Doc+Ruby · · Score: 4, Interesting

    A really distributable system would include only the scheduler in the kernel, with an "outer" layer of secure (crypto signed, sealed and delivered) APIs for submitting process and data requests, and an "inner" core for hardware access, including CPU, data (storage/network such as ethernet, USB, IDE, RAM, BIOS ROM, etc) and presentation (monitors, keyboards, mice, soundcards, printers, etc). Such a nanokernel would be tiny, highly efficient, and mix/matchable with many other apps and OS'es. Privileges would be part of a comprehensive security model, with IPC filtered through access control, whether within a single memory segment, LAN, or WAN. All domains would be virtualized. And such symmetry and simplicity would set the stage for flexible inter-kernel load balancing and failover.

    We're talking open-ended scalability. Security. Performance. Reliability. The OS is no longer just a privileged app, but a smaller, more focused critter, serving apps rather than being served by them. With this new scheduler framework, let a billion nanokernels bloom.

    --

    --
    make install -not war

    1. Re:nanokernel: scheduler by Ignignot · · Score: 2, Interesting

      And you've just created the need for a "sub-kernel" which would do your actual work. We're talking more complexity. More bugs. Less performance. Less security. You haven't made the kernel smaller, you've just pulled a part of it out because you think it'd be cool to hold its beating heart in your hand while it still runs.

      --
      I submitted this story last night, and it didn't get posted.
    2. Re:nanokernel: scheduler by sploo22 · · Score: 4, Informative

      You mean an exokernel?

      --
      Karma: Segmentation fault (tried to dereference a null post)
    3. Re:nanokernel: scheduler by Doc+Ruby · · Score: 4, Informative

      No, those other jobs removed from the nanokernel run as independent apps. We've simplified the architecture, by reducing the inner complexity of the kernel, and moving its functions into the rest of the system. IPC and HW access are the only roles which need unique centralization, even in our "OS" paradigm. That means bugs are easier to identify, with fewer hidden dependencies (they're more overt). And performance is more granular, with less extra functions required to be loaded, and logically considered, to perform a specified task. Better security from the reduced trust among processes and the kernel, with a simpler model for their interoperation.

      The kernel is much smaller. The old kernel's functions are distributed in more, optional, dynamically (runtime) includable objects, so they might even be spread among a larger total object population. But that isn't necessary to specific operations, and that can be tuned at runtime, by some of the processes themselves. Just another step down the road away from the monolithic, single-task computer, for increased parallelism and manageability.

      --

      --
      make install -not war

    4. Re:nanokernel: scheduler by Bloater · · Score: 3, Informative

      > A really distributable system would include only the scheduler in the kernel

      No, a kernel doesn't even need to have a scheduler in it. It needs to be able to take instructions to transfer control to another cpu context, and be able to create the necessary page tables, and that's about it.

    5. Re:nanokernel: scheduler by burki · · Score: 3, Funny

      it's probably called HURD. and will be ready next year.

    6. Re:nanokernel: scheduler by sql*kitten · · Score: 3, Interesting

      You are describing the IBM AS/400.

    7. Re:nanokernel: scheduler by Anonymous Coward · · Score: 2, Informative
      No, it does sound like he's describing a microkernel (or "nanokernel", ugh) rather than an exokernel. The defining characteristic of an exokernel is that it provides no abstraction at all. It multiplexes hardware, but does not abstract it. There are no OS "system calls" in the traditional sense. There is no message passing; there are no OS provisions at all, really. Your "application" (which is really an operating system) deals with hardware, with the exokernel checking to make sure the device is being multiplexed properly.

      The mentioning of "API" or "IPC" or "message" (let alone access control!) entirely excludes any exokernels.

  5. Scheduler Development? by Anonymous Coward · · Score: 4, Funny

    I thought everone was using MS project. No?

  6. Re:I hope I'm not alone when I ask... by Timesprout · · Score: 3, Funny

    Depends how you pronounce it. If you pronounce it properly then it a vital part of the OS determining how the various running processes interact with the system, resources and each other. If you use the American pronunciation then its something KDE are probably working on to organise your time.

    --
    Do not try to read the dupe, thats impossible. Instead, only try to realize the truth
    What truth?
    There is no dupe
  7. Re:I hope I'm not alone when I ask... by The+Analog+Kid · · Score: 3, Informative
  8. Re:I hope I'm not alone when I ask... by AuMatar · · Score: 2, Informative

    A scheduler is a part of the OS which decides what process has access to a given resource at a given time. The main scheduler is the process scheduler, which allocates which process gets to use a CPU right now. There's also schedulers for disk access, sending ethernet packets, and anything else that can only be accessed 1 at a time.

    --
    I still have more fans than freaks. WTF is wrong with you people?
  9. I hope ... by The-Bus · · Score: 2, Funny

    I hope this succeeds, then gets improved upon in a second version. How cool would it be to be a part of the next project called... "Bossa Nova"?

    --

    Small potatoes make the steak look bigger.

  10. Interrupts by LordMyren · · Score: 3, Interesting

    I have a sneaking support its poor interrupt handling which makes my ALSA skip like Riverdance every time i start a file copy in mid song.

    Even buffering the full song, it still skips. I've tried XMMS, moosic, mpg321. I've sought ever high priority trick i can. No matter what I try, start that file copy and WHAMO, instant pain and suffering.

    Myren

    1. Re:Interrupts by plam · · Score: 2, Informative

      Have you turned on DMA for your hard drive? (hdparm -d 1 /dev/hda).

    2. Re:Interrupts by cr0sh · · Score: 2, Informative
      hdparm has to honestly be one of the most undelooked at tools for increasing performance in a Linux system. Everything will get a boost from using hdparm - just remember to update your /etc/rc* scripts to reinstate your choices on reboot. A good document for hdparm can be found here:

      Speeding up Linux Using hdparm

      --
      Reason is the Path to God - Anon
    3. Re:Interrupts by MarcQuadra · · Score: 2

      I have to agree with cr0sh. You're thinking about things a bit too simplistically.

      Imagine that hdparm HASN'T been set properly, your drive reads don't appear to be slow or hog the CPU, and you're happy. But when things get 'hairy' and your'e doing a full-blast copy and trying to play an MP3 (which only needs about 1MB/minute) things are getting all choked-up. I'll bet this wouldn't happen if...

      hdparm -A1 -a64 -c1 -d1 -m16 -u1 -W1 /dev/hda

      (that reads:

      enable hardware lookahead, enable 64-sector software precaching, enable 32-bit controller-cpu IO, enable DMS (less interrupts), move 16 sectors (8K) at a time instead of 1, unmask other IRQs while doing disk stuff (probably YOUR problem), and enable writes to be committed to the drive buffer instead of disk.

      )

      My guess is that your disk access is throwing a LOT of interrupts because you haven't set hdparm right and the sound card is starving for attention behind the wall of data. This wouldn't show up as an overtaxed drive or an overtaxed CPU but it WOULD affect your playback.

      I've got an ancient 10GB IDE drive here and I can play MP3s back smoothly while opening OpenOffice.org AND copying files up to the server at 100Mbits/sec. You don't have 'scheduler' issues, your system is like a racecar running 87-octane gas.

      --
      "Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
  11. while(1) loops by vijaya_chandra · · Score: 3, Funny

    10 penguins for anyone who can suggest a scheduler that taxes the stupid newbie programs with

    while(1);

    (/me too is a newbie)

  12. Re:I hope I'm not alone when I ask... by Waffle+Iron · · Score: 2, Funny
    What the hell is a scheduler?

    The scheduler is the person on your project team who, after each week's schedule slips, tapes together the latest updated Gantt chart printouts into long banners and hangs them the walls.

  13. Re:eeek (OT) by slashjames · · Score: 2, Interesting
    Working with semaphores and locks... *shudder* keep the bad man away!!

    Just remember that it's those semaphores, mutexes, and locks that allow multi-threaded applications work. A class that is supposed to be "thread safe" but isn't will make your life rather... interesting.
  14. Re: locks &semaphores, "nothing I hated more" by nusratt · · Score: 3, Insightful

    "Working with semaphores and locks... *shudder* keep the bad man away!!"
    http://slashdot.org/comments.pl?sid=11388 0&cid=964 7043

    Actually, it's quite interesting. It forces you to conceptualize with a certain kind of rigor, and to code with particular vigilance for potential deadlocks, race-conditions, and mis-serialization of resource-updates.

    I can't tell you the number of times that I've read or listened to a description of a design or architecture, and immediately said "uh-oh", to the surprise and dismay of developers who really should know better.

    In a time of increasing processor parallelism, this is a skill-set which must be mastered by anyone who truly aspires to become (or remain) a serious professional-level developer. It's not just for kernel-hackers.

  15. Re:I hope I'm not alone when I ask... by vadim_t · · Score: 5, Informative

    It's the part of the OS that determines what task to run.

    The problem is this: You want to run tasks A, B, and C, and need to do it in the most optimal way possible.

    The simplest way is FIFO, like say, DOS. If A starts first, then A runs until completion, then B starts. This is bad for many reasons, like that if A gets stuck for any reason nothing gets done, and that while A is waiting for input the free resources can't be used for anything else.

    A simple way of multitasking would be simply alternating between A, B and C every fixed interval. But that's not very good either, perhaps B doesn't need to do any processing now, and only wastes time, while C could really use that.

    A better way is to do it by priorities, but then you need to find a good way of calculating this priority.

    One of the reasons there's so much talk about them is that most become slower when you have more tasks. Most of the simple ones need to examine every running task to determine what should run now, and if you want to launch 10000 processes at once, that's not good. O(1) schedulers are a solution for that, because they use algorithms that always take the same amount of time to execute, irregardless of whether there's 3 or 10000 tasks. That doesn't mean a simpler scheduler wouldn't work faster for 10 tasks, though.

    The other problem is how to determine how to distribute CPU time. Say, for servers you mostly want fast and fair3o determine which processes are interactive, and which are on the background and won't mind some interruption.

  16. schedulers as modules by rsd · · Score: 4, Interesting

    Using a plugin^Mmodule interface to load schedulers at runtime wouldn't generate
    a performance impact in the scheduler? (as opposed to have the scheduler compiled
    inside the kernel).

    I AFAIK, the scheduler has to be as compact (optimized) as possicible to reside as long as possible
    in the cpu's cache. This way it can check the memory pages map as fast as possible to [de]allocate,
    switch process as fast as possible.

    Using a module scheduler, wouldn't make it have to derreference each function address each time
    each function is called?
    And probably sometimes derreferencing derreferences few times to get the correct address?

    Couldn't this hurt performance?

    I agree that loading an efficient scheduler to handle a situation better than the defalt scheduller would
    compensate for that, but still...

  17. Re:who cares by angulion · · Score: 4, Insightful
    Either way---say what you will, but windows has always been more responsive on the desktop.

    And Linux has been more responsive in the server-room. A server doesn't even need a desktop.
    A desktop user like you might not care if your scheduler degrades after you have 800 processes running, but I can assure you people dealing with large server systems does.
    Windows is in large part more responsive on the desktop since it is in part intergrated into the kernel - the downside being that what would be a application crash can bring down the whole OS. Also you have less privilege separation. (windows desktop is "unsafe" area, but that is another story).

    (What directX equivalent is there on Linux?)

    OpenGL, SDL, OpenAL for starters. Guess what they all can do that DirectX can't?! Yepp, run on different systems and architectures, be it Linux, Windows or a Mac.

    If Linux developers don't stop diddling around with something that was solved years ago, Linux will just go away.

    Linux developer base is quite large and the developers like to "diddle" around to find the best/most effective solution, not the one that takes the least amount of time to make, like it often tends to be when deadlines are pushed in companies.

    Linux doesn't even have a program that can do half of what programs like dreamweaver can do.

    I admit that there is nothing quite like dreamweaver for Linux, but I'm convinced that Quanta can do more than half. :P Anyway, as Linux gets more acceptance, programs like Dreamweaver will eventually be ported.. what will you say then?

    For the developer, what app is as integrated as Visual Studio? KDevelop? Pshaw.

    Are there any development tools that is more restricted to one platform and project management than VS?
    KDevelop and Anjuta might not be as integrated, but they can use CVS, SubVersion etc. and compile on Linux even for Windows (or the other way around). Oh, and they are free.

    I'm not flaming you---but the article. Users don't give a shit about schedulers if there's no applications with them.

    No, not at all :P
    You know, there are alot of people that acctually are interested in kernel-schedulers, allthough they do not tend to be your average desktop user.
    More over, one place where schedulers matter most, ie. in servers, there is absolutely no shortage of applications on Linux.. I'd more like to say there is a shortage of applications in this are for Windows, if anything.

  18. Impact of scheduling on threads by ThePhin · · Score: 3, Interesting

    Xavier Leroy is one of the authors of Ocaml, an efficient (within 2x of C) variant of the ML functional programming language. In the Caml-list mailing list recently, he had this to say about scheduling:

    The 2.6 Linux kernels changed the behavior of sched_yield() in a way that causes the unfairness you observed. Other threaded applications are affected, including Open Office (!). My belief is that it's really a bug in 2.6 kernels and that the new behavior of sched_yield(), while technically conformant to the POSIX specs, lowers the quality of implementation quite a lot.

    (I seem to remember from my LinuxThreads development days that this isn't the first time the kernel developers broke sched_yield(), then realized their error.)

    The question I'm currently investigating is whether the call to sched_yield() can be omitted, as it's just a scheduling hint. Initial experiments suggested that this would hurt fairness (in Caml thread scheduling) quite a lot on all platforms other than Linux 2.6. More careful experiments that I'm currently conducting suggest that it might not be so bad. One can also play games where sched_yield() isn't called if there are no other Caml threads waiting for the global Caml mutex.

    So at least one class of user is forced to be aware of the scheduler, to refer to another poster's assertion that users shouldn't even need to know...