Slashdot Mirror

← Back to Stories (view on slashdot.org)

libkse to libpthread switch on FreeBSD

Posted by Hemos on from the moving-forward dept.

Dan writes "Daniel Eischen says that libkse has been renamed back to libpthread and is now the default threads library. The gcc-pthread option has also been changed to link to libpthread instead of libc_r. For alpha and sparc64 machines, libkse has not been renamed and links are installed so that libpthread points to libc_r. FreeBSD GNOME team's Joe Marcus Clarke confirmed that the ports system will switch to using libpthread as the default for PTHREAD_LIBS shortly. A patch set is currently being tested, once that completes, the necessary port hooks will be in place to easily build applications linked to libpthread."

26 comments

  1. BSD may be dying... by Ianoo · · Score: 3, Insightful

    ... according to some, but I think it's rather useful. The old saying "Don't put your eggs in one basket" applies. (Friendly) Competition is good for open source software - witness KDE and Gnome's attempts to out-innovate eachother.

    I don't think for one minute that SCO's claims about Linux are legitimate, but imagine if they were: and some idiot judge ordered that Linux distribution was stopped. I think all the BSD trolls might be glad of an alternative kernel and userland under those circumstances!

    1. Re:BSD may be dying... by FutureShoks · · Score: 3, Interesting
      BSD is not dying. I remember reading a survey not a month ago that surveyed some top ISPs. FreeBSD running Apache was, according to the article, the most popular and widely used ISP platform there is. Period.

      Linux is trying to "do a Windows" and be all things to all men as quickly as possible.

      FreeBSD is more like a working pick-up truck. They don't have a wide range of paint colors and picking up chicks with them can be hard - but they do what they are built for, and they do it very well.

      --
      ___FutureShoks___
    2. Re:BSD may be dying... by zangdesign · · Score: 1, Offtopic

      They don't have a wide range of paint colors and picking up chicks with them can be hard - but they do what they are built for, and they do it very well.

      You mean staring at UFO's and conjugal relations with not-quite-second cousins?

      Sorry. Someone had to say it - might as well be me.

      More on point - if FreeBSD is dying - why is everyone so determined to point out that it is dying (which it isn't)? Wishful thinking?

      --
      To celebrate the occasion of my 1000th post, I will post no more forever on Slashdot. Goodbye.
    3. Re:BSD may be dying... by Carnildo · · Score: 0, Offtopic

      More on point - if FreeBSD is dying - why is everyone so determined to point out that it is dying (which it isn't)? Wishful thinking?

      I think that, these days, the "BSD is dying" posts are more a traditional thing to do on Slashdot than any sort of expression of fact or desire.

      --
      "They redundantly repeated themselves over and over again incessantly without end ad infinitum" -- ibid.
    4. Re:BSD may be dying... by anthonyrcalgary · · Score: 1, Offtopic
      Some Linux people like to think that it's just them, Windows, and the commercial Unixes.
      The trolls are probably a small number of people that have their own reasons that wouldn't make sense to us. As for the rest... I have a friend that thinks Linux will gradually force everything out because it's good enough at everything, rather than really good at any particular thing. He's not a zealot, but that's what he thinks. I think that's a common attitude.

      (many) Linux supporters see the OS world as a tripod: themselves, Windows, and commerical UNIXes. The BSDs occupy the same space as Linux. Using a license that allows work to be "stolen", competing with them for talent and mindshare. That's not a comfortable position to be in. I mean, there are those that really get honestly angry over the BSD license.

      Frankly, the whole attitude that there should be one OS to do everything confuses me. Some goals are mutually exclusive.
      --
      When someone might yell at me, it has to be OpenBSD.
    5. Re:BSD may be dying... by drinkypoo · · Score: 1
      I agree with your friend. Linux doesn't have to be the best at everything to relegate almost everything else to bargain bin status. Of course I doubt it's going to force windows out any time soon but there's room for it to just wander around dominating everything. Its desktop market share is increasing (from nothing to just over nothing, so far) and it's made huge inroads in both the server and embedded markets. Linux scales pretty well from the very small to the fairly large.

      Ultimately I see it all but completely displacing all other operating systems on the high end. I think that the tiniest stuff will never run linux, because we will continue to have tiny machines with little ram and no mmu, they'll just get smaller and smaller and more and more ubiquitous, until every button on your shirt has a processor in it. It's going to be a long time before linux would be useful in a system that small, by which time something else should have come along and stomped on it, in turn. Of course this isn't necessarily true, we have genetic Unixes that are descended from some of the oldest operating systems around alive and well today. Linux could in fact be around forever.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    6. Re:BSD may be dying... by Random+Frequency · · Score: 1

      Linux is trying to "do a Windows" and be all things to all men as quickly as possible.

      This statement doesn't make sense. It implies that there is one group taking an operating kernel and trying to stretch it out everywhere?

      LOTS of people are trying to take various distributions of systems running on linux and making it do everything.

  2. In translation... by edhall · · Score: 5, Informative

    "Kernel threads" will now be the default instead of "user threads." (Many BSDers cringe at this use of the term "kernel threads," since to them it represents the misapplication of a term apropriate only to a thread that runs entirely within the kernel. But the above is a usage common in the Linux world.)

    This is a key step on the way to finishing the work for FreeBSD 5 and moving on to FreeBSD 6. It incorporates a highly sophisticated (some would say over-sophisticated) M:N threading system; this is where a new kernel-scheduled context is created only for threads that the userland scheduler thinks may block as opposed to having a kernel context for each thread like Linux does. It remains to be seen if the theoretical advantages of this approach will be turned into real-world advantages. I suspect that it will be a while before we know; although libkse has proven stable of late, there will be lots of additional experience acquired now that it is the default and no doubt this will result in further tuning.

    Congrats to the FreeBSD team! This and the (not entirely unrelated) SMPng subsystem were the biggest steps on the way to getting FreeBSD 5 ready to take the FreeBSD mantle. It was a gamble going this route rather than the "safe" alternative of a 1:1 model, and there were times when a number of folks wondered if libkse would ever be finished. Well, now it is!

    -Ed
    1. Re:In translation... by Anonymous Coward · · Score: 1, Informative

      Well, the M:N approach has already been done well on Solaris, so it's not really a mystery what the pros/cons are. On Linux, the M:N approach was rejected because the performance advantages were non-existant. LWPs are light enough on Linux that presumably the extra overhead of mapping U-threads to K-threads was counter-productive. In any case, Linux application developers are warned against creating excessive numbers of K-threads in the first place.

      Despite all those arguments, I think an M:N model is still useful for those applications that really need to spawn zillions of mostly non-contending threads (do they exist? Java might be one candidate, but Java's user thread implementation is already a better choice than kernel threads on most platforms). It remains to be seen if the FreeBSD implementation bears this out.

    2. Re:In translation... by Anonymous Coward · · Score: 0

      It might prove to be a bigger win to FreeBSD because their context switching is quite a bit heavier than Linux.

    3. Re:In translation... by thufir · · Score: 1

      Linux 2.6 uses NGPT (next gen posix threads), which is an M:N system.

      LinuxThreads was 1:1 but is now deprecated.

    4. Re:In translation... by thufir · · Score: 1

      My bad, 2.6 uses NPTL (new posix thread library) which is 1:1.

  3. Sun now discourages M:N by Anonymous Coward · · Score: 0

    With the latest OS release, Sun's default
    thread library is 1:1. It seems Sun got
    tired of fighting with the complexity.
    The real-world complexity tends to eat up
    any theoretical performance advantages of
    an M:N model.

    Sure there isn't some obscure ioctl() that
    will block a thread or cause some other sort
    of trouble when you generate a new thread?

    1. Re:Sun now discourages M:N by Anonymous Coward · · Score: 1, Insightful

      It's intersting that the architects of Windows NT came to the same conclusion back in the 1980s.

      I personally don't believe that what's best for NT, Linux and now Solaris is necessarily best for FreeBSD, but the fact that the leading OS (NT), leading open-source OS (Linux) and leading UNIX OS (Solaris) all decided on 1:1 (belatedly in the case of Solaris) is certainly an endorsement of it.

      IIRC, one of the main reasons Sun chose an M:N model for Solaris was to ease the porting of old UNIX software that assumed user-mode threads libraries in lieu of real threads (this sort of software ran notoriously badly when ported to NT). With most UNIXes (and even Linux) having had real threads for some time now, this sort of software has become less common, and there isn't such a need to support it, hence the move to 1:1.

      I'd be interested to know why the FreeBSD team chose an M:N design, and in what areas their design is expected to be superior to a 1:1 model, or to the old Solaris M:N model.

    2. Re:Sun now discourages M:N by Anonymous Coward · · Score: 0

      He probably means all NT-derived OSes, including Windows 2000, XP, and 2003.

    3. Re:Sun now discourages M:N by Anonymous Coward · · Score: 0

      Yes, I meant OSes running on the NT kernel, not the product called 'Windows NT'. Actually, though, the 'Old Technology' Windows kernel (the one in Windows 9x/Me) might still have a bigger installed base than NT. At any rate, it also uses 1:1 scheduling.

    4. Re:Sun now discourages M:N by Anonymous Coward · · Score: 0

      Oh leading in installed userbase. Yeah thats fair, sorry for the misunderstanding.

    5. Re:Sun now discourages M:N by julian_el · · Score: 5, Informative
      Well, thanks for asking... We chose M:N because with scheduler activations it wasn't really a very difficult thing to do. And we came from a culture of trying to not use kernel resources for everything..

      We have the option of running in 1:1 mode or in M:N mode with the same code, and we can support 30,000 or so threads in userspace (depending on tuning) and not use 30,000 ses of kernel resources.

      As I mentionned in another response, The code we ended up with can run in both 1:1 and M:N modes and the library can be compiled to operate in either mode.

      To be fair, we took a long time to do this and SUN changed thir minds on the topic when we had already decided where to go, but either way we have good threads..

      It really is quite cute to hit 'H' in top and see Mozilla split into 3 threads..

      We have quite a bit of cleaning up to do.. for example the cpu affinity is mostly a fiction at this time, (at least between threads) and the scheduler interractiosn need work..

      As I have pointed out.. what we implemented was t thread framework that can be used to implement almost any threading model.. For example we will be able to emulate linux threads (well, we do in linux mode) but linux will not be able to emulate our threading unless they add SAs to their kernel..

      We can do:

      * process based (clone) threading (e.g. linuxthreads)

      * process internal threads (the old single process threads)

      * 1:1 system scope threads

      * 1:1 process scope threads

      * M:N theads with async syscalls. (SAs) We'll see which survives in 5 years :-)

      Hopefully this makes a good platform for people to do thread research.. That was a big design goal.. provide general solutions that people can use their imagination to use in ways we hadn't thought of.. That's where libthr came from.. it was a surprise to us.. people whipped it up in a few weeks using the infrastrucure we'd made. (they didn't know how close we were to releasing working code so their stopgap and kse were released almost together :-)

  4. KSE is Kernel Schedulable Entities by Anonymous Coward · · Score: 0

    KSE is Kernel Schedulable Entities, and it's most definitely not 1:1.

    FreeBSD has just moved from default user space threading to default scheduler activations.

    Scheduler actications can be used to get M:N threading, which is the most commonly expected use, and they can also be used for 1:1.

    Effectively, this announcement indicates that libthr, which is the native 1:1 threading for FreeBSD by Jeff R. lost out to libkse, and libc_r (user space threading) lost out to both of them.

    -- FYI

    1. Re:KSE is Kernel Schedulable Entities by edhall · · Score: 1

      I think some folks would disagree that libthr wasn't successful -- libkse seemed for all intents and purposes stalled until libthr came along and gave it a kick in the pants. In addition, libthr helped show how the KSE mechanism had a clean enough separation of policy and implementation to support alternative models. If at some point it's decided that the M:N model doesn't work out -- even for just a limited number of applications -- an alternative could quickly be made available. Although KSE's still aren't entirely without controversy (they complicates development and upkeep in several areas of the kernel), libthr did a lot to calm people's fears at a critical point.

      -Ed
    2. Re:KSE is Kernel Schedulable Entities by julian_el · · Score: 2, Informative

      This is a simplification..
      KSE is not just M:N but also 1:1

      The code to generate new "kernel threads" on the fly if an existing thread blocks is only a small part of what was done (though it was tricky I admit).

      Re: KSE and 1:1 vs M:N,
      It can be used in either way.
      firstly the library can be compiled to create a separate kernel thread for each user thread,
      and secondly, the thread creation process can be done in a way that guarantees a kernel thread bound to the new thread. even if the library was compile "M:N".

      libthr is another arrow in the quither, but
      libkse (as it was called) can do most that libthr
      can do. (I think the kernel supported mutex that
      libthr can use is probably more efficient than what libkse uses for the same thing (but I haven't
      benchmarked them)