Slashdot Mirror


OpenBSD SMP In The Works

Cajal writes "Four students at the University of Waterloo are working to add SMP support to OpenBSD as part of the Spinlocks project. More information is available in a story at the OpenBSD Journal's site. They expect to have an initial working MP kernel in January."

25 of 259 comments (clear)

  1. Wow! by Jeffrey+Baker · · Score: 5, Funny

    In related news, the Egyptians are on the cusp of discovering Construction, which will allow them to build Aqueduct and Coliseum. However, this is not expected to improve the odds of their feared Chariot against invading Mechanized Infantry.

    1. Re:Wow! by ajs · · Score: 5, Insightful

      Ok, I grant your comment is funny, but I'm a Linux user (and sympathizer :) who grew up on BSD, and it really pains me to think that any OS (Solaris, Linux, HP/UX, etc) needs to be viewed as a competitor to BSD rather than a fellow citizen in the realm of UNIX and UNIX-derived OSes.

      Each has its niche, and while some of those niches wane over time (e.g. SCO, IRIX, DG/UX), others flourish (Linux, FreeBSD, Solaris) and that's a good thing. They continue to flow into the containers that they define, rather than having to attack eachother as many products do.

  2. Great news by ekrout · · Score: 4, Insightful

    And it makes for a good research project as well.

    But I ask here, as an honest interested person, why one would wait until SMP is correctly and efficiently implemented into OpenBSD when they could simply use any old recent version of Windows or Linux on SMP hardware to get symmetric multiprocessor support for a high-load server?

    I understand that Research -> Products -> Corporate $$$, but is this perhaps too little too late for OpenBSD?

    --

    If you celebrate Xmas, befriend me (538
    1. Re:Great news by bahwi · · Score: 5, Informative

      Because OpenBSD is about security, not the lastest and greatest features. Linux is about the latest and greatest features. Since the economy went south, most of the peopl working on any of the BSD's lost their jobs or were unable to continue working on the BSD's during corporate time. Where the BSD's have corporate backing and private backing, Linux is mostly private backing, i.e. people at home working on it. Again, things are changing, but everyone has their preferences. No one is going to simply give up OpenBSD to go to Linux, if they need SMP, that is the best route. But from OpenBSD's web page:

      "Only one remote hole in the default install, in more than 7 years!"

      So they all have their uses, Linux, FreeBSD, OpenBSD, and NetBSD. =) Live together, work together, don't kill each other.

    2. Re:Great news by jfedor · · Score: 4, Insightful

      But I ask here, as an honest interested person, why one would wait until SMP is correctly and efficiently implemented into OpenBSD when they could simply use any old recent version of Windows or Linux on SMP hardware to get symmetric multiprocessor support for a high-load server?

      Because you like OpenBSD and would like to help them test the SMP-enabled version so that one day it runs properly?

      What's an "old recent version of Windows", BTW? :)

      -jfedor

    3. Re:Great news by Anonymous Coward · · Score: 4, Funny

      MS-DOS - No root exploits, no patches since 1981!

    4. Re:Great news by jfedor · · Score: 5, Funny

      "Only one remote hole in the default install, in more than 7 years!"

      What puzzles me is how they jumped from "nearly 6 years" to "more than 7 years" in less than a year. :)

      -jfedor

    5. Re:Great news by joib · · Score: 5, Insightful

      I'd say the main reason for Linux development continuing rapidly despite the economy is that the Linux market is orders of magnitude larger than the *BSD market, so the distro makers (and other companies who employ Linux kernel hackers) have the money to keep Linux kernel hackers employed.

  3. It's about friggin time they did... by CoolVibe · · Score: 5, Informative
    Although, the question remains if Theo will accept these patches...

    The last time I spoke to Theo in person, he wasn't too keen on SMP. That wasn't too long ago.

    1. Re:It's about friggin time they did... by aridhol · · Score: 4, Interesting
      Possibly he wasn't keen on the time investment required to implement SMP. If these guys do all the work, it may well make it in.

      OTOH, it may be that SMP code is more difficult to audit, and that this is the reason it won't make it in. Remember, SMP allows for the possibility of race conditions within the kernel itself, which would be a nightmare to validate for security.

      --
      I can't say that I don't give a fuck. I've just run out of fuck to give.
    2. Re:It's about friggin time they did... by CoolVibe · · Score: 4, Insightful
      Possibly he wasn't keen on the time investment required to implement SMP. If these guys do all the work, it may well make it in.

      It might. Even if Theo doesn't accept these patches, the patches will still be available. I just hope they keep maintaining them and keep them up to date if Theo says no for any reason.

      OTOH, it may be that SMP code is more difficult to audit, and that this is the reason it won't make it in. Remember, SMP allows for the possibility of race conditions within the kernel itself, which would be a nightmare to validate for security.

      If Theo would deny this work, it would probably be on those grounds. It was indeed one of the reasons he mentioned to me.

      The most likely place for race conditions to occur on SMP systems is with threading. I have yet to see a totally solid threading implementation that is totally devoid of race conditions of any kind wrt locking/freeing/semaphores/etc. Usually kernel developers solve most of the problems by passing one big lock around (like linux does, and FreeBSD (ever heard of Giant?))

      All in all good news that these guys are working on it indeed. My main concern is seeing this project die because it has the chance of being shot down by Theo. I really hope they persist in pushing Theo to accept it. I also hope they have a lot of patience while dealing with Theo, he's also not the easiest to get along with :)

  4. Re:The problem with OpenBSD.. by jfedor · · Score: 5, Insightful

    It doesn't matter which CD set you buy, what's important is that the project gets the money.

    You can always get the latest release by FTP.

    So why don't you just buy the current release now.

    -jfedor

  5. Re:Hooray! by Daniel_Staal · · Score: 4, Informative

    They have ipv6. OpenBSD ships with ipv6 active and operating. They've been looking at/working on SMP for some time, but they (read: Theo) wants to make sure it meets the standards of the rest of the OS. SMP adds quite a few (theoretically at least) security problems to deal with, and they want to be sure those problems are fully addressed.

    --
    'Sensible' is a curse word.
  6. Re:The problem with OpenBSD.. by jfedor · · Score: 5, Informative

    Isn't it the OpenBSD folks who are telling people not to make ISOs because the codebase changes frequently enough?

    No.

    Perhaps you are confused by this.

    Why would you purchase a set of discs to perform multiple installs when OpenBSD developers recommend against using a static copy?

    They don't. OpenBSD releases come at regular 6 months intervals (3.2 was a month early). That's what you should be using. You can use the snapshots or even the current CVS if you feel brave.

    Sure, I can understand buying copies to support OpenBSD. I buy Redhat for the same reason, it's more principle than the actual material in the box.

    You are correct. There's a slight difference, though, OpenBSD is not trying to turn a profit, just cover the development costs.

    -jfedor

  7. XP and Linux comparisons are pointless by flinxmeister · · Score: 5, Interesting

    I've been using OpenBSD in several mission critical networking roles for 3 years now, and I can safely say that I haven't needed SMP.

    The conventional wisdom that an operating system should be judged according to it's bells and whistles is what's wrong with the software industry. An OS should be judged by two things: Does it do the job I require of it, and does it do it well?

    There are many many jobs that do not require SMP. There are many many jobs being done on SMP boxes that do not require SMP. As the price of processors has diminished, SMP is just a cool thing to buy. I'd be willing to put money down saying that 75% of the SMP boxes out there aren't needed (if that was measurable).

    So, if you want to judge your OS based on features you don't need, then go for it. I use OpenBSD because it is the best choice for that particular need. If you want to assume that one OS is the Uber-OS because of the back panel of the box, then go for it. I'll assume a particular OS is best for the task at hand, and go with that.

    I'm not part of the OpenBSD project (nor do I play one on TV), but one of the central points behind it is that they don't put in things unless they are needed. So far it doesn't seem like SMP has been justified in the great scheme of things (no surprise given the actual need in the wild). I'd much rather have them working on things I'm going to be using instead of evaluating other products based on things I won't.

  8. Re:Hooray! by bahwi · · Score: 4, Insightful


    Troll.

    OpenBSD has had IPv6 since version 2.7 out in June 2000.

    And for the record, FreeBSD has had IPv6 since March, 2000, version 4.0

    And let's not forget who brought you OpenSSH.

    SMP isn't the top priority. Giving up stability and security for the latest and greatest features are not what everyone wants. A friend of mine complained about FreeBSD not having good SMP support, I asked him if he had an SMP machine, he said "No." I hope that is enough to illustrate my point.

    Sorry to go off on this, but mod the parent down if you mod me down please. People always trounce on any of the BSD's while praising Linux here.

    Hear that? That's my karma in the toilet. Flush.

  9. A long wait... by evenprime · · Score: 5, Informative
    There's been talk of doing this since 1997. In the past there was concern about the cost of SMP hardware to develop on and also on the huge amount of time needed to do it right:
    SMP is a big deal. OpenBSD does things, and it does them RIGHT. To do SMP right, we'd need to make the kernel fully-reentrant. This means that we'd clean up the kernel I/O functions so that they don't wait on one another (that's a really dumbed-down, bad explanation of it.) By making the kernel re-entrant, we wouldn't have the problem of spinlocks (one processor waiting on the other to finish I/O, etc.) This would mean almost a COMPLETE re-write of the kernel. This would be a six+ month ordeal for quite a few coders working 40-60 hour weeks. Remember, such a huge task needs to include not only the re-writing of existing code, but checking it to make sure it works on all supported platforms without breaking all the great existing features of OpenBSD.
    That bit about doing things the Right Way is a major consideration for the OpenBSD team. In 1998 jkatz pointed out that they probably wouldn't just use the code from another BSD because they wanted to make sure that OpenBSD's solution was more scaleable.
    --

    "Weapons should be hardy rather than decorative" - Miyamoto Musashi
    I think that goes for OS's too
  10. Very smart... by Dahan · · Score: 4, Interesting
    Good to see that they're using NetBSD as a reference for this... OpenBSD is basically a less-active branch of NetBSD from a couple years back, so it should be a pretty straightforward process to merge in the SMP stuff from NetBSD (which, like just about every other OS, has had SMP for quite some time now).

    OpenBSD is a very promising OS, and SMP support will finally let it play with the big boys in the free *nix playground :)

    1. Re:Very smart... by LizardKing · · Score: 4, Interesting

      OpenBSD may have been branched from NetBSD, but there is practically no resemblance left anymore. Both, in source code and userland, there have been so many changes that the differences between Net and Open are bigger than the difference between either of them and FreeBSD.

      I applaud your attempt to counter the accusation that OpenBSD is "less active" than Net, but you've got it a little wrong. The userland between the three *BSDs is very similar, and the kernels have similar subsystem layouts. Without this similarity, things like softdeps, systrace and IPv6 wouldn't have percolated so quickly into all three. Finally, note that this new OpenBSD SMP work builds on Bill Studenmunds NetBSD code.

      Chris

  11. Re:Waterloo? by thirty-seven · · Score: 4, Informative
    The University of Waterloo, eh? Well, knowing them...

    Which apparently you don't.

    the versions of OpenBSD with SMP support will require a Windows XP activation key...

    Or maybe they figured out a way to port OpenBSd to Windows. Or something. Waterloo?

    I assume you're referring to the stories from several months ago about a proposed deal where UW's Computer and Electrical Engineering department would, as part of a larger research sponsorship deal with MS, agree to make C# the language used in a first year class for CompEng students. There was a huge outcry against this by most CS and CompEng students and profs. Also, note that the School of Computer Science, in the Faculty of Mathematics, had nothing to do with this deal.

    It is my impression that there are many UW students who use or contribute to Open Source projects. Profs are more than willing to make an occasional joke in class at Microsoft's expense. And most CS students (I can't speak for CompEngers) don't touch any MS products for programming projects past first year, by far preferring to use the provided unix labs.

    Do I think the CompEng department's decision regarding C# bad? Very much so. But as I understand it, this decision was made by a few key people who stretched their authority, when really they should have consulted with more people. In fact, Engineering profs have called UW administration on this decision, leading to a decision that before the MS deal can be finalized, it must be approved by "the Electrical and Computer Engineering Department Program Committee, Faculty of Engineering Admissions Committee, Faculty of Engineering Academic Committee, Faculty of Engineering Undergraduate Studies Committee, Year 1 Implementation Committee, Senate Undergraduate Council, UW Registrar's Office and the senate as required by UW policy and practice." This, in my opinion, effectively has killed the deal. See this article for more about this. [Note that the article implies that the proposed MS C# deal would have affected all first year programming classes. This is untrue: only first year CompEng classes would have been affected; CS students would have been fine.]

    The very reason that this decision was such as big deal at UW is that it goes so very much against the prevalent attitude there. And the very large amount of negative feedback they got from UW students and profs in the CompEng and CS departments should ensure that something like this doesn't happen again at UW.

    --

    Atheism is a religion to the same extent that not collecting stamps is a hobby.

  12. This would be nice by QuietRiot · · Score: 5, Interesting
    I'm curious how much of a rework will be required [by the OpenBSD core developers] once these guys are done. 4 guys on a one-year project. SMP. Good luck. Will this be a patch-type thing? Will the core team accept it, or reject it outright? Will the core team use some portion of it - cleaning it up along the way, or will it take a major rewrite?

    It's strange how things like this end up changing would would have been. Do it right the first time, because if it gets adopted, and it wasn't done right, efforts will be diluted.

    I'm glad to see it's happening though. At least somebody's throwing some brainpower at it rather than waiting around for Theo & friends. (no fault to Theo, I know SMP is "in the works" - OpenBSD is secure, first and foremost. That's what I, and many others, care about most. Kudos to you and your team on this! You have a highly-regarded, ultra secure OS that has kept many cracker-types and script-kiddies at bay for many years. You have saved many people many thousands or millions of dollars with the protection your software project has provided. You have given nothing to the headache medicine providers of the IT industry.)

    One more processor for my dual-capable Sun SS20 and I'll have a grand-ole time playing with this. Just too bad it comes with only a single 10-speed ethernet port. Anybody know about S-bus fast ethernet cards?

    To these brave deveopers: Way to go! Thanks for getting the ball rolling and best of luck with your project (and dealing with the publicity! :)

  13. Re:Not to be trollish.... by Tuzanor · · Score: 5, Informative
    Its OpenBSD that has "taken so long". FreeBSD has had SMP for ages now, and until very recently it better scaled than linux's. Keep in mind that NetBSD and FreeBSD forked in the early ninties and both had different priorities. FreeBSD became a stable high performance platform. It only ran on x86 (now alpha, soon to be Ultrasparc and Itanium). They eventually added SMP with other various features.

    NetBSD was more focused on portability. They were aimed at the embeded market (which Wasabi systems is in business in) where there isn't SMP. When Theo forked OpenBSD off of NetBSD they still didn't have it and it still wasn't a priority. Now there is more interest in it, especially now that SMP hardware is becoming so cheap.

  14. Re:Uses of SMP? by drinkypoo · · Score: 4, Informative
    In addition to what the siblings of this comment currently say: Everything that requires lots of CPU, or lots of processes running at the same time. A single thread is usually not relocated across processors, so the following is more or less true in the case where you have lots of processes of vaguely equal size and more than one processor: The number of context switches which have to be done is divided by the number of processors. That's one benefit. The other is that (in theory but not in practice of course) you can execute twice as many instructions at once. Obviously you still have operations which have to wait on things other than other instructions being retired (Though that too) which I suspect is the main reason why usable "speed" (let's say cycles for the sake of saying SOMETHING) does not tend to scale linearly with additional processors.

    Anyway anything that is relatively (or entirely) cpu-bound and involves lots of processes or threads will be sped up significantly. Also one process doesn't tend to monopolize your processor so badly.

    --
    "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  15. Re:a bsd question by cant_get_a_good_nick · · Score: 4, Insightful

    There are some ego questions, Theo used to work on NetBSD. As smart as he is, and productive as he is, they found him abrasive and let him go. He hasn't changed much, and isn't likely to, and the world's a better place for having OpenBSD. From what I understand FreeBSD and NetBSD forked around the same time from the original BSD code, and 386/BSD. 386/BSD was kind of showing its rough edges, and people wnated to clean it up. FreeBSD went the x86 route, NetBSD always tried to be multiplatform.

    That said, there is a lot of code sharing. The USB core is exactly the same in both Net and Free, probably OpenBSD as well. systrace was originally slated for NetBSD, ended up in Open. 5.0 is getting a new /etc/rc startup, pioneered in NetBSD. A lot of the userland is the same, and there's some action on getting a standard ports system.

    In some respects, there's more sharing in BSD than in some of the Linux distros. Since all the owners of the BSDs are essentially non-commercial, there's no real incentive to make proprietary stuff. In some situations, the BSD license is easier to share stuff, but it really doesn't in this case - if they were all GPL they could share things.

    I understand what you're saying, that it diverts attention and resources. But you also have to realize they pick up thing as well. There's some cross pollination. I believe the SMP stuff is kind of taken from BSDi, if not the actual code then at least the general idea.

    The other thing is "one-size fits all" gives you a huge XXXXL product. If all the things went into just one or the other, it would be pretty bloated. The focus of FreeBSD (optimized for Intel) is in respects incompatible with NetBSD (ultra-portability). Ask anyone who's worked on gcc about the problems of optimizing portable code.

  16. Re:Note: Announcement Not From OpenBSD.org by ISWalker · · Score: 5, Informative

    Finally someone who is correct. I'm one of the students working on this. It is our computer engineering project. The plan is to have it somewhat working by January. We decided to this because we needed an idea for project and we thought it would be fun and allow us to learn a lot in the process. When the project is complete, we plan to release the code we have and if OpenBSD wants to use it he can, however, that wasn't necessarily the original intent.