Slashdot Mirror


/bin And /sbin Now Dynamically Linked In FreeBSD

Dan writes "Gordon Tetlow just committed a patch in FreeBSD current to change /bin and /sbin from statically to dynamically linked. The reason to do this is two-fold. This feature brings support for loadable PAM and NSS modules to base system utilities located in those directories. It also reduces the storage requirements for the root filesystem due to the use of shared libraries. This feature can be disabled in a buildworld by defining the Makefile (make.conf) variable WITHOUT_DYNAMICROOT. Note that statically-linked, crunched executables are available in the /rescue directory for use during system repair and recovery operations."

58 of 172 comments (clear)

  1. Bad if not included with non-dyn executables by Creepy+Crawler · · Score: 1, Insightful

    The reason behind having /bin and /sbin as static linked is so that if you upgraded the libc and it failed, you still have a working system.

    Be aware that now with this syetm, you wont be able to mv the crunched static exec's to /bin or /sbin . With the old system, you didnt have to shut down to do a libc recovery. Now you do.

    Baaaad.

    --
    1. Re:Bad if not included with non-dyn executables by JohnFluxx · · Score: 2, Insightful

      why not? /rescue/bin/mv /rescue/bin* /bin

      etc.

    2. Re:Bad if not included with non-dyn executables by polymath69 · · Score: 1
      Did you read the summary? Static executables will be available in /recover (that's a new one on me.)

      But I don't understand this myself. /sbin means "static /bin" and it's that way for recovery purposes. Why make it dynamic and add a new directory to be the new statically linked recovery directory? This seems on the surface to just push the original problem back a step. Is the next new thing to be /srecover?

      --

      --
      I don't want to rule the world... I just want to be in charge of mayonnaise.
    3. Re:Bad if not included with non-dyn executables by __past__ · · Score: 4, Informative
      But I don't understand this myself. /sbin means "static /bin" and it's that way for recovery purposes.
      Wrong. man 7 hier:
      /sbin/
      system programs and administration utilities fundamental to both single-user and multi-user environments
      Think "system administration", not "static". After all, /bin was static too, so the distinction doesn't much sense (and I think /sbin has been part of Unix earlier than dynamic linking anyway)
    4. Re:Bad if not included with non-dyn executables by Anonymous Coward · · Score: 1, Interesting

      Sir, several Linux distributions (such as Slackware) upgrade their libc's using dynamic /bin and /sbin, and without statically-linked package management tools (upgradepkg/makepkg/explodepkg/installpkg/removepk g). I would think it would be possible with FreeBSD as well.

    5. Re:Bad if not included with non-dyn executables by 0x0d0a · · Score: 1

      It's pretty easy to have a couple of static tools on the system to fix things if an upgrade fails.

      I believe RH packages sash and busybox. They don't have any problem, and once, long ago, after borking up ld.so (worse than just libc), I had to use 'em to recover.

      How often do you have a libc upgrade fail, though, in this day of package managers and tested distributions?

      Another poster asked what the point of this is -- its memory savings.

    6. Re:Bad if not included with non-dyn executables by Brandybuck · · Score: 3, Funny

      How often do you have a libc upgrade fail, though

      In FreeBSD, never. In Linuxland, everytime GNU bumps the minor version of glibc. To be fair, the upgrade works fine. It's just that everything else on the system breaks.

      --
      Don't blame me, I didn't vote for either of them!
    7. Re:Bad if not included with non-dyn executables by Anonymous Coward · · Score: 1, Insightful

      Fortunately, Linux is not prone to this exploitation. After reading the 10th SCO article of the day, the BSD administrator nodded knowingly and walked silently into the night....

    8. Re:Bad if not included with non-dyn executables by thogard · · Score: 1

      I don't know when that came about but the early AT&T systems with dynamic linking used /sbin to mean static-bin. I think the s becaome overloaded early on and if your mounting a file system and have /bin/mount and /sbin/mount which one should the init script use? I think thats where the "s is for system" came into meaning.

  2. Cool by __past__ · · Score: 4, Insightful
    Even with having statically linked versions of the tools needed for system repair in /rescue, the whole stuff still takes less place then before. So you really get the best of both worlds: The base system is smaller, more flexible and potentially even faster (has anybody measured it yet?), but static binaries are still around when you hosed you linker or libs.

    I'm still somewhat surprised that this got committed now, shouldn't 5.2 be released Really Soon Now? This looks like something that ought to be tested in -CURRENT for a good while.

    1. Re:Cool by rtaylor · · Score: 4, Informative

      Indeed.. According to the schedule the tree is to be frozen today.

      --
      Rod Taylor
    2. Re:Cool by metamatic · · Score: 1

      If (dynamic linked /bin and /sbin) + (statically linked /rescue) is smaller than the old statically linked /bin and /sbin, why not move /rescue into /bin and /sbin and reduce size even further?

      --
      GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
    3. Re:Cool by __past__ · · Score: 4, Informative

      Because size is not the only reason for the dynamic /bin. One big reason was better support for dynamically loadable PAM/nsswitch modules in the base utilities - so there is a good reason to have multiple versions of some binaries, one that always works and one that can do more fancy stuff.

    4. Re:Cool by shlong · · Score: 4, Informative

      I'm still somewhat surprised that this got committed now, shouldn't 5.2 be released Really Soon Now? This looks like something that ought to be tested in -CURRENT for a good while.

      This is of course a very valid concern. It should be noted, however, that this was the last phase in the overall change. The 'heavy lifting' parts were done months ago (creating /rescue, moving libraries to /lib and /libexec). It also had been extensively tested by many developers, and was already an optional switch when building world.
      In the end, we decided that a dynamic root filesystem was the future of 5.x, and that the 5.3 cycle was already overbooked with significant changes. We still have nearly a month before the release date of 5.2 to iron out any bugs. So after some final testing for a few days last week, I asked Gordon to Throw The Switch.

      --
      Cat, the other, tastier white meat.
    5. Re:Cool by realdpk · · Score: 1

      Static binaries, in cases where you want to run them somewhat rapidly (>100/s), are much, much faster than dynamic binaries.

    6. Re:Cool by gnu-sucks · · Score: 5, Insightful

      if anything, there should be a /static. You see, 99% of all users will not see ANY improvement in this new scheme. It would be better to 'tack on' this idea as an option to the 2% user base that would actually use this to their advantage.

      Do I really want to set my default shell to /rescue/sh and set all my important tools similarly? Heck no.

      I want those users using PAM and whatnot to specify /static for THEIR SPECIFIC needs, and let the rest of the FreeBSD users maintain /bin and /sbin as they are, and should be.

      This is sort of like an isp optomizing all connections for SNMP, because two users said it would be a good idea for what they do all day. And the isp tells all the other users, "well, we have an alternative dialup connection you can use, though its only at 9600, for better web browsing"

      So sure, you can either 1) recompile FreeBSD, or 2) suffer the slings and arrows of /rescue.

      Or, FreeBSD could just NOT BE STUPID OUT OF THE BOX.

    7. Re:Cool by gnu-sucks · · Score: 1

      When was the last time you thought to yourself, "hmm, I think ls is running too slow.."?

      Sure, for the 1% of FreeBSD users that actually thought that at one time, this is great. But wouldn't it make more sence for those users to just recompile their specific binaries, instead of forcing nearly all sane FreeBSD users to recompile their ENTIRE /bin and /sbin?

    8. Re:Cool by gnu-sucks · · Score: 1

      erm... just realized what you were saying. ignore my parent post :P

    9. Re:Cool by Professor+Bluebird · · Score: 1

      Also, in theory, it makes upgrading easier, especiallly if there is a flaw or bug fix or some other _minor_ change in the library. Just drop in the library, and no need to recompile everything. In practice though, this does occasionally break stuff. (*cough glibc*)

  3. Re:Ant this news is ... by torpor · · Score: 5, Insightful

    Of course it is.

    How the /bin && /sbin binaries are treated in a release as significant as FreeBSD, is definitely news for nerds, stuff that matters.

    The static nature of /sbin has been a significant issue in Unix-user/-admin land for many, many years. This change now sets in place a new scheme for administrators to understand and work around - with the new linking flag, admins can set up systems in new and interesting ways.

    Any FreeBSD admin who now doesn't understand the reasoning and potential problems of this new linking scheme will *definitely* need to learn it and understand it if they want to continue doing a good admin job in the future.

    One good way to learn about this and use it is to read the comments from slashdot readers about this issue ... and thus, this *IS* important news for slashdot.

    Just because SCO/MICROSOFT/LINUX isn't in the article, doesn't mean its not important ...

    --
    ; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
  4. Re:Ant this news is ... by Jerf · · Score: 1, Insightful

    You do realize this was posted in the BSD section of Slashdot, right?

  5. Re:Ant this news is ... by flex941 · · Score: 1

    So why they did it now not centuries ago? I mean dynamic linking has been known widely used concept for a while now.

  6. Re:Ant this news is ... by torpor · · Score: 1

    Because, if you read the article, the script to make this possible in the FreeBSD build process has been recently introduced, and is a good tool to do this with.

    It doesn't matter if 'dynamic linking' is old technology or not - the fact that this change has been made (and, it appears, generally accepted) is news.

    --
    ; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
  7. Re:Ant this news is ... by IM6100 · · Score: 1

    It might even serve as a 'news item' that is a clear warning to some FreeBSD users who don't follow all the newsgroups and discussion threads, but who would view this as a negative thing for FreeBSD when made aware of it. For which reason it's an excellent topic heading/article for Slashdot to cover.

    It's always seemed to me like FreeBSD is the 'compromise for Performance' BSD version. They make no pretense of being widely cross-platform, and do things like this, which break basic 'rules' of the UNIX system, for tweaky/performance reasons. The thing this sort of thing reminds me of is when Microsoft pulled the graphical subsystem into the Windows NT kernal layer 'to enhance performance.'

    --
    A Good Intro to NetBS
  8. Re:what a dumb idea by Goo.cc · · Score: 1

    I don't agree with you. This will make patching and updating the system far easier than before and it will reduce disk space requirements, which should always be a goal no matter how big hard drives get. My only concern would be in regards to a performnace hit.

    And yes, you do sound like a troll.

  9. Security advantage by ebcdic · · Score: 4, Insightful

    One of the main advantages of this is that you can replace libraries with security loopholes without having to rebuild everything. There have been several cases of bugs in the standard libraries that have required the statically compiled programs in /bin to be rebuilt.

    1. Re:Security advantage by cpghost · · Score: 1

      That is a very good point! I know many people who had been bitten by this a few times in the past. With fast machines, making the world every now and then is not such an obstacle as it used to be some years ago (security by recompiling). But sometimes, especially on colocated machines, applying a libc fix can still be a life-saver!

      --
      cpghost at Cordula's Web.
    2. Re:Security advantage by cperciva · · Score: 2, Informative

      This isn't really a big deal -- if you don't want to rebuild things yourself, you can just use FreeBSD Update.

    3. Re:Security advantage by neuroscr · · Score: 2, Interesting

      and introduce them just as fast. Now you can hack all the binaries in one go wtihout a recompile...

    4. Re:Security advantage by drinkypoo · · Score: 1

      In my ever so humble opinion, what's in /bin and /sbin should be static, because it should be things required to boot, and those should have as few dependencies as possible. Everything a normal user should use should be in /usr, and that includes administrative tools and daemons.

      There's no reason not to have dynamic versions of everything that's in /bin in /usr/bin, and restrict /bin only to root. If you can't mount /usr you probably can't go multiuser and other users can't log in anyway.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  10. Purpose of / by martycombs · · Score: 3, Interesting

    I thought *nix was designed as:

    / - minimum required to boot and repair system

    /usr - other binaries included on default distro

    /usr/local or /opt - packages added above and beyond the distro

    If your system ever became damaged, you booted to / and fixed it. If / is too large, then audit what's in there and make sure it contains the bare minimum required.

    Adding /rescue is unnecessarily cluttering up the system.

    1. Re:Purpose of / by Brandybuck · · Score: 4, Informative

      / - minimum required to boot and repair system

      It still is. /rescue is for repairing the system. /bin and /sbin are a comfortable but minimal set needed for booting.

      While there may be stuff in /bin and /sbin you might never use, all of it is used by someone or another at boot, single or rescue time.

      --
      Don't blame me, I didn't vote for either of them!
  11. Re:Ant this news is ... by CmdrTHAC0 · · Score: 1

    Is the Original Unix Way the One True Way? Let us go back to before dynamic linkers ever existed. Let us give up SMP. Let us return to our roots, as technologically advanced as Windows 95.

    To stop evolving is to die. They knew having a dynamically linked /bin and /sbin could be harmful, so they put static (but crunched) binaries in /rescue. Stick ANY Linux distribution in and see if they do the same. No, Linux just screws you over if you hose ld-linux or libc. (Unless you've had the foresight to install sash.)

    Rather than camping out in the past and watching the world race on ahead of them, FreeBSD is providing flexibility (at a small cost to performance, in this case) while taking care to avoid erasing past wisdom. I can't see how that compares to putting GDI into the Windows kernel.

    --
    __CmdrTHAC0__
    In Soviet Russia, Spanish Inquisition doesn't expect YOU!!
  12. Re:what a dumb idea by CmdrTHAC0 · · Score: 1

    Yeah, you sure do sound like a troll, with such an adversarial tone. Also by claiming to know what 99% of users want. (Don't forget, 87% of statistics are made up on the spot.)

    If I were to say anything about 99% of users, it's that they'll never hose the system to the point where they'll need to care.

    And incidentally, for those upgrading by CVSup from an earlier release, they'll be recompiling the system anyway.

    Finally, would you mind justifying exactly why this is so bad, for us poor, unenlightened masses?

    --
    __CmdrTHAC0__
    In Soviet Russia, Spanish Inquisition doesn't expect YOU!!
  13. Re:what a dumb idea by acidtripp101 · · Score: 1

    You have obviously never installed a freeBSD system.
    The FIRST thing that I do after installing from a release is to cvsup the latest sources.
    This requires me to rebuild the world.
    From my experience with freebsd, well over 75% of admins do this. And the other (less than) 25% probably don't even patch, so they won't care anyway.
    SO, if you don't want it, put the option in right before you update.

    --
    Not Free(as in beer). Free(as in "I'm free to beat you over the head for being a dumbass")
  14. Re:Ant this news is ... by 0x0d0a · · Score: 1

    Let me guess -- you don't understand why the article is important, therefore you feel the need to insult the article poster.

  15. Re:The Editor war is over : Vim won! by fsmunoz · · Score: 3, Funny

    ...With less and less distrobutions shipping emacs...

    M-x ispell-buffer

  16. Amiga by jafuser · · Score: 1

    Didn't the Amiga Operating System do this nearly two decades ago?

    If not, how does this differ?

    --
    Please consider making an automatic monthly recurring donation to the EFF
    1. Re:Amiga by Brandybuck · · Score: 2, Informative

      If not, how does this differ?

      The difference is that the Amiga is dead, while FreeBSD is alive, well and kicking. FreeBSD could have done this way back in 1.0, but there was no pressing need to. In the immortal words of some anonymous software designer: "don't prematurely optimize."

      --
      Don't blame me, I didn't vote for either of them!
    2. Re:Amiga by cant_get_a_good_nick · · Score: 3, Informative

      You don't understand the post. The point is not that FreeBSD is going boldly into the 80s with dynamic linking, it's that the default world now has binaries living in /bin and /sbin as dynamic instead of static. For a long time they were all static linked because of recovery issues. Now they are dynamically linked (with an option to be static if you're paranoid). They probably link against only stuff in /lib which will be on the root partition. Anything in /usr/lib would be dangerous if /usr was toasted - your shell would go bye bye.

    3. Re:Amiga by mdwh2 · · Score: 1
    4. Re:Amiga by kigrwik · · Score: 1

      In the immortal words of some anonymous software designer: "don't prematurely optimize."

      Well, I've always heard "Premature optimization is the root of all evil" attributed to Knuth, so it's not *that* anonymous ! :)

      --
      -- don't discount flying pigs until you have good air defense
  17. Isn't /sbin for static binaries? by kriston · · Score: 2, Interesting

    I thought one of the reasons we have /sbin is so that we can run the binaries there without having dynamic libraries involved.

    Reasons being:

    1) Size: Running in single-user mode or small kernels that don't use dynamically-linked libraries.

    2) Security: No risk of library-path-based security exploits.

    Am I missing something here? Why isn't /sbin statically-linked, anyway?

    Kris

    --

    Kriston

  18. Re:Help me :) by Anonymous Coward · · Score: 1, Insightful

    How would you go about doing all this yourself in some Linux distribution?

    Just recompile all the system tools with dynamic linkage and you're done?

    Anyway, my root partition is only 20 Mb, and that's including root's home(not that there is anything there now...). Do I really have to?


    How do you get this into Linux? Why, do what they've been doing for years -- copy the BSD source, remove the copyright, and call it Linux.

  19. ummm... by shaitand · · Score: 2, Interesting

    okay, maybe it's just me, and maybe I'm wrong. But I was under the impression that /bin /sbin's primary reason for existance was the same hole this /rescue directory will be filling? And how does it use LESS space (as if space were an issue anymore compared to speed in which case static is USUALLY, not always, better anyway), to simply move the static versions of the binaries in /bin and /sbin to /rescue and add dynamic versions to /bin and /sbin.

    Since /rescue now becomes as fundementally critical as /bin and /sbin have been before it certainly counts as part of the base system. If you move the static binaries, and add something, isn't that BIGGER than just the static binaries?

  20. Re:Ant this news is ... by bovinewasteproduct · · Score: 1

    They make no pretense of being widely cross-platform, and do things like this, which break basic 'rules' of the UNIX system, for tweaky/performance reasons.

    Huh???

    No, I would say that FreeBSD is the 'Stable over features'BSD version. The interface has not changed, your shell scripts, mounts, etc, etc will work just fine.

    BTW, this was done for anything but performance. If anything, this will hurt performance of those binaries (but then again, how often do you really really use those binaries anyway???). It was done to allow more functionality(sp) for the system.

    BWP

  21. Re:This article is from 1998 by uncoveror · · Score: 1

    BSD isn't dead. The next Windows will be based on it. Read more.

    --
    The Uncoveror: It's the real news.
  22. Did you actually try it? by siobHan · · Score: 4, Insightful

    I wonder if anyone screaming about what a HORRIBLE idea this is and how it will cause cancer in kittens, has actually tried it? Obviously the FreeBSD developers are not dumb, and root will always be able to get on to use /rescue, and the advantages are a lot more sophisticated than just saving some space on the root partition.

    It's pretty knee-jerk to scream about it if you don't actually know how it's been implemented.

    J

  23. Doesn't work w/ 5.x, non-Intel by Xenophon+Fenderson, · · Score: 1

    "FreeBSD update" doesn't work for those of us tracking 5.x-RELEASE, and it only provides binary updates for the IA32 (i386-family) port.

    --
    I'm proud of my Northern Tibetian Heritage
    1. Re:Doesn't work w/ 5.x, non-Intel by cperciva · · Score: 1

      FreeBSD Update doesn't care what you're running. But you're mostly right -- nobody is publishing updates for anything other than i386 4.(7|8|9)-RELEASE.

      In the near future (next week, maybe the week after) I'll start publishing updates for i386 5.x as well; non-i386 platforms... well, I haven't heard from anyone interested in using FreeBSD Update on those, so that will probably wait until FreeBSD Update becomes part of the base system (around 5.5-RELEASE, maybe?) and the Project takes on the task of building the updates.

    2. Re:Doesn't work w/ 5.x, non-Intel by Xenophon+Fenderson, · · Score: 1

      What does it take to create binary updates? I'd be more than willing to build binary updates for FreeBSD/alpha 5.1-RELEASE on a soon-to-be-mine AlphaServer 2100.

      --
      I'm proud of my Northern Tibetian Heritage
    3. Re:Doesn't work w/ 5.x, non-Intel by cperciva · · Score: 1

      Building updates requires a box which doesn't object to having its clock shifted forwards by 400 days from time to time. Having people use the updates you build requires that people trust you.

      When I get the code written -- which is probably going to be about a year from now -- that last requirement will be dropped, since it will be possible for several buildboxes to cross-verify and cross-sign. (This is the primary reason I'm keeping FreeBSD Update out of the base system for now.)

      But feel free to download the build code and try it out -- I haven't tested it on non-i386 systems, but it should work (and if it doesn't, I'd be very happy to hear about it).

  24. Speed by dcs · · Score: 2, Informative

    Actually, no. Dynamically linked binaries are slower than statically linked ones.

    --
    (8-DCS)
    1. Re:Speed by dcs · · Score: 1

      No, they are ALWAYS slower. It might not be _relevant_ if the time it takes to setup the process in memory is insignificant compared to how long the application runs, but the setup time will ALWAYS be slower for dynamically linked applications, and there is NO factor that makes them faster than statically linked ones.

      The only hypothetical scenario where a system with all-dynamically linked apps would perform better than the former cases is one where lots of _different_ statically compiled apps run for a long time, and the memory available is not enough to run them all without trashing. But, first, that's a problem of lack of memory, not a problem of speed of execution; and two, we are talking about /bin and /sbin, which do not fit this scenario.

      --
      (8-DCS)
  25. Re:what a dumb idea by stripes · · Score: 1
    I don't agree with you. This will make patching and updating the system far easier than before and it will reduce disk space requirements, which should always be a goal no matter how big hard drives get. My only concern would be in regards to a performnace hit.

    I agree that patching will be easier, unless you patch a library bug that shows up in the new /recovery tools. I'm not so sure about making the overall disk space requirement smaller, if the /recover tools are even close to the full set of stuff that is now dynamically linked the total disk use is higher. The only saving grace there is you might not put /recover on the same disk partition as /. Not really a win.

    So patching is in some cases better, total disk space usage is probably up, and performance is probably down a little bit (unmemorable unless you are looking for it). Not in my opinion a good change, but not bad enough to send me scurrying off to AnotherBSD, not by a long shot. In the grand scheme of things it is a tiny change.




    The last paper I read on dynamic linking showed about a 2% penalty on RISC CPUs and a bit over 10% on CISCs (losing one of 16+ registers to hold a libs base pointer hurts less then losing one of "about 4 to 8" registers!). That was a long time ago though, around when Sun added them to SunOS, maybe the early 90s or late 80s. Very short running programs (ls and echo) were hurt a lot more. Oddly enough some very small programs (like cat and echo) were larger as dynamic linked code (more disk space used to record the static code to invoke the dynamic linker, and to hold the symbols to link).

    It mattered to me more then too. I didn't have SunOS source so any lib bug I wanted to fix, or lib call I wanted to improve wasn't gonna happen for the static programs.

  26. Re:what a dumb idea by tigga · · Score: 1
    You have obviously never installed a freeBSD system. The FIRST thing that I do after installing from a release is to cvsup the latest sources. This requires me to rebuild the world. From my experience with freebsd, well over 75% of admins do this. And the other (less than) 25% probably don't even patch, so they won't care anyway.

    Why do you do it?
    Just to have something new and shiny in your hands?

    In case of production system one have to use RELEASE, as it was thorough tested. The need for upgrade arise in case of security issues or performance improvements. And then one have to recompile only parts affected.

    So far I do not know anybody who rebuild the world for production system.

    In case one tracking CURRENT and it's definitely not a production - it may be some support system, though, it's just easier to cvsup and make world. And just revert back in case something went wrong - it happens...

  27. Re:what a dumb idea by gnu-sucks · · Score: 1

    YOU obviously haven't seen my server racks. And, you don't know who I am either. I have been using unix longer than you've been alive, likely, and I've certainly installed FreeBSD a few times.

    Anyway, as others have stated, in a production enviroment, its incredably stupid to run with the bleeding edge.

    When you have several thousand customers accessing your servers, you can't exactly turn them off, recompile the OS, and reboot.

    Sure, you could build extra redundant servers, grab new sources on off-peak hours... but by then you could have just used another system instead.

    And if you seriously think its a good idea to only grab new sources once (upon install), then well, thats just silly!

  28. Re:Ant this news is ... by neuroxmurf · · Score: 1

    > but then again, how often do you really really use those binaries anyway???

    You're right. I never use /bin/ls! Surely no one could ever care about its performance!