Was I the only one who read that as PHP ass framework?
No, a lot of people read it like that, and this word play was deliberate - it actually helps the marketing by attracting extra blog posts ("hey, I found something stupidly named" and the like).;-) However, the official spelling is "phpass", with no caps, where "pass" is for password or pass (successful authentication).
I previously made a similar spelling joke in naming popa3d, a POP3 server. Russians get this one.
As to the bug, as other people noted the typical alternatives to not using phpass and crypt_blowfish would have been far worse. This is not an excuse. I do feel embarrassed for the bug. But I am also being realistic such that I and others learn the right lessons from this.
In practice, most uses of phpass that I am aware of don't actually use crypt_blowfish (and thus are not affected) - those choose to use phpass' "portable hashes" instead. For passwords without 8-bit chars, those are weaker than crypt_blowfish, but they do avoid the bug. (And with 8-bit chars it is not obvious which are weaker, even despite of the bug.)
(Apparently, I forgot to submit this comment the first time around. To avoid re-typing, I ptrace(2)-dumped my Firefox process memory to a file, then grepped it for "deliberate" - and here the comment is. I am used to doing things like that, e.g. when investigating abusive processes on compromised shared hosting accounts of customers. Maybe someone will find this tip useful or curious.)
It seems like Ulrich Drepper was right opposing, in rather harsh words, my suggestions to include bcrypt in glibc. My bad.
I also briefly thought of where we would be if Ulrich accepted bcrypt into glibc. I have several points to make:
1. It is likely that adjusting the crypt_blowfish code to glibc conventions would happen to remove the bug - just like it happened with Perl's Crypt::Eksblowfish (it's based on crypt_blowfish, but the bug is gone). Yes, this does mean that those coding conventions are maybe superior, although it is easier for them to be such when only a more limited number of platforms is considered. There is a lesson for me to learn here.
2. bcrypt is not only crypt_blowfish. glibc could also use OpenBSD's code (lacking this bug) if it looked more suitable by whatever criteria.
3. I just took another look at Ulrich's SHA-crypt.txt, sha256crypt.c, and sha512crypt.c. I don't see any 8-bit characters in passwords in the test vectors. Unless I have missed those, it looks like a bug in SHA-crypt causing similar misbehavior would go unnoticed. No, I do not find it likely that such a bug exists there, but then I also did not find it likely for crypt_blowfish. Anyone wants to test and confirm that there's no 8-bitness bug in SHA-crypt? Please do. But what to test Ulrich's SHA-crypt against? Does Solaris use the same code or a reimplementation? (I don't know.) If it's the same code, and no reimplementation exists, then you'd have to try causing collisions or something like that, perhaps with low "rounds" (to make this test reasonably quick). Or create that reimplementation for testing. (BTW, I did the latter in phpass, for testing the correctness of my implementation of its "portable hashes".)
4. SHA-crypt is reasonably good, especially for acceptance due to its use of NIST-approved SHA-2 family functions. However, it does have its drawbacks. bcrypt turned out to be GPU-unfriendly, whereas we should see reasonably efficient implementations of SHA-crypt for GPUs soon (this is being worked on and I see no major obstacles). In neither case a GPU is usable for password hashing on an authentication server (there's too little parallelism in one instance of a bcrypt or SHA-crypt hash computation), even if you had a GPU there, so GPU-unfriendliness is an advantage of bcrypt if you compare it against SHA-crypt.
5. Finally, there have been plenty of security bugs in glibc.
It does seem odd that people haven't run fuzzed data against a number of different implementations of blowfish and not noticed differing output. I'd have thought that would be a fairly normal thing for someone developing a crypto algorithm implementation to do.
Of course. But there are reasons why this didn't happen to a sufficient extent in this case. None of these reasons are valid excuses. There are lessons to learn from this.
There were test vectors, and the implementation behaved exactly like OpenBSD's on those. The code was in John the Ripper, and it was correctly cracking password hashes from OpenBSD. So it felt like the code was working just as it should have, because it actually did (on the tests that were thrown at it). There was no expectation that specifically 8-bit characters would cause any difference.
As to fuzzing vs. OpenBSD's implementation, I wish I did that, and I wish I included 8-bit characters. I felt that JtR essentially was an equivalent of this test, due to people using it to crack password hashes from OpenBSD. The code existed in JtR for 2-3 years before I made it available separately for reuse. Apparently, 8-bit characters in weak passwords (and in wordlists) were just too uncommon for anyone to notice any passwords not getting cracked where they should have been.
ping/traceroute only need to be able to open a raw socket etc.
Olaf Kirch's implementation of traceroute, which we use in Owl, does not need a raw socket. It uses Linux 2.4+ features (that is, they've been available in mainstream kernels for a decade) to do exactly what the usual LBNL traceroute did (send UDP packets, detect ICMP errors), only without requiring special privileges. We install it mode 755 and it just works for all users.
I fully agree with you that "anyone can be wrong".
I also agree with your comments on a large percentage of CPU time typically being spent in shared libraries, and I agree that they're normally PIC anyway. So, yes, 6% slowdown on main program code when going PIE does not translate to a 6% slowdown of the entire system. We typically use Owl rebuilds from source as our benchmark, so this is likely what we will use to see the overall effect of going PIE as well (that is, run a second rebuild on a system already rebuilt to use PIE, including gcc, etc.)
Our bzip2 does not use shared libbz2, specifically for the reason you mention. Here's the exact text of the Owl bzip2.spec changelog entry dated Feb 1, 2002: "Package the bzip2 binary that is statically-linked against libbz2 for better performance on register-starved architectures such as the x86." IIRC, we actually ran benchmarks of both kinds of builds of bzip2 at the time. I've just double-checked - the bzip2 binary in Owl 3.0 still uses shared libc only, whereas libbz2 is linked in statically (the version of bzip2 has been updated several times since 2002, of course).:-)
It was nice talking to you. I think you could actually contribute to post-3.0 Owl development if you wanted to (hint).;-)
How "passwd" works with root group r-- and no SUID is a mystery to me, I'll have to look later.
Please take a look at our presentation slides, it only takes a few minutes. Then you might have more specific questions on the implementation, which I'd be happy to answer.
Please note that what I wrote in my reply to your comment was quite different. I never said anything about your experience. Now that you raised this topic, I can say that you appear to be familiar with Unix security. You simply had not looked at our stuff before you wrote your comment, that's all.
I find your comment re: the cost of PIE interesting, thanks! It leaves many questions, though. 6% on 32-bit x86 is the number I had been aware of before. Your statement that "the affected code was actually running less than 2% of the time" is curious and potentially very useful. However, this, assuming that it's true, does not yield your claimed 0.0012%; it yields 0.12%. Of course, 0.12% may be considered acceptable (far more likely than 6% at least). I am curious about the details of your test; I guess, some systems will actually run "the affected code" 100% of the time - e.g., this happens if I build John the Ripper as PIE and run it for days (which is why I dislike Gentoo building JtR like that, and have to recommend JtR users to make their own builds).;-) I'd appreciate it if you e-mail me (solar at openwall.com) more details on your test, and we'll discuss from there. We're considering PIE by default for post-3.0 Owl-current (and next release).
Your comments about cron/at are similar to what we thought of when we designed the system. Just like our per-user shadow files, the crontab files are per-user (and they were prior to our changes as well). Our crond does check crontab file ownership, and then it will only run the cron jobs found in the file as the file's owner. Cron and at jobs running as root are supported - of course, as long as the corresponding files are root-owned.
Actually, there are at least two things I've been planning to add to the website (but we never got around to doing that): an interface to browse our man pages and another one to browse our packages (list and contents - at least the metadata). I don't know whether this is a higher priority than a mere facelift or not.
Re: immutable/append-only flags -- you are obviously wrong, chattr -i and chattr -a are your friends to remove these flags in a normal multi-user runlevel, indeed, you need to be root to do it, though.
Actually, it is possible to configure a Linux system such that "chattr -a" won't work without having to reboot first. With ancient kernels, we had BSD-style securelevel for this. With recent ones, we have the capability bounding set instead (/proc/sys/kernel/cap-bound). But very few sysadmins make use of this functionality, and it is tricky and painful to use it in a way that would achieve much (need to make almost the entire "base system" immutable - the individual files - or watch for any unexpected reboots and consider those signs of a potential intrusion, but that just wastes time). I tried this with securelevel on Linux 2.0 kernels in late 1990s, but I am not doing this anymore.
Khopesh - you're welcome, and thank you for your constructive comments. We might create a FAQ for Owl based on the questions/comments we've received.
Agreed re: insufficient documentation on owl-control. This is something for us to improve. Also add our own web interface to our man pages, like *BSD's have - frankly, this has been on my to-do list for years... but there was always something more important or/and more urgent.
I have replied to your comment's GGP with the info you have requested.
(ALT Linux have imported owl-control for their distributions, and they have contributed some changes back to us. They also happen to have placed this man page on the web.)
I'm afraid that this is not going to help you very much, though, because owl-control is very generic and abstract, and so is its man page. Perhaps we should add a few usage examples to illustrate owl-control's purpose.
I'll try to explain what it's for. Some packages on Owl provide what we call, in owl-control terms, "facilities". An example of such facility is su, provided by the SimplePAMApps package. The package also provides the file/etc/control.d/facilities/su, which defines several possible settings for su. In this case, the settings are: public, wheel, wheelonly, restricted - as shown by "control su list". The default for su on Owl 3.0 is restricted, which corresponds to/bin/su being mode 700 (usable by root only). If you issue "control su wheelonly",/bin/su changes to mode 4710, with group wheel. Issuing "control su" then confirms that the current setting is indeed wheelonly. Other related programs are control-dump and control-restore: these are used by scriptlets of our RPM packages. For example, our SimplePAMApps.spec invokes "control-dump su" in %pre and "control-restore su" in %post. This lets a possibly customized control setting for su persist over upgrades of the SimplePAMApps package.
Some other facilities are: bind-debug (controls whether it's possible to have BIND produce debugging logs or not), ntpd (can be set to "server" or "client"), postfix (can be set to "server" or "local"), and many others. So this is not limited to changing permissions on files - it also works for configuration file contents - but we only use it for trivial changes (a line commented-out or not) that need to persist over system upgrades. Also, such control'able changes are perfectly compatible with manual edits to the corresponding config files. If a sysadmin ends up editing a file such that control's facility script does not recognize one of the supported settings, control will start reporting "unknown" for that facility's state - but that is not any worse than not having control would have been.
As you can see, this is a very nice security-relevant feature for sysadmins, but it is not central to the Owl security model. Things such as our tcb suite (making the passwd command run without requiring root privs) are more essential to our security model.
If you can't su or sudo, how you get anything done?
We normally login directly as a root user for sysadmin tasks (e.g., r_john for a sysadmin named John) and also directly as a non-root user (e.g., john) for other tasks. This applies to both console and remote logins (ssh). This is the approach we've been using for years, and it works well for us.
As I have mentioned, those who prefer the traditional su approach, despite of its added risks (including compromise propagation from a sysadmin's non-root account to root), may "control su wheelonly".
I'm interested in this for a VMware guest OS, as a possible alternative to m0n0wall. Have the authors thought about that kind of implementation
Yes, Owl 3.0 works in VMware out of the box. We mostly run it in QEMU and VirtualBox for our VM-based testing, though.
BTW, you might find it curious that when you run Owl in a VM like this, you can further create OpenVZ containers with multiple instances of Owl and with other Linux distros inside that single running copy of Owl. Such container-based virtualization has no further performance overhead.:-)
How much RAM does it need to run adequately?
128 MB is plenty, probably a lot less will do. I have my QEMU set to its default of 128 MB when I do install-tests with new Owl ISOs. Also, my Owl-based router and small fileserver at home has 128 MB RAM (runs the default-enabled set of Owl services plus ntpd, named, pppd/pppoe, openvpn, squid, NFS, and a sometimes a few lftp's and ctorrent's under screen).
On the other hand, of course you may need a lot more RAM for your specific uses of Owl - e.g., to run many OpenVZ containers, to do web hosting, etc. We currently administer several 32 GB RAM machines running Owl and one 64 GB RAM machine, as well as a countless number of 4-16 GB machines - so we know Owl has no problem with these larger RAM sizes as well (and with the many OpenVZ containers running various Linux distros actually making use of this RAM).
Openwall... secure? webbased interfaces can be very vulnerable.
There's no web-based interface in Owl.
I don't think Openwall supports grsec, containers, Cuda etc. ? I bet they still use deprecated software like Snort.
Are you just trolling? I'll provide a serious response nevertheless: we don't include the grsecurity patch, but our userland will run just fine with a grsecurity-patched kernel - this is up to our users to choose if they like. We do support containers out of the box, with OpenVZ. Owl can act both as an OpenVZ host (it includes the kernel and vz* tools) and guest (we provide pre-created OpenVZ container templates, and there's "make vztemplate" to build your own ones if you like). CUDA has nothing to do with a server/appliance Linux distro at this time (although I can think of some obscure uses). Since when is Snort deprecated? Anyway, we don't include it, but it can be easily installed on top of Owl.
The exploits named in the summary are mostly for locally connected users, which means academic environments.
Yes, local users and pseudo-users. No, this is not limited to academic environments. Think shared web hosting - hundreds or thousands of local users on a system. We're actually setting up Owl-based shared web hosting systems for our clients. Besides the different user accounts belonging to different shared hosting customers, privilege separation between the accounts is also needed because one of the websites may get compromised (via a web app vulnerability) and we would not want such a compromise to easily propagate onto all other websites hosted on the same system (not even if they're of the same customer).
Additionally, even on a dedicated server for a single website (or, say, on a mail server) there are pseudo-users. Even the sshd service from OpenSSH uses a pseudo-user for its built-in privilege separation. This is needed to mitigate the impact of vulnerabilities in certain parts of code in those services (and in libraries that they use). You don't want any vulnerability anywhere in sshd or in parts of libc/libcrypto/libz that it uses (and in a lot of other libraries on riskier Linux distros such as RHEL/CentOS/Fedora - just see "ldd/usr/sbin/sshd" on those and be very scared) to result in a full-blown root compromise. The extra barrier between the less-privileged sub-process (doing lots of things) and the parent root-privileged process (doing fewer things) is of some help. Ditto for other services that implement privilege separation - and this is the kind of service implementations that we include in Owl (OpenSSH, vsftpd, OpenNTPD, Postfix all have privsep implemented upstream; telnetd was missing privsep, but we introduced privsep into it).
some sort of permissions escalation daemon ("owl-control" perhaps?)
owl-control is merely a tool to ease system administration - change settings and have your changes persist over system upgrades. It is one of the nice security-related features of Owl, but it is NOT key to running a reasonable Owl system without SUIDs (this is achieved by other means - SGIDs and changes to various system components). owl-control is not what you thought it was (not a "permissions escalation daemon", but merely a set of scripts).
I do agree with you that we need to present our documentation (such as man pages) in a form more accessible to someone who is not an Owl user yet. The only reason why we did not do that yet is lack of time (small development team working on a mostly non-commercial project, so we constantly have to choose which of the pending tasks we make progress on).
Basically if you exploit something with 'shadow' (i.e. passwd) you add a root user account to/etc/passwd and su to it. if you exploit crontab or at, you add a crontab that adds a root level account or runs a command as root or creates a SUID program. It requires some hacker creativity, but doesn't make anything secure.
That's poor analysis on your part, so your conclusion is completely wrong. Please refer to our presentation slides for an explanation of how things actually work and why the attacks you describe would not work:
BTW, the announcement specifically mentioned that "a vulnerability in crontab(1) or at(1) can't result in a root compromise without a vulnerability in crond(8) or in a critical system component relied upon by crond(8)." Did you not read that? Or do you disagree, thereby stating that we're inexperienced in the stuff we've been doing for 10 years?
Since a lot of people are confused just like you are, I'd be happy for any suggestions on how we could explain what we do and what we have achieved better. I did include that quote in the announcement specifically because I knew of common misconceptions about our work, but apparently that was not enough? Thanks!
Speaking of ping in particular, I have to admit that in Owl 3.0 it is simply restricted to invocation by root by default (mode 700) - not great, but usually acceptable. It can be re-enabled with "control ping public", and this setting will persist over upgrades, but it re-introduces the risk. We're working towards a better solution - specifically, we're testing a Linux kernel patch implementing non-raw ICMP sockets, which we intend to submit to LKML soon. (It already works for us, including via our patched ping, but this stuff was too cutting-edge to include it in the release.)
traceroute just works on Owl out of the box, though - we're using an implementation that makes use of Linux-specific kernel features (upstream, no patches) such that it does not need any special privileges to run (it is installed mode 755 and it works for all users). (This is similar to what we're trying to implement for ping, but it's already a reality.)
The new system allows/bin/su to have permission to write to/etc/passwd, but not to do other root things like read/write to/root or mount filesystems not enumerated in/etc/fstab. It is "granular root".
You're talking of Fedora's approach with fscaps (and there are errors in your description). We don't do that. We're smarter than that! So your comment does not apply to Owl, at all. I've explained what we actually do in further comments. It is also shown in our presentation slides:
Yes, our distro doesn't encourage users to use su or sudo. The reason is that escalating privileges from a less privileged account to a more privileged account is bad from security standpoint.
Exactly. And our solution to the "accountability" problem when there's more than one sysadmin is multiple root accounts - we typically prefix their usernames with "r_" for clarity, and we keep the main "root" account locked. We even have our own msulogin program, replacement for sulogin, to allow for single user mode console logins under any one of multiple root accounts that might exist on the system. The rest of Linux tools happened to just work with multiple root accounts fine, with no changes needed on our part.
That said, if anyone truly wants to use the traditional absurd approach of su'ing to root, they can re-enable su with "control su wheelonly" (there are other possible settings as well). Of course, this makes/bin/su SUID root... but you asked for it. This setting will persist over Owl upgrades. See "man control", "control" (just run it with no options for a list of facilities and their possible settings).
Of all the Linux vulnerabilities in the past few years, how many would have been stopped by this?
There were in fact cases where other Linux distros had to issue updates and advisories - e.g., for an issue in crontab (to use the same example that I used for some other comments here) - whereas on Owl not only the vulnerability did not exist in the first place, but also it would have been mitigated due to the greatly reduced privileges of the crontab program (to use the same example again). To give another example, the recent glibc vulnerability with LD_AUDIT and $ORIGIN (discovered by Tavis Ormandy) not only did not affect Owl for a couple of reasons (including due to a glibc hardening patch), but also would be mitigated by not having a single SUID program if the vulnerability did exist on Owl (it did exist on almost all other distros). As you can see from this, we often have multiple layers of security hardening that help (with this being one of the layers) whereas others sometimes happen to have none - and are vulnerable.
At this point, our weakest link is the Linux kernel - large and monolithic - so that's pretty much the only place where critical vulnerabilities that do affect us are still being found... There's little we can do about this (while staying Linux), but it may be a focus area for further security hardening now that we have an almost perfect userland. Also, Owl differs from other/major distros as it relates to exposing the Linux kernel to attack - we expose less of it - e.g., we exclude kernel module auto-loading, we have saner/safer permissions on device files, and we have no SUIDs by default (which mitigates some classes of potential vulnerabilities in the Linux kernel's program startup code - such vulnerabilities were found/fixed in the past).
That was regarding "no SUIDs". As to the detection of log record spoofing, it does not stop attacks, but it is a security feature. When you review logs, you typically want to have a way to know whether the records are genuine, at least under the assumption that the system has not been compromised yet. On traditional Unix/Linux/BSD/..., you had to assume that any (pseudo-)user could have trivially produced any of the log records, pretending to be any system service or even the kernel.
> BTW Fedora 15 is also dropping SUID, so while Openwall is the only current distro. It's by no means the only one in development.
Right, however Fedora's approach is (1) completely different and a lot worse than ours (in my opinion, indeed) and (2) either won't result in complete removal of SUID programs or will leave many with root-equivalent capability sets. This means that they will continue to expose the dynamic linker, libc startup, and relevant parts of the Linux kernel to the usual risks associated with SUID root program startup - something that we avoid.
> Ubuntu is also removing SUID, but I don't know their timetable.
They're aware of the drawbacks of Fedora's approach, so they don't hurry to implement it. They're also aware of our proposed alternative.:-)
> Here's one of the better criticisms of dropping SUID, and it's from an Openwall developer. These criticisms are echoed by almost everyone thinking about removing SUID.
I am glad that you liked my criticism of Fedora's approach, however it appears that you misunderstood me. I criticized their specific approach with fs capabilities, not the idea of getting rid of SUIDs in general. The approach taken by us in Owl (many years ago, but only widely publicized now) and the one being taken by Fedora now are completely different (SGID and changes elsewhere in the system in Owl vs. fscaps in Fedora). The purpose of my criticism was to make other folks, including Fedora developers, consider the issues and the alternatives. It was not to discourage them from taking the move, but rather to give them an opportunity to consider our approach, which we consider to be the better and "real" one.
With our approach implemented in Owl, a used-to-be-SUID program runs SGID to a group dedicated to a certain purpose - e.g., editing a user's crontab. Other parts of the system have been modified such that privileges of that group are sufficient for but can't be expanded further than its intended purpose - e.g., permissions on/var/spool/cron and extra checks in crond (for crontab file ownership and more) are such that a possible compromise of group crontab would not give the attacker almost anything - no ability to edit another user's crontabs without also finding a (second) vulnerability in crond or in system components relied upon by crond. Yes, we could as well not use SGID - just make the crontab program fully unprivileged and the/var/spool/cron directory writable by anyone (the sticky bit prevents messing with another user's existing crontabs). The reasons why we chose to use SGID and a group are: (1) this is needed to prevent some DoS attacks (such as taking up another user's would-be crontab filename) and (2) it is an extra layer of security (so direct attacks on crond are not possible without compromising crontab first). Our changes to crontab/crond have since been adopted by ALT Linux, OpenBSD, and even by upstream ISC (Vixie) cron (but most distros somehow don't hurry to make use of this functionality, continuing to install crontab SUID root...)
Fedora's approach is to replace SUID root with fscaps - that is, the programs are still granted a lot of privilege, just somewhat less than they were with SUID root. In many cases (perhaps even in most, where ping might be the only exception), such capability sets are in fact root-equivalent, so this is sweeping SUIDs under the rug. Also, there's no second layer of security. See the difference?
> I wrote a program once that needed SUID for about a hundred asm instructions.
Sure the program may be tiny, but it exposes parts of the dynamic linker, libc startup, and even extra parts of the Linux kernel code for attack. All of these components have historically had vulnerabilities exploitable specifically via SUID programs. The point behind not having a single SUID program in a default install of Owl is primarily to mitigate potential vulnerabilities in those components. I do agree that, as the distro level, it would otherwise be possible to live with a few tiny SUID programs, but the risk with other exposed components is very real - so we avoid it.
> CAP would be wasted as chroot() is one of those known to grant root power.
Right. Red Hat are going to "waste" capabilities like this. We are not doing that (we use SGIDs and changes elsewhere in the system instead). (To be fair, there is a subtle reason why even root-equivalent capability sets may be slightly better than directly running as root when we consider some specific vulnerability classes. This is just too complicated to discuss in a comment like this.)
Slashdot Mirror
Was I the only one who read that as PHP ass framework?
No, a lot of people read it like that, and this word play was deliberate - it actually helps the marketing by attracting extra blog posts ("hey, I found something stupidly named" and the like). ;-) However, the official spelling is "phpass", with no caps, where "pass" is for password or pass (successful authentication).
I previously made a similar spelling joke in naming popa3d, a POP3 server. Russians get this one.
As to the bug, as other people noted the typical alternatives to not using phpass and crypt_blowfish would have been far worse. This is not an excuse. I do feel embarrassed for the bug. But I am also being realistic such that I and others learn the right lessons from this.
In practice, most uses of phpass that I am aware of don't actually use crypt_blowfish (and thus are not affected) - those choose to use phpass' "portable hashes" instead. For passwords without 8-bit chars, those are weaker than crypt_blowfish, but they do avoid the bug. (And with 8-bit chars it is not obvious which are weaker, even despite of the bug.)
(Apparently, I forgot to submit this comment the first time around. To avoid re-typing, I ptrace(2)-dumped my Firefox process memory to a file, then grepped it for "deliberate" - and here the comment is. I am used to doing things like that, e.g. when investigating abusive processes on compromised shared hosting accounts of customers. Maybe someone will find this tip useful or curious.)
It seems like Ulrich Drepper was right opposing, in rather harsh words, my suggestions to include bcrypt in glibc. My bad.
I also briefly thought of where we would be if Ulrich accepted bcrypt into glibc. I have several points to make:
1. It is likely that adjusting the crypt_blowfish code to glibc conventions would happen to remove the bug - just like it happened with Perl's Crypt::Eksblowfish (it's based on crypt_blowfish, but the bug is gone). Yes, this does mean that those coding conventions are maybe superior, although it is easier for them to be such when only a more limited number of platforms is considered. There is a lesson for me to learn here.
2. bcrypt is not only crypt_blowfish. glibc could also use OpenBSD's code (lacking this bug) if it looked more suitable by whatever criteria.
3. I just took another look at Ulrich's SHA-crypt.txt, sha256crypt.c, and sha512crypt.c. I don't see any 8-bit characters in passwords in the test vectors. Unless I have missed those, it looks like a bug in SHA-crypt causing similar misbehavior would go unnoticed. No, I do not find it likely that such a bug exists there, but then I also did not find it likely for crypt_blowfish. Anyone wants to test and confirm that there's no 8-bitness bug in SHA-crypt? Please do. But what to test Ulrich's SHA-crypt against? Does Solaris use the same code or a reimplementation? (I don't know.) If it's the same code, and no reimplementation exists, then you'd have to try causing collisions or something like that, perhaps with low "rounds" (to make this test reasonably quick). Or create that reimplementation for testing. (BTW, I did the latter in phpass, for testing the correctness of my implementation of its "portable hashes".)
4. SHA-crypt is reasonably good, especially for acceptance due to its use of NIST-approved SHA-2 family functions. However, it does have its drawbacks. bcrypt turned out to be GPU-unfriendly, whereas we should see reasonably efficient implementations of SHA-crypt for GPUs soon (this is being worked on and I see no major obstacles). In neither case a GPU is usable for password hashing on an authentication server (there's too little parallelism in one instance of a bcrypt or SHA-crypt hash computation), even if you had a GPU there, so GPU-unfriendliness is an advantage of bcrypt if you compare it against SHA-crypt.
5. Finally, there have been plenty of security bugs in glibc.
It does seem odd that people haven't run fuzzed data against a number of different implementations of blowfish and not noticed differing output. I'd have thought that would be a fairly normal thing for someone developing a crypto algorithm implementation to do.
Of course. But there are reasons why this didn't happen to a sufficient extent in this case. None of these reasons are valid excuses. There are lessons to learn from this.
There were test vectors, and the implementation behaved exactly like OpenBSD's on those. The code was in John the Ripper, and it was correctly cracking password hashes from OpenBSD. So it felt like the code was working just as it should have, because it actually did (on the tests that were thrown at it). There was no expectation that specifically 8-bit characters would cause any difference.
As to fuzzing vs. OpenBSD's implementation, I wish I did that, and I wish I included 8-bit characters. I felt that JtR essentially was an equivalent of this test, due to people using it to crack password hashes from OpenBSD. The code existed in JtR for 2-3 years before I made it available separately for reuse. Apparently, 8-bit characters in weak passwords (and in wordlists) were just too uncommon for anyone to notice any passwords not getting cracked where they should have been.
We're getting close to an official fix, which includes a quick self-test on every use (not just by "make check"), with both 7- and 8-bit chars: http://www.openwall.com/lists/oss-security/2011/06/21/2
ping/traceroute only need to be able to open a raw socket etc.
Olaf Kirch's implementation of traceroute, which we use in Owl, does not need a raw socket. It uses Linux 2.4+ features (that is, they've been available in mainstream kernels for a decade) to do exactly what the usual LBNL traceroute did (send UDP packets, detect ICMP errors), only without requiring special privileges. We install it mode 755 and it just works for all users.
I fully agree with you that "anyone can be wrong".
I also agree with your comments on a large percentage of CPU time typically being spent in shared libraries, and I agree that they're normally PIC anyway. So, yes, 6% slowdown on main program code when going PIE does not translate to a 6% slowdown of the entire system. We typically use Owl rebuilds from source as our benchmark, so this is likely what we will use to see the overall effect of going PIE as well (that is, run a second rebuild on a system already rebuilt to use PIE, including gcc, etc.)
Our bzip2 does not use shared libbz2, specifically for the reason you mention. Here's the exact text of the Owl bzip2.spec changelog entry dated Feb 1, 2002: "Package the bzip2 binary that is statically-linked against libbz2 for better performance on register-starved architectures such as the x86." IIRC, we actually ran benchmarks of both kinds of builds of bzip2 at the time. I've just double-checked - the bzip2 binary in Owl 3.0 still uses shared libc only, whereas libbz2 is linked in statically (the version of bzip2 has been updated several times since 2002, of course). :-)
It was nice talking to you. I think you could actually contribute to post-3.0 Owl development if you wanted to (hint). ;-)
How "passwd" works with root group r-- and no SUID is a mystery to me, I'll have to look later.
Please take a look at our presentation slides, it only takes a few minutes. Then you might have more specific questions on the implementation, which I'd be happy to answer.
http://www.openwall.com/tcb/
http://www.openwall.com/presentations/Owl/mgp00020.html
http://www.openwall.com/presentations/Owl/mgp00021.html
http://www.openwall.com/presentations/Owl/mgp00022.html
http://www.openwall.com/presentations/Owl/mgp00023.html
... he was more experienced than me ...
Please note that what I wrote in my reply to your comment was quite different. I never said anything about your experience. Now that you raised this topic, I can say that you appear to be familiar with Unix security. You simply had not looked at our stuff before you wrote your comment, that's all.
I find your comment re: the cost of PIE interesting, thanks! It leaves many questions, though. 6% on 32-bit x86 is the number I had been aware of before. Your statement that "the affected code was actually running less than 2% of the time" is curious and potentially very useful. However, this, assuming that it's true, does not yield your claimed 0.0012%; it yields 0.12%. Of course, 0.12% may be considered acceptable (far more likely than 6% at least). I am curious about the details of your test; I guess, some systems will actually run "the affected code" 100% of the time - e.g., this happens if I build John the Ripper as PIE and run it for days (which is why I dislike Gentoo building JtR like that, and have to recommend JtR users to make their own builds). ;-) I'd appreciate it if you e-mail me (solar at openwall.com) more details on your test, and we'll discuss from there. We're considering PIE by default for post-3.0 Owl-current (and next release).
Your comments about cron/at are similar to what we thought of when we designed the system. Just like our per-user shadow files, the crontab files are per-user (and they were prior to our changes as well). Our crond does check crontab file ownership, and then it will only run the cron jobs found in the file as the file's owner. Cron and at jobs running as root are supported - of course, as long as the corresponding files are root-owned.
every bit of info is there
Actually, there are at least two things I've been planning to add to the website (but we never got around to doing that): an interface to browse our man pages and another one to browse our packages (list and contents - at least the metadata). I don't know whether this is a higher priority than a mere facelift or not.
Re: immutable/append-only flags -- you are obviously wrong, chattr -i and chattr -a are your friends to remove these flags in a normal multi-user runlevel, indeed, you need to be root to do it, though.
Actually, it is possible to configure a Linux system such that "chattr -a" won't work without having to reboot first. With ancient kernels, we had BSD-style securelevel for this. With recent ones, we have the capability bounding set instead (/proc/sys/kernel/cap-bound). But very few sysadmins make use of this functionality, and it is tricky and painful to use it in a way that would achieve much (need to make almost the entire "base system" immutable - the individual files - or watch for any unexpected reboots and consider those signs of a potential intrusion, but that just wastes time). I tried this with securelevel on Linux 2.0 kernels in late 1990s, but I am not doing this anymore.
Khopesh - you're welcome, and thank you for your constructive comments. We might create a FAQ for Owl based on the questions/comments we've received.
Agreed re: insufficient documentation on owl-control. This is something for us to improve. Also add our own web interface to our man pages, like *BSD's have - frankly, this has been on my to-do list for years... but there was always something more important or/and more urgent.
I have replied to your comment's GGP with the info you have requested.
You can view the formatted control(8) man page here:
http://docs.altlinux.org/manpages/control.8.html
(ALT Linux have imported owl-control for their distributions, and they have contributed some changes back to us. They also happen to have placed this man page on the web.)
I'm afraid that this is not going to help you very much, though, because owl-control is very generic and abstract, and so is its man page. Perhaps we should add a few usage examples to illustrate owl-control's purpose.
I'll try to explain what it's for. Some packages on Owl provide what we call, in owl-control terms, "facilities". An example of such facility is su, provided by the SimplePAMApps package. The package also provides the file /etc/control.d/facilities/su, which defines several possible settings for su. In this case, the settings are: public, wheel, wheelonly, restricted - as shown by "control su list". The default for su on Owl 3.0 is restricted, which corresponds to /bin/su being mode 700 (usable by root only). If you issue "control su wheelonly", /bin/su changes to mode 4710, with group wheel. Issuing "control su" then confirms that the current setting is indeed wheelonly. Other related programs are control-dump and control-restore: these are used by scriptlets of our RPM packages. For example, our SimplePAMApps.spec invokes "control-dump su" in %pre and "control-restore su" in %post. This lets a possibly customized control setting for su persist over upgrades of the SimplePAMApps package.
Some other facilities are: bind-debug (controls whether it's possible to have BIND produce debugging logs or not), ntpd (can be set to "server" or "client"), postfix (can be set to "server" or "local"), and many others. So this is not limited to changing permissions on files - it also works for configuration file contents - but we only use it for trivial changes (a line commented-out or not) that need to persist over system upgrades. Also, such control'able changes are perfectly compatible with manual edits to the corresponding config files. If a sysadmin ends up editing a file such that control's facility script does not recognize one of the supported settings, control will start reporting "unknown" for that facility's state - but that is not any worse than not having control would have been.
As you can see, this is a very nice security-relevant feature for sysadmins, but it is not central to the Owl security model. Things such as our tcb suite (making the passwd command run without requiring root privs) are more essential to our security model.
If you can't su or sudo, how you get anything done?
We normally login directly as a root user for sysadmin tasks (e.g., r_john for a sysadmin named John) and also directly as a non-root user (e.g., john) for other tasks. This applies to both console and remote logins (ssh). This is the approach we've been using for years, and it works well for us.
As I have mentioned, those who prefer the traditional su approach, despite of its added risks (including compromise propagation from a sysadmin's non-root account to root), may "control su wheelonly".
I'm interested in this for a VMware guest OS, as a possible alternative to m0n0wall. Have the authors thought about that kind of implementation
Yes, Owl 3.0 works in VMware out of the box. We mostly run it in QEMU and VirtualBox for our VM-based testing, though.
BTW, you might find it curious that when you run Owl in a VM like this, you can further create OpenVZ containers with multiple instances of Owl and with other Linux distros inside that single running copy of Owl. Such container-based virtualization has no further performance overhead. :-)
How much RAM does it need to run adequately?
128 MB is plenty, probably a lot less will do. I have my QEMU set to its default of 128 MB when I do install-tests with new Owl ISOs. Also, my Owl-based router and small fileserver at home has 128 MB RAM (runs the default-enabled set of Owl services plus ntpd, named, pppd/pppoe, openvpn, squid, NFS, and a sometimes a few lftp's and ctorrent's under screen).
On the other hand, of course you may need a lot more RAM for your specific uses of Owl - e.g., to run many OpenVZ containers, to do web hosting, etc. We currently administer several 32 GB RAM machines running Owl and one 64 GB RAM machine, as well as a countless number of 4-16 GB machines - so we know Owl has no problem with these larger RAM sizes as well (and with the many OpenVZ containers running various Linux distros actually making use of this RAM).
Openwall... secure? webbased interfaces can be very vulnerable.
There's no web-based interface in Owl.
I don't think Openwall supports grsec, containers, Cuda etc. ? I bet they still use deprecated software like Snort.
Are you just trolling? I'll provide a serious response nevertheless: we don't include the grsecurity patch, but our userland will run just fine with a grsecurity-patched kernel - this is up to our users to choose if they like. We do support containers out of the box, with OpenVZ. Owl can act both as an OpenVZ host (it includes the kernel and vz* tools) and guest (we provide pre-created OpenVZ container templates, and there's "make vztemplate" to build your own ones if you like). CUDA has nothing to do with a server/appliance Linux distro at this time (although I can think of some obscure uses). Since when is Snort deprecated? Anyway, we don't include it, but it can be easily installed on top of Owl.
The exploits named in the summary are mostly for locally connected users, which means academic environments.
Yes, local users and pseudo-users. No, this is not limited to academic environments. Think shared web hosting - hundreds or thousands of local users on a system. We're actually setting up Owl-based shared web hosting systems for our clients. Besides the different user accounts belonging to different shared hosting customers, privilege separation between the accounts is also needed because one of the websites may get compromised (via a web app vulnerability) and we would not want such a compromise to easily propagate onto all other websites hosted on the same system (not even if they're of the same customer).
Additionally, even on a dedicated server for a single website (or, say, on a mail server) there are pseudo-users. Even the sshd service from OpenSSH uses a pseudo-user for its built-in privilege separation. This is needed to mitigate the impact of vulnerabilities in certain parts of code in those services (and in libraries that they use). You don't want any vulnerability anywhere in sshd or in parts of libc/libcrypto/libz that it uses (and in a lot of other libraries on riskier Linux distros such as RHEL/CentOS/Fedora - just see "ldd /usr/sbin/sshd" on those and be very scared) to result in a full-blown root compromise. The extra barrier between the less-privileged sub-process (doing lots of things) and the parent root-privileged process (doing fewer things) is of some help. Ditto for other services that implement privilege separation - and this is the kind of service implementations that we include in Owl (OpenSSH, vsftpd, OpenNTPD, Postfix all have privsep implemented upstream; telnetd was missing privsep, but we introduced privsep into it).
some sort of permissions escalation daemon ("owl-control" perhaps?)
owl-control is merely a tool to ease system administration - change settings and have your changes persist over system upgrades. It is one of the nice security-related features of Owl, but it is NOT key to running a reasonable Owl system without SUIDs (this is achieved by other means - SGIDs and changes to various system components). owl-control is not what you thought it was (not a "permissions escalation daemon", but merely a set of scripts).
I do agree with you that we need to present our documentation (such as man pages) in a form more accessible to someone who is not an Owl user yet. The only reason why we did not do that yet is lack of time (small development team working on a mostly non-commercial project, so we constantly have to choose which of the pending tasks we make progress on).
From some googling and the announcement.
Basically if you exploit something with 'shadow' (i.e. passwd) you add a root user account to /etc/passwd and su to it. if you exploit crontab or at, you add a crontab that adds a root level account or runs a command as root or creates a SUID program. It requires some hacker creativity, but doesn't make anything secure.
That's poor analysis on your part, so your conclusion is completely wrong. Please refer to our presentation slides for an explanation of how things actually work and why the attacks you describe would not work:
http://www.openwall.com/presentations/Owl/
BTW, the announcement specifically mentioned that "a vulnerability in crontab(1) or at(1) can't result in a root compromise without a vulnerability in crond(8) or in a critical system component relied upon by crond(8)." Did you not read that? Or do you disagree, thereby stating that we're inexperienced in the stuff we've been doing for 10 years?
Since a lot of people are confused just like you are, I'd be happy for any suggestions on how we could explain what we do and what we have achieved better. I did include that quote in the announcement specifically because I knew of common misconceptions about our work, but apparently that was not enough? Thanks!
Speaking of ping in particular, I have to admit that in Owl 3.0 it is simply restricted to invocation by root by default (mode 700) - not great, but usually acceptable. It can be re-enabled with "control ping public", and this setting will persist over upgrades, but it re-introduces the risk. We're working towards a better solution - specifically, we're testing a Linux kernel patch implementing non-raw ICMP sockets, which we intend to submit to LKML soon. (It already works for us, including via our patched ping, but this stuff was too cutting-edge to include it in the release.)
traceroute just works on Owl out of the box, though - we're using an implementation that makes use of Linux-specific kernel features (upstream, no patches) such that it does not need any special privileges to run (it is installed mode 755 and it works for all users). (This is similar to what we're trying to implement for ping, but it's already a reality.)
The new system allows /bin/su to have permission to write to /etc/passwd, but not to do other root things like read/write to /root or mount filesystems not enumerated in /etc/fstab. It is "granular root".
You're talking of Fedora's approach with fscaps (and there are errors in your description). We don't do that. We're smarter than that! So your comment does not apply to Owl, at all. I've explained what we actually do in further comments. It is also shown in our presentation slides:
http://www.openwall.com/presentations/Owl/
Specifically:
http://www.openwall.com/presentations/Owl/mgp00013.html
http://www.openwall.com/presentations/Owl/mgp00020.html
(and further slides, just click "next").
Yes, our distro doesn't encourage users to use su or sudo. The reason is that escalating privileges from a less privileged account to a more privileged account is bad from security standpoint.
Exactly. And our solution to the "accountability" problem when there's more than one sysadmin is multiple root accounts - we typically prefix their usernames with "r_" for clarity, and we keep the main "root" account locked. We even have our own msulogin program, replacement for sulogin, to allow for single user mode console logins under any one of multiple root accounts that might exist on the system. The rest of Linux tools happened to just work with multiple root accounts fine, with no changes needed on our part.
That said, if anyone truly wants to use the traditional absurd approach of su'ing to root, they can re-enable su with "control su wheelonly" (there are other possible settings as well). Of course, this makes /bin/su SUID root... but you asked for it. This setting will persist over Owl upgrades. See "man control", "control" (just run it with no options for a list of facilities and their possible settings).
On the other hand, i believe they use capabilities ...
No, we don't! We're smarter than that. I've explained what we do in further comments (in the thread about my criticism of Fedora's approach).
Of all the Linux vulnerabilities in the past few years, how many would have been stopped by this?
There were in fact cases where other Linux distros had to issue updates and advisories - e.g., for an issue in crontab (to use the same example that I used for some other comments here) - whereas on Owl not only the vulnerability did not exist in the first place, but also it would have been mitigated due to the greatly reduced privileges of the crontab program (to use the same example again). To give another example, the recent glibc vulnerability with LD_AUDIT and $ORIGIN (discovered by Tavis Ormandy) not only did not affect Owl for a couple of reasons (including due to a glibc hardening patch), but also would be mitigated by not having a single SUID program if the vulnerability did exist on Owl (it did exist on almost all other distros). As you can see from this, we often have multiple layers of security hardening that help (with this being one of the layers) whereas others sometimes happen to have none - and are vulnerable.
At this point, our weakest link is the Linux kernel - large and monolithic - so that's pretty much the only place where critical vulnerabilities that do affect us are still being found... There's little we can do about this (while staying Linux), but it may be a focus area for further security hardening now that we have an almost perfect userland. Also, Owl differs from other/major distros as it relates to exposing the Linux kernel to attack - we expose less of it - e.g., we exclude kernel module auto-loading, we have saner/safer permissions on device files, and we have no SUIDs by default (which mitigates some classes of potential vulnerabilities in the Linux kernel's program startup code - such vulnerabilities were found/fixed in the past).
That was regarding "no SUIDs". As to the detection of log record spoofing, it does not stop attacks, but it is a security feature. When you review logs, you typically want to have a way to know whether the records are genuine, at least under the assumption that the system has not been compromised yet. On traditional Unix/Linux/BSD/..., you had to assume that any (pseudo-)user could have trivially produced any of the log records, pretending to be any system service or even the kernel.
> BTW Fedora 15 is also dropping SUID, so while Openwall is the only current distro. It's by no means the only one in development.
Right, however Fedora's approach is (1) completely different and a lot worse than ours (in my opinion, indeed) and (2) either won't result in complete removal of SUID programs or will leave many with root-equivalent capability sets. This means that they will continue to expose the dynamic linker, libc startup, and relevant parts of the Linux kernel to the usual risks associated with SUID root program startup - something that we avoid.
> Ubuntu is also removing SUID, but I don't know their timetable.
They're aware of the drawbacks of Fedora's approach, so they don't hurry to implement it. They're also aware of our proposed alternative. :-)
> Here's one of the better criticisms of dropping SUID, and it's from an Openwall developer. These criticisms are echoed by almost everyone thinking about removing SUID.
I am glad that you liked my criticism of Fedora's approach, however it appears that you misunderstood me. I criticized their specific approach with fs capabilities, not the idea of getting rid of SUIDs in general. The approach taken by us in Owl (many years ago, but only widely publicized now) and the one being taken by Fedora now are completely different (SGID and changes elsewhere in the system in Owl vs. fscaps in Fedora). The purpose of my criticism was to make other folks, including Fedora developers, consider the issues and the alternatives. It was not to discourage them from taking the move, but rather to give them an opportunity to consider our approach, which we consider to be the better and "real" one.
With our approach implemented in Owl, a used-to-be-SUID program runs SGID to a group dedicated to a certain purpose - e.g., editing a user's crontab. Other parts of the system have been modified such that privileges of that group are sufficient for but can't be expanded further than its intended purpose - e.g., permissions on /var/spool/cron and extra checks in crond (for crontab file ownership and more) are such that a possible compromise of group crontab would not give the attacker almost anything - no ability to edit another user's crontabs without also finding a (second) vulnerability in crond or in system components relied upon by crond. Yes, we could as well not use SGID - just make the crontab program fully unprivileged and the /var/spool/cron directory writable by anyone (the sticky bit prevents messing with another user's existing crontabs). The reasons why we chose to use SGID and a group are: (1) this is needed to prevent some DoS attacks (such as taking up another user's would-be crontab filename) and (2) it is an extra layer of security (so direct attacks on crond are not possible without compromising crontab first). Our changes to crontab/crond have since been adopted by ALT Linux, OpenBSD, and even by upstream ISC (Vixie) cron (but most distros somehow don't hurry to make use of this functionality, continuing to install crontab SUID root...)
Fedora's approach is to replace SUID root with fscaps - that is, the programs are still granted a lot of privilege, just somewhat less than they were with SUID root. In many cases (perhaps even in most, where ping might be the only exception), such capability sets are in fact root-equivalent, so this is sweeping SUIDs under the rug. Also, there's no second layer of security. See the difference?
> I wrote a program once that needed SUID for about a hundred asm instructions.
Sure the program may be tiny, but it exposes parts of the dynamic linker, libc startup, and even extra parts of the Linux kernel code for attack. All of these components have historically had vulnerabilities exploitable specifically via SUID programs. The point behind not having a single SUID program in a default install of Owl is primarily to mitigate potential vulnerabilities in those components. I do agree that, as the distro level, it would otherwise be possible to live with a few tiny SUID programs, but the risk with other exposed components is very real - so we avoid it.
> CAP would be wasted as chroot() is one of those known to grant root power.
Right. Red Hat are going to "waste" capabilities like this. We are not doing that (we use SGIDs and changes elsewhere in the system instead). (To be fair, there is a subtle reason why even root-equivalent capability sets may be slightly better than directly running as root when we consider some specific vulnerability classes. This is just too complicated to discuss in a comment like this.)