Slashdot Mirror

For a start, consider using LIDS

(The name is a misnomer because it prevents alteration of protected components (even as root).)

i can point to things on the intrusion detection side of things... i only bother with the people who have a stick up their arses about security on wireless networks. i have no problem letting my neighbors use my network, it's more than speedy enough for all of us, but wardrivers who decide they want to "inform me of the risks" deserve some of their own medicine. http://www.lids.org/ i wont help you with the getting them back part, that's illegal.

Cisco Security Agent is a close analog to the sort of comprehensive kernel security hooking that something like LIDS does on Linux. If you can do some research to determine how they're doing it, that'll be a start. They hook all sorts of things, from file and network opens to attempts to sniff keystrokes and executing dynamically modified memory.

Alternatively you can try LIDS. One of the many features of LIDS is ACLs that are applicable to root as well. Configuring LIDS section of their FAQ gives a number of interesting examples of putting a heavy "lid" on access to the system, while keeping it useable.

Alternatively you can try LIDS. One of the many features of LIDS is ACLs that are applicable to root as well. Configuring LIDS section of their FAQ gives a number of interesting examples of putting a heavy "lid" on access to the system, while keeping it useable.

When I attempt to read from a raw disk device as a non privileged user, I get "access denied". Can you be more specific and provide links or proof that this is the current state of affairs with OpenBSD?

I see you deliberly cut my sentence off at a convenient point. I suggest that if you genuinely want to know more about what it means to control access to devices and files using MAC (including for software run as root, i.e. via expolits) I refer you to software such as L.I.D.S. and Pitbull.

Put simply:

On a system providing live services in real world scenarios (such as web, mail, ftp, ldap, samba, database software etc.) it's inherently less secure than a trusted operating system or one that at least partically impliments MAC, POSIX.1e, or that otherwise places restrictions such as on what specific software is allowed to perform privileged operations such as bind to a given port, or access a given file (etc) no matter what user the software in question it's run as (i.e. code run via software exploits, this means you).

I realise that OpenBSD does not have all worthy modern security features, but to say that OpenBSD security is just hype which boils down to OpenSSH and switched off services is just not right. It's not like all these things that you have failed to mention are useless.

I didn't say they were 'just hype', you seem to be trying quite hard to twist my words. However, I certainly stand by my assertion it's association with being a 'secure' OS is primarily based on those two factors, it's certainly not based on it's feature support. Despite how much Theo De Ratt belittles and misrepresents the work of others from a professional security standpoint OpenBSD's feature set really isn't that impressive compared to other free and commercial alternatives, including what can be done with Linux or even compared with features in newer versions of FreeBSD.

Re: OpenSSH on other platforms:

Can you back this up with a link? I find it a little hard to believe given his absolute freedom stance. Especially since OpenSSH is everywhere nowdays.

Theo's statement that he they were not interested in providing OpenSSH for any platform other than OpenBSD was how the whole argument with Alex de Joode started. See my previous post above, the Slashdot archives, or Google, for a link.

This appears to be carefully worded. I guess what you are saying here is that OpenBSD is a secure platform, but less than useful? Sure, it is not the most useful system out there, but for what it does well, it does very well.

Any OS will all it's network services turned off by default, is secure from remote exploits. However (and this is the reason I was choosing my words carefully) if you actually intended to run services (such as Web, Mail or FTP) on that system or allow local user accounts then OpenBSD is not what I would consider a particularly secure platform and there are certainly far superior alternatives (both free and commercial), that are far more deserving of credit and consideration.

Theo's often false derision (and deliberate misrepresentation) of the efforts of other OS vendors is something I find utterly unpalatable.

I'd love to be able to do more than that though.

Like being able to "noexec" on an entire directory, without having to break up the filesystem into separate partitions. 'chattr +n /var/tmp' or something like that.

Making an entire directory immutable would be great too. Eg 'chattr +i /bin' -- sadly thats not possible either.

grsecurity and lids are both capable of some of these things, but they are big scary kernel patches and each have a different approach to things, with their own configs and toolsets. And both can be pretty confusing.

What is more interesting here is the derrivative. The perception of Windows is improving rapidly, the perception of Linux is pretty static.

Eh, you're kidding... right?

I've not seen that AT ALL. Windows security is an oxymoron, and people complain about it BITTERLY to me. I've been delivering Linux-only services for years, and it's all I can do to keep up with all the projects on my plate.

One of my clients is on the verge of switching about 50% of their desktop systems in use by their staff to Linux. They're evaluating it now. Issues I know of are: MS-Word (Hello Crossover Office!) and printing.

What "security action" should be going on in the Linux world that isn't? I have a modest number of servers on the 'net. The only one with security issues is one with a bazillion, ancient CGI scripts on it. (that for various reasons, I can't just have removed - ugh)

But, just in case, do you remember SELinux? Or perhaps LIDS?

Heck googling for "Linux Security" produced a few interesting results, right on the home page!

Next time, listen BEFORE you speak...

There are far more effective solutions available here and here. In fact, you could get ahead of the curve for when people start trying to write spyware for Linux. Do a front-end to LIDS. Install it with a restrictuve ruleset, and then the front-end monitors the warning logs. If it detects something then it pops up a box saying "blah just tried to write to directory foo, do you wish to authorise this?". If the user clicks yes then add a new rule and restart LIDS. Obviously this isn't perfect as you would then have to re-run the command. It would be better to write hooks into LIDs itself for this purpose.

Phillip.

It would be nice if operating systems could protect applications from each other.... Are there any operating systems that do that?

My prayers have been answered, yes there are and discusing them on slashdot should have heaponed eons ago... With computers taking more sick days then people you would think people would be asking for a secure OS when they buy a new pc at compusa.

Its called capability based acces control (first implemented in the 70`s). Its just a fancy way of saying that rather then having a program get rights becouse of whoever executes it it gets all sorts of rights all by itself.... yes thats an improvement security wise becouse this way a process can get only the rights it needs.

Ofcourse you could go and build an all new operating system for this priciple. However many operating systems have been hacked to do tiny bits of this already. In fact many personal firewalls do it for windows (I never though I would be advocating something called a firewall considering I tend to call firwalls "stupid packet filters", and claim they do little for security) Ofcourse open operating systems have plenty of implementations of this idea. Now if only people were to ask microsoft for stuff like this. Windows is full of crazy features that are there becouse big customers needed them. With microsoft giving up on their "(backwards) compatibility before anything else" idea (XP sp2) structural changes might someday make it into windows. Ofcourse thats only if paying customers want them.

I have used GR Security for quite some time, and its not that great loss.

OpenWall was mentioned, but I preffer LIDS as a replacement to GRSecurity. The itens below where taken from GRSecurity site. All listed features are at LIDS either:
# Change root (chroot) hardening
# /tmp race prevention
# Extensive auditing
# Prevention of entire classes of exploits related to address space bugs (from the PaX project)
# Additional randomness in the TCP/IP stack
# A restriction that allows a user to only view his/her processes
# Every security alert or audit contains the IP address of the person that caused the event
Besides, LIDS has a clever ACL schema for file protection and a master password, that if an attacker gets root privileges, it could not exploit the machine completly.

...in those attacks, like they have in the numerous Microsoft leaks. Imagine the strife we'd be in if they stole the source to Debian!

But seriously, how shall I put this? ChkRootKit, TripWire, AIDE, FICC, ProSum, Toby, msec, Nessus, LSAT, Saint, LIDS and of course if you want totally proactive, try SELinux, Medusa DS9 or OpenWall. That's hardly an exhaustive list, but it does hit many of the highlights. Boy, youse bin livin in a monoculture too damn long!

Why not run something like LIDS. You can lock access to your logfiles so that only certain processes can run them.

It looks like a bit of work to set up and administer but you'd think that an organization like Debian would make sure all their computers would be running it.

There are various other methods currently being researched, typing speeds/patterns is the main one that could be used here but its not really reliable enough currently. For a major project like debian implementing some kind of smartcards wouldn't be beyond the realms of possibility (or budget anymore).

It is possible to limit break-ins like this using one of the various sets of ACL's around.

I'm not sure I agree with the point that "its not about the OS being secure".....well the OS has users, who tend to do things like write their passwords down, lose them or get themselves socially engineered. A truly secure OS should take this into account and have appropriate measures to limit the damage such a user could do.

when a M$ compromise comes to light

You mean, "when Microsoft have no choice but to annouce a break-in". With the loss of share price such announcements would cause don't you think they'd just keep them quiet? The problem is that Joe Public, and Joe Ceo both think this means that they are more secure because CNN doesn't carry stories about them being hacked. This is a problem with society, not open or closed source.

Damn straight.

Although, one thing needs to stay clear: Linux is only secure if you know what the hell you're doing. 51% of all known successful root compromises occur under Linux. (Linux has more than 51% of the market share, IIRC, so it's not a very fair comparison. If anybody has market share data, please provide it so we can look at ratios.)

I prefer running Linux, of course. At least I know I can secure it.

What I find most troublesome is that Microsoft seems to be taking the lead in providing a means of control that goes beyond the ACL approach that has been traditional until now. It's an astute move for M$. If the rest of the world doesn't come up with an alternative, it will become all that much harder to dislodge Windows from the corporate desktop.

It tends to be complex to implement due to a lot of flexibility, and definitly has different goals (securing your systems, as opposed to securing profit of others), but it isn't as Microsoft would be the only one to improve their security model.

It doesn't need any of those privileges, but Linux has no mechanism to protect you on that level

Obligatory reference to LIDS. LIDS lets you specify exactly which users and applications have any combination of an extensive list of privileges, including read or write privileges at the file level and opening sockets. A common example would be hiding /etc/shadow from everyone (even root) *except* /bin/login, who has read-only access.

My old quote used to be "can't get root if there is no root", but you weren't claiming that Linux suffers from multiple privilege escalation vulnerabilities. All told, you are right about native Linux, but there is at least one fix.

Intrusion Tolerance is already being practiced, although another term for it is defense in depth.

Another poster has described how defense in depth and fault tolerance apply to firewalls, network infrastructure, etc. I'd like to mention host-based measures to slow an attacker down and limit the damage they can do.

One of the oldest host-based D-i-D measures is chroot jails. A 'chroot' in Unix means that an application is run with access to only a limited subset of the filesystem, one which does not contain interesting, useful, or leveragable files. This makes it harder for an attacker to leverage, say, user-level access via a buggy network daemon into root-level access, access to the system passwd/shadow file, or access to system binaries.

chroot isn't perfect; the process still shares access to the OS kernel and the network, and can leverage those.

LIDS is a Linux-specific solution. LIDS allows capabilities on a system to be locked down beyond the capability of even root to modify. For example, you can set /usr/bin/* to be read-only, and not even root can override that without first disabling LIDS. The ability to bind to network ports can be controlled; e.g. only /usr/sbin/sendmail can bind to port 25 (and /usr/sbin/sendmail can be made read-only). The ability to load modules into the kernel and access devices to do similar things (e.g. /dev/kmem) can be blocked. In other words, the ability of an attacker who gains root access on the host to rootkit it is severely degraded. There are still openings, though, e.g. root can access user's files.

Security-Enhanced Linux is the next step. Rather than emasculating root as LIDS does, it "has no concept of a 'root' super-user, and does not share the well-known shortcomings of the traditional Linux security mechanisms...." Privileges can be carefully handed out to protect the system from the users and the users from each other.

Even Windows can benefit from some careful configuration. Consider how NIMDA used the Windows TFTP.EXE binary to bootstrap its access up - why is TFTP.EXE executable by anyone on the system? Set ACLs on system binaries. Make sure the IIS web root isn't on the OS drive to block directory traversal attacks. Remove things that aren't needed.

I can't remember the attribution, but someone summed Intrusion Tolerance up by saying, "If you can't prevent it, you sure as hell better be able to detect it." Keeping the bad guys off the server may be impossible, but every little roadblock you put in to slow them down will give you a better chance of detecting them and stopping them before they capture the flag and end the game.

How the hell is this insightful!? It's FUD!

Windows XP has USB 2.0

Wow, what an innovation... they have drivers for USB 2.0 devices. Wow... So do we.

it has low-latency audio

Let's see, does Linux...? Yep, we got that too.

it can play DVDs

Wow, do you think other platforms could do that? Yes, I think so.

it has translucent windows

Well actually, it doesn't. However, these guys have had it for a while and these guys are pretty close.

built-in NAT

Linux has had this since version 2.0. It worked great even back in 1999.

drag-and-drop CD recording

OKay, I'll conceed on this point, but I'll definitely mention you could find this here before Microsoft ever had a clue. As for XP drag and drop CD recording... it still doesn't work right.

an MPEG-4 media player

Once again, I reference these guys again. What's so impressive about that? Microsoft aren't even the people who introduced it.

it has an encrypted, compressed file system

Well, let's see here... yes, we definite have that too. As a matter of fact, I've been using encrypted file systems in Linux for years. As far as I recall, I was doing that before Windows was. No wait, Windows still doesn't offer encryption beneath the file system. Weak.

they have fine-grained access controls

Only very recently. Linux has enjoyed ACL from here and here as well.

they have a common language runtime

Funny thing is it was implemented by the open source community faster than Microsoft did.

They are pushing and developing modern programming languages so that we aren't all stuck programming in C.

A language is a tool Some languages are good for some tasks, some are better for others. For example, you couldn't quite write an operating system in Lisp like you could with C. To make this point shows how much of a fool you are. By the way, GCC compiles langauges other than C too.

Some of this technology sucks, and most of it they didn't invent, but they are pushing new technology.

Yes, most of it does suck but none of it is new. Microsoft only pushes regurgitations of what the rest of the industry has had (often for years).

(I also know that most of this stuff is available on linux, but it's also kind of a pain in the ass.)

I don't see any problems. None of what you mentioned was hard to find nor is any of it any harder to use than in Windows. For example, I play a DVD by loading my DVD player and press the button with the little triangle on it (play).

Your "points" fall down to absolutely nothing. Microsoft offers no advantages, just disadvantages over open source technologies.

You sir are a major corporate whore, completely deceived, clueless, not too bright, and giving free marketing hype to a multibillion dollar company. How does that make you feel?

grsecurity

LIDS

As far as the NSA planting a back door into SELinux, I really doubt it. A backdoor in open source code would be discovered eventually, and the NSA would have a very hard time denying it.

It seems much more likely that they would put back doors into closed source products, which do not receive as much scrunity.

The installation howto for LIDS says that you can turn it off by appending security=0 as a kernel parameter in your boot loader. This seems silly since they go to a lot of trouble to ensure that even the root user can't kill its processess and stuff. What is stopping the root user from just editing the boot loader conf and rebooting with these parameters.

> Isn't Trusted Solaris basically just this? At an OS level, you associate trust levels that permeate throughout your network.

I think this kind of approach is better than creating virtual machine sandboxes that still run the old weak UNIX security model. If someone 0wns your sandboxed apache, they can still likely cause
a DoS with it, or propogate worms, or pretty much anything really.

Good use of iptables and linux "capabilities" can help a lot with limiting what an application can do, but still don't go far enough, IMHO.

Look at projects like SELinux, LIDS, RSBAC, LOMAC
for examples of "free software" alternate security models.

And yeah, they're a pain for development, but then so is trying to program securely :)

- MugginsM

That's a good list. The only thing I'd add is to suggest using something like LIDS. The basic notion is that you add something to the kernel so that root is no longer a trusted account. Instead, you give very specific permissions to each program that needs them.

That way even if somebody does compromise, say, named, they won't be able to screw with much. And since LIDS logs attempts to violate security, as soon as they try to, say, view the password file or alter the logs, I'll know about it.

Why don't webhosters allow SSH, no FTP or telnet, but pre-patch their servers with LIDS?
Oh and don't allow outgoing packets that aren't part of the reply to either incoming 22, 80, and 443.

Because it is run as root, you are leaving your machine wide open to anything the trojan wants to do.

How about LIDS? LIDS lets you set r/w perms on files and directories and restrict actions that all users, even root, can perform. With a tight LIDS setup, root isn't even root. A simple example is setting /var/log/messages to read-only for everyone (that's users AND executables) except syslog and cron, and cron gets read-write perms only during the 1 minute a week it needs to rotate the log file.

That's pretty tight.

SuxOS introduces a revolutionary security structure, using among others, the Linux Intrusion Detection System to enforce MAC (Mandatory Access Control), the grsecurity kernel patch, to enhance overall security by putting restrictions on various parts of the /proc filesystem, preventing common buffer overflows, TCP/IP stealth code et cetera, plus the valuable protection from format string vulnerabilities given by FormatGuard. Other than that, Pluggable Authentication Modules are used for resource limiting and authentication. All this, together with the fact that SuxOS only includes applications and servers that are known to have a history of few or none security flaws, gives the administrator unsurpassed security and control over the system.

The Linux Intrusion Detection System makes it possible to make an incredibly fine grained set of Access Control Lists, thus making it virtually impossible for even a skilled cracker to penetrate the strong security layers of SuxOS. LIDS provides the ability to control all access to system resources, even preventing a root compromise from subverting the security of the entire system. The default Access Control Lists in SuxOS, has been set up in a very secure fashion, by locking up the system completely, and then explicitly granting access to the applications that need it. The outcome of this is extremely fine grained access control, unsurpassed by any other known Linux distribution today.

Security of the host itself has been significantly improved. Enforcement of longer passwords, insecure protocols non-existent, and extensive logging and auditing provide a solid foundation to build a complete corporate Internet presence.

• They're unlikely to get the first password; 99% of breakins involve getting root through insecurities, not through password compromise.
• And how would that happen? The password is only used during OS maintenance, which should happen rarely on a production server. LIDS itself protects the LIDS binaries from being trapdoored. Use it only from the console to remove a number of network risks.
• Then do something more! This is all open-source software; if you need SecureID cards, thumbprint scanning, and blood sampling, you can put that in, too.

Or better yet, no swap at all. For all of the real-time (as opposed to batch) server stuff I write, I make sure the hardware should never need to swap. Once you allow swapping, response time goes into the toilet, usually causing queuing and spiralling delays. Yuck.

Of course, root can still walk through RAM, but there are ways to fix that. I've lately been trying LIDS, which adds a more complex permission model. You can make it so that root is normally pretty limited, requiring a separate maintenance password to do anything dangerous.

Create a box running Apache SSL and have it firewalled / protected like crazy and locked down with LIDS or the NSA patches to linux. Use this box as the "password server" and have access to each and every password logged. And have each NOC employee be part of access groups that say "router access" or "colo access" or something so they can ONLY access data available for their group.

On the logging tables in the database, make sure they aren't readable or writeable by the web-user. They should only allow INSERT queries.

This might be the best way.

x

Stacking modules (loading more than one module at once) is problematic, because security policies are known to not be composable in general. However, if the modules have been designed to be stacked, then LSM will let you stack them.

Crispin
----
Crispin Cowan, Ph.D.
Chief Scientist, WireX Communications, Inc.
Immunix: Security Hardened Linux Distribution
Available for purchase

Right. Try "rm -rf /" as a user...

Anyway, there is the LIDS project that you might be interested in... that it it's primary goal!

-Ben

(Do we *really* need to be running our network services as root just so that they can bind to a low-numbered port?)

I suspect you knew this already (and just added that comment for effect), but that problem has been solved at least twice, with the LIDS patch and NSA's SELinux. Both accomplish this by untying capabilities from the user-id (such that, for example, a program can be given permission to bind to low-numbered ports while receiving no other advanced privileges).

A whitepaper can be found for the NSA's solution, and reasonable documentation can be found for the LIDS project.

(Do we *really* need to be running our network services as root just so that they can bind to a low-numbered port?)

I suspect you knew this already (and just added that comment for effect), but that problem has been solved at least twice, with the LIDS patch and NSA's SELinux. Both accomplish this by untying capabilities from the user-id (such that, for example, a program can be given permission to bind to low-numbered ports while receiving no other advanced privileges).

A whitepaper can be found for the NSA's solution, and reasonable documentation can be found for the LIDS project.

To protect against unknown exploits, there are kernel patches like LIDS . With LIDS, you can enforce any program to only access some files. For instance, you can enforce Bind to only read his configuration files, and nothing else. So even if an exploit is found, your system will be safe.

It works amazingly well, and for almost everything on your system.

But does it apply to SSH and FTP? Probably not. When you give FTP access to customers so that they can upload web pages, the FTP server needs read/write access to everything in /home. So it means that if an exploit is found, even with a properly configured LIDS barrier, the attacker can change the content of any customer file. And that's really dangerous. And LIDS can hardly avoid this.

Linux Kernel Pillow Talk

(Linux Kernel Pillow Talk: Page1of1)
By Moshe Bar

October 29, 2001

And you thought the netherworlds of dry kernel engineering were free of politics, egos, and prima-donnas? Guess again. The events of the last four to six weeks and the e-mails flying to and from the Linux kernel mailing list show how Byzantine and complex the dynamics of decision finding, features design, and implementations can be. Go to http://www.tux.org/lkml/ to subscribe to the kernel mailing list, but be careful: This is a very high-traffic list. Subscribe only if you really want to follow every single detail of the Linux kernel, or instead read the weekly digest at Linux Kernel Cousin at http://kt.zork.net/kernel-tra ffic.

Sure, the lively debates have always existed. In the past there have been disputes about the Linux firewalling code, networking code, scheduler, installer, driver model, and many more. One recurrent theme has always been the Virtual Memory (VM) manager. Nothing determines the peculiar behavior, the feel -- even the ultimate success or failure of an operating system -- like its virtual memory design. Sometime during the development cycle leading up to the Linux 2.4.0 kernel, in other words in 2.3.xx times, Rik Van Riel (http://www.surriel.com), a Dutch kernel hacker working for Brazil-based Conectiva (one of the smaller Linux distributions), introduced a radically new VM code. It was based on what seemed to be new and advanced algorithms for efficient finding, allocation, and disposal of virtual memory pages requested by programs. Rik later introduced an interesting new kernel feature called the "OOM killer." OOM stands for Out Of Memory. The OOM killer attempts to locate a killable process when memory runs out in the system. Without such a feature the whole machine can go nuts or enter a vicious cycle of swapping out a few pages, realizing immediately after that those pages are needed, and searching again for swappable page candidates, keeping the kernel busy doing only this instead of letting user processes run.

Rik is a gifted hacker, and among other things he has been trying to improve the efficiency and speed of maintenance of those lists in the kernel responsible for managing all the virtual memory pages in the system. One of the main questions to address in every operating system VM code is: "How do you choose which page to steal next when there is a RAM shortage?"

In the 2.4.0 release, the Linux kernel scans the process page and decides which page to remove. The problem with this approach is that sometimes a lot of process tables have to be scanned to free just one page, or very few pages. Also, this approach does not guarantee that the pages stolen are only those that will not be needed again very soon. Some UNIXes introduced the notion of the working set; that is, the minimum amount required by a process to function efficiently. This solution is, however, limited to per-process pages only and does not consider other kinds of pages, such as filesystem caching. Stealing from these pages might in some cases even prove counter-productive. Very often in VM theory, a solution to one problem can worsen another; that's why kernel programming is difficult.

Rik van Riel and I have variously discussed another approach, called "reverse mapping," which implements a reverse-lookup between the page and process table. Once you have reverse-mapped pages, the VM can simply scan the pages for the ones to be freed. Naturally, some extra fields need to be added to the appropriate control tables to allow this reverse mapping. My own implementation has an overhead of 14 bytes and is therefore certainly a lesser solution than Rik's -- his overhead is just 8 bytes.

Other extremely talented kernel hackers such as Marcelo Tosati and Ben LaHaise have made other important contributions to the Linux VM.

However, even though all these intelligent people tried hard to make the Linux VM fast, efficient, and powerful, user reports since the 2.4.0 release indicated poor Linux kernel performance and erratic and unstable behaviors. Up to kernel 2.4.7, for instance, on machines with small memory footprints (less than 40-MB RAM), sudden swap storms could erupt which would virtually freeze the system while it inexplicably started swapping pages in out and like crazy. In some cases, the aforementioned OOM Killer would choose the wrong process to kill; I have seen the all-important init process killed erroneously. Many fringe kernel projects, like my own Mosix project or others such as Win4Lin, suffered because users accused these projects of unstable operations, assuming that a released kernel like 2.4.0 must be free of such nasty bugs. Even though the kernel had gradually evolved from 2.4.0 to 2.4.9, it was evident that the VM design was more of a liability than an advantage.

Linus himself said in a recent kernel list mailing that he wasn't happy yet with the VM. These problems were enough for many Linux shops to resist the migration to the 2.4 kernels and instead continue using the 2.2.19 kind of kernels. Obviously, compared to 2.4., the 2.2. series has many shortcomings -- like no zero-copy networking, the division of page cache and buffer-cache in filesystem operations, big spinlocks (serializations of kernel execution paths for computers with more than one CPU) for many parts of the kernel, and so on.

A simple C program like the one below shows how kernels up to 2.4.9 had problems dealing with stress workloads on the VM system. If, after running this program, you turned the swap partition off with swapoff, your server or workstation would become totally unresponsive for up to 15 minutes.

/* based on a code originally proposed by Andrew Tanenbaum, later by Derek Glidden and many others since */ #include void main(void) { /* in the next line we allocate 200MB, but since the virtual memory page is not actually allocated by the kernel until we use it, we also have to create an access to. The amount of allocated pages should reflect the total RAM on your computer. This test runs well with machines of, say, 256MB */ void *p = (void *)calloc(50000000, sizeof(int)) ; /* In the next line we let the system calm down a bit after allocating pages*/ sleep(12); /* and now re release it all again */ free(p); }

Back in February 2001, I ran an informal and unscientific benchmark comparing FreeBSD 4.1.1 to kernel 2.4.0 (visit http://ww w.byte.com/documents/s=558/byt20010130s0010/) on exactly the same hardware and with exactly the same subsystems versions (MySQL, Sendmail, Apache, and others). The results clearly showed that, indeed, there were major problems with the efficiency and speed of the early 2.4 kernels. A New VM

Then, on September 24, with the kernel standing at version 2.4.9, everything suddenly changed. Andrea Arcangeli, an Italian kernel hacker (read my interview with him two years ago at http://ww w.byte.com/documents/s=287/byt20000229s0008/) and a very prolific contributor, decided that enough was enough. He sat down and in one of those marathon hacking bouts completely rebuilt the VM from scratch. In short succession he sent to Linus Torvalds over 150 patches to the 2.4.9 kernel, to implement a new VM engine. This is an extremely remarkable feat. A VM is a major piece of software and by nature very complex. One needs to satisfy many opposed objectives: Simultaneously efficient handling for server-type loads and interactive-type loads; ease of implementation and at the same time, optimized use of every last and small feature of the CPU. The VM must also be able to run well on Intel CPUs spanning 4 or 5 generations, as well as on AMD chips, Alphas, MIPSes, Sparcs, ARMs, and what have you. Andrea, by the way, does all his development on a Compaq AlphaServer with 2 500-MHz CPUs and 3-GB RAM.

Out of the blue, Linus accepted the new VM and incorporated it into the official Linux kernel tree.

Recently, I spent two days with Andrea giving speeches. During the two days, over many bottles of beer, we had plenty of time to discuss his new VM. I was mainly interested in how the new VM affects Mosix. Because Mosix must migrate virtual memory pages belonging to the program's address spaces between cluster nodes, it is important to correctly understand the VM and interface efficiently to it.

Specifically, Andrea took exception to the following problems in the 2.4 VM:

• kswapd looping forever on DMA or NORMAL class-zones.
• swap+ram will be almost all available address space (modulo when the swap cache serves to avoid swapin of shared anonymous memory after a fork).
• swapout storms.
• benchmarks, when run repeatedly, gradually slow down.

The new VM is much simpler and faster. Let me explain how it works.

The old 2.4 VM had a major design problem that manifested itself mainly when freeing physically dirty pages (remember dirty pages are the frames of 4-KB memory in the RAM whose contents have been modified by one of the virtual memory pages residing in it). The last owner of the page (usually the VM, except in swapoff) has to clear the dirty flag before freeing the page. When being swapped off in swapoff it may be a little more complicated -- we may need to grab the pagecache_lock to ensure nobody starts using the page while we clear it.

So, Andrea went and did the following: All physical pages are now divided into active and inactive pages. These two are further divided into dirty and clean for both active and inactive. When the active dirty pages become about 66 percent of the total number of pages, the VM starts to scan them for the oldest ones to be put into inactive dirty and then, later still, from there to the swap when memory becomes tight. This part is very central to the new VM and its simplicity is...well, simply stunning.

This elegant mechanism totally changes the behavior of the 2.4.10 kernel under heavy load and also makes for much better predictability of the system. Another very important change is that the swap is now additional to the RAM, just like in 2.2 times. All earlier 2.4 kernels (since 2.3.12) needed at least the same amount of RAM in swap and then more to give you additional virtual memory. This meant that on an 8-GB server, you needed to put aside almost a full 9-GB disk just to be able to swap, similar to some versions of Solaris or other UNIXes.

Finally, the page scanner doesn't page scan if there are theoretically no freeable pages, whereas before it did. Oh, and the OOM killer never really worked, so Andrea disabled it, as I did for all my kernels. In 2.4.12 it is enabled again; this time, however, it works much better. Try it with the above program to see it in action.

Arcangeli's VM is stable, acts predictably -- something that the old VM never really achieved -- and it makes the swap space look like it did in 2.2 days. Additionally, the design is much simpler and easier to understand. People will catch up fast with it.

However, many kernel hackers disagree. Upon the release of kernel 2.4.10, a virulent and sometimes aggressive debate flamed up, with many people trying to show why one of the two was a good VM and the other not. Some comments got a bit out of control, and only in the last two weeks or so has some calm been restored.

However, one nasty side effect stays. Alan Cox, the number two man after Linus Torvalds, does not yet like the new VM and in his own kernel tree (called the "ac tree") he still continues to use and patch the old VM. As a consequence, users and system administrators now find themselves facing two very different kernel trees to choose from: the official Linux tree and the Alan Cox tree. Quite often, latest patches to drivers and new features are only in Alan Cox's tree. Those who want to go with the official Linux source code may find themselves unable to apply the patches due to the different VM code all over. It is acceptable for the two trees to be different for a few days on such important subsystems like VM, but it is not acceptable to have them different for months and across many kernel versions.

Nobody has yet dared to speak of a Linux source fork, but this is dangerously close to one.

It became obvious that the VM up to 2.4.10 was a design liability. You can try to fix something that was designed badly, but it will never become a beauty. I think Linus' decision to scrap the old VM and go with the Arcangeli VM was courageous and right. Having a functioning and stable Linux box should not be deferred to 2.5 when we can do it already with 2.4. Kernel Preemption

But apart from the VM issues, there are other lively debates in the kernel community. There was an interesting interview at h ttp://kerneltrap.com/article.php?sid=328&mode=thre ad&order=0 with Robert Love, who is leading one of two projects trying to make the Linux kernel fully preemptible. Making the kernel preemptible means making it possible to interrupt whatever the kernel is doing (say, executing a system call) to process some other outstanding task and then return to its original task. Linux, as a multiprocessing OS, obviously always did that for user-land processes. However, many, just like Robert Love, feel that the fact that Linux up to now would not let itself be interrupted contributed to poor latency. Latency describes how quickly you can expect a response from your kernel when you actually need something from it. Note that Linux is not designed as a real-time OS (though there is at least one Linux real-time implementation somewhere), and therefore does not explicitly guarantee latency. User-land programs must be aware of this as, especially with kernel preemption, latencies can be very unpredictable.

Theoretically, an OS will answer faster if it can be interrupted. What does suffer from kernel preemption is the global throughput. If you have a task that gets n seconds within the kernel to complete (let's say executing a given system call takes 0.005 seconds), then all the interruptions add some overhead to switch from one kernel task to another. So, finishing the execution of that system call (in our example) will finally require n+op where p is the frequency of switching and o the static overhead for one switching operation. Notice that kernel context switching does not invalidate the CPU cache, and is therefore not as expensive as process switching. However, kernel preemption will surely lead to a higher rate of switching from kernel space to user space, because upon preemption the scheduler might decide to give higher priority to a user process.

In other words, kernel preemption does decrease latency but slows down overall throughput. It's the math: nothing to be done against it.

Furthermore, in his interview, Robert Love heavily criticized Linus Torvalds for adopting Andrea Arcangeli's new VM in 2.4.10 and dropping the old van Riel VM.

Well, I did try the patch with kernel 2.4.12 and with pre13. While accurate measurement (which Robert Love provides with the preemption kernel patches) does indeed report an improvement in latency, for the life of me I have not noticed it on an empirical basis.

I really do appreciate Love's work, but I do not fully agree with some of his comments in the interview. First, as Linus himself said, if latency sucks in the kernel then we should check why it sucks, with or without preemptive scheduling. If the latency is bad in the stock kernel, then it should be fixed anyway.

The preemptive kernel 2.4.12 worked fine on my laptops and on my SGI 550 workstation where I do interactive work. The MP3 player very rarely skipped beats when doing heavy background work such as kernel compiling or opening large files in the editor. But for my servers and clusters, the decrease in performance and the unpredictability of latency is a problem. Also, some important patches will not apply to a Love-patched kernel. Mosix, the clustering kernel extension, does not patch correctly, and neither do some versions of the LIDS intrusion detection system.

It is up to each individual user to decide whether or not to use the patch, but is important to understand the implications of using it. Linux and FreeBSD Revisited

Upon returning home the other week after meeting with Andrea, I went to my lab and searched for the disk images of the server comparison I ran back in January of this year (of FreeBSD 4.1.1 versus Linux 2.4.0). I took the Compaq ML500 server I have been reviewing (2x 1-GHz CPUs, 2-GB RAM) and upgraded both the FreeBSD disk image to 4.4-Stable and the Linux version to 2.4.12. Then, I changed the memory down to 192-MB RAM so as to stress the VM system more. I also upgraded to the latest stable versions of Sendmail (8.12.1) and MySQL (version 3.23.42). Finally, I compiled everything with the latest version of gcc, 3.0.2, and tuned the two instances to the best of my knowledge (softupdates and increased maxusers for FreeBSD, and untouched default values for Linux).

The results were very interesting indeed. Since this benchmark is too much to be handled in this article, Byte.com will post it here soon for you to read.

The story of this article is that the 2.4 kernel has finally grown up with the 2.4.10 release. Not many users outside the relatively small kernel community realize that. Now you know about it, too. Spread the good news and immediately install 2.4.12 on your busy server. The server will thank you for it.

Moshe Bar is a systems administrator and OS researcher who started learning UNIX on a PDP-11 with AT&T UNIX Release 6, back in 1981. Moshe has a M.Sc and a Ph.D. in computer science and writes UNIX-related books.

For more of Moshe's columns, visit the Serving With LinuxIndex Page . Page1of1

Together with a chrooted install of Apache, the security level is becoming rather comfortable.

This is looking very nice. They're putting hooks into lots of places in the kernel. If the hooks themselves are accepted into the core kernel, then many of the different Linux security projects (like LIDS) will be able to work with little (or even no) kernel patching. It also has clean seperation between it's various components, so that anyone can plug in their own implentation of any of the sub-systems; thus, just like in Perl, ther'll be More Than One Way To Do It.

LIDS.

Don't use password authentication with SSH / OpenSSH .
The beauty of *SSH is that you can use crypto keys for authentication. With ssh-keygen -tdsa , you create a pair of keys. id_dsa is the private one (keep it on your computer), and id_dsa.pub is the public one. You can copy the public one in ~/.ssh/authorized_keys2 on every server you are willing to access. The public key can be given to everyone, you can even put it in your signature so that people can grant you access to their machines if they want to.
When the public/private keys matches, you can log in. No need to enter any password. Simple. Fast. Easy. Really handy (especially with scp). And it's secure. Don't tell me "yeah but if your client computer gets rooted, the bad guy can grab your private key". First, LIDS can hide the private key. Then, you can add a passphrase if you want. Next, if your computer gets rooted, the bad guy can always install a keyboard sniffer.
Try DSA authentication. It rules. And it solves the problem of people chosing trivial passwords. Once you only use DSA authentication, you can disable password authentication in your sshd_config file.

-- Pure FTP server - Upgrade your FTP server to something simple and secure.

www.lids.org
Very nice implementation of MAC. Not as flexible as the NSA's scheme but it's useable right now and greatly limits the amount of damage an intruder with root access can do. Highly recommended for any system.

If people stopped giving root God-like powers then problems like this wouldn't crop up. Patches like LIDS help put root in a jail. Someday we can pray that root, and all the trust and power that goes along with UID 0, will go away completely.

www.lids.org

I am running it on a test system and I am extremely impressed. It implements capabilities allowing you to assign least priviledge so if someone gets root on the box they still can't do anything. No longer do you need to open yourself up to attack just because a program needs to bind to a low number port, for example. It's a huge boost to the security of any Linux system. This plus the standard techniques used to secure a box can really lock things down.

I'm looking forward to learning more about the FreeBSD security feature, though. This is the first I've heard about it and I thank everyone for the pointers thus far. This has been a really interesting discussion.

A couple links to throw your way before the rest of you start spewing all the grandios wonders of the ACL systems. BTW, NT's ACL lists don't fit the bill anyway. I've worked with them, and they're a pain in the behind.

• EROS OS Essays on Capabilities Systems
• Linux Intrusion Detection System

EROS has some serious potential, folks! And if you want serious Linux security, look into LIDS...

Has anyone ever come up with the mods to Linux or one of the BSDs to allow for OS-level triggers on the filesystem? I would much prefer to have a syslog entry get fired off the moment that anyone opens /bin/login in O_RDWR mode than have tripwire discover it minutes later.

Yes, there is such a thing for Linux, and it is called LIDS. It is basically a kernel patch that, along with a user-mode administration program, does what you just described and much more. For example it can send a message (directly from the kernel!) to somebody on some IP address (which you preconfigured) instead of logging to syslog, also with the help of the capability support in the kernel you can limit the access to certain group of system calls. Check it out!

I really don't think this is accurate; I know there were a number of local exploits in the past 6 months that affected all BSDs, including OpenBSD.

Now, this might just be a matter of hair-splitting; perhaps OpenBSD doesn't install any of the vulnerable BSD utils by default.

If that's the case, it's not a fair comparison, since RedHat has a number of different installation levels available.

That said, I'd like to see things like LIDS incorporated into the Linux kernel, available for all to use. That would go a long way towards helping make Linux distributions more secure, if they'd at least turn on some of the openwall stuff (which has supposedly been incorporated into LIDS).

--

www.lids.org

I'm a little confused. While I've looked for details on how to verify that a system is B1 compliant, I only seem to find vauge references.

I don't see the practical difference between a system that's B1 compliant and one where ACLs are added (1) (2) and all accounts including root are severly restricted -- and both of these security measures avaiable under Linux.

I realize that verification testing is done on a per-installation basis, so many of the details will be specific to that installation. ...yet, all security is specific to the tasks and installation, so this doesn't clear up anything.

The general goals including many specific "the system shall" type directives, should be readily available...but where?

Tools like LIDS under Linux and other kernel patches to handle ACLs seem to do the job...so what's missing?

I'm a little confused. While I've looked for details on how to verify that a system is B1 compliant, I only seem to find vauge references.

I don't see the practical difference between a system that's B1 compliant and one where ACLs are added (1) (2) and all accounts including root are severly restricted -- and both of these security measures avaiable under Linux.

I realize that verification testing is done on a per-installation basis, so many of the details will be specific to that installation. ...yet, all security is specific to the tasks and installation, so this doesn't clear up anything.

The general goals including many specific "the system shall" type directives, should be readily available...but where?

Tools like LIDS under Linux and other kernel patches to handle ACLs seem to do the job...so what's missing?

http://bastille-linux.sourceforge.net/

http://dwheeler.com/ secure-programs/Secure-Programs-HOWTO.html

http://i30www.ira.uka.de/SawMill/index. html

http://immunix.org/

http://medusa.fornax.sk/

http://oss.sgi.com/projects/ob1/index.ht ml

http://soledad.cs.ucdavis.edu/

http://users.ox.ac.uk /~mbeattie/linux/ANNOUNCE.mac30-20000214

http://www.data.slu.se/bifrost/index.en .htm

http://www.guug.de/~winni/posix.1e/

http://www.lids.org/

http://www.rsbac.de/rsbac/

http:// www.securecomputing.com/archive/press/2000/nsa_faq _secure_linux.html

http://www.trinix.org