Slashdot Mirror
How Well Do Most OSes Handle Resource Management?
schlika asks: "After getting some troubles with a highly loaded Web server running Linux 2.2.12 I read some infos about its max thread/processes limitation and 'not-so-great' virtual memory management. Could some of you comment such issues with other OS's such as FreeBSD (which everyone says it's better for that task without ever giving explanations), OpenBSD, NetBSD, Solaris, Linux2.4 and others? Please give real world examples/comparisons if possible."
Well, FreeBSD (and others) have login.conf for specifying resource limits.
Admittedly, these only operate per-user, not per-process. A lot of people choose to run important processes under a specific user anyway (news, bind).
Cheers,
Si
Disk caches statically set at boot-time, to change the cache size needing a reboot. Calculating how much memory your processes would need, and using the rest for cache. Finding the optimum value was not easy, particularly if process sizes varied over the day.
We also had to cope with resource restrictions such as concurrent socket connections, concurrently opened files, Inode tables and so on. This could only be changed by tweaking a kernel parameter, and some could not even be monitored easily. It wasn't always easy to guess the best value to set these to, given that increasing them increased the memory used by the kernel, the memory could not be used elsewhere, and memory was not as cheap and plentiful as it is now.
They don't know they're born today! And yes, I am a Yorkshireman (born on Yorkshire day too!).
I starting working at a FreeBSD shop (flyingcroc.com, parent company of sextracker.com) coming from a linux background.
We do some pretty amazing things with freebsd. We have tons of servers doing between 100-300 reqs/second with apache, others that have done 40-50Mbit sustained (on a 100Mbit NIC, Intel hardware), and all kinds of other crazy stuff.
I've ssh'd into various machines that were getting hammered and the load was in the 700s with disk/swap/ram/cpu all taking a beating, and I was still able to do what I needed. When I've dealt with linux machines under conditions nowhere near as bad as that, it was a total nightmare to even get logged in, let alone do anything.
I still use linux for my workstation, because I love the desktop goodies, games, and debian, but for high performance servers it's hard to beat freebsd.
Unix allocates CPU time clumsily, nice and pbind are about as much control as a sysadmin has over a running process, other than stopping it altogether and restarting it. Contrast this with OS/390 or VMS where the sysadmin can control exactly how much CPU a process gets, the size of its working set, migrate processes around between nodes in a cluster. IBM have a tool called the "Work Load Manager". It is able to configure your system based on what you want to do, not how you want to do it. For example, you say that this batch job must complete by this time in the morning, this class of transaction must complete within this time, and this group of users get mo more than 10% of CPU in the morning, and 30% in the afternoons, and WLM will configure your cluster, if it is physically possible, to do it. You can run a mainframe class OS at 90% of the machines capability consistently, a Unix system rarely exceeds 30% of its capacity when averaged over a period of time, it simply spends too much time either waiting for things or trying to manage its own workload. And what's worse, the CPU gets involved in every I/O in a Unix system, because of the way buffers work. Every disk block gets transferred by a CPU through an operating system buffer. When you edit on a UNIX box, every single character goes to the CPU and gets echoed back. And on the network, even character gets a packet sent back and forth. VMS deals with "record" - whole lines of text, even at the network protocol level.
And don't even get me started on the lost+found directory. You don't get that on an industrial grade file system, because it's journalled to ensure consistency.
Thankyou for listening.