Slashdot Mirror

1.) You did not understand what I said,
I didn't say that you should scrape your DMZ for secure apps.

I said that developers should focus on security, because even a DMZ can be broken into, AND THEREFORE DEVELOPERS AND USERS SHOULD FOCUS ON SECURITY TO CONTAIN THE IMPACT OF THEIR HOLE IN THEIR DMZ CONFIGURATION.

And DMZ should not equal to a free-for-all zone.

I know that security is relative, however if you can put the bars higher with relatively low effort you should do it.

2.) You like your features till the day you are blamed for the major fuck up.

3.) I hope you have realized that the trend in politics and future law making is going towards accountability in security and therefore this topic will get much more important for the survival of companies.

4.) And btw. performance and security is possible[1], if you just reduce the middle-ware-middle-ware and do it right for the first time.

[1] https://www.fefe.de/gatling/

[1] http://www.fefe.de/

That is the most Spartan website I've seen ever, and I am talking about the source too.

Why are they much better alternatives?
The software written in (*) has no vulnerabilities?

Choosing a language does not really address security, because that choosing will affect how the programmer thinks about security and possibly the less experienced programmers will slack on "programming for safety" paradigm .. because the language does everything for the programmer.

For example:
Please have a look at fefe's gatling[1], an incredible fast http-server, with only very few security problems in the past - written entierly in "C". Also the funny thing is that certain of these highlevel languages will use bindings to these older libraries written in C.

So you will be bitten again.

From all information I overlook I can say, yes in "C" it is incredible easy to make simple errors with hugh consequences - choosing types for example. However "C"-programming can be made more secure with a strict application of certain rules especially on "forbidden" & dangerous constructions. The missconception why "C" is deemed as an insecure language is that much of the code in use stems from the "ancient" times, when such code was mostly not exposed to the raw unforgiving "force" of the internet.

Also there was not such a "zoo" for other different programming languages, so much of the software was implemented using "C". This effect is similar to todays "I use java now, I don't need to take care of security".

The different incarnations of "C" standards also play their part, similar to the "Perl-Mageddon" if you do not have a concise standard about how a programming language will be "interpreted" or "translated" you are deemed to introduce errors. Imagemagik is bloated & ancient, two aspects that are problematic. Fefe adheres to his own standards, that bloat and complexity are the real threats for security. (dietlibc vs. libc). And he is often correct on this topic.

[1] http://www.fefe.de/

The point with using c++ is that gcc can already be compiled with a c++ compiler so they can now start using the c++ without a total rewrite/redesign.

No.

C and C++ diverged many years ago. C++ is not a strict superset of C. You can not compile a C program with a C++ compiler and expect it to behave correctly in general. Yes, there may be ways of hacking around it, but that would by definition lead to complex and unmaintainable code.

C++ is not even a solution looking for a problem. It is too difficult for human beings to use effectively, and there are still many deficiencies in the official language.

gcc is a critical piece of infrastructure in the Free and Open Source world. This is a disaster waiting to happen.

While most programs do use libc others try to either avoid libc altogether (for example: http://blog.ksplice.com/2010/03/libc-free-world/ ) or use other "diet" versions of libc (for example: http://www.fefe.de/dietlibc/ )

What about those programs? Will they get to enjoy what the article talks about?

Okay, I've now had time to do a bit of digging. I think I was thinking more of these graphs regarding performance:
http://bulk.fefe.de/scalability/
I do concede, however, that (1) these have nothing to do with netcraft, (2) far from show FreeBSD as the winner in all cases, and (3) are more than a few years old. I have not been keeping up to date.

I still haven't found the info on "server market share"; I'm positive I used to have good info that a respectable number of truly large systems, but I also concede that eBay, Amazon, and Google are now all apparently linux-hosted.

I'm sorry, but I don't see the pertinence of the link to the "most reliable hosting companies". The other link is interesting; the preceding pages simply show that various OS'es lead for various tasks; there clearly is no overall leader (the GCrypt test on page 3 is appalling).

So yes, I was over the top up there. I stand corrected.

It's been done before, depends what you need from the library : http://www.fefe.de/dietlibc/, http://www.uclibc.org/, etc (but these are clearly targetted at embedded systems).

That being said, dependency hell is the main reason Linux cannot get ahead of Windows or Mac for the masses - the abstraction layer may not be as optimisable as on Linux, but you can distribute small binaries and be _sure_ they work out of the box with no issues.

bmo is right, just stay...you have no real reason to switch, unless you want a different package management system, which doesn't seem to be the case. I was thinking about switching to BSD myself and decided not to because I was happy with my Fedora install and the BSD's are a tad bit behind on performance compared to Linux according to these benchmarks ( I know they are a little outdated, but do include kernel 2.6).

When are we going to start countering the current trend?

When the tech-hostile ultraconservative 60-somethings from whom the parties that bring up such laws draw the majority of their voters have died off. The CDU/CSU parties which pushed this horrendous law is highly popular amongst people over 60 with low education. (Source: Zeit.de, screenshots courtesy of this excellent blog)

And I suspect the situation in other countries to be similar. Those people do not understand what the Internet is and how it works, and they have an unwavering trust in the state and government. A terrible combination.

When are we going to start countering the current trend?

When the tech-hostile ultraconservative 60-somethings from whom the parties that bring up such laws draw the majority of their voters have died off. The CDU/CSU parties which pushed this horrendous law is highly popular amongst people over 60 with low education. (Source: Zeit.de, screenshots courtesy of this excellent blog)

And I suspect the situation in other countries to be similar. Those people do not understand what the Internet is and how it works, and they have an unwavering trust in the state and government. A terrible combination.

When are we going to start countering the current trend?

When the tech-hostile ultraconservative 60-somethings from whom the parties that bring up such laws draw the majority of their voters have died off. The CDU/CSU parties which pushed this horrendous law is highly popular amongst people over 60 with low education. (Source: Zeit.de, screenshots courtesy of this excellent blog)

And I suspect the situation in other countries to be similar. Those people do not understand what the Internet is and how it works, and they have an unwavering trust in the state and government. A terrible combination.

SPD

Also take a look at the excellent stralloc which is also part of libowfat.

Better yet user better, more convenient string APIs in the first place. One of them is in libowfat. As you pointed out strn* doesn't protect you from buffer overflows. There are a mighty fine number of good programmers who you'd call braindead. People make mistakes, and if an API helps in that regard then that API is better than the stdio garbage. That's why OpenBSD uses strl* functions instead of strn*.

Because uClibc has(had) inferior threading and performance. And it is(was) missing the GNU extensions that many popular FOSS projects depend on.
There is also newlib and dietlibc. In many ways I find newlib to be better than uClibc, although I still tend to use uClibc for projects because it's good enough and we already use it.

And for those kind words I'm going to post a follow up to my original post of more relevant changes between the 2.4 and latest 2.6 kernels (I'll try and add a few more words after each point).

Kernel configuration was overhauled. Outside support for more GUI menus, it now means you no longer have to do make dep after changing something. Further building modules outside the kernel tree is now not so baroque. The time to build and partially rebuild kernels also dropped. Building a kernel in parallel (i.e. using more than one CPU during the build process) works better.

Better support for configuring out unneeded parts of the core kernel on embedded systems. You can see the seeds of this going mainline in a git commit on 2.5.70. There is an outside project called Linux Tiny that produces patches aimed at being able to configure out features not needed for embedded systems. Over the course of 2.6 many of these patches have trickled into the mainstream kernel.

I mentioned that 2.6 scales better under load in my previous post. Here are some benchmark comparison graphs of 2.4 versus 2.6 kernels (the graphs also include comparisons against the BSDs but you can see that Linux 2.4 had some serious problems that Linux 2.6 addressed).

The kernel is now (on systems where there is reliable device discovery) able to automatically load the modules it needs to drive hardware. No more having to adjust static lists of which modules need to be loaded.

udev was introduced. This change meant that the entries in /dev were no longer static. In 2.4 all possible device entries (even for devices you didn't have) were shown in /dev and their major/minor numbers were fixed (which was causing problems as new devices were turning up - what major/minor number do you give them?). Additionally the other dynamic /dev system (devfs) was whittled away and killed off.

FUSE support (LWN article about FUSE). Allows filesystem drivers to be written in userspace. Currently the best Linux NTFS driver is written using FUSE and it allows fun things like sshfs. Might be handy if you need users to be able to configure where data is stored remotely, you are writing your own filesystem or you need to support writing to NTFS formatted USB disks...

There is better CFS (Samba/SMB/Windows File Sharing) support. NFS version 4 support was also added.

cpufreq support. The kernel can clock down the CPU speed (usually by changing voltages via some hardware interface) to save large amounts of power. This can be done in response to work load so you run at full speed as often as possible and then when things are quiet you scale down to the lowest setting (you often save the most power when doing absolutely nothing so it pays to finish things as quickly as possible).

Any switch from 2.4 to 2.6 will of course require userspace changes (updated modutils, udev, later gcc, later glibc).

There is also davej's post Halloween document discussing changes from 2.4 to 2.5. This is very detailed and is another excellent reference.

Many many other things have changed too (e.g. ALSA support for sound has been added) but I have tried to keep the ones mentioned at least tangentially related to the original scenario :)

There seem to be a few BSD tools with the aim of building smaller collections of binaries in a similar fashion to BusyBox.

You've mentioned crunchgen but there is also embutils (which can be smaller than busy box but requires dietlibc) and there also seems to be something called beastiebox (which allows different amounts of linking). Finally there is Cauldron which seems to be a collection of tools for creating embedded BSD environments.

A couple of things that have changed, from the top of my head:

  - Those old machines had an OS dedicated to support exactly the hardware that was shipped. OTOH, Linux distros are usually generic, and detect hardware on boot.

  - There's a huge difference between a BASIC interpreter and a full-fledged multi-user OS with networking, GUI, etc.

  - PC hardware spends a lot of time during boot before actually handing control to the software. Server hardware is often a lot worse.

  - The code involved in booting those old machines was probably written by Real Programmers, who knew the ins and outs of the hardware they were programming for. It's probably horrible to maintain, but ridiculously fast. By contrast, most of the code involved in booting your Linux distro is written in a variety of languages, by a whole lot of people, few of whom are Real Programmers, and it's probably optimized for genericity and maintainability, rather than efficiency.

Having said all that, you _can_ boot Linux ridiculously fast. I used to have a 486 where the boot process basically consisted of (1) POST (2) Load GRUB (3) Load kernel (4) Run init (5) Run login. Pretty much a normal boot process. What was special about it is that the kernel had all required modules compiled in, and nothing else. init (IIRC) only spawned a single tty and ran login. And both init and login were BusyBox statically linked against dietlibc. From GRUB to login prompt took less than 5 seconds, IIRC. You could further improve on this by foregoing the BIOS in favor of something like coreboot.

IPV6 patch for DJBDNS

Personally, I have no need for it or the IXFR functionality at this point.

However, my understanding is that BIND's IXFR implentation breaks if you use hand-written or tool-generated zone files.

... I still recall this exhaustive report comparing several kernels' performance back in 2003 in which NetBSD pretty much beat the pants off of everybody else (note the two updates with separate graphs)...

Oops, it needs a trailing slash: http://bulk.fefe.de/scalability/

Isn't NetBSD the system filled with academics who insist upon clean, manageable, and portable code above all other standards? Too bad the NetBSD kernel didn't get judged here, I suspect it would have taken the cake.

I still recall this exhaustive report comparing several kernels' performance back in 2003 in which NetBSD pretty much beat the pants off of everybody else (note the two updates with separate graphs). The initial poor performance was due to an old revision, and upon seeing that there were some places in which the newer revision wasn't so hot, the developers fixed them and in only two weeks, NetBSD beat out FreeBSD on every scalability test. Their pragmatism and insistence on quality code finally paid off.

Ever since seeing those charts, I've been waiting for Debian/NetBSD to come out...

Let's have multiple forks of glibc then!

You mean like uClibc, klibc, dietlibc, etc?

And the kernel!

You mean like all those various kernel patches people use? Or maybe the BSDs vs Darwin vs Linux vs Solaris?

and coreutils!

You mean like BusyBox?

A foolish consistency is the hobgoblin of little minds.

For years I have personally found NetBSD to outperform Linux, sometimes the difference is dramatic. NetBSD's unified buffer-cache is a phenomenal performer and NetBSD scalability is awesome too. Plus add to that the fact that NetBSD is developed as a whole system, with kernel, user land, file system layout, package management and documentation as a complete system. The BSD's are not bolted together. Each part is designed to work with the others from the outset by the same group of like-minded people.

That actually really shows when you compare with any Linux which has a separately developed kernel, separately developed user land, separately developed file systems and file system layouts, separately developed package management and separately developed documentation. Linux does not come together as cleanly as the BSD's do.

Try using a BSD for a few months and you might only turn back on the odd occasion that you need to use Linux for some fringe requirement.

Take a look at some Linux vs NetBSD benchmarks:

http://bulk.fefe.de/scalability/

Search for "re-check NetBSD" on that page, to see incredible improvements which the NetBSD project made in just weeks. Taking NetBSD beyond performance that the Linux crowd has been working on for years.

NetBSD is by far the best Xen platform too:

http://users.piuha.net/martti/comp/xendom0/xendom0.html

These benchmarks are indicative of what I personally see with my various uses for UNIX-like systems when performance is apparent.

Apparently, this topic has been discussed on /. before, and the most intriguing tool I became aware then was minit (http://www.fefe.de/minit/). Here's a nice posting about its most important properties: http://developers.slashdot.org/comments.pl?sid=808 17&cid=7115996

OpenBSD fast? Is this a joke?

Given these benchmarks, I doubt that speed is a major concern here, at least with current operating systems. I'm not sure about resource usage, though.

About the experience, this should IMHO be solvable by an appropriate library.

There've been discussions here and here.

The idea is to load services in parallel, and replace the ancient Init system. Windows NT supposed does this; still, it's no match for a hand-tuned minimal init system, such as this or this. I tried this back in 2003, and had my desktop open in a maximum of 5 seconds. Bios ~1 sec, Kernel load time ~ 1 sec, Services+GUI ~3 secs. This was on a 1Ghz Celeron, with 128MB RAM. It also helped that I was running Sawfish, with no desktop widgets(no menus, toolbars, anything). Just a bunch of keyboard shortcuts to launch the apps I used. I stopped installing any new distros after that - none of them could offer a better user experience than what I had. It also helps in gaining a better understanding of your system - what are each services used for, how to do file system recovery, and esoteric details.

Nope, he's right. This tells the sad story of OpenBsd very well. http://bulk.fefe.de/scalability/

Linus could have chosen some other libc to link the kernel with (just as the person in your example could have chosen to live somewhere other than in the city).

It's not the "next big thing." In fact, it's the old big thing that kept me running Windows XP on a machine at home.

That sounds like it's a bad thing.

Seriously, everything which reduces the pains of using Windows and keeps people from switching is not good. Read this: http://www.fefe.de/nowindows/

- Hurga

However, giving people a way to work around bugs in Windows makes them stay longer with Windows. That's why I consider porting software to Windows sabotage. It does not help people under Windows, in the contrary. It makes them stay longer with Windows. And while they stay, they will put pressure on others to also use Windows. It only helps Microsoft.

Correction: when _you_ start using up a lot of memory Linux totally sucks. When I start using up a lot of memory, Linux acts exactly as I expect, and better than FreeBSD. (PDF reference) Hrm. Looks like FreeBSD panics under load in it's default configuration. So sad.

From a stability point of view, Linux and NetBSD worked stable all the time, FreeBSD 5.1-RELEASE panicked under load (that went away with 5.1- CURRENT) and OpenBSD crashed and panicked even in 3.4-CURRENT. OpenBSD also surprised me with "interesting" syslog messages like "/bsd: full".

The most important OS offering async I/O is FreeBSD.
Linux 2.4 scales badly for mmap and many processes.

(FreeBSD) kqueue is older than epoll. I think Linux should simply have implemented the kqueue API instead of inventing epoll, but the Linux people insist on doing all the mistakes of the other people again. For example, the epoll guy initially thought he could get away without level triggering. The performance of epoll and kqueue is very similiar.

I'm not an expert on any of this,

That's obvious.

but what I do know is that when you start using up a lot of memory Linux totally sucks.

Correction: when _you_ start using up a lot of memory Linux totally sucks. When I start using up a lot of memory, Linux acts exactly as I expect, and better than FreeBSD.

http://bulk.fefe.de/scalable-networking.pdf

Hrm. Looks like FreeBSD panics under load in it's default configuration. So sad.

Meanwhile, I have some systems that constantly run with a run-queue length above 100.0 and are still (albeit somewhat) responsive.

Wow, this latest move is sure to rocket OpenBSD to the top.

I mean, the network performace will likely still suck, especially compared to the competition, but at least now we can monitor our servers!

Big Brother's given us this capability for years. Nothing to see here, move along.

What's with the Microsoft Excel style graphs? They're not very precise or professional-looking.
You would have thought the author would use something better like gnuplot?

The author's opinion "Personally, I still choose XFS for filesystem performance and scalability."
is largely irrelevant here and sounds like bias, although the author acknowledges this.

There is no discussion of the results. The text between the graphs only mentions superficially
what is obvious to anyone looking at the graphs.

Seems a far cry from the very nicely done BSD and Linux benchmark at http://bulk.fefe.de/scalability/

Because OpenBSD isnt' scalable.

I don't want any of my work to give anyone a reason to support companies like Microsoft who try to limit people's freedoms. That's why I develop my software on a completely free platform. So I know it works on a completely free platform. Many people using Windows don't care about their freedom. They do care about quality software and for that reason try to replace all the user space software from Microsoft with better free alternatives. This is the sole reason for the existance of cygwin.

However, giving people a way to work around bugs in Windows makes them stay longer with Windows. That's why I consider porting software to Windows sabotage. It does not help people under Windows, in the contrary. It makes them stay longer with Windows. And while they stay, they will put pressure on others to also use Windows. It only helps Microsoft.

While this text singles out Microsoft, other companies are equally evil. For example, porting the diet libc to Solaris would help Sun, noone else. Don't do it.

In the same line of argumentation, I will not modify any of my software so it works better with proprietary development platforms like Visual C++, even if I sacrifice great amounts of performance by not exploiting their features. And I ask you to do the same.

This text was taken from http://www.fefe.de/nowindows/.

-- AC

Personally, I would be happier if you didn't need an antivirus and firewall on windows to start with.

You need an antivirus on every maschine - nearly every OS has its problems (ok, MacOSX has 0 known viruses, but that's OT). Under Windows you do not nessesary need a personal firewall. I collected a few links for you ;)
http://www.fefe.de/pffaq/
http://www.iks-jena.de/mitarb/lutz/usenet/Firewall .en.html
http://www.ntsvcfg.de/ntsvcfg_eng.html
http://www.stud.tu-ilmenau.de/~traenk/zaweg.htm (this one's evil...)

Don't get me wrong: Firewalls are great - just personal firewalls aren't (IMHO). A firewall is a concept and not a program. Get a fine proxy and configure a packet-filter on it - this will give you more security than any personal firewall could!

Fefe - is that you? o_O;

np: Barbara Morgenstern - Mjisnjedschaz (Fjorden)

"It's no wonder that DragonFlyBSD is now becoming the premiere production BSD.." Dragonfly is nowhere near production quality yet. It may or may not be better than FreeBSD in the future, but your fanatism (earning you +5 insightful apparently) blinds has blinded you to the fact that not even its developers recommended for production use.

"Like it or not, DragonFlyBSD is bound to take the role FreeBSD has.." Seeing how Dragonfly has a different set of goals than FreeBSD, I cannot see how it would take FreeBSD's role ... provided it becomes better, which is not proven yet! This is like saying that Open~ or Net~ will take FreeBSD's role in the future! It is stupid.: "Meanwhile, systems like FreeBSD which have failed to make the transition to a far more threaded kernel design will lose the performance race." Just as silly as the rest - FreeBSD 5/6 now shows very good performance on MP systems. Last time a more or less objective comparison was made, FreeBSD 5.x proved to be more scalable than 4.x - and the difference by which linux won was quite negligible, if you read the whole article. So, what you wrote is one of the silliest rants I have recently read.

It's worth it, for the better network performance

Why are distributing FUD?
You are in error or it's just intentional lies?
Check those benchmarks again -http://bulk.fefe.de/scalability/

If you want to seriously compare two open-source Unix-like systems, the only instrinsic difference is the kernel. Arguing that one system is better because of the default configuration of network services, the package system, the organization of the rc scripts, and so on, is a red herring, because there is no reason you can't take all of the userspace from one system and run it on top of the kernel from the other -- and there are projects which do this.

In that light, these benchmarks are the most enlightening comparison I have seen to date. Some BSD users have attacked the methodology, but none of them has gone on to do alternative tests of their own, and the author has been very conscientious about addressing some of the criticism. The bottom line is that FreeBSD is, whichever version you choose, at best equal to Linux in low-level kernel performance, and usually slower.

When you also take into account the greater ease of use of most common Linux distributions, broader hardware support, greater availability of commercial software (yes, you may be able to run it under FreeBSD's Linux emulation layer, but the vendor is unlikely to officially support that, which matters to large corporations), and better scalability, it really isn't suprising that most people considering a free Unix-like operating system choose some distribution of Linux.

Undoubtedly for a long time, perhaps until the 2.4 kernel came out, FreeBSD probably was superior, and had a well-deserved reputation as a better choice for serious usage. For some purposes (there are some routing benchmarks that FreeBSD people always bring up, which I can't find right now) it may still be. But through some combination of the AT&T lawsuit, media coverage, and pure chance (licensing may also have played a part), the commercial support and developer mindshare swung decisively to the Linux kernel, and today it is clearly the best choice for most uses. We can wonder what would have happened if FreeBSD had won out instead -- the resulting kernel might very well be better than either Linux or FreeBSD is today -- but that doesn't change the facts about which is the better choice today.

FreeBSD, it is a server OS

Well, according to this benchmark linux 2.6 outperforms all BSDs on most network related metrics.
Admittedly the thing is quite old now, anyone know how current BSD performs vs linux?

It's not as fast as Linux 2.6 though, which pitted against OpenBSD, NetBSD and FreeBSD came out top in almost all tests.

http://bulk.fefe.de/scalability/

On the other hand, BSD does DOMINATE Linux. Everything that matters is better in BSD....who cares about a MS-imitating desktop OS? IBM and HP should get their crap together and start supporting the BSDs instead of Linux.

And someday, maybe openBSD won't be so slow.