OpenBSD 3.0 Release, Interview with Theo

mvw writes: "Here is an interview with OpenBSD's Theo de Raadt. Interesting is his comment on Soft Updates and the comparison to the rivaling Journaling file systems technology. Further he links to a very interesting paper by some Soft Updates researchers." And although OpenBSD 3.0 has an "official" release date of December 1 for whatever reason, it seems to be available by FTP or CD already. Lots of changes since 2.9.

Comments

This is a very good thing!

[InterruptDescriptorT]

SECURITY FIX: fix buffer overflow reading queue file in lpd

For those running OpenBSD, especially as a gateway/firewall/NAT box, this is an important fix. I am running 2.9 with this patch added, and my snort logs tell me (judging from the number of attempts) that this exploit is a fairly commonly tried one. In November alone, there were at least 30 lpd overflow attempts on my machine. Granted, not most people have lpd open to the world, but I can imagine a few people might want to do remote printing from work, etc.

Re:This is a very good thing!

[smack.addict]

Why in the name of all that is holy would anyone have lpd running on a firewall?

Re:This is a very good thing!

[Greg W.]

1) logging to paper; so the cracker can't totally erase his trail

That doesn't require lpd. Just add the line printer's device name as an additional target in syslog.conf.

Or run a teletype console, and log everything important to the console. (I've actually seen a setup that used that. In production. In 1996.)

Even if you do use the Unix print spooling subsystem on your firewall, you should not have the lpd port (515/tcp) open on the public network interface(s).

Fixes

[jeriqo]

Actually, OpenBSD 3.0 was available for download since nov 25th, and a few patches (security fixes) are already available.
Here is the list: http://www.openbsd.org/errata.html

Don't forget to update to OpenSSH 3.0.1

-J

Release Date

[Accipiter]

And although OpenBSD 3.0 has an "official" release date of December 1 for whatever reason, it seems to be available by FTP or CD already.

Probably because they want to avoid a fiasco like the last tremendous release mess that michael caused.

It's not uncommon for "official" releases to be after the initial release. It's like when a large department store has a "GRAND OPENING". In many cases, the GRAND OPENING is about a week after the store actually opens. Or if the store opens during the week, the GRAND OPENING will be on that weekend.

The origin of OpenBSD

[Gopher]

As I sit here waiting for my copy of OpenBSD 3.0 to arrive, I've been reading the exchange of emails between Theo and the NetBSD core team, which is a history of how OpenBSD came to be.

If you haven't read them before, it's quite a read, and a good lesson of how personal politics can fragment a collaborative project.

Here's the link: http://zeus.theos.com/deraadt/coremail

Re:The origin of OpenBSD

[Syberghost]

What was amazing to me about them is the fact that Theo proudly links to them as proof that he was being entirely reasonable and they were being discriminatory, but the emails show quite clearly that he was completely unwilling to make a simple promise not to be an asshole after having demonstrated a history of pissing people off.

He's got a right to be an asshole, and god knows I'm the pot calling the kettle black, but to link to those emails and think they provide vindication is heavily disconnected from reality.

Re:The origin of OpenBSD

[rho]

I think Theo's abrasive nature is just that--his nature. He isn't willing to change his way of dealing with people.

Often I'll wish people would just simply way what they think, with no prevarication; and when somebody (like Theo) does exactly that, I get squeamish all of a sudden. "Ooh, I can't believe he said that..."

Theo has a habit of speaking his mind. Dealing with him is probably a chore, but a worthwhile one.

Re:The origin of OpenBSD

[dghcasp]

Notably absent from the email exchange are any of the emails, ICB logs, or anything that show the basis for the whole problem.

Basically, Theo had a history of being abusive and petty to anyone who didn't meet his standards of cluefulness. He pretty much admits this himself in the interview. This was alienating a large number of NetBSD developers who ended up leaving the project (I was one of them.)

The Core team repeatedly asked him to tone it down; their feeling seemed more of a "anyone who wants to help with NetBSD will be welcome," instead of "You must be this elite to code NetBSD." Theo maintained that he was doing nothing wrong.

Eventually, they shut Theo down, which is where the email thread starts. A large part of the thread deals with Theo's requests to regain CVS access. The Core group was willing to submit his code as patches themselves, but Theo would only submit code if he could have CVS write access. Core was worried that Theo might decide to get "revenge" by damaging the CVS tree; This might seem worry-warting, except they all knew that Theo had been previously fired from a SysAdmin job at the U of C for doing something like that.

Eventually, Theo started OpenBSD and now has his own sandbox where nobody can tell him what to do. In the end, I guess that's good, because both OpenBSD and NetBSD regularly crib from each other's trees anyways and people now get the choice of whether they want to deal with Theo or not.

Re:The origin of OpenBSD

[aussersterne]

I read them and got exactly the opposite view. It sounded to me like he was a regular guy getting the shaft and not wanting to take it lying down. And that little clip from IRC where he said:

Then I guess you are just stupid.

That made me laugh like mad. I love it. Sounds like me. Sounds like my friends. Hey, he cycles. He caves. He founds OpenBSD. He speaks his mind. He has a sense of humor. He sounds cool, not like an asshole at all.

Some of the other people I was reading... Like the guy who kept on about professionalism and representing your organization, even in private e-mail... sound like pricks/assholes to me. I've had to deal with people like that -- people who feel like the dollars and the "drive to succeed" are all that matter and that individuality and honesty have no place in America.

But then, I will never sell me soul to my employer or anyone else, no matter how much cash or recognition it would get me. Guess that makes me a commie. ;) Of course, the whole open-source world has been accused of being nothing more than a communist plot...

Rant, rant, yaddah, yadda...

I dig Theo. OpenBSD just scored personality points in my book.

Re:The origin of OpenBSD

[scrytch]

Then I guess you are just stupid.

That made me laugh like mad. I love it. Sounds like me. Sounds like my friends. Hey, he cycles. He caves. He founds OpenBSD. He speaks his mind. He has a sense of humor. He sounds cool, not like an asshole at all.


This sort of social stuntedness is what you find novel, fresh, and daring? Cripes, it's just the typical petulance one normally comes to expect from this guy. Churchill could be quite an asshole, but he had style (e.g. "when I wake up, I'll be sober") Theo's an organizational genius, not a half bad coder, he's probably even nice to his own team ... but he is not only utterly intolerant, he is vindictive, and it's precisely why NetBSD gave him the boot.

There are a lot of stupid people out there. Most of them aren't even worth dealing with. But it certainly doesn't make one an iconoclast to throwing around petty insults to prop up one's feelings of superiority. It makes for a pathetic maladjusted loner ... or for those who have to witness this behavior day after day, just an asshole.

MandrakeBSD?

[timothy]

a) Theo and company (good company) don't need or seek new users just to be popular. They like doing what they do -- I know that. Don't take what I'm about to say as marketing advice to them, so much as a pleasant wish. It doesn't impose an obligation or demand on the OpenBSD guys, and I know it. Still ...

b) I'm surprised (not to say hurt, disappointed and disconsolate) that no one (am I wrong?) has come out with the equivalent of Mandrake to at least one of the BSDs -- and by equivalent I mean in a certain superficial but important way: user-friendly, pretty install, emphasis on user experience, intelligibility.

c) Really, I'm just talking about the install. Something with some graphical flair, built-in help system for new users, and a game or two, or a little slideshow, or some interesting history text files, *something* built in to play while slow parts of the install proceed. No accounting for taste, but I think there are a lot of good graphic artists (all the Ximian stuff, for instance, and many great KDE examples) working in the world of free software. (Hey, I also like the BSD art, so obviously I am open for attack by the art critics;)).

I name Mandrake as my prototype here, just because I happen to like their stuff -- RH also makes a pretty install, not quite as cute, and so do several other distros. But Mandrake is in Walmart, which suits my example ("Walmart: making things accessable to the masses")

Cheers,

Tim

Re:MandrakeBSD?

[wiredog]

Why? Mandrake is aimed straight at the Desktop. RedHat aims at Windows NT users. The BSD's aim at unix sysadmins who Know What They Are Doing. Open/Free/Net don't need a User Friendly graphical install interface because their current interface is friendly to the users they aim at.

Re:MandrakeBSD?

[Pope Slackman]

Besides, adding GUI doesn't necessarily represent a "dumbing down" of the product. The installer can still ask the same questions, but in a more user friendly manner.

I see that as being a pointless waste of effort.
Why would it be good to work on a graphical installer (which entails difficult and failure-prone things such as video device detection) that does exactly the same think as a console installer?
The only benefit is that it would look prettier - installation would still remain just as "difficult"[1].
The utilitarian console installer works fine, and I see no reason waste man-hours on changing it, when that same time could be spent improving important things.
Cute graphical installers are just frippery.

C-X C-S
[1] Difficult in quotes because I've done several OpenBSD installs and never found it to be any more difficult to install than Linux or NT.

Re:MandrakeBSD?

[Pope Slackman]

And that is why the *BSDs are an "also-ran".

Believe what you like.
I'm not even sure the OS race has started yet, let alone ended.
Besides, not everyone is after "world domination", some people just want a secure, reliable OS.

The same can be said of cute ncurses-based installers. Why not just make everybody edit a text file on the boot floppy?

Some things (NIC detection, for example) are better when interactive, and a console-based installer provides that capability with a minimum of work on the coder's end, and maximum compatibility for the user.
(But in some cases (like setting up a bunch of identical boxes) a text install config file can be /really nice/ for automation.)

Or of EMACS; why not just use ed? All that extra functionality is just frippery.

Where'd that come from? The original post was talking about making a GUI installer
that had no more inherent functionality than the console installer.
Your analogy is bogus, as it misses my original point.

C-X C-S

Re:MandrakeBSD?

[Jason Earl]

VMS is the original anti-UNIX. It later added some general POSIXy behaviour simply because everyone was using UNIX. Windows NT also had the stated goal of becoming "a better UNIX than UNIX," but they certainly haven't spent much time actually trying to be Unix compatible. Their POSIX layer is a joke, and they don't even have a decent way to fork() for crying out loud.

Besides, while Microsoft almost certainly is looking into "borrowing" portions of BSD code (which will then magically become innovative), they aren't ever likely to actually release an OS that is Unix like. Part of the fun of the BSDs, Linux, and Commercial Unixen is that it usually isn't too much trouble to port your software from one of these platforms to a different one. This is precisely what Microsoft wants to avoid. Microsoft wants the equivalent of a one way valve when it comes to software portability. They want for it to be easy to port from Unix to Windows, but they want it to be impossible to port from Windows to Unix. Clearly shifting to a BSD based OS would work against them.

Re:MandrakeBSD?

[psxndc]

Lets get over ourselves and the secret handshakes to get in the clubhouse where non geeks are not allowed.

While I see where you are coming from, you need to understand that some people do not feel this way for good reasons. Lowering the barrier for entry decreases the signal to noise ratio.

Imagine a super hard to use OS where only five people can use it. Any conversation about it among the five users would most likely be useful banter. Now imagine they lower the barrier for entry. Now joe bob idiot is asking "How do I use my printer?" "Why do I need to login?". Conversations devolve into hand-holding sessions and the original five now can't spend the time doing what they used to do: be productive.

It really comes down to the question of "Do you want your OS to be useful to you or useful to others?" and if you choose the latter, who are the others? Joe PhD engineer, billy high school smartkid, or grandma who thinks TV's are new fangled devil boxes? As far as OpenBSD is concerned, I got the impression Theo wanted to make it for himself (and his group of developers). If other people wanted to use it, great, but they are on their own if they don't take the time to delve into the nitty gritty like Theo and company do (not the extent obviously, but you get my point)

psxndc

Re:As much as I

[^chuck^]

sigh, its been well explained that you don't need a journaling filesystem to be safe with transfering data to the harddrive. In fact, if you're clever enough, you can even get away safely writing without having to hold the entire system up (hence, softupdates). If you actually look through the interview, you'll find Theo actually pointing you to resources that quite seriously make this point (journaling not needed).
take a look at this
it can be frustrating being right, all journaling really seems to do is attempt to fix the problems ext2fs has by laying another piece of code on top of it, instead of fixing the primary problem, that is that ext2 is broken as far as the BSD hackers are concerned.
Is waiting for fsck to finish really that much of a problem for you?

pf : an excellent packet filter

[chrysalis]

The big new feature in OpenBSD 3.0 is pf.

• Interesting stuff in pf over ipf : the configuration file accepts a very similar syntax, but with very handy shortcuts, especially expansion. For instance you can write { pop,smtp,imap } in a rule to specify a list of ports, instead of creating multiple rules. It also accepts macro substitutions. You can easily write very clean configuration files.
• Interesting stuff in pf over ipfw/ipfiler/iptables :
  • scrub : just give an interface name, and pf will "normalize" everything coming to this interface. Packets will get cleaned up and reconstructed : your local network will only see clean packets, nothing that could be dangerous for badly written IP stacks.
  • tcp state modulation : this feature dynamically remaps tcp sequence numbers, to give the excellent entropy of OpenBSD stack to all your traffic. It means that servers running Windows, badly configured Solaris or older FreeBSD versions can be protected from session hijacking, even through their stack has weak sequence randomization.

pf seems to be very stable so far. Just don't forget to apply the related errata if you're planning to use IPv6.
Another great feature of OpenBSD 3.0 regarding network filtering/routing is the integration of AltQ, that brings quality of service to your IP traffic. It basically has the same (but very flexible and efficient) algorithms and class system that Linux has. But it's very nice to see it in OpenBSD.

Re:pf : an excellent packet filter

[chrysalis]

* For NAT:

OpenBSD 3.0 has a transparent ftp proxy called "ftp-proxy". You have to run it through inetd (or any super server. I use it with tcpserver) . It listens to a local port, and you just have to redirect outgoing traffic for port 21 to the local ftp proxy port. It allows active and passive connections to NATed internal hosts.

If it can help, my /etc/nat.conf file is :

rdr on vr1 proto tcp from any to any port 21 -> 127.0.0.1 port 8081
nat on vr0 from 10.1.1.0/24 to any -> 195.132.209.36

I start ftp-proxy like this :

/usr/local/bin/tcpserver -H -R -q 127.0.0.1 8081 /usr/libexec/ftp-proxy &

*WARNING*
ftp-proxy has a nice security feature to only accept anonymous sessions (-A). But don't trust it : clients can bypass the restrictions with some buggy servers (the flaw works with proftpd and ncftpd. it doesn't work with pureftpd) .

* For firewalling (without NAT) :

You have to explicitely open some ports for active connections. For the minimum number of ports : choose at least twice the max number of simultaneous sessions you need. Open them on the firewall. Then, force your FTP server to only use these ports. On Pure-FTPd, it's with '-p :', example :

pure-ftpd -4 -p 50000:51000 &

(don't forget '-4' for OpenBSD) .

ISO download

[Syberghost]

As Theo says himself in his interview, people who don't like his model of selling the ISOs are free to make their own. This will hopefully quiet the stupidity that usually follows this announcement:

As usual, ISO images here.

I'm waitting on our 3 OpenBSD CDs

[alexhmit01]

We use OpenBSD exclusively for our web servers. We moved our database servers from OpenBSD to Linux. I look foward to migrating our systems (some 2.8, some 2.9, one that I broke trying to do a fancy 2.8->2.9 upgrade...) when our CDs arrived. We figured that we use OpenBSD a lot, owning a bunch of CDs was worth it. Alas, it is is still cheaper than the copies of RedHat that we pick up.

OpenBSD has a real problem that I was never able to resolve, this makes it worthless for a database server. The machine is quite "efficient" with memory, which let it run with very little memory. However, with a lot of memory (our db servers normally have 1.5GB -> 2GB, I LIKE giving PostgreSQL lots of buffers and sortmem) there is little documentation on tweaking the system. I even contacted the developers in charge of the SysV memory support, etc., and they thought I hit the crack rock a little to hard.

For web servers, however, I'm quite comfortable with our OpenBSD servers sitting open on the Internet. I'm terrified of a RedHat box not being hidden. As a result, I keep the database nice and hidden.

Linux blows OpenBSD's performance away. This is a matter of Linux focusing on performance. However, for web servers (that for us just run PHP, mod_rewrite, and some other toys) I don't care... When I need more web serving power, I buy another web server for $2K. Having SSL built in to Apache is nice, and the ports is too fucking slick.

BTW: OpenBSD seems to run quite nicely on my Penguin Computing 1U servers... :)

Alex

I expect to keep our production servers on 2.9 for 2-3 months, but move development to 3.0.

Re:As much as I

[greygent]

> Is waiting for fsck to finish really that much of a problem for you?

Yes, actually, when you're dealing with servers with 100's of gigs.

Re:file systems

[mvw]

Some links:

• The article given does some comparisons:
  http://www.usenix.org/publications/library/proceed ings/usenix2000/general/full_papers/seltzer/seltze r_html/index.html

• Then we have the Matt Dillon Interview, where Soft Updates covered as well (point 3):
  http://www.osnews.com/story.php?news_id=153

• Here is a nice french article:
  http://www.freebsd-fr.org/docs/fr/others/systeme-f ichier/

• And this is a series of articles on file systems:
  http://www-106.ibm.com/developerworks/linux/librar y/l-fs7/

• This is the original FFS paper:
  http://docs.freebsd.org/44doc/smm/05.fastfs/paper. html

Re:file systems

[WasterDave]

In the beginning were file systems. A file system took what you wanted it to write and put it on the disk for you. And it was good.

But the users moaned "speed, we must have more speed" and indeed their call was echoed by the admins. So write ahead caching was invented so the users calls would return sooner, and once again all was peaceful with filesystems.

But then one day someone tripped over the power cable and the OS died. On recovery it was discovered that the filesystem was completely borked (due to some of it being in the write ahead cache when the power died) and lots of data was lost. There was much wailing and gnashing of teeth so the journal was invented. A journal writes a list of things that the file system will do when it gets around to it, but writes this list to the drive so it doesn't get lost when the power is lost. Because the list is all in one place the journal is fast and once again there was peace.

Over the years slowly everyone, even Microsoft and even the Linux kernel made themselves journals but the BSD hackers (Greg Lehey?) realised you didn't really need one if you were careful about the order in which you wrote to the disk. And hence softupdates were invented, and are (arguably) very slightly faster. But mostly just different. Like Reiser, but that's another story entirely.

Gottit? Synchronous writes good, but slow. Async writes bad, but fast. Journaled writes good, and fast. Softupdates good and fast without a journal.

Dave

The reason for the early release:

[snake_dad]

was that the cd's were available earlier than expected, according to this message from Theo at the OpenBSD Journal.

Btw, the headlines from this site are available as a slashbox, just check the box in your /. preferences.

Snake_dad (who runs Linux, Winedose, Novell 3.12 and ... OpenBSD :-)