Slashdot Mirror
OpenBSD 4.9 Released
An anonymous reader writes "The release of OpenBSD 4.9 has been announced. New highlights included since 4.8: enabled NTFS by default (read-only), the vmt(4) driver by default for VMWare tools, SMP kernels can now boot on machines with up to 64 cores, support for AES-NI instructions found in recent Intel processors, improvements in suspend and resume, OpenSSH 5.8, MySQL 5.1.54, LibreOffice 3.3.0.4, and bug fixes."
Also in BSD news, an anonymous reader writes "DragonFly BSD 2.10 has been released! The latest release brings data deduplication (online and at garbage-collection time) to the HAMMER file system. Capping off years of work, the MP lock is no longer the main point of contention in multiprocessor systems. It also brings a new version of the pf packet filter, support for 63 CPUs and 512 GB of RAM and switches the system compiler to gcc 4.4."
Why is NTFS always read only. It shouldn't be so hard to make a proper file system driver what the hell?
wake me when they have:
1) start/stop scripts, so I don't have to ps|grep|kill|...crap, what were those flags for the daemon again... to manage running processes or daemons
Well, for this one:
New rc.d(8) for starting, stopping and reconfiguring package daemons:
The rc.subr(8) framework allows for easy creation of rc scripts. This framework is still evolving.
Only a handful of packages have migrated for now.
rc.local can still be used instead of or in addition to rc.d(8).
WARNING! This girl exceeds the MAXIMUM SAFE standards established by the FDA for BRATTINESS
BSD.ru confirms Netcraft is dead!!!!!
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
as for #2, you build the patched release files on another server and deploy on production, procedure 5.4 Building a Release is in the (very nicely done) docs http://www.openbsd.org/faq/faq5.html#Release
why, they're not necessary. The flags for starting a daemon are in /etc/rc.conf and /etc/rc.conf.local, and the pid of running daemons are in /var/run or use ps ea for them. Simple and clean with no cruft is why I like OpenBSD for applicances and routers so much.
Couldn't it be said that OpenBSD and DragonflyBSD are just different distributions of BSD?
It couldn't, because they have very different kernel and base system (source code wise). They have descended from the same codebase, yes, but it was a very long time ago.
Slack and Ubuntu use the same Linux kernel, albeit with a certain combination of patches in case of Ubuntu.
Netcraft confirms it, BSD jokes are dead.
Atomic ops are limited to 64-bits for the most part (though maybe 128 bits w/fp insns we can't really depend on that). There are several subsystems in the kernel which rely on atomic ops to test and manipulate cpu masks which would have to be reformulated.
The main issue there is one of performance. We don't want to have to use a spinlock for cases where cmpxchg solves the problem because spinlock collisions can get VERY expensive once you have more than 8 cpus in contention.
Similarly the stolen bit for the pmap spinlock (reducing the limit from 64 to 63) is there to deal with a race where one thread needs to do a SMP invtlb style operation just as a new cpu tries to switch-in a thread using the same pmap (adding another cpu to the mask of cpus that need their TLBs to be invalidated). It's a fairly rare race but it has to be dealt with properly. Also fixable with some work.
The 512GB memory limit only exists because we still populate the DMAP entries manually and it is currently hard coded for 512G (one DMAP pte). A good programmer could fix that issue in about 2 hours but we're not going to worry about it unless we actually get hardware to test with with > 512G of dram populated. That much dynamic ram is a bit beyond our budget, not to mention the 1000W+ (~8A) of power it would eat.
-Matt
The basic mobo support for large N-way configurations has gotten cheap. Power management still has a long ways to go on these beasts, though. Our monster.dragonflybsd.org box is using the quad-socket supermicro mobo with four 12-core opterons (48 cores) and 64G of ram, and I think all told cost around $8000 or so.
The limitation for for these sorts of boxes is basically just power now. The 12-core opterons are effectively limited to 2GHz due to power issues, and these big beasts are really only high performers in environments where all the cores can be used concurrently with very little conflict.
By comparison, a PhenomII x 6 or an Intel I7 runs 6 cores for the PhenomII and 4 x 2 cores for the I7 but automatically boosts the base ~3.2GHz clock to almost 4 GHz when some of the cores are idle. These single chip solutions also have a MUCH faster path to memory than multi-chip solutions, particularly the Intel Sandybridge cpus, and much faster bus locked instructions. So if your application is only effectively using ~4-6 cores concurrently it will tend to run at least twice as fast as it would on a high-core-count monster.
That means that for most general server use a single-chip multi-core solution is what you want. The latest single-chip mobos for Intel and AMD support 16G-32G of ram and 5 or more SATA-III (6GHz) ports. Throw in a small SSD and you are suddenly able to push 400MBytes/sec+ in random-accessed file bandwidth out your network using just ONE of those SATA-III ports. That's in a desktop configuration! So today's modern desktop mobos is equivalent to last year's server mobos at 30-50% the power cost.
A modern high-end configuration as above eats ~60W idle where as the absolute minimum power draw on a 48-core Supermicro box w/ 64G of ram (the ram eating most of the power) is ~250-300W. Big difference.
So lots of cores is not necessarily going to be the best solution. In fact, probably the only really good fit for a 48+ core box is going to be for virtualization purposes.
-Matt
Well, you don't run your toaster 24x7. In fact, most residental homes use less than 1000W of power averaged 24x7 for the entire home.
Running 1000W 24x7 is ~$180-$240/month in electricty depending on where you live. Commercial power isn't much cheaper (and due to improvements in density most colo facilities now also charge for power or include only a few amps in the lowest-tier of service).
It adds up fairly quickly. The DragonFly project has 7 core production machines. Six of those in my machine room together eat around ~3.4A of power 24x7 (idle), and a lot more when they are busy. The last one is colocated and eats ~2A. There are another 2-3 essentially dedicated colocated boxes which are donated and another ~12 boxes on the third tier which donate mirroring and bandwidth. And DragonFly is a very small project.
For small projects... and here I'm not talking just about BSD projects but also many Linux projects, running your own machines requires either a fat purse somewhere or a sponser. FreeBSD gets a lot of sponsorship to help cover continuing costs.
For DragonFly we get some sponsership in the form of a few remote colocated boxes with reasonable bandwidth but mostly there are just two of us funding ongoing operations. I also fund getting ~4 new almost-bleeding-edge single-socket machines every year to keep us up-to-date on hardware and post the old boxes to various developers in need as they get replaced by new boxes, in a sort of pipeline. But it's taken 3 years to build that pipeline. New boxes come in and operate as test machines for ~1 year, then production machines for ~1-2 years, then get rotated out.
This situation has gotten a little better over the years as small projects can now run their boxes on real machines at home with a reasonable amount of upstream bandwidth, then drill a VPN through to a colocated IP service to route the IPs without having to deal with ISP filters (ISP-allocated static IPs tend not to work very well because AT&T and COMCAST's stateful filters can mess up your TCP connections when you have a lot of concurrency).
Even so it seems to me that a lot of projects don't even have that... they either rent time on a virtual machines or depend on sharing space with other larger projects. It's possible to do a lot with virtualized resources, up to a point, but rented virtualized resources tend to have very non-deterministic resources and you can wind up in trouble if you get a demand spike.
-Matt
actually, various unofficial rc.d projects by various people have been available for openbsd for at least 10 years including port of the netbsd one. Most OpenBSD users say "ick" because of the normal use of OpenBSD...
OpenBSD primarily gets used on boxes with very focused purpose, so just a few daemons to manage and I'd rather have single file to control them than runlevels and rc.d
ZFS has a large team of people behind it and resources that I don't have. That said HAMMER wasn't really designed to try to compete against it. HAMMER was designed to solve similar problems, but it wasn't designed to replace RAID as ZFS was. But ZFS is no panacea, and anyone who uses it can tell you that. The IP is now owned by Oracle, the license isn't truly open-source. ZFS itself is an extremely heavy-weight filesystem and essentially requires its ARC cache and relatively compatible workloads to work efficiently... and a veritable ton of memory.
HAMMER has a tiny footprint by comparison, gives you fine-grained automatic snapshots, and most importantly gives you near real-time queueless mirroring streams that makes creating backup topologies painless. Among many other features. Frankly ZFS might be the filesystem of choice if you are running dozens of disks but HAMMER is a much better fit otherwise.
People scream the RAID mantra all the time but the vast majority of people in the open-source world don't actually need massive RAID arrays to put together a reliable service. Often it takes just one 2TB HD and one 80G SSD x a few servers and in DragonFly HAMMER + swapcache fits that bill extremely well.
Our ultimate goal is real-time multi-master clustering. HAMMER doesn't get us quite there, primarily owing to the topology mismatch between HAMMER's B-Tree and OS filesystem cache topologies (mostly the namecache), but as the work progresses it will eventually achieve that.
In anycase, there's a huge difference between the people who do the actual design and implementation of these filesystems and the people who merely use them. Our goals as designers and programmers are not necessarily going to match the goals of the typical end-user who wants a magical black box that does everything under the sun with maximal performance in all respects and works without having to life a finger. ZFS can't even achieve that!
-Matt
DragonFly is still being designed, but the stated end goals are 1. Single system image clustering 2. providing multiple isolated environments in userland, 3. providing highly available clustered filesystem with multi-mastered mirroring/backup, de-duplication, snapshots