DragonFly BSD 3.0 Released

Not the big one by laffer1 · 2012-02-25 23:49 · Score: 3, Interesting

This release is interesting, but the rest of the year is dedicated to HAMMER2 and that will be the real story with DragonFly next. Most of the work on this release was incremental. Some interesting benchmarks were posted against FreeBSD in the last few months for PostgreSQL. There was some coverage on OSNews on this

http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved

--
MidnightBSD: The BSD for Everyone

Re:Not the big one by laffer1 · 2012-02-26 00:18 · Score: 4, Informative

If you're writing stories about DragonFly, then you want to cover all of it's distributed systems. The whole point of DragonFly is getting it ready for clustering. That's what Matt Dillon is into.
Some of the features of HAMMER & HAMMER2 are duplicated in other file systems, but most of them have much less friendly licenses. Even ZFS is under CDDL, which isn't terrible but precludes it from being used in Linux (the kernel). From my perspective, HAMMER could be the file system that everyone could use due to the license.
HAMMER is clearly the biggest feature of DragonFly that originated there. I think that constitutes coverage.

--
MidnightBSD: The BSD for Everyone
Re:Not the big one by rgbrenner · 2012-02-26 03:48 · Score: 4, Informative

Your benchmark compares HAMMER, ZFS, UFS, and EXT, when really HAMMER is most similar to ZFS. And in the benchmark, those two are pretty similar. The difference between the two: ZFS expects virtually unlimited RAM and will consume GBs easily; HAMMER will work with as little as 256MB of RAM.
Re:Not the big one by m.dillon · 2012-02-26 07:55 · Score: 2

Every open-source filesystem to-date has had serious pitfalls. Very serious pitfalls. In the Linux space it comes down to either significant bugs under heavy loads or extremely poor performance. I don't use Linux in production myself but I have several friends that do and they have yet to find any solution that doesn't occasionally explode in their faces. People talk about a lot of these linux filesystems as if they were the best thing since sliced bread but that's really only on paper. Every linux filesystem to-date has had and still has serious issues... everything from pseudo-commercialization or licensing to serious bugs when pushed... it's a mess.
In the BSD space there is basically no viable choice other than HAMMER1 (DragonFly) or ZFS (FreeBSD). And, no, I don't consider UFS w/softupdates and logging (let alone 'background fsck' or its very limited snap features) to be a viable choice.
HAMMER1 and ZFS also have serious deficiencies. For HAMMER1 its excessive seeking to access meta-data. For ZFS its excessive kernel memory use and the need for a lot of tuning to match the workload (and good luck with mixed workloads). With UFS you begin to hit major issues the instant kern.maxvnodes is hit, or the moment the directory hash cache limit is reached.
For DragonFly users, HAMMER1's meta-data issue is fairly easily solved. One big lesson we learned was that it doesn't actually take a whole lot of cache to cache the meta-data for even a modestly large filesystem (~several terrabytes), so DragonFly's generic swapcache feature coupled with a small SSD solves the meta-data problem very neatly. DragonFly also doesn't have a maxvnodes issue for caching purposes with the HAMMER+SSD combination and it solves it WITHOUT having to integrate the SSD into the filesystem like ZFS does. I learned a number of other lessons from HAMMER1 as well, particularly when it came down to the level of sophistication required to manage HAMMER1's B-Tree and the vulnerability (for any filesystem) of depending too much on the free block map.
However, even with all the features HAMMER1 has (automatic fine-grained history, trivial snapshots, trivial streaming incremental backups, etc)... it couldn't get us to our goal.
HAMMER2 is going to give us numerous additional features while at the same time solving the limitations of HAMMER1 that prevented it from being easily extended to cluster setups. HAMMER2 will have all the features of HAMMER1 plus also writable snapshots, multi-branching snapshots, a copies mechanism that ought to work considerably better than ZFS's, larger checks (up to 192 bits), block compression, a better de-dup implementation, and numerous other features. Plus it will be better matched for the clustering features we want. And, on top of all of that, HAMMER2's code base is actually going to be less complex than HAMMER1's code base was.
The biggest lesson learned from the HAMMER1 work is that meta-data is easy to cache, even for super-huge filesystems. We are taking advantage of that realization to greatly simplify the allocation scheme to make snapshot management & features utterly trivial to implement. Most free space management will be disconnected from production access paths (reading AND writing).
-Matt

Re:Will Try it by hairyfeet · 2012-02-26 01:07 · Score: 2, Insightful

Why not use one of the bazillion free download managers? I assume by your sig you don't use Windows but I assume Linux has similar software.

As for TFA, how does this compare to the other major OSes, like OSX, Win 7, Ubuntu, or even PC-BSD? What advantages does it give over the others? What are its best features? Why would you recommend this over other OSes? This is why i hate announcements like TFA because they don't give someone who doesn't use the OS a reason why we should care or try it.

--
ACs don't waste your time replying, your posts are never seen by me.

Re:Will Try it by Anonymous Coward · 2012-02-26 01:27 · Score: 2, Informative

wget --limit-rate=20k --continue http://mirror-master.dragonflybsd.org/iso-images/dfly-i386-3.0.1_REL.iso.bz2

If you don't have wget, get it. You can get it for pretty much anything. Linux. Other Unixes. Windows. OS/2 Warp. Macs. Android. 20 year old Amigas. Atari STs. Commodore 64s.

Re:Will Try it by kestasjk · 2012-02-26 02:11 · Score: 4, Informative

No-one (very few) care about whether BSD is "genuine UNIX" or "genuinely free software".

I administer three UNIX servers, all FreeBSD, and here's what I can tell you about the differences between it and Linux and Windows Server (which are also decent server OSes):

It's free
BSD is really simple; the kernel and OS are maintained by the same group, so they go step by step.
It doesn't change much; this is as much a great thing for servers as it is a terrible thing for everything else.
The ports system. This is a really big plus for BSD; I've tried many *nix distros and none are quite as consistent and reliable (for servers) as the ports system
pf. Although originally an OpenBSD thing this is a firewall which has a beautifully simple syntax. It's just so easy to express solid firewall rules, with queuing and everything. (Tbh iptables is probably at least as configurable, but last I checked pf definitely offered more power / learning-effort.)
Good community: You'll almost always find the solution to your problem, and it'll almost always be tailored to your BSD installation, rather than this or that flavor of Linux.

YMMV, Im sure many people here maintain great Linux servers, but for my humble needs I really like my three FreeBSD servers.

--
// MD_Update(&m,buf,j);

Re:Will Try it by rev0lt · 2012-02-26 02:39 · Score: 3, Informative

FreeBSD changes quite a lot between major releases, but usually doesn't break/remake what already exists. There are some few exceptions.
The ports system is also available on Linux; The pkgsrc system (that is used by DragonFly and NetBSD) is available for both Linux and Solaris.
PF is an awesome user-friendly firewall, but it has its limitations on high traffic systems (pf isn't multi-core friendly). Probably NPF will be an option soon.
The documentation is existing, up-to-date and usually accurate.
For servers, there are 4 key awesome technology components ATM - Jails (Linux has namespaces, but I don't know if it's funcional yet), CARP (pf-based redundancy), ZFS and HAST (somewhat equivalent do DRDB).

Re:Will Try it by Anne+Thwacks · 2012-02-26 03:02 · Score: 2

Never dare suggest BSDs should be easier or user-friendlier

BSD is very user friendly - its just kind of selective about who its friends are!

--
Sent from my ASR33 using ASCII

Re:good guy; bad choice by rev0lt · 2012-02-26 03:13 · Score: 3, Informative

If I recall correctly, there were some major conflicts regarding the design decisions of FreeBSD 5.0 branch. If I recall correctly, Dillon wanted to continue the 4.X work and gradually remove the giant lock from kernel, and other developers wanted to rewrite/re-engineer the kernel torwards multiprocessor support.
Dillon left the team and started working on DragonFlyBSD.
It is interesting, all this years later, that it seems Dillon was right. According to the Dfly 2.13 benchmarks, FreeBSD and DFly are close enough to be considered equivalents, and with DFly taking a lead in some tests. AFAIK PostgreSQL isn't threaded so for at least process-based applications, both Dillon's vision and the FreeBSD team turned out to be equivalent. (But the "breaking of things" and funcionality that started with the 5.0 branch was a huge long-term benefit, as it forced the reimplementation of key infrastructure components - network, storage, etc).

Re:Will Try it by unixisc · 2012-02-26 03:48 · Score: 4, Insightful

The BSDs - FreeBSD, OpenBSD, et al, would have to go through an official and formal certification w/ the Open Group in order to be certified as Unix. I don't doubt that they'd pass, but then, I don't doubt that Linux would pass either. That too, every version would have to be certified separately. I doubt that any distro would want to go thru the expense of doing it, and so the only certified Unixes out there are the ones like Solaris, HP/UX, AIX and OS-X.

The licensing issue is also somewhat tangential here - if a BSD has something that Linux hasn't, a customer will have no issues working w/ BSD, since BSD code can be incorporated in and released as a part of anything from proprietary to GPL3 software. If Linux has something that BSD hasn't, customers who need it will work around it, like Google did w/ Android. On the Linux side, I can see it getting confusing, since Linux is not going to become GPL3, but the things it uses - glibc, gcc, etc have become GPL3, which is a source for potential confusion.

Aside from that, I agree w/ the others like kestasjk below - few will care about whether it's genuine Unix or genuinely free software.

Re:good guy; bad choice by rgbrenner · 2012-02-26 04:06 · Score: 2

The reason the BSDs have been festering for the past decade

As a FreeBSD user since 3.4, I resent that statement. Just recently I was trying to get a FreeBSD domU working on XenServer... 7.x was unusable; 8.x was a little better but still unusable/unstable.. but 9.0 works and has been stable (with a couple of minor problems). It's only 2012, and FreeBSD already has near production-quality virtualization. FreeBSD is really on the cutting edge of this 'virtualization' tech...

And just a couple of major versions ago, we got BINARY UPDATES.

Things are getting really exciting. Watch out Linux. FreeBSD is creating some really cutting-edge software.

Re:Will Try it by ABCC · 2012-02-26 05:30 · Score: 2

Obviously substitute the url for which ever one you decide to actually d/l, but the following ought to work:

wget --limit-rate=20k http://www.dragonflybsd.org/download/#index1h1

Re:Will Try it by blade8086 · 2012-02-26 08:31 · Score: 2

Also - forgot to discuss this:

"
For instance, PC-BSD too is a FreeBSD derivative, fine-tuned for use as a desktop, w/ a choice of user interfaces, a new PBI packaging system, USB3 support and so on. So I can see a desktop BSD user prefering PC-BSD to FreeBSD. What sort of BSD users would want to use DragonFly over FreeBSD? Ones that have SMP systems?
"

This is not an 'apples to apples' comparison - PC-BSD is basically a FreeBSD 'distribution' -
FreeBSD base OS rebuilt with added user-friendly features that you mention - a FreeBSD binary will run on PC-BSD and vice-versa - and as I understand it you can update the base PC-BSD system by pulling
FreeBSD source and rebuilding/reinstalling as the PC-BSD stuff is 'added on' - you can still use FreeBSD ports with PC-BSD, etc.

DragonFlyBSD is a FreeBSD fork - so a totally different OS, with common heritage. DragonFlyBSD is no longer directly binary compatible with FreeBSD (without using the binary emulation layer for old FreeBSD 4 binaries), and the kernel architecture has diverged quite a bit w/r/t threading, various device driver api's, networking etc being quite different (but still close enough that code can be ported back and forth without too much trouble). People (myself included) who would prefer DragonFlyBSD share the same philosophy w/r/t the design of the system and related goals. Purpose wise the two are quite similar - general purpose for desktop/server on mainly/exclusively x86/x86-64 platform - but my personal take and alot of the community is that DragonFly has taken the 'right approach' to 'next gen' features such as SMP, etc, and focusing on clean internals for these features has left things more flexible for future work. The 'fruits' of this effort can be seen in similar performance for the initial lock-free SMP in the last release compared to FreeBSD despite a much much smaller amount of developer time - with more performance improvments likely as the changes leading to 3.0 are tuned further.

I got into DragonFlyBSD because at the time (~2006) I was quite interested in OS internals / design, and the SMP work design goals seemed to 'seem right' as compared to the FreeBSD5+ approach - the small DragonFly community made it easy to get involved and seems much more 'cosy', so I stuck with it, esp. because at the time the VKernel work was just taking shape which was very exiting to have a BSD derived OS with native, BSD licensed full system virtualization (if not hardware assisted) - which is still unique among the BSD derivatives.

GPL vs BSD by ninejaguar · 2012-02-26 09:24 · Score: 2

Both are free in cost and use, but only Linux is Free.

The GPL license, which the Linux kernel is under, limits the freedom of developers to limit the freedom of other developers to make use of changes from derived code. This is effectively done when Developer A takes GPL'd code from Developer B to benefit from Developer B's work. If distributing the derived work, Developer A must release any changes made to Developer B's work so that other developers, including Developer B, ARE also in turn Free to benefit the same way that Developer A benefited. This is called reciprocity, and is a form of cooperation (something which most parents hope their children learn). A GPL license by Developer B ensures Developer A behaves in a selfless or altruistic manner at the cost of not allowing Developer A the choice to be selfish to others, including to Developer B. For the convenience of Developer A, this requirement is only triggered when Developer A distributes the derived work originally based on Developer B's GPL'd work. The use of work already under the GPL is a completely voluntary choice for Developer A to make. The freedom of choice as to which type of licensed code to take is not limited, and Developer A can instead look for other work already under the BSD license to take for personal benefit while restricting the same benefit to others by closing the source of their changes.

The BSD license, which the BSD kernel is under, allows developers to limit the freedom of other developers to make use of changes by closing the source of a derived change, limiting the benefit of the change to only the initial closer of the derived source. This doesn't just stop the first generation of developers who could've benefited from the change, but it also stops any later developers from benefiting from and contributing to further generation of changes to the derived work. This is effectively done when Developer A takes BSD'd code from developer B to benefit from developer B's work. If Developer A distributes the derived work, and Developer A doesn't release any changes made to Developer B's work, then other developers, including Developer B, are NOT Free to benefit from the changes made by Developer A the way that Developer A benefited from Developer B's work. This is called selfishness, and is an example of non-cooperation. The BSD license allows the choice to be selfish at the cost of depriving the choice by others to utilize derived changes originally based on the work of others. The freedom of choice as to which type of licensed code to take is not limited, and Developer A can instead look for other work already under the GPL license to take for personal benefit without restricting the same benefit to others.

In either case, it is up to the original developer, Developer B, to decide which type of behavior to allow by choosing the license.

= 9J =

Re:Will Try it by smash · 2012-02-26 15:04 · Score: 3, Insightful

I tell you what, the BSDs would be a lot more likely to pass than Linux, as they're directly descended from the AT&T unix code base, and continue to do things the "Unix way". Mac OS X (largely FreeBSD userland) has been certified as Unix. Linux is a clusterfuck of NIH syndrome and GPL software that is often different for the sake of being different.

And the GPL is NOT free. It contains restrictions on what others can do with the code you release (i.e., they can't close it). Just because you might not like the possibility of code being closed, restricting people from doing that is not more free than allowing people to do anything with it.

--
I run: Windows, OS X, Linux, FreeBSD. Just because you have a hammer, doesn't mean everything is a nail.

16 of 102 comments (clear)