Slashdot Mirror
Btrfs Is Getting There, But Not Quite Ready For Production
An anonymous reader writes "Btrfs is the next-gen filesystem for Linux, likely to replace ext3 and ext4 in coming years. Btrfs offers many compelling new features and development proceeds apace, but many users still aren't sure whether it's 'ready enough' to entrust their data to. Anchor, a webhosting company, reports on trying it out, with mixed feelings. Their opinion: worth a look-in for most systems, but too risky for frontline production servers. The writeup includes a few nasty caveats that will bite you on serious deployments."
It says "experimental." They appreciate you helping them test their file system out. I appreciate it too, so please do. But remember that you are testing an experimental filesystem. When it eats your data, make sure you report it and have backups.
I've been happily using the XFS file system since the early-to-mid-2000s and have never had a problem. It is rock solid and much faster than ext3/ext4 in my experience, tested a lot longer than Btrfs, and handles the millions and millions of small files on redditmirror.cc very effectively.
Slashdot Valentines Beta Massacre: iT WORKED! The boycotts killed Beta!!
Does submitter even work in IT?
I think we need to talk about the oracle in the woodpile - ie, Oracle. BTRFS is an Oracle project. What happens when it goes the way of MySQL? Will Monty Wideanus appear on a white steed to save us?
Do you even lift?
These aren't the 'roids you're looking for.
Meanwhile ZFS announced that it was ready for production last month.
http://zfsonlinux.org/
Isn't that a common trait with experimental systems?
A few nasty caveats? When we are talking about a file system, there can be no nasty caveats. If the file system isn't bullet proof, it is unusable!
Ugh, I'm really sorry about this post, Slashdot. I really didn't think it was going to a "First post." What I really meant to post was
Aside from ideological reasons, why would anyone choose btrfs over ZFS? The latter seems to be superior from both a theoretical and practical standpoint, and has more real-world stability testing. This is especially true if you're not specifically wedded to Linux and are willing to use FreeBSD or a derivative for your file hosting server(s).
That will probably get 95% done before the next cool kid on the block takes over.
Filesystems on linux never seem to be 100% done. By that I mean, they are bulletproof and reasonably fast.
Just think, if instead of having dozens of primary fileystems (ext[2,3,4], reiserfs, ocfs, ocfs2, xfs, jfs. gfs2, zfs) over the last 15 years, someone had actually spent the time and designed/researched a general purpose filesystem, implemented and slogged through all the issues on one of them. Linux might have a good filesystem. The ext2/3 combination appeared to be on that path, but instead of fixing the fsync problems, and a couple other issues, it was basically abandoned in favor of ext4 which while maintaining the ability to mount ext3, and is named the same could have just been called "we took some bad ideas from XFS for performance reasons and now we have the same data integrity problems".
Those distros such as SuSE Linux Enterprise Server, that claim it was production ready and have it in the install, should be shunned. Don't entrust your data to them
So what is this 'Full' limit? In the ZFS world it's accepted to keep the pool (volume) under 80% usage to prevent issues. Is this something that should be applied as a 'best practice' to btrfs?
Please just STFU if you don't know what you are talking about.
Both CERN and Fermilab have been using XFS to manage **PETABYTES** of data for many years now. Fermilab switched from ReiserFS, which should give you some indication of the time frame we are talking about.
The problem with "XFS" eating data wasn't with XFS - it was with the Linux devmapper ignoring filesystem barrier requests.
Gotta love this code:
Martin Steigerwald wrote:
> Hello!
>
> Are write barriers over device mapper supported or not?
Nope.
see dm_request(): /*
* There is no use in forwarding any barrier request since we can't
* guarantee it is (or can be) handled by the targets correctly.
*/
if (unlikely(bio_barrier(bio))) {
bio_endio(bio, -EOPNOTSUPP);
return 0;
}
Who's the clown who thought THAT was acceptable? WHAT. THE. FUCK?!?!?!
And it wasn't just devmapper that had such a childish attitude towards file system barriers:
Andrew Morton's response tells a lot about why this default is set the way it is:
Last time this came up lots of workloads slowed down by 30% so I dropped the patches in horror. I just don't think we can quietly go and slow everyone's machines down by this much...
There are no happy solutions here, and I'm inclined to let this dog remain asleep and continue to leave it up to distributors to decide what their default should be.
So barriers are disabled by default because they have a serious impact on performance. And, beyond that, the fact is that people get away with running their filesystems without using barriers. Reports of ext3 filesystem corruption are few and far between.
It turns out that the "getting away with it" factor is not just luck. Ted Ts'o explains what's going on: the journal on ext3/ext4 filesystems is normally contiguous on the physical media. The filesystem code tries to create it that way, and, since the journal is normally created at the same time as the filesystem itself, contiguous space is easy to come by. Keeping the journal together will be good for performance, but it also helps to prevent reordering. In normal usage, the commit record will land on the block just after the rest of the journal data, so there is no reason for the drive to reorder things. The commit record will naturally be written just after all of the other journal log data has made it to the media.
I love that italicized part. "OMG! Data integrity causes a performance hit! Screw data integerity! We won't be able to brag that we're faster than Solaris!"
See also http://www.redhat.com/archives/rhl-devel-list/2008-June/msg00560.html
There's a lot more out there if you care to look.
Toss in other things like the way Linux handles NFSv2 group membership (More than 16? Let's just silently drop some!) and lots of fanbois wonder why I view Linux as little better than Windows. Hell, Microsoft may fuck things up six ways from Sunday, but they're not CHILDISH when it comes to things like data integrity.
I still use reiserfs of my mdadm raid-5 (Installed right after Katrina ate my old sever)
And i have reisrfs on an older mdadm raid-5 from around 2000.
both have been rock solid running 24x7 and in reality that is the ultimate test.
(Though I would like to see a new BeFS driver created for fun.)
Actually I'm being serious. This is why I come to /.
Python: 'And then suddenly you have a language which says "we're all stuck with whatever the whiniest coder wants".'
I still prefer XFS ;-).
And on the Eighth Day, Man created God.
The age-old "unite and consolidate" argument rears its pointless head once again. And once again, the answer is that the people who work on open source software do so because they benefit from it -- whether for experience, for money, or for plain old personal satisfaction -- not because you benefit from it.
Open source software is not charity work. It may be free, and you may benefit from it, but it wasn't designed FOR you.
I'm going to break this to you as politely as I can. No matter what you say or do, the people who work on these projects will continue to devote their time and effort to the things which they find valuable, not the things which you find valuable. There is one thing, however, that can change that: you can pay them.
ZFS us vastly superior to Btrfs--yes, even on Linux. It's all about licensing crap. This is yet another example of open source foolishly generating waste heat on projects for dubious reasons.
Installed Xubuntu 12.10 last October(ish) on USB2 stick (jetflash 32G) with Btrfs (only /boot had EXT2 partition, no swap)
Reason: 24/7 machine. It's a notebook - always spinning harddrive is a drag: spins up cooling fun; so I went solid state for primary OS drive.Needed filesystem that spreads wear and does checksums - hence Btrfs.
Usage - downloading stuff (to the stick itself, not the harddrive) plus some NASing. Data volume: wrapped around those 32gigs few times already.
Observations so far: no problems at all.
Other details: Had to play with I/O scheduler (I think settled on CFQ. Interestingly, NOOP sucked). Had to install hdidle (I think) otherwise couldn't force sda to go to sleep (bug (?)).
I've just lost ( an our ago ) the entire FS tree in archlinux installation ( I forgot to prepare meself with the btrfsprogs ) - I thought I was safe to use btrfs - I wasn't even able to boot ubuntu-live (13.04) because [the screwed] btrfs partitions made the kernel (btrfs module from the Ubuntu live) crash at boot :-)
- destroy offeding partition
- restart from scratch but NEVER-EVER use btrfs again!
I have my lesson - don't touch anythings you do NOT know very well. :-)
Therefore, it is a pleasure for me to re-install archlinux ( using ext4 this time!!! ) again
Long live archlinux :-)
hahahaha
File systems are well-understood. What the **** is taking them so long to reach stability? Either it passes regression tests or it doesn't, and it should take a few weeks to write. That was many, many years ago. What have they been doing since then?
I don't see how that's any different: "Show stoppers" means "things that are unacceptable in our scenario".
I tried btrfs as my main laptop filesystem:
nice features, speed ok, but i happened to unplug by mistake the power supply, without a battery. bad crash... I tried using btrfsck, and other debug tools, even in the "dangerdon'teveruse" git branch, they just segfaulted. at the end my filesystem was unrecoverable, I used btrfs-restore, only to find out that 90% of my files had been truncated to 0... even files i didn't use for months....
now, maybe it was the compress=lzo option, or maybe I played a little too much with the repair tools (possible), but untill btrfs can sustain power drops without problems, and the repair tools at least do not segfault, I won't use it for my main filesystem...
btrfs is supposed to save a consistent state every 30 seconds, so I don't understand how I messed up that bad.... maybe the superblock was gone and the btrfsck --repair borked everything, I don't know.... luckily for me: backups :)
"I was gratified to be able to answer promptly, and I did. I said I didn't know." -- Mark Twain
To be honest, there are many projects that are just this - a rewrite of working code just because the license doesn't match what you want. BDB => GDBM for some reason pops in the mind first. Usually it's mostly a waste of resources as it takes time to build up the feature set of the copied code and avoid the bugs that were revisited because they ignored the design of the copied code. I'm still waiting for my FSF Skype clone.
My guess is that humans want to be architects, not maintainers. It's fun to be bold and create "new" things with the partial safety of it following a known framework than go and try to fix that annoying bug in someone else's code that only shows up on Toshiba hardware with the 2976G chipset and NOT the 2976F chipset and when Obama wears a red tie. This is not of course all of it, there are some legit license reasons for some forks, but underneath methinks this is always a secondary reason.
I bet the power never goes out at CERN.
When all you have is a hammer, every problem starts to look like a thumb.
if linux is a toy (your basic argument)
Wrong. I admin linux servers professionally. I am a database developer who makes extensive use of open source software in a production environment.
On the contrary, my "basic argument" was that calling for other people to unite and consolidate is pointless. If they wanted to unite and consolidate, they would have already done so. Naturally, the people working on open source software have already considered everything you said, and they have already said "no".
FYI, I have been a linux geek since '97, and people have been saying the exact same things as you, continually, for the entire 15 years. The reality is that consolidation will happen if and when it makes sense to the people doing the actual work -- not when it makes sense to you.
I'm still waiting for my FSF Skype clone.
Eh? Free SIP clients have been around for much longer than Skype. SIP hardware has been around for much longer than Skype hardware. I have one sitting on my desk right now. Just because Skype became popular, does that mean the entire rest of the industry should abandon their existing IP communications protocol?
bit rot file system? No thanks.
If you are "trusting your data" to *ANY* file-system, you are likely to be disappointed.
I have run btrfs off and on for maybe 3 or 4 years because I don't *HAVE* to trust my data to it. I have good backups that run daily. If btrfs screws the pooch, I'm not really out that much.
Note though, my backup servers run ZFS. :-)
Honestly, it seems to me that btrfs has gotten worse over the last few years rather than better. 4 years or so ago when I first started using it, it actually worked pretty well and I was fairly happy with it, including taking automatic snapshots, but I never had a data loss. ISTR that I switched away from it because I upgraded to a new distro and had to reformat, for various reasons. Newer versions I've tried have been barely usable and I've had brtfs wedge itself a few times. Some of the issues were distro integration issues I think, like 12.04 seemed to *ALWAYS* run a full fsck on boot, and I think it took a snapshot when I tried to do an upgrade to 12.10, which somehow caused it to think that it had space available when it didn't and it ran out of disc space during the upgrade...
I really want btrfs to get production ready, but I'm half thinking that by the time it is HAMMER2 will be out and I'll be infatuated with it. Note that btrfs and HAMMER started around the same time, maybe HAMMER had a 6 month lead. HAMMER has been "production stable" and has been the default Dragonfly BSD filesystem for several years. Dillon seems to know how to build a file-system...
I am running the latest kernel and I created a plain (no RAID, no fancy stuff) btrfs partition on a 1TB SATA disk today.
Within an hour I had 3 kernel panics while copying a bunch of files to the btrfs partition.
The same disk works great with NTFS or ext4 (no problems whatsoever).
I don't believe that btrfs would be ready even for home systems for many years to come (there aren't many developers working on it). Lets not even talk about production systems ;)
tux3 is being actively developed, and is nearing merge to mainline. It already outperforms ext4 in most tests, all while providing stronger data guarantees.
I tried to use a SIP client for Skype-style usage once. Maybe I just had a terrible one recommended to me, but I did not know it was possible to make a messenger app that confusing. Does anyone really expect average Skype users to convert to that?
All I know is that "Btrfs" makes me think of a drunk trying to say the word "barf" in a sentence. Maybe they should come up with a better name, like "Anchor" (lifted right off the story).
The only meaningful difference is that the Skype application comes pre-configured for their network. SIP softphones have to be manually configured by their users.
I've lost data with ext3.
... and backups.
I've had data corruptions with reiserfs.
I've lost data with ext4 (which happened to be the most frustrating, tedious, and complete failure of all).
Most recently, I had some HD failures on a fully RAID-1'd server running entirely on XFS, and had to re-install the OS from scratch and restore from backups. The new install was onto btrfs.
I've had partitions running on btrfs for a little over a year, and have not yet lost data on these, but it's just a matter of time; I will lose data. I used to blame it on cheap drives, but I've seen SMART failures on young Seagates so I'm now convinced there's no such thing as a high quality, high density drive. At the moment, I find btrfs easy to use (intuitive and simple), and full-featured, so it's what I'm currently using. But I suffer from no illusions; at some point, I will have FS corruptions and have to restore from backups, and I can only hope that any FS corruptions won't go undetected and be propagated to my backups for very long before that happens. Failures are inevitable no matter what I use, so now I value simplicity, convenience, speed
Right now, btrfs beats the alternatives for convenience and features. I put my trust in backups, not file systems, and value is in features and convenience, not some false perception of safety or reliability.
This post just emphasises again that we need engineering standards and measures for software.
I know there are only a few diehard holdout BeOS geeks still out there, and I know we have a terrible secret the world has never uncovered: BeFS. This file system, coded and deployed (production) in 1992, is 64-bit, multi-threaded, and fully journaled â" attributes taken for granted today, but only futuristic buzzwords for other OSes of the day. Hard drives deployed on R4, an Intel x86 or PPC OS, were typically 6GB IDE drives. BeFS can handle single files of up to 18,000 petabytes - all of recorded human history at the time was only ~100 petabytes. BeFS is built on an OODB. It's tough, reliable, and well documented (there are even three venerated O'Reilly books on the subject â" two dedicated to *just* the filesystem). It's what zfs and btfs want to be when they grow up. And today, it's discarded. While Linux, OS X, BSD and other OSes could be compiled with kernel support, they aren't. Running it essentially means putting a virtual FS in a file. Tragic â" another example of reinventing the wheel.
"Adventure? Excitement? A Jedi craves not these things."
If you get a client with your network, it comes preconfigured.. so that's not a difference.
There *are* a lot of sucky sip clients. There are also some excellent ones.