I still have Mac Mini (Freescale PowerPC G4) which I used for Debian development for half a decade, and which is now idle with a FreeBSD 10.2 install at present, and while I went to Intel and AMD for my last two systems, I'd certainly welcome a return to an affordable POWER system. I've been pretty disappointed in the state of open hardware for a good while.
I was looking at the offer for an OpenPOWER system from Tyan (http://www.tyan.com/campaign/openpower/) but I'd prefer a workstation rather than a rackmount unit. If it can run FreeBSD, then even better. The only rub is the graphics support; if I can stick in an AMD board and have it work with OpenFirmware and the current open drivers, I'd be quite happy.
Because you might be streaming content into the texture after its creation e.g. with GLTexSubImage2D (or 3D). You might already be running the render loop a few times over before it's fully filled. Ideally the driver would only ever give you blanked memory so it would be transparent and imperceptible.
Filtering out ultrafine particulates is hard. In a lab setting, most types of filters will clog immediately, and the backpressure is huge. You wouldn't be able to put one inline in an exhaust and not have it fail (rupture), clog or stop the engine in a matter of minutes. There are other ways of filtering, but doing so in a hot continuous exhaust stream is not a trivial problem to solve.
Your prefix should be constant and should remain the same across reconnects. If you want the remainder to be constant, it should be constant with SLAAC, being based on the MAC address. If it's changing and you don't want it to, try disabling privacy extensions? Or you could use DHCPv6 or static allocation if that wasn't sufficient.
It's odd that the ISP doesn't provide a router which can use the services they provide!
I got a generic Thomson/Alcatel router from my ISP which does v4 and v6. I had the same model from my previous ISP and it was IPv4 only, so just a firmware difference between the two.
My ISP gives me an IPv4 address and an IPv6/64. So each machine internally gets a NATed IPv4 address (as is usual) and one or more global IPv6 addresses (some are configured statically e.g. server with DNS entry; clients use SLAAC and privacy extensions).
There's no charge for the 2^64 addresses. They aren't scarce and "valuable" like IPv4 addresses.
You can make it a business case very easily. Call your ISP and ask them for some concrete timelines for IPv6 service. If they haven't got any, then cancel your subscription; when they ask why you're leaving, tell them that you want IPv6 and they aren't providing it. They'll get the message when they lose enough business.
I moved to another ISP with IPv6 service (aaisp) specifically because my existing ISP at the time had promised it was coming for two years without delivering, and put it on the back burner. I now pay a bit more, but have excellent service which includes IPv6 by default.
I have native IPv6 from my ISP. The ISP-supplied router handles v4 and v6 automatically. The router's firewall handles IPv6 exactly the same as it handles IPv4; if you want to open up ports, allow inbound/outbound connections, etc., it's all configured pretty much exactly as it was for v4. If people can handle configuring IPv4, then they can handle IPv6. Being "constantly pwned" is seriously overstating the risks.
https://bugs.freedesktop.org/s... is certainly one of the issues. But there are other situations which *do* cause log data loss. Google shows up plenty with little effort.
I'm actually using the correct terminology here. They are required to return the site to a "greenfield" state. And no land stays brown; it grasses over naturally very quickly so it's rather more accurate than brownfield.
I'm not sure what this has to do with NFS. Fails on Ubuntu 15.10 for me. But zero inodes doesn't look like a useful setting. Works with a positive count or if unspecified.
The NFS issue is likely it mounting the NFS filesystem before all the necessary RPC services are working, or something like that.
Seriously, have you even thought about this in any depth?
What's more safe and secure, many tens of thousands of lines of C running in PID0 with root privs, or interpreted scripts each running as root or with reduced privs in their own separate process?
It's obviously the latter. The only compiled code is the shell interpreter, which is well tested and used all over the place with root privs already. And the shell scripts being run are trusted. Now, you can argue all the other points about the downsides of shell vs unit files. But as for security, this one is plain and obvious. All that C is just dying for a trivial buffer overflow or some other standard exploit to allow a full compromise of the system from some valid or even user-supplied untrusted input. Yes, all that code is long overdue for a thorough audit; we're living on borrowed time as it is. How long to you think it will be until the first major exploit surfaces? Maybe then we'll have a rethink about the appalling design practices the systemd "geniuses" have foisted upon us. Having so much code running in the most critical and vulnerable part of the system is idiocy of the first order.
Sounds good... right up to the point when there's a problem. Then what happens? systemd notices that the log is corrupt, and... deletes it. No log of the problem. See: the long and angry (and unfixed) bugzilla tickets.
As for log compression, you don't want the actively written log to be compressed. If there's a problem, even as small as a single bit error, then the log will be unrecoverable. That's the tradeoff you make with compression. logrotate compromises by only compressing older logfiles. If there are any minor errors in the active log then you can still read it just fine.
As for tampering. It's of minor importance only. While the systemd people and their fanbois might harp on about it, they are catering for a problem which is of far less importance than hardware failure or power loss. Right now, all that foward hashing is so useless. If a simple power failure causes the checks to fail and the whole log to be discarded, it's a net negative. You threw away all my bloody logs! Like many of the systemd features, there's a whole fanfare about how essential it is and how everything else is awful and insecure. And as usual, there's some small credence to the claims. But it's massively overblown, and it has significant downsides. The "scary" things you mention are all of low likelihood--they only apply if your system has been compromised; there are rather more likely and less nefarious things to care about before these. I'd rather have a guarantee that my logs won't be deleted on a whim. Never, ever delete my logs!
As an example for the problems with troubleshooting, I've recently installed a few Ubuntu 15.10 systems (bare metal and in VMware VMs). In both scenarios, NFS filesystems fail to be mounted at boot. Why? I have no bloody clue. There's nothing in the logs. If I umount them (they are showing as mounted but give immediate I/O error on use) and remount by hand it all works just fine. No idea how to troubleshoot it. With sysvinit I could boot to an interactive shell in the late initramfs and step through every script by hand, and pinpoint exactly where something was failing. With systemd, even with correctly configured systems, I still experience occasional unrecoverable hangs at boot as it screws up its nondeterministic boot ordering and waits forever on something or other (who knows what it might be). systemd has absolutely not improved boot reliability. If I boot a FreeBSD system with exactly the same NFS configuration, it mounts everything perfectly at boot every damned time.
Another thing, is the broken compatibility with what came before. Example: I edit/etc/nfsidmap.conf to configure NFSv4 id mapping. Previously I'd reload/restart the nfs-common service with 'service' or 'invoke-rc.d'. Job done. What's the systemd equivalent? I don't know, and I can't see any obvious candidate. Since both these commands now forward to systemd, a diligent engineer would have made the old service names forward to the new target(s) to retain compatibility, maybe with a helpful message indicating what the new names were. There's nothing. I don't see any obvious nfsidmap or generic rpc or nfs units. Yes, it's partly me lacking familiarity with the new way of doing things; but the lack of care in having a smooth transition for admins makes for a terrible time, and the mess of units makes the whole thing difficult to comprehend as a whole and baroquely overcomplicated.
That's the reprocessing facility, Sellafield. And there's a reason for that particular building 30 being a mess if you read the history, not that it makes it right. Fuel is not left at power stations.
This isn't the US though and spent fuel won't be left lying around at the rector site indefinitely. As the article states the fuel will be offloaded and sent away to be reprocessed. Afterward, like other similar sites, they'll remove the turbine hall and other ancillary buildings, then leave the reactor to sit for a few decades to allow the residual radioactivity to decay to almost nothing before (carefully) demolishing it entirely. In a relatively short time, it will be a greenfield site you would never know had a reactor on it.
To be fair, the race existed in udev prior to the systemd merge as well. When lvremove randomly stops working, it's a bit surprising, and it took a while to pinpoint udev as the culprit keeping the snapshot devices open and preventing their removal. "Helpful" such behaviour is not. We had to move all the debian buildds from using lvm snapshots to unpacking tar files as a result (btrfs being too fragile as mentioned).
They definitely are. But it doesn't scale well. The time taken to scan the files and their contents on the source and destination system becomes overwhelming. The largest I've taken it to is a few terabytes, consisting of many thousands of directories each containing thousands of files (scientific imaging data). It ends up taking hours, where with ZFS it would take a few seconds. It also thrashes the discs on both systems as it scans everything, and uses a lot of memory. ZFS does none of these things--the send/recv is a "simple" streaming operation.
That said, with tools based upon rsync, like unison, it becomes possible to do two-way synchronisation which is pretty powerful. ZFS send/recv only works one-way. But again, the scan time with unison becomes prohibitive.
Without some kind of incremental snapshot, with read-only privileges after the snapshot, straight replication is next to useless if someone does "rm -rf/". And it happens *all the time*.
So, exactly what ZFS provides then... You take periodic snapshots (hourly, daily, weekly, or whatever), then send the deltas between the snapshots to the destination system. You can easily put that in a cron job and have a regular push to a backup system (hey, exactly like what the tool in TFA is doing...). If someone does wipe out all their files, you have the snapshot(s) containing it on both the source and destination system, depending upon your schedule for dropping old snapshots. However you decide to manage things, you can recover the removed files so long as they are present in an older snapshot.
ZFS originated within Sun, which was bought by Oracle. Oracle then laid off most (all?) of the ZFS developers, who then went to work for other companies. The current ZFS development is no longer inside Oracle, and nor is it owned by them. They own the copyright on the original CDDL releases. Big deal. Not using it because of the historic association with Oracle would be a little... extreme.
Your Synology "reference" is a classic "appeal to authority", only it's a really bad choice of authority due to its complete lack of any technical detail or substance of any kind. That link is to a marketing page for a company which makes money selling hardware. It's just a few bullet points (snapshotting, checksumming in essence), without any discussion of the actual tradeoffs or comparison with other systems. It's worthless. It's only purpose is to tick a feature box to act as an incentive to purchase their systems; as for the actual performance and reliability of those features--that's the customer's problem. Caveat emptor.
I've done more than casual work and development with Btrfs. For example, from back when I was a Debian developer, here's the original inital support for Btrfs snapshotting in schroot. This lets you create virtual environments from Btrfs snapshots, as well as other types such as LVM and overlays. You can then plug this into other tools such as sbuild, and then build the whole of Debian using snapshotted clean build environments. Doing this, Btrfs fails hard around every 18 hours, going read-only. Why? Creating and deleting 18000 snapshots for 8 parallel builds quickly unbalances the filesystem, requiring a manual rebalance. You don't see that unfortunate detail in the Synology fluff page, do you?
You can also get snapshots and decent recovery (albeit without block-level checksums) from LVM and mdraid. In my experience, its recovery behaviour after real hardware failure is vastly more reliable than Btrfs. Simply put, it has always resynched the data without problem, while Btrfs has caused irrecoverable data loss, despite it theoretically being much better. LVM snapshots have very different tradeoffs as well. And on modern Linux with udev, we had to abandon using them due to races in udev/systemd making them randomly fail.
The point I'm making is that the reality of the chosen tradeoffs between performance, reliability and featureset of the different filesystems is a subtle one. You can't reduce it down to "Btrfs is better" or "ZFS is better". That's marketing. But I have spent over seven years pushing Btrfs to its limits, and have found it sorely lacking. It's unacceptable that it unbalances itself to the point of unusability. It's unacceptable that it has led to irrecoverable dataloss on several occasions. It's also unacceptable that in its eight years of existence, none of the developers could be bothered to write any decent documentation. The dataloss was down to bugs, some of which are fixed, but it does leave you in a position of lacking trust in it in the face of such problems. If you compare this with ZFS, while it's not fair to say it has been totally bug free, it has been almost bug free, and the number of dataloss incidents is small. I've yet to encounter any problems with ZFS myself, but I've encountered many serious issues with Btrfs.
Anyone who uses Btrfs or ZFS on a NAS system does so at their own risk after researching the various options and their tradeoffs. Just because a vendor decides to make and market a system using Btrfs does not make that system the best choice. It just means they thought they could make some profit from it.
Er, no. Btrfs may one day make feature parity with ZFS, and it may also achive the reliability of ZFS, but it has a long, long, way to go in both areas to get to those points.
The on-disc structures might have been declared "stable", but what does that mean, really? That you'll be able to mount current filesystems on future kernels, yes. That the frozen design was correct and contains no design flaws? No. Personally, I think they froze it way too early. There are a number of fairly fundamental issues with the Btrfs design which compromise its performance (fsync) and integrity (unbalancing, data loss on recovery), and in some cases place arbitrary limits upon things (e.g. the hardlink issue). Some can be mitigated, while others can not. These and other issues are easily found and researched.
Seriously, I've been using Btrfs since very near the beginning for a variety of tasks. But I've been objective about it, rather than a blinkered fanboi. It's an interesting filesystem with some good ideas. But it has/always/ been a case of "next year it will be stable", and the performance is dire. Progress has been painfully slow, and the bugs I've encountered along the way have been numerous and show-stopping. Maybe it will "get there", but I think your assertion that "once BTFS userland side gets stable" that it will replace ZFS is incredibly naive. It assumes that there are no major issues remaining on the kernel side, and it also assumes that the only thing needing doing on the user side is stability. Based on its history to date, the likelihood of the kernel side being bug-free is close to zero. On the user side the tools are primitive, feature-incomplete and almost completely undocumented, containing little information and no examples. On the ZFS side, the tools are feature complete and are properly documented, with examples, and with whole sets of training material on top of that.
If you needed to make a decision on which to use for a serious deployment, or even just for a smaller scale home NAS, right now if you objectively compare the two, the choice is quite clear, and it's not Btrfs. Based upon the development history of the two, it's unlikely that this will change much in the next few years. Remember also that ZFS development is very active, perhaps even moreso than Btrfs. But who knows, maybe by 2020 Btrfs will surpass it.
There is no grey area with respect to the licensing. It's CDDL, a free software licence. It's 100% Free.
It might be incompatible with the GPL, but that's a non-issue. The userland tools are fine under this licence. The kernel modules are fine under this licence. Now, it means that the kernel modules aren't going to appear in a kernel release anytime soon, but that in no way makes for any legal problems in using them as loadable modules, today. It works fine from a technical point of view, and it's also fine from a legal point of view.
Slashdot Mirror
I still have Mac Mini (Freescale PowerPC G4) which I used for Debian development for half a decade, and which is now idle with a FreeBSD 10.2 install at present, and while I went to Intel and AMD for my last two systems, I'd certainly welcome a return to an affordable POWER system. I've been pretty disappointed in the state of open hardware for a good while.
I was looking at the offer for an OpenPOWER system from Tyan (http://www.tyan.com/campaign/openpower/) but I'd prefer a workstation rather than a rackmount unit. If it can run FreeBSD, then even better. The only rub is the graphics support; if I can stick in an AMD board and have it work with OpenFirmware and the current open drivers, I'd be quite happy.
It's all available online as well: https://www.gov.uk/guidance/wo...
Because you might be streaming content into the texture after its creation e.g. with GLTexSubImage2D (or 3D). You might already be running the render loop a few times over before it's fully filled. Ideally the driver would only ever give you blanked memory so it would be transparent and imperceptible.
Filtering out ultrafine particulates is hard. In a lab setting, most types of filters will clog immediately, and the backpressure is huge. You wouldn't be able to put one inline in an exhaust and not have it fail (rupture), clog or stop the engine in a matter of minutes. There are other ways of filtering, but doing so in a hot continuous exhaust stream is not a trivial problem to solve.
Your prefix should be constant and should remain the same across reconnects. If you want the remainder to be constant, it should be constant with SLAAC, being based on the MAC address. If it's changing and you don't want it to, try disabling privacy extensions? Or you could use DHCPv6 or static allocation if that wasn't sufficient.
It's odd that the ISP doesn't provide a router which can use the services they provide!
I got a generic Thomson/Alcatel router from my ISP which does v4 and v6. I had the same model from my previous ISP and it was IPv4 only, so just a firmware difference between the two.
My ISP gives me an IPv4 address and an IPv6 /64. So each machine internally gets a NATed IPv4 address (as is usual) and one or more global IPv6 addresses (some are configured statically e.g. server with DNS entry; clients use SLAAC and privacy extensions).
There's no charge for the 2^64 addresses. They aren't scarce and "valuable" like IPv4 addresses.
You can make it a business case very easily. Call your ISP and ask them for some concrete timelines for IPv6 service. If they haven't got any, then cancel your subscription; when they ask why you're leaving, tell them that you want IPv6 and they aren't providing it. They'll get the message when they lose enough business.
I moved to another ISP with IPv6 service (aaisp) specifically because my existing ISP at the time had promised it was coming for two years without delivering, and put it on the back burner. I now pay a bit more, but have excellent service which includes IPv6 by default.
I have native IPv6 from my ISP. The ISP-supplied router handles v4 and v6 automatically. The router's firewall handles IPv6 exactly the same as it handles IPv4; if you want to open up ports, allow inbound/outbound connections, etc., it's all configured pretty much exactly as it was for v4. If people can handle configuring IPv4, then they can handle IPv6. Being "constantly pwned" is seriously overstating the risks.
https://bugs.freedesktop.org/s... is certainly one of the issues. But there are other situations which *do* cause log data loss. Google shows up plenty with little effort.
I'm actually using the correct terminology here. They are required to return the site to a "greenfield" state. And no land stays brown; it grasses over naturally very quickly so it's rather more accurate than brownfield.
I'm not sure what this has to do with NFS. Fails on Ubuntu 15.10 for me. But zero inodes doesn't look like a useful setting. Works with a positive count or if unspecified.
The NFS issue is likely it mounting the NFS filesystem before all the necessary RPC services are working, or something like that.
Seriously, have you even thought about this in any depth?
What's more safe and secure, many tens of thousands of lines of C running in PID0 with root privs, or interpreted scripts each running as root or with reduced privs in their own separate process?
It's obviously the latter. The only compiled code is the shell interpreter, which is well tested and used all over the place with root privs already. And the shell scripts being run are trusted. Now, you can argue all the other points about the downsides of shell vs unit files. But as for security, this one is plain and obvious. All that C is just dying for a trivial buffer overflow or some other standard exploit to allow a full compromise of the system from some valid or even user-supplied untrusted input. Yes, all that code is long overdue for a thorough audit; we're living on borrowed time as it is. How long to you think it will be until the first major exploit surfaces? Maybe then we'll have a rethink about the appalling design practices the systemd "geniuses" have foisted upon us. Having so much code running in the most critical and vulnerable part of the system is idiocy of the first order.
Sounds good... right up to the point when there's a problem. Then what happens? systemd notices that the log is corrupt, and... deletes it. No log of the problem. See: the long and angry (and unfixed) bugzilla tickets.
As for log compression, you don't want the actively written log to be compressed. If there's a problem, even as small as a single bit error, then the log will be unrecoverable. That's the tradeoff you make with compression. logrotate compromises by only compressing older logfiles. If there are any minor errors in the active log then you can still read it just fine.
As for tampering. It's of minor importance only. While the systemd people and their fanbois might harp on about it, they are catering for a problem which is of far less importance than hardware failure or power loss. Right now, all that foward hashing is so useless. If a simple power failure causes the checks to fail and the whole log to be discarded, it's a net negative. You threw away all my bloody logs! Like many of the systemd features, there's a whole fanfare about how essential it is and how everything else is awful and insecure. And as usual, there's some small credence to the claims. But it's massively overblown, and it has significant downsides. The "scary" things you mention are all of low likelihood--they only apply if your system has been compromised; there are rather more likely and less nefarious things to care about before these. I'd rather have a guarantee that my logs won't be deleted on a whim. Never, ever delete my logs!
As an example for the problems with troubleshooting, I've recently installed a few Ubuntu 15.10 systems (bare metal and in VMware VMs). In both scenarios, NFS filesystems fail to be mounted at boot. Why? I have no bloody clue. There's nothing in the logs. If I umount them (they are showing as mounted but give immediate I/O error on use) and remount by hand it all works just fine. No idea how to troubleshoot it. With sysvinit I could boot to an interactive shell in the late initramfs and step through every script by hand, and pinpoint exactly where something was failing. With systemd, even with correctly configured systems, I still experience occasional unrecoverable hangs at boot as it screws up its nondeterministic boot ordering and waits forever on something or other (who knows what it might be). systemd has absolutely not improved boot reliability. If I boot a FreeBSD system with exactly the same NFS configuration, it mounts everything perfectly at boot every damned time.
Another thing, is the broken compatibility with what came before. Example: I edit /etc/nfsidmap.conf to configure NFSv4 id mapping. Previously I'd reload/restart the nfs-common service with 'service' or 'invoke-rc.d'. Job done. What's the systemd equivalent? I don't know, and I can't see any obvious candidate. Since both these commands now forward to systemd, a diligent engineer would have made the old service names forward to the new target(s) to retain compatibility, maybe with a helpful message indicating what the new names were. There's nothing. I don't see any obvious nfsidmap or generic rpc or nfs units. Yes, it's partly me lacking familiarity with the new way of doing things; but the lack of care in having a smooth transition for admins makes for a terrible time, and the mess of units makes the whole thing difficult to comprehend as a whole and baroquely overcomplicated.
That's the reprocessing facility, Sellafield. And there's a reason for that particular building 30 being a mess if you read the history, not that it makes it right. Fuel is not left at power stations.
This isn't the US though and spent fuel won't be left lying around at the rector site indefinitely. As the article states the fuel will be offloaded and sent away to be reprocessed. Afterward, like other similar sites, they'll remove the turbine hall and other ancillary buildings, then leave the reactor to sit for a few decades to allow the residual radioactivity to decay to almost nothing before (carefully) demolishing it entirely. In a relatively short time, it will be a greenfield site you would never know had a reactor on it.
To be fair, the race existed in udev prior to the systemd merge as well. When lvremove randomly stops working, it's a bit surprising, and it took a while to pinpoint udev as the culprit keeping the snapshot devices open and preventing their removal. "Helpful" such behaviour is not. We had to move all the debian buildds from using lvm snapshots to unpacking tar files as a result (btrfs being too fragile as mentioned).
They definitely are. But it doesn't scale well. The time taken to scan the files and their contents on the source and destination system becomes overwhelming. The largest I've taken it to is a few terabytes, consisting of many thousands of directories each containing thousands of files (scientific imaging data). It ends up taking hours, where with ZFS it would take a few seconds. It also thrashes the discs on both systems as it scans everything, and uses a lot of memory. ZFS does none of these things--the send/recv is a "simple" streaming operation.
That said, with tools based upon rsync, like unison, it becomes possible to do two-way synchronisation which is pretty powerful. ZFS send/recv only works one-way. But again, the scan time with unison becomes prohibitive.
You do realise that Btrfs originated within Oracle, right? ZFS was merely acquired by them.
Without some kind of incremental snapshot, with read-only privileges after the snapshot, straight replication is next to useless if someone does "rm -rf /". And it happens *all the time*.
So, exactly what ZFS provides then... You take periodic snapshots (hourly, daily, weekly, or whatever), then send the deltas between the snapshots to the destination system. You can easily put that in a cron job and have a regular push to a backup system (hey, exactly like what the tool in TFA is doing...). If someone does wipe out all their files, you have the snapshot(s) containing it on both the source and destination system, depending upon your schedule for dropping old snapshots. However you decide to manage things, you can recover the removed files so long as they are present in an older snapshot.
Er, OpenZFS...
ZFS originated within Sun, which was bought by Oracle. Oracle then laid off most (all?) of the ZFS developers, who then went to work for other companies. The current ZFS development is no longer inside Oracle, and nor is it owned by them. They own the copyright on the original CDDL releases. Big deal. Not using it because of the historic association with Oracle would be a little... extreme.
Are you for real AC, or just trolling?
Your Synology "reference" is a classic "appeal to authority", only it's a really bad choice of authority due to its complete lack of any technical detail or substance of any kind. That link is to a marketing page for a company which makes money selling hardware. It's just a few bullet points (snapshotting, checksumming in essence), without any discussion of the actual tradeoffs or comparison with other systems. It's worthless. It's only purpose is to tick a feature box to act as an incentive to purchase their systems; as for the actual performance and reliability of those features--that's the customer's problem. Caveat emptor.
I've done more than casual work and development with Btrfs. For example, from back when I was a Debian developer, here's the original inital support for Btrfs snapshotting in schroot. This lets you create virtual environments from Btrfs snapshots, as well as other types such as LVM and overlays. You can then plug this into other tools such as sbuild, and then build the whole of Debian using snapshotted clean build environments. Doing this, Btrfs fails hard around every 18 hours, going read-only. Why? Creating and deleting 18000 snapshots for 8 parallel builds quickly unbalances the filesystem, requiring a manual rebalance. You don't see that unfortunate detail in the Synology fluff page, do you?
You can also get snapshots and decent recovery (albeit without block-level checksums) from LVM and mdraid. In my experience, its recovery behaviour after real hardware failure is vastly more reliable than Btrfs. Simply put, it has always resynched the data without problem, while Btrfs has caused irrecoverable data loss, despite it theoretically being much better. LVM snapshots have very different tradeoffs as well. And on modern Linux with udev, we had to abandon using them due to races in udev/systemd making them randomly fail.
The point I'm making is that the reality of the chosen tradeoffs between performance, reliability and featureset of the different filesystems is a subtle one. You can't reduce it down to "Btrfs is better" or "ZFS is better". That's marketing. But I have spent over seven years pushing Btrfs to its limits, and have found it sorely lacking. It's unacceptable that it unbalances itself to the point of unusability. It's unacceptable that it has led to irrecoverable dataloss on several occasions. It's also unacceptable that in its eight years of existence, none of the developers could be bothered to write any decent documentation. The dataloss was down to bugs, some of which are fixed, but it does leave you in a position of lacking trust in it in the face of such problems. If you compare this with ZFS, while it's not fair to say it has been totally bug free, it has been almost bug free, and the number of dataloss incidents is small. I've yet to encounter any problems with ZFS myself, but I've encountered many serious issues with Btrfs.
Anyone who uses Btrfs or ZFS on a NAS system does so at their own risk after researching the various options and their tradeoffs. Just because a vendor decides to make and market a system using Btrfs does not make that system the best choice. It just means they thought they could make some profit from it.
Er, no. Btrfs may one day make feature parity with ZFS, and it may also achive the reliability of ZFS, but it has a long, long, way to go in both areas to get to those points.
The on-disc structures might have been declared "stable", but what does that mean, really? That you'll be able to mount current filesystems on future kernels, yes. That the frozen design was correct and contains no design flaws? No. Personally, I think they froze it way too early. There are a number of fairly fundamental issues with the Btrfs design which compromise its performance (fsync) and integrity (unbalancing, data loss on recovery), and in some cases place arbitrary limits upon things (e.g. the hardlink issue). Some can be mitigated, while others can not. These and other issues are easily found and researched.
Seriously, I've been using Btrfs since very near the beginning for a variety of tasks. But I've been objective about it, rather than a blinkered fanboi. It's an interesting filesystem with some good ideas. But it has /always/ been a case of "next year it will be stable", and the performance is dire. Progress has been painfully slow, and the bugs I've encountered along the way have been numerous and show-stopping. Maybe it will "get there", but I think your assertion that "once BTFS userland side gets stable" that it will replace ZFS is incredibly naive. It assumes that there are no major issues remaining on the kernel side, and it also assumes that the only thing needing doing on the user side is stability. Based on its history to date, the likelihood of the kernel side being bug-free is close to zero. On the user side the tools are primitive, feature-incomplete and almost completely undocumented, containing little information and no examples. On the ZFS side, the tools are feature complete and are properly documented, with examples, and with whole sets of training material on top of that.
If you needed to make a decision on which to use for a serious deployment, or even just for a smaller scale home NAS, right now if you objectively compare the two, the choice is quite clear, and it's not Btrfs. Based upon the development history of the two, it's unlikely that this will change much in the next few years. Remember also that ZFS development is very active, perhaps even moreso than Btrfs. But who knows, maybe by 2020 Btrfs will surpass it.
There is no grey area with respect to the licensing. It's CDDL, a free software licence. It's 100% Free.
It might be incompatible with the GPL, but that's a non-issue. The userland tools are fine under this licence. The kernel modules are fine under this licence. Now, it means that the kernel modules aren't going to appear in a kernel release anytime soon, but that in no way makes for any legal problems in using them as loadable modules, today. It works fine from a technical point of view, and it's also fine from a legal point of view.