It's not too hard to do with a script. My snapshots are daily and they rotate every month leaving the first of every month as a longer term backup. It's granular enough for my data but hourly or even every minute would work fine too.
The recommendations vary a lot, mostly because it also depends on whether you're accessing a subset of your files more frequently than others. 1Gb per Tb seems good for most loads but ZFS will use as much as you throw at it.
Except FreeBSD can't do Xen + ZFS. BSD is good for a lot of things and so is Linux. A good sysadmin picks the right tool for the job. I'd like to think the BSD project benefits from more people using ZFS.
Have another look at the docs, you can upgrade capacity one disk at a time but, of course, throughput suffers because you now have asymmetric storage. You can also change level to some degree by grouping sets (eg turning a mirror into a stripped mirror)
But isn't that the whole point, ZFS is designed to avoid the most common failure modes but it relies on reducing the errors in the data it is using for check summing. Thats why ECC is important, RAM errors are the 2nd most common bit flip error and if that's your comparator, it needs to be accurate.
You could still use ZVOLs. I use ext4 on top of ZFS for some virtual machines. If you make sure to use the same block size in both the performance hit is small (approx 10%) and you will have ext4 with proper check summing and snapshot capabilities under it.
Because ZFS allows you to flexibly allocate, administer, snapshot and backup data in a coherent way. You can even use other file system on top of it if you want (zvols). Hardware level checksumming on drives is not aware of your file structure and a look at SMART info on a drive will reveal quite a few Unrecovered errors (density is a bitch). So I wouldn't rely on your drive to correct it's own errors.
You're talking about different levels. I use a zpool on SSDs for my databases AND give the databases lots of RAM. The two are not mutually exclusive but if course your RAM should be utilised as close to your query as possible to increase throughput. ZFS is for integrity first, squeezing speed out of it is certainly possible but not it's primary purpose.
ZFS-fuse is not ZFSonLinux! It uses an older version and is much buggier. Also which HDD controller are you using? This really matters cos ZFS should be talking to your drives directly and bad controllers sometimes do nasty things.
Actually it's pretty friendly on resources but likes lots of RAM to perform well (1Gb per Tb of storage is a good minimum). One of my servers runs on an atom processor (8x 3TB drives in equivalent to RAID 6 gets throughput of about 200MB/sec)
Adding disks is also a strength. You can grow data sets quite easily but naturally performance degrades until you update the whole drive set. A lot of RAID controllers can be put in HDA mode so you may be lucky. However the Adaptec controllers go cheap 2nd hand ($100).
One should also use SSDs with capacitors if they are backing a ZIL but the only things that are absolutely essential are decent HDD controllers (I've had good results with Adaptec SAS controllers, even using a mixed SAS/SATA drive set on separate channels)
The technical descriptions I've read say that you absolutely should use ECC because ZFS will eventually hit a checksum mismatch. This could result in valid data being flagged as corrupt. ECC RAM is not much more expensive these days but you do need a mobo that supports it.
ZFS is a layer below LVM. It's best to give it direct control over your drives (no hardware RAID). The reason for this is to allow it to do data integrity checks on the actual data being written. It's similarly fast compared to hardware RAID but guarantees data integrity in a much more compete fashion. I use a striped mirrored setup which is similar to RAID 10 (over 4x 3TB drives with caches on a pair of SSDs). If you cache like this, frequent reads don't need to go to the spindles. It also had built in compression and deduplication. The best thing IMO is instant snapshots though, that's one feature I can't believe I lived without.
I've been using ZFSonLinux for a year in production. No problems at all. It's my storage back end for Xen Virtual machines. Just make sure you use ECC RAM and a decent hard disk controller. Instant snapshots and ZFS send/receive functions are awesome, have reduced my backup times by an order of magnitude. I use a Debian Wheezy/Unstable hybrid.
Actually email intradomain is relatively secure. If it's send from a gs.com to gs.com address from an internal computer, the email should never leave the building (assuming they have their SMTP server onsite). This is one of the main advantages of a federated protocol
I agree, carbon sequestration technology requires use of the end products in ways that don't return the CO2 to the carbon cycle (or at least slow down turnover).
This is no different for any technology (including tree biotech). However, there is no technology that I have seen that is more efficient in life-cycle analysis than the well considered use of biological methods of CO2 harvesting.
Water use is also a key requirement of other CO2 harvesting techniques so not a specific negative of the use of plants.
It's lucky that we have so many different trees (and plants in general) that given a start can colonise almost any environment.
The 6% figure comes from Zhu, Ort, and Long 2008, see this post for some discussion: http://biology.stackexchange.c...
While the solar illumination -> biomass conversion is only about 6%, one needs to consider the whole life cycle of the technology. Trees have a number of difficult to beat efficiencies. Firstly, they do not require manufacturing (which uses a significant amount of energy and materials). Secondly, they do not require transport to site. They also self replicate which is a huge bonus that other technologies cannot currently compete with. Also, trees produce a variety of highly useful materials. Yes there is some need to refine these end products. To your second point, one does not want to use wood as fuel if the aim is to harvest CO2. Use as a building material, however is ideal and also saves more CO2 as concrete or steel are much more carbon intensive. If you look at the whole life cycle of a CO2 removal system, it is hard to beat, but please post back some figures if you think there is a tech with better efficiency considering these factors.
There is an amazing piece of technology that harnesses sunlight, converts water and CO2 into complex carbohydrates, useful proteins and even medicines. It self propagates and can be installed in a variety of environments. There is an existing harvesting infrastructure and it also produces an essential building material.
It is known as trees.
Should have had a look at the production curves, tight, shale and fracked sources have very rapid decline. It's pretty much the definition of unsustainable.
Slashdot Mirror
Actually it has 32GB of reg ECC RAM but that's not all for ZFS!
It's not too hard to do with a script. My snapshots are daily and they rotate every month leaving the first of every month as a longer term backup. It's granular enough for my data but hourly or even every minute would work fine too.
The recommendations vary a lot, mostly because it also depends on whether you're accessing a subset of your files more frequently than others. 1Gb per Tb seems good for most loads but ZFS will use as much as you throw at it.
You explained that so much better than I did!
Except FreeBSD can't do Xen + ZFS. BSD is good for a lot of things and so is Linux. A good sysadmin picks the right tool for the job. I'd like to think the BSD project benefits from more people using ZFS.
Have another look at the docs, you can upgrade capacity one disk at a time but, of course, throughput suffers because you now have asymmetric storage. You can also change level to some degree by grouping sets (eg turning a mirror into a stripped mirror)
But isn't that the whole point, ZFS is designed to avoid the most common failure modes but it relies on reducing the errors in the data it is using for check summing. Thats why ECC is important, RAM errors are the 2nd most common bit flip error and if that's your comparator, it needs to be accurate.
You could still use ZVOLs. I use ext4 on top of ZFS for some virtual machines. If you make sure to use the same block size in both the performance hit is small (approx 10%) and you will have ext4 with proper check summing and snapshot capabilities under it.
Because ZFS allows you to flexibly allocate, administer, snapshot and backup data in a coherent way. You can even use other file system on top of it if you want (zvols). Hardware level checksumming on drives is not aware of your file structure and a look at SMART info on a drive will reveal quite a few Unrecovered errors (density is a bitch). So I wouldn't rely on your drive to correct it's own errors.
You're talking about different levels. I use a zpool on SSDs for my databases AND give the databases lots of RAM. The two are not mutually exclusive but if course your RAM should be utilised as close to your query as possible to increase throughput. ZFS is for integrity first, squeezing speed out of it is certainly possible but not it's primary purpose.
ZFS-fuse is not ZFSonLinux! It uses an older version and is much buggier. Also which HDD controller are you using? This really matters cos ZFS should be talking to your drives directly and bad controllers sometimes do nasty things.
Good sane description here: http://ianhowson.com/do-you-re...
Because ZFS has far better features than BtrFS http://rudd-o.com/linux-and-fr...
Actually it's pretty friendly on resources but likes lots of RAM to perform well (1Gb per Tb of storage is a good minimum). One of my servers runs on an atom processor (8x 3TB drives in equivalent to RAID 6 gets throughput of about 200MB/sec) Adding disks is also a strength. You can grow data sets quite easily but naturally performance degrades until you update the whole drive set. A lot of RAID controllers can be put in HDA mode so you may be lucky. However the Adaptec controllers go cheap 2nd hand ($100).
One should also use SSDs with capacitors if they are backing a ZIL but the only things that are absolutely essential are decent HDD controllers (I've had good results with Adaptec SAS controllers, even using a mixed SAS/SATA drive set on separate channels)
The technical descriptions I've read say that you absolutely should use ECC because ZFS will eventually hit a checksum mismatch. This could result in valid data being flagged as corrupt. ECC RAM is not much more expensive these days but you do need a mobo that supports it.
ZFS is a layer below LVM. It's best to give it direct control over your drives (no hardware RAID). The reason for this is to allow it to do data integrity checks on the actual data being written. It's similarly fast compared to hardware RAID but guarantees data integrity in a much more compete fashion. I use a striped mirrored setup which is similar to RAID 10 (over 4x 3TB drives with caches on a pair of SSDs). If you cache like this, frequent reads don't need to go to the spindles. It also had built in compression and deduplication. The best thing IMO is instant snapshots though, that's one feature I can't believe I lived without.
I've been using ZFSonLinux for a year in production. No problems at all. It's my storage back end for Xen Virtual machines. Just make sure you use ECC RAM and a decent hard disk controller. Instant snapshots and ZFS send/receive functions are awesome, have reduced my backup times by an order of magnitude. I use a Debian Wheezy/Unstable hybrid.
me too, another vote for the E3000
Actually email intradomain is relatively secure. If it's send from a gs.com to gs.com address from an internal computer, the email should never leave the building (assuming they have their SMTP server onsite). This is one of the main advantages of a federated protocol
I agree, carbon sequestration technology requires use of the end products in ways that don't return the CO2 to the carbon cycle (or at least slow down turnover). This is no different for any technology (including tree biotech). However, there is no technology that I have seen that is more efficient in life-cycle analysis than the well considered use of biological methods of CO2 harvesting. Water use is also a key requirement of other CO2 harvesting techniques so not a specific negative of the use of plants.
It's lucky that we have so many different trees (and plants in general) that given a start can colonise almost any environment. The 6% figure comes from Zhu, Ort, and Long 2008, see this post for some discussion: http://biology.stackexchange.c...
While the solar illumination -> biomass conversion is only about 6%, one needs to consider the whole life cycle of the technology.
Trees have a number of difficult to beat efficiencies.
Firstly, they do not require manufacturing (which uses a significant amount of energy and materials). Secondly, they do not require transport to site. They also self replicate which is a huge bonus that other technologies cannot currently compete with. Also, trees produce a variety of highly useful materials. Yes there is some need to refine these end products.
To your second point, one does not want to use wood as fuel if the aim is to harvest CO2. Use as a building material, however is ideal and also saves more CO2 as concrete or steel are much more carbon intensive.
If you look at the whole life cycle of a CO2 removal system, it is hard to beat, but please post back some figures if you think there is a tech with better efficiency considering these factors.
There is an amazing piece of technology that harnesses sunlight, converts water and CO2 into complex carbohydrates, useful proteins and even medicines. It self propagates and can be installed in a variety of environments. There is an existing harvesting infrastructure and it also produces an essential building material. It is known as trees.
Should have had a look at the production curves, tight, shale and fracked sources have very rapid decline. It's pretty much the definition of unsustainable.
http://www.theoildrum.com/node...