However, the most obvious reading of the poster's comment seemed to insinuate a belief that Voyager's present velocity is being maintained by its power source.
I didn't want to obfuscate the very basic physics point—that Voyager running out of power has nothing to do with its velocity. I briefly considered adding the provisos you listed but decided it wasn't indicated in that context.
Anyway, like I said, perhaps the poster meant something else.
Voyager 1 won't escape the Oort cloud (really the outer Oort cloud) for another 14,000 - 28,000 years. (Probably due to running out of power in the next 10 to 15 years.)
Perhaps I have misinterpreted your statement, but are you aware of Newton's First Law of Motion? Voyager has no need for power to continue its journey; running out of power will have no effect on its velocity.
My guess is that, aside from attitude adjustment, Voyager hasn't fired its thrusters since its encounter with Titan in 1980.
Rare earths are – well – rare. (Well, not really – but they are tricky to mine and refine without causing pollution.)
FTFY. Rare earths are not particularly rare; we have massive reserves in the US. It's just that our last mine closed several years ago due to the difficulty complying with our environmental regulations. Last year I read about one company that thinks they have developed a viable, minimally polluting extraction process which may allow our domestic reserves to be tapped again.
In effect, right now other countries are displacing their pollution to China vis-a-vis rare earths. The worst China could do in this trade war is cause other countries to decide to allow temporary pollution regulation waivers which would bring the countries' own domestic reserves back online.
That's not exactly the high-tech device apocalypse that the media typically makes it appear to be.
I had to load about six different ROM images onto my phone before I found one that would let me use GPS without leaking information to Google.
Interesting, how did you verify this? EtherApe?
Personally, I just loaded DroidWall blacklisted the kernel and the GPS process and called it good. Didn't bother to check further, because I rarely enable GPS anyway.
OpenPDroid takes care of the rest for me. I just block apps from being able to access my GPS or network location. Sometimes I like to have fun: I have OpenPDroid return false data to individual apps... "Why yes, I *am* at the Tunguska Event site in Siberia, and—wow!—my phone number coincidentally is 123-456-7890! What special promotions do you have for me today? Oh, you want to access my camera? Sorry, it's mysteriously offline right now.".
Use a CyanogenMod, AOKP, or AOSP ("vanilla") ROM with OpenPDroid injected into it. Check the screenshots for the PDroid Manager app (configures the OS-framework hooks) to see what I mean.
Yes, you usually have to inject it yourself using the auto-patcher utility, but OpenPDroid solves all your privacy concerns (and far better than any iOS solution can ever do).
So, this solution requires the ability to use a command line (and install cygwin if you are a windows user); however, you're unlikely to ever find a mainstream device OS that is as privacy-secured for the user as you are able to achieve with OpenPDroid.
If you are really paranoid, also use DroidWall to prevent certain apps from having access to the network via iptables rules.
I never liked the HOSTS file approach to blocking ads, even on Android.
Instead, what I did was to setup a VPS with dnsmasq that is configured with entries from the latest ad-server namelists; now all my devices and machines can hit a single, poisoned DNS that is simple to maintain. As a bonus, it reduces latency because I also have httpd configured with mod_rewrite; the poisoned DNS entry in dnsmasq is the IP of the httpd instance which returns a single-byte-of-content, HTTP-200 (OK) response for any URL that is requested. Furthermore, most webpages reflow nicely if they get a basically-empty HTTP 200 response for a resource request, which makes this approach superior to the standard HOSTS file implementation that points back to localhost (ie. there are no "server is not listening" error messages in iframes, etc).
I cron'd the VPS to pull multiple ad-server namelist sources daily, update the poisoned entries in dnsmasq, and kick the service.
Unlike GOA, the NRA's record for supporting and protecting rights is abysmal. It seems the NRA rarely encounters a compromise on our rights that they don't enthusiastically endorse.
The NICS federal background check system for gun sales? An NRA idea.
Gun free school zones? NRA supported those for years.
Stripping depressed/PTSD veterans of their right to buy firearms without due process? NRA partnered with Charles Schumer on that one (WTF? That's like the NOW partnering with Mike Huckabee on an abortion restriction law)
The list goes on and on. I was honestly surprised when the NRA decided to take a stand against the current encroachment attempts, given that they almost supported the 1994 Crime Bill "assault weapons" and magazine capacity bans.
(Disclaimer: I am a GOA lifetime member, so feel free to proceed with your ad hominem attacks)
Please re-read my comment. I was suggesting what I perceive to be a better argument for your position, because the obvious (fallaciously simplistic) retort to a remark about a 25% drop in value is to cite the 200+% return so far in 2013. It saves effort to leapfrog the specious counterpoint by citing the obvious bubble that will inevitably crash soon.
Haha, not everyone who replies to you is a True Believer In The Inevitability Of Bitcoin.
Every 6 months we get a story about how the value of BTC has dropped 25% again and theres been a hacking incident at some exchange or something, but its OK because its mathematically proveable and thus a viable currency.
True, but I think a better argument for you is the fact that despite these periodic "corrections" the price is still continuing to rise precipitously. The volatility is insane, but how viable is it that the price on 1 Jan 2013 was 1 BTC/~$15, and right before this latest glitch, it was pushing ~$50 per BTC?
Look at the chart. See the exponential growth? The value has more than tripled in the 71 days so far in 2013. If the present bubble is extrapolated that represents roughly a 49,000% APY; clearly, that growth is unsustainable even in the face of "pigheaded wishful thinking".
Want to buy some tulip bulbs? I hear they are a "can't lose" investment!
Are you certain you didn't lose track of the units?
I calculate five IPs per person, even if we force all the poor 1.5E14 kg chunks of mass of the earth behind NAT so we can steal their routable addresses for human use.
I was bitter about the whole thing for a long time, until I learned the failure part...which made a tiny part of me very happy indeed.
Ah, schadenfreude—it has always seemed historically inevitable to me that the word with this definition would have German etymology. "There's a word for this? And it's German? Quelle surprise."
There's been a lot of saber rattling between Japan and China in the last few months over minor islands of little value.
The Senkaku Islands dispute is less about dick waving and ultimately more about oil reserves. International maritime law allows for exclusive economic zones for some distance off a country's territorial shores. Thus, these otherwise-worthless islands actually have some value beyond distracting both countries' populations from domestic issues. Also, like every good territorial dispute requires, both sides have their own preferred "true name" for the disputed territory.
This dispute is somewhat analogous to the US' Aroostook War over Maine's borders or the Oregon Boundary Dispute ("54 40' or Fight!")—both of which were solved without war despite the bellicose rhetoric.
Hey, I didn't notice your comment about being employee 0 at Digital Convergence—that's cool. I saw you said you were opposed to the invasive tracking that caused the uproar. Haha, ironically the DC tracking probably would have been less invasive than what modern ad networks do.
However, if they *hadn't* done the tracking, etc, what would have been the business model? Give the CueCats away at a loss, but make it up in volume?
I suppose if the CueCats had smart card chips for encryption with server-generated keys to prevent replay attacks (or something similar) then they might have been able to force users to only use their site for the barcode lookups. Otherwise, I imagine the endgame would have been the same.
What privacy invading issues might you be referring to?
Each CueCat has a unique identifier that is appended to the scanned encrypted data. The original software was designed to track you based on everything you scanned.
Unfortunately for Digital Innovations, their ub3r 1337 h4x0r engineers decided that "base64 encoding + constant XOR == encryption". Fail. So, alternate software was quickly created to decode CueCat output, and the CueCats were thus rendered simple, free barcode scanners.
In retrospect, this whole debacle may have been the first lolcat. Heh.
OMG, could have I been right 15 years ago when I perceived sudo as a security hole?
Yes, sudo is a security hole.
I never seemed any gain in using sudo, especially when most sysadmin allow users to do "sudo bash".
Not only is it usually allowed, it's actually very difficult to prevent a user with sudo rights from being able to escalate that to a root shell. Read the sudo man page regarding shell escapes. "Due to the large number of programs that offer shell escapes, restricting users to the set of programs that do not is often unworkable. and "Note that restricting shell escapes is not a panacea."
... and what if some user uploads a statically compiled bash binary (cleverly named something else) and sudo's *that*?
it doesn't protect you from anything compared to plain su.
Whoa, there... that's a bridge too far.
sudo prevents you from having to share a single root password with everyone who needs to su. It's much easier to revoke an individual user's sudo rights than it is to come up with a new root password and communicate it with "everyone but the now-revoked person". Besides, people are more likely to share a root password that "everyone knows" than they are to share their personal login creds.
Besides, you can setup nice logging with sudo that gives you per-user security audits. This is useful in professional circumstances, because it allows you to enforce policies like "do not sudo a shell, or you will be fired". And if someone *does* "sudo su" and then nukes the security audit log to cover their tracks then that's not only a termination-worthy offense, it's also likely a criminal act in many jurisdictions.
And, yes, I have configured sudo on production machines for users who were restricted to executing only a handful of explicitly designated scripts, so sudo *was* far more secure than giving them su in that context.
Sudo vs. su on your machine at home for which you are the only person with a login? Meh. It doesn't really matter, so go with your preference.
What happens when the disk is full? In other filesystems, I can change the contents of a sector on a full disk, but I can't do anything that requires allocating more space.
That's a very degenerate use case, unless you are referring to an esoteric situation I can't envision at the moment.
But to answer your question, at the moment my ZFS pools are configured to have 4k blocks; obviously, there is file system overhead for the ZFS uberblock metadata, copy-on-write space, etc. However, are you seriously suggesting it's a significant advantage to be able to squeeze out a few more handfuls of KB space on a TB-sized pool?
Besides, I would gladly trade an "edit files on a completely filled disk" capability for the "free snapshot" capability that ZFS offers due to copy-on-write.
Likewise, how does it update file system metadata and actual file data in an atomic manner? Unless those are stored on the same sector, then even gathering them up into a single multiple-sector write operation is not atomic, as the first sector can be written, and then failure happens.
A simplified example of copy-on-write goes something like this (no doubt ZFS has some specific variations on the theme): 1) A three block file (A,B,C) is going to have block B overwritten by a program. 2) A new block, D, is allocated and has a copy of B's data written to it. 3) The program's write data changes the data in D. 4) The block pointer for the file's block chain is changed so that the second block in the file points to D. The block pointer is small enough that it is a single, atomic write operation at the hardware level. 5) The file is now A,D,C, and B is marked reusable.
The concept is trivially extensible to ensure multiple-sector writes are atomic... just do a chain of COW blocks and only update the original file's block pointer atomically as the last step, to swap one block chain linked-list pointer for another. As you can see, if you interrupt the write procedure at any point before completion then the original file hasn't been changed. Filesystem metadata is similarly protected by ZFS internal transactional semantics.
As for whether multiple-write operations are flushed at any given moment, that depends on your filesystem and/or process' strategy. On one extreme there is PostgreSQL, which calls fsync() after every operation. On the other extreme, there's ext4 which had (has?) a default transaction flush window of 30+ seconds unless an explicit fsync() was invoked. These pending writes haven't even made it to the controller yet, because they are being aggregated by the OS. This is intended to increase performance, but leads to data loss unless the OS can be kept running quiescently for at least 30 seconds or put through an orderly shutdown. Also, by default ext4 will "helpfully" reorder writes for you, which also can cause massive data loss in this scenario.
Now, if you were referring to what happens if a user process has only generated 75% of a new file when the power plug is yanked, then that's a problem that process resumption logic, a UPS, and/or an orderly shutdown are intended to solve.
Also, you can see how COW makes cheap, point-in-time snapshots trivial... just clone the original file block pointers and now you have a snapshot of the file as it was. This only requires space on the order of the diff size between the filesystem dataset's current state and its state at the time of the snapshot. Essentially, taking a snapshot prevents the "mark the old blocks reusable" step from happening until the snapshot is destroyed.
Although there are many ways to make the chance of inconsistent data be extremely rare, even if hardware tells the truth and doesn't return from a write request until it is really written, there is no way to completely eliminate it.
Stipulated. I believe you are referring, at least in part, to the Two Generals' Problem.
At some point, the data on any writable disk is inconsistent, even with a journal system.
ZFS is copy-on-write, ie. it is not a journaled filesystem. It is transaction-based and atomic; it does not traverse an inconsistent state because it does not overwrite the data during a write.
but data can be lost, simply because data that hasn't been written to disk yet can't be if power is lost.
Right, no doubt you have deduced I am making a distinction between data inconsistency (corruption) and "lost writes". Note that because ZFS is a "rampant layering violation", lost writes cannot cause inconsistency/corruption of the filesystem itself (except in degenerate scenario 2 below).
The primary data "loss" scenarios for ZFS during power loss are:
1) File writes that haven't been flushed yet. Note that this does not lead to filesystem inconsistency, just lost writes. A battery-backed write cache won't fix this—if this scenario is a concern, then you really need a UPS for the machine and an orderly shutdown. That is to say, if pending asynchronous writes being flushed is a concern then you're probably also concerned about currently-executing application processes that may not be complete.
2) Hardware that lies/breaks spec and acks a synchronize-cache command when it hasn't really done so. This causes much rage in the ZFS developers, because there's no true fix for it besides suggest people not use substandard hardware. Note that this *can* cause ZFS to lose data. Would this scenario corrupt a system with battery-backed RAID write cache as well? The drive fraudulently swears it has flushed the data... would the controller double-check later by reading the data back in just to see if it was lying? Even if the controller did try to check, how would it know that the drive wasn't responding from its own cache?
This is where battery-backed cache can help, as the OS thinks it has been written, but it hasn't yet.
I presume you are referring to enabling write caching on your controller, and asserting it is safe to do so due to the battery-backed cache.
I specced an IBM System x3650 M3 with a similar system for a production PostgreSQL server. God, I loved that machine... you could even make a 96 core/1 TB RAM Voltron-like single logical machine by plugging four of them together. Anyway, the intention of the cache was to be able to safely disable fsync() in PG without risk of data corruption. The IBM system took it one step further: the battery-backed cache had the ability to be transferred to another x3650 and replay the cache on the other machine in case of catastrophic failure. Anyway, I never bothered to enable the feature because we never became I/O bound on writes with fsync() enabled while I was there. It did give us headroom, though.
Anyway, the obvious perf advantage is to allow the hardware to lie and assert sync has happened even though it hasn't yet.
On my home ZFS array I have a allocated part of my OCZ Vertex 4 SSD for my RAID zpool's ZIL, which serves a similar acceleration function for synchronous writes (only). The ZIL acts like a protected write cache, but only in the case of power failure. Essentially, the in-RAM ZFS pending write cache is mirrored to the ZIL device(s) (if you have them). If the ZIL device acks a sync of the ZFS cache data written to it, then ZFS counts the overall write to the destination volume as complete (because the transactions are now replayable if power is lost) and therefore allows the OS sync() to return.
Also note that I misspoke, because the Adaptec 5805Z that I have uses a capacitor, not a battery.
By design, the ZIL is never read unless there is a power-failure. The Vertex 4 is supposed to have supercaps to ensure flush, but given
With battery-backed cache, you don't really need to worry about this. With battery-backed cache that dumps to flash (what I use), you really don't need to worry about this.
Haha, fair enough. I suppose my rejoinder would be that with RAID-Z my data on the array is never in an inconsistent state while writing, so I don't have to worry about obtaining a battery-backed cache or a UPS and then ensuring the batteries are still good after a few years.
what do you use for raid-z, raid-z2? (i.e. mdadm equivalent)?
ZFS has been called a "rampant layering violation" by Linux kernel devs because it's vertically integrated from the block device to the user-visible filesystem. All ZFS (filesystem, RAID-Z, RAID mirroring, filesystem checking, etc) is handled by the two ZFS tools: zpool and zfs.
zpool is the equivalent of mdadm, Linux LVM, fsck, etc. zfs handles dataset management, snapshotting, is the default replacement for fstab for ZFS volumes, etc.
ZFS seemed intimidating to me at first, but I discovered that even configuration via the command line is trivial.
For example: zpool create myArray raidz2/dev/sd1/dev/sd2.../dev/sdn
...will create a RAID-Z2 pool called myArray, using the devices you listed, allocate it, format it, and mount it automatically. It is smart enough to tolerate non-deterministic device enumeration when you reboot/hotplug.
Features like snapshots (these are really cheap in terms of disk space due to ZFS copy-on-write) are similarly simple: zfs snapshot myArray@20130306_weekly
...will create a read-only snapshot of your whole array at the instant it is executed that is named "20130306_weekly". This requires no disk space until you start making changes to data on myArray. So, it's like Time Machine in that regard (easy restore if you accidentally rm -rf/myArray/important_stuff, and the snapshot command is trivially cron'able). ZFS theoretically supports up to 2^64 snapshots at once. You can also clone a read-only snapshot to make a read-write copy if you want to do experimentation on something. The only disk space required for that will be the diffs between the clone and the current state of the snapshotted zpool.
zpool scrub myArray
...will execute an integrity check of the pool, repair any bitrot in a redundant pool (RAID-Z or mirror), etc. As I said, this happens asynchronously while the device is online and available for RW I/O. It doesn't appreciably affect system throughput, because it's at low priority. Obviously cron'able as well.
However, like I said, FreeNAS lets you do the ZFS administration via a dialog-based web GUI if you prefer. By default, FreeNAS boots from a USB stick, though you can install it on a disk.
FreeNAS is based on FreeBSD, because the ZFS CDDL isn't compatible with the GPL. So, ZFS is unlikely to ever be included in the Linux kernel, though the ZFS On Linux project is attempting to provide source rpms to compile into a standard Linux distro.
The new Linux filesystem "alternative" to ZFS will be btrfs, but because it "respects boundaries" it is unlikely to ever have anything like RAID-Z, which would require btrfs to "wrongly" assume responsibilities handled by the lower-level Linux LVM/mdadm.
I realize the risk of UREs increases with larger drives, but as far as I know this isn't exacerbated with RAID.
You are correct. For example with the WD Red drives I am using, they cite a URE of "<1 per 10^14 bits", which works out to be "less than one per 11.3 TB" (how much less?). At first glance, that doesn't seem like a big deal. That's just a little bit-rot, right? Well, drives are sector-based, so let's say you hit an average of one 4K sector with a URE per 11.3 TB.
In normal RAID-5 operation, the system will be able to correct for that error. However, what happens when you've dropped a drive and are attempting to restripe the array onto a spare/replacement? You need to be able to read all the data on all the remaining drives in order to reconstruct the lost drive. In your case, it appears that would be a total read of 16 TB. It has to read all sectors on the remaining four 4 TB drives because standard, non-RAID-Z arrays are below the filesystem, so the array has no idea whether you are actually using 15 MB or 15 TB of the space your RAID-5 array. While reading in the 16 TB, your remaining HDDs will hit an average of one URE, and now your RAID rebuild has failed by definition. Again, though, who cares if you lose just one stripe in the array per RAID-5 rebuild?
Well, it depends on how your RAID system treats it. If it is programmed to halt the rebuild and mark it as failed, then you are left hoping you can backup all the data and nuke/recreate the array before another drive is lost. At the very least, that would be a significant inconvenience compared to the expected "plug in new drive and wait for rebuild".
Hopefully this 20TB RAID will last like 5 years, and I'll move the data to a new storage system using some better technology that's cheaper, faster, and safer.
I recommend you consider RAID-Z when you are weighing alternatives (haha, obvious plug is obvious). RAID-Z is single parity and so tolerates a single drive failure like RAID-5, RAID-Z2 is double parity like RAID-6, and RAID-Z3 is triple parity.
There are a lot of benefits that RAID-Z offers that can only be achieved by pushing the redundancy considerations into the filesystem, such as only having to read the data in your array that you've actually used when you are attempting to recover from a lost drive (rather than the whole disk). It's also demonstrably safer than RAID-5/6 equivalents, offers simple snapshotting of live volumes to make in-place time machine type backups, the ability to do the equivalent of chkdsk/fsck while the volume is online and accepting RW I/O, allows the addition of SSD cache devices to automatically accelerate the pool, etc. The principal drawbacks today are hardware requirements (performance is best if you have 1GB RAM per TB of disk and a fast, 64-bit CPU) and OS support (though FreeNAS makes RAID-Z effectively painless already and has lots of cool features).
All this isn't to bash RAID-5. I had a hardware-based 4x80GB RAID-5 at home that served me well for over a decade. Then again, when I recently booted up that server to finally offload the data I discovered it had dropped a drive and another refused to enumerate, so the array was toast.
Even double-parity RAID isn't worth it with consumer drives. You'll find out you lose multiple disks due to point failures too easily. RAID-Z2 or bust IMO.
Right, I decided to select the WD Red drives with TLER even though I am implementing RAID-Z2 and it isn't clear that TLER is necessary for ZFS. Anything to avoid resilvering a 20 TB encrypted array...
As a unexpected bonus, these WD Red drives run far cooler than my WD RE4-GP (which seems ironic, given the marketing).
Anyway, my point is that RAID-5 isn't the level of data integrity assurance it once was, even ignoring the RAID-5 write hole risk that is inherent to the RAID-5 and RAID-6 concept.
Slashdot Mirror
Here, if this ameliorates your outrage then this link works as well:
http://yro.slashdot.org/story/13/03/20/0256259/possible-chemical-weapons-use-in-syria
As does this:
http://developers.slashdot.org/story/13/03/20/0256259/possible-chemical-weapons-use-in-syria
And this:
http://diceholdingssucks.slashdot.org/story/13/03/20/0256259/possible-chemical-weapons-use-in-syria
DNS wildcard fun!
Certainly.
However, the most obvious reading of the poster's comment seemed to insinuate a belief that Voyager's present velocity is being maintained by its power source.
I didn't want to obfuscate the very basic physics point—that Voyager running out of power has nothing to do with its velocity. I briefly considered adding the provisos you listed but decided it wasn't indicated in that context.
Anyway, like I said, perhaps the poster meant something else.
Voyager 1 won't escape the Oort cloud (really the outer Oort cloud) for another 14,000 - 28,000 years. (Probably due to running out of power in the next 10 to 15 years.)
Perhaps I have misinterpreted your statement, but are you aware of Newton's First Law of Motion? Voyager has no need for power to continue its journey; running out of power will have no effect on its velocity.
My guess is that, aside from attitude adjustment, Voyager hasn't fired its thrusters since its encounter with Titan in 1980.
Rare earths are – well – rare. (Well, not really – but they are tricky to mine and refine without causing pollution.)
FTFY. Rare earths are not particularly rare; we have massive reserves in the US. It's just that our last mine closed several years ago due to the difficulty complying with our environmental regulations. Last year I read about one company that thinks they have developed a viable, minimally polluting extraction process which may allow our domestic reserves to be tapped again.
In effect, right now other countries are displacing their pollution to China vis-a-vis rare earths. The worst China could do in this trade war is cause other countries to decide to allow temporary pollution regulation waivers which would bring the countries' own domestic reserves back online.
That's not exactly the high-tech device apocalypse that the media typically makes it appear to be.
I had to load about six different ROM images onto my phone before I found one that would let me use GPS without leaking information to Google.
Interesting, how did you verify this? EtherApe?
Personally, I just loaded DroidWall blacklisted the kernel and the GPS process and called it good. Didn't bother to check further, because I rarely enable GPS anyway.
OpenPDroid takes care of the rest for me. I just block apps from being able to access my GPS or network location. Sometimes I like to have fun: I have OpenPDroid return false data to individual apps... "Why yes, I *am* at the Tunguska Event site in Siberia, and—wow!—my phone number coincidentally is 123-456-7890! What special promotions do you have for me today? Oh, you want to access my camera? Sorry, it's mysteriously offline right now.".
Use a CyanogenMod, AOKP, or AOSP ("vanilla") ROM with OpenPDroid injected into it. Check the screenshots for the PDroid Manager app (configures the OS-framework hooks) to see what I mean.
Yes, you usually have to inject it yourself using the auto-patcher utility, but OpenPDroid solves all your privacy concerns (and far better than any iOS solution can ever do).
So, this solution requires the ability to use a command line (and install cygwin if you are a windows user); however, you're unlikely to ever find a mainstream device OS that is as privacy-secured for the user as you are able to achieve with OpenPDroid.
If you are really paranoid, also use DroidWall to prevent certain apps from having access to the network via iptables rules.
I never liked the HOSTS file approach to blocking ads, even on Android.
Instead, what I did was to setup a VPS with dnsmasq that is configured with entries from the latest ad-server namelists; now all my devices and machines can hit a single, poisoned DNS that is simple to maintain. As a bonus, it reduces latency because I also have httpd configured with mod_rewrite; the poisoned DNS entry in dnsmasq is the IP of the httpd instance which returns a single-byte-of-content, HTTP-200 (OK) response for any URL that is requested. Furthermore, most webpages reflow nicely if they get a basically-empty HTTP 200 response for a resource request, which makes this approach superior to the standard HOSTS file implementation that points back to localhost (ie. there are no "server is not listening" error messages in iframes, etc).
I cron'd the VPS to pull multiple ad-server namelist sources daily, update the poisoned entries in dnsmasq, and kick the service.
Solved once, solved everywhere.
Eat that, ad networks.
Unlike GOA, the NRA's record for supporting and protecting rights is abysmal. It seems the NRA rarely encounters a compromise on our rights that they don't enthusiastically endorse.
The NICS federal background check system for gun sales? An NRA idea.
Gun free school zones? NRA supported those for years.
Stripping depressed/PTSD veterans of their right to buy firearms without due process? NRA partnered with Charles Schumer on that one (WTF? That's like the NOW partnering with Mike Huckabee on an abortion restriction law)
The list goes on and on. I was honestly surprised when the NRA decided to take a stand against the current encroachment attempts, given that they almost supported the 1994 Crime Bill "assault weapons" and magazine capacity bans.
(Disclaimer: I am a GOA lifetime member, so feel free to proceed with your ad hominem attacks)
Indeed, I might expect that this entire paragraph could be accurately surmised in one or two German words.
Yes, but I still believe it's cheating when they can compound nouns to make words like Donaudampfschiffahrtselektrizitätenhauptbetriebswerkbauunterbeamtengesellschaft.
Cheating, but awesome.
Please re-read my comment. I was suggesting what I perceive to be a better argument for your position, because the obvious (fallaciously simplistic) retort to a remark about a 25% drop in value is to cite the 200+% return so far in 2013. It saves effort to leapfrog the specious counterpoint by citing the obvious bubble that will inevitably crash soon.
Haha, not everyone who replies to you is a True Believer In The Inevitability Of Bitcoin.
Every 6 months we get a story about how the value of BTC has dropped 25% again and theres been a hacking incident at some exchange or something, but its OK because its mathematically proveable and thus a viable currency.
True, but I think a better argument for you is the fact that despite these periodic "corrections" the price is still continuing to rise precipitously. The volatility is insane, but how viable is it that the price on 1 Jan 2013 was 1 BTC/~$15, and right before this latest glitch, it was pushing ~$50 per BTC?
Look at the chart. See the exponential growth? The value has more than tripled in the 71 days so far in 2013. If the present bubble is extrapolated that represents roughly a 49,000% APY; clearly, that growth is unsustainable even in the face of "pigheaded wishful thinking".
Want to buy some tulip bulbs? I hear they are a "can't lose" investment!
Are you certain you didn't lose track of the units?
I calculate five IPs per person, even if we force all the poor 1.5E14 kg chunks of mass of the earth behind NAT so we can steal their routable addresses for human use.
I was bitter about the whole thing for a long time, until I learned the failure part...which made a tiny part of me very happy indeed.
Ah, schadenfreude—it has always seemed historically inevitable to me that the word with this definition would have German etymology. "There's a word for this? And it's German? Quelle surprise."
There's been a lot of saber rattling between Japan and China in the last few months over minor islands of little value.
The Senkaku Islands dispute is less about dick waving and ultimately more about oil reserves. International maritime law allows for exclusive economic zones for some distance off a country's territorial shores. Thus, these otherwise-worthless islands actually have some value beyond distracting both countries' populations from domestic issues. Also, like every good territorial dispute requires, both sides have their own preferred "true name" for the disputed territory.
This dispute is somewhat analogous to the US' Aroostook War over Maine's borders or the Oregon Boundary Dispute ("54 40' or Fight!")—both of which were solved without war despite the bellicose rhetoric.
Hey, I didn't notice your comment about being employee 0 at Digital Convergence—that's cool. I saw you said you were opposed to the invasive tracking that caused the uproar. Haha, ironically the DC tracking probably would have been less invasive than what modern ad networks do.
However, if they *hadn't* done the tracking, etc, what would have been the business model? Give the CueCats away at a loss, but make it up in volume?
I suppose if the CueCats had smart card chips for encryption with server-generated keys to prevent replay attacks (or something similar) then they might have been able to force users to only use their site for the barcode lookups. Otherwise, I imagine the endgame would have been the same.
What privacy invading issues might you be referring to?
Each CueCat has a unique identifier that is appended to the scanned encrypted data. The original software was designed to track you based on everything you scanned.
Unfortunately for Digital Innovations, their ub3r 1337 h4x0r engineers decided that "base64 encoding + constant XOR == encryption". Fail. So, alternate software was quickly created to decode CueCat output, and the CueCats were thus rendered simple, free barcode scanners.
In retrospect, this whole debacle may have been the first lolcat. Heh.
Can you make string cheese from the goat milk?
Yes, and as a bonus it also flosses your teeth while you eat it.
OMG, could have I been right 15 years ago when I perceived sudo as a security hole?
Yes, sudo is a security hole.
I never seemed any gain in using sudo, especially when most sysadmin allow users to do "sudo bash".
Not only is it usually allowed, it's actually very difficult to prevent a user with sudo rights from being able to escalate that to a root shell. Read the sudo man page regarding shell escapes. "Due to the large number of programs that offer shell escapes, restricting users to the set of programs that do not is often unworkable. and "Note that restricting shell escapes is not a panacea."
... and what if some user uploads a statically compiled bash binary (cleverly named something else) and sudo's *that*?
it doesn't protect you from anything compared to plain su.
Whoa, there... that's a bridge too far.
sudo prevents you from having to share a single root password with everyone who needs to su. It's much easier to revoke an individual user's sudo rights than it is to come up with a new root password and communicate it with "everyone but the now-revoked person". Besides, people are more likely to share a root password that "everyone knows" than they are to share their personal login creds.
Besides, you can setup nice logging with sudo that gives you per-user security audits. This is useful in professional circumstances, because it allows you to enforce policies like "do not sudo a shell, or you will be fired". And if someone *does* "sudo su" and then nukes the security audit log to cover their tracks then that's not only a termination-worthy offense, it's also likely a criminal act in many jurisdictions.
And, yes, I have configured sudo on production machines for users who were restricted to executing only a handful of explicitly designated scripts, so sudo *was* far more secure than giving them su in that context.
Sudo vs. su on your machine at home for which you are the only person with a login? Meh. It doesn't really matter, so go with your preference.
What happens when the disk is full? In other filesystems, I can change the contents of a sector on a full disk, but I can't do anything that requires allocating more space.
That's a very degenerate use case, unless you are referring to an esoteric situation I can't envision at the moment.
But to answer your question, at the moment my ZFS pools are configured to have 4k blocks; obviously, there is file system overhead for the ZFS uberblock metadata, copy-on-write space, etc. However, are you seriously suggesting it's a significant advantage to be able to squeeze out a few more handfuls of KB space on a TB-sized pool?
Besides, I would gladly trade an "edit files on a completely filled disk" capability for the "free snapshot" capability that ZFS offers due to copy-on-write.
Likewise, how does it update file system metadata and actual file data in an atomic manner? Unless those are stored on the same sector, then even gathering them up into a single multiple-sector write operation is not atomic, as the first sector can be written, and then failure happens.
A simplified example of copy-on-write goes something like this (no doubt ZFS has some specific variations on the theme):
1) A three block file (A,B,C) is going to have block B overwritten by a program.
2) A new block, D, is allocated and has a copy of B's data written to it.
3) The program's write data changes the data in D.
4) The block pointer for the file's block chain is changed so that the second block in the file points to D. The block pointer is small enough that it is a single, atomic write operation at the hardware level.
5) The file is now A,D,C, and B is marked reusable.
The concept is trivially extensible to ensure multiple-sector writes are atomic... just do a chain of COW blocks and only update the original file's block pointer atomically as the last step, to swap one block chain linked-list pointer for another. As you can see, if you interrupt the write procedure at any point before completion then the original file hasn't been changed. Filesystem metadata is similarly protected by ZFS internal transactional semantics.
As for whether multiple-write operations are flushed at any given moment, that depends on your filesystem and/or process' strategy. On one extreme there is PostgreSQL, which calls fsync() after every operation. On the other extreme, there's ext4 which had (has?) a default transaction flush window of 30+ seconds unless an explicit fsync() was invoked. These pending writes haven't even made it to the controller yet, because they are being aggregated by the OS. This is intended to increase performance, but leads to data loss unless the OS can be kept running quiescently for at least 30 seconds or put through an orderly shutdown. Also, by default ext4 will "helpfully" reorder writes for you, which also can cause massive data loss in this scenario.
Now, if you were referring to what happens if a user process has only generated 75% of a new file when the power plug is yanked, then that's a problem that process resumption logic, a UPS, and/or an orderly shutdown are intended to solve.
Also, you can see how COW makes cheap, point-in-time snapshots trivial... just clone the original file block pointers and now you have a snapshot of the file as it was. This only requires space on the order of the diff size between the filesystem dataset's current state and its state at the time of the snapshot. Essentially, taking a snapshot prevents the "mark the old blocks reusable" step from happening until the snapshot is destroyed.
Although there are many ways to make the chance of inconsistent data be extremely rare, even if hardware tells the truth and doesn't return from a write request until it is really written, there is no way to completely eliminate it.
Stipulated. I believe you are referring, at least in part, to the Two Generals' Problem.
dave@console:~ ssh dave@hal-787
[dave@hal-787 ~]$ echo "1" >
echo: I'm sorry, Dave, I can't do that.
[dave@hal-787 ~]$ sudo echo "1" >
dave@console:~ Connection to hal-787 lost.
At some point, the data on any writable disk is inconsistent, even with a journal system.
ZFS is copy-on-write, ie. it is not a journaled filesystem. It is transaction-based and atomic; it does not traverse an inconsistent state because it does not overwrite the data during a write.
but data can be lost, simply because data that hasn't been written to disk yet can't be if power is lost.
Right, no doubt you have deduced I am making a distinction between data inconsistency (corruption) and "lost writes". Note that because ZFS is a "rampant layering violation", lost writes cannot cause inconsistency/corruption of the filesystem itself (except in degenerate scenario 2 below).
The primary data "loss" scenarios for ZFS during power loss are:
1) File writes that haven't been flushed yet. Note that this does not lead to filesystem inconsistency, just lost writes. A battery-backed write cache won't fix this—if this scenario is a concern, then you really need a UPS for the machine and an orderly shutdown. That is to say, if pending asynchronous writes being flushed is a concern then you're probably also concerned about currently-executing application processes that may not be complete.
2) Hardware that lies/breaks spec and acks a synchronize-cache command when it hasn't really done so. This causes much rage in the ZFS developers, because there's no true fix for it besides suggest people not use substandard hardware. Note that this *can* cause ZFS to lose data. Would this scenario corrupt a system with battery-backed RAID write cache as well? The drive fraudulently swears it has flushed the data... would the controller double-check later by reading the data back in just to see if it was lying? Even if the controller did try to check, how would it know that the drive wasn't responding from its own cache?
This is where battery-backed cache can help, as the OS thinks it has been written, but it hasn't yet.
I presume you are referring to enabling write caching on your controller, and asserting it is safe to do so due to the battery-backed cache.
I specced an IBM System x3650 M3 with a similar system for a production PostgreSQL server. God, I loved that machine... you could even make a 96 core/1 TB RAM Voltron-like single logical machine by plugging four of them together. Anyway, the intention of the cache was to be able to safely disable fsync() in PG without risk of data corruption. The IBM system took it one step further: the battery-backed cache had the ability to be transferred to another x3650 and replay the cache on the other machine in case of catastrophic failure. Anyway, I never bothered to enable the feature because we never became I/O bound on writes with fsync() enabled while I was there. It did give us headroom, though.
Anyway, the obvious perf advantage is to allow the hardware to lie and assert sync has happened even though it hasn't yet.
On my home ZFS array I have a allocated part of my OCZ Vertex 4 SSD for my RAID zpool's ZIL, which serves a similar acceleration function for synchronous writes (only). The ZIL acts like a protected write cache, but only in the case of power failure. Essentially, the in-RAM ZFS pending write cache is mirrored to the ZIL device(s) (if you have them). If the ZIL device acks a sync of the ZFS cache data written to it, then ZFS counts the overall write to the destination volume as complete (because the transactions are now replayable if power is lost) and therefore allows the OS sync() to return.
Also note that I misspoke, because the Adaptec 5805Z that I have uses a capacitor, not a battery.
By design, the ZIL is never read unless there is a power-failure. The Vertex 4 is supposed to have supercaps to ensure flush, but given
With battery-backed cache, you don't really need to worry about this. With battery-backed cache that dumps to flash (what I use), you really don't need to worry about this.
Haha, fair enough. I suppose my rejoinder would be that with RAID-Z my data on the array is never in an inconsistent state while writing, so I don't have to worry about obtaining a battery-backed cache or a UPS and then ensuring the batteries are still good after a few years.
what do you use for raid-z, raid-z2? (i.e. mdadm equivalent)?
ZFS has been called a "rampant layering violation" by Linux kernel devs because it's vertically integrated from the block device to the user-visible filesystem. All ZFS (filesystem, RAID-Z, RAID mirroring, filesystem checking, etc) is handled by the two ZFS tools: zpool and zfs.
zpool is the equivalent of mdadm, Linux LVM, fsck, etc.
zfs handles dataset management, snapshotting, is the default replacement for fstab for ZFS volumes, etc.
ZFS seemed intimidating to me at first, but I discovered that even configuration via the command line is trivial.
For example: /dev/sd1 /dev/sd2 ... /dev/sdn
...will create a RAID-Z2 pool called myArray, using the devices you listed, allocate it, format it, and mount it automatically. It is smart enough to tolerate non-deterministic device enumeration when you reboot/hotplug.
zpool create myArray raidz2
Features like snapshots (these are really cheap in terms of disk space due to ZFS copy-on-write) are similarly simple:
...will create a read-only snapshot of your whole array at the instant it is executed that is named "20130306_weekly". This requires no disk space until you start making changes to data on myArray. So, it's like Time Machine in that regard (easy restore if you accidentally rm -rf /myArray/important_stuff, and the snapshot command is trivially cron'able). ZFS theoretically supports up to 2^64 snapshots at once. You can also clone a read-only snapshot to make a read-write copy if you want to do experimentation on something. The only disk space required for that will be the diffs between the clone and the current state of the snapshotted zpool.
zfs snapshot myArray@20130306_weekly
zpool scrub myArray
...will execute an integrity check of the pool, repair any bitrot in a redundant pool (RAID-Z or mirror), etc. As I said, this happens asynchronously while the device is online and available for RW I/O. It doesn't appreciably affect system throughput, because it's at low priority. Obviously cron'able as well.
However, like I said, FreeNAS lets you do the ZFS administration via a dialog-based web GUI if you prefer. By default, FreeNAS boots from a USB stick, though you can install it on a disk.
FreeNAS is based on FreeBSD, because the ZFS CDDL isn't compatible with the GPL. So, ZFS is unlikely to ever be included in the Linux kernel, though the ZFS On Linux project is attempting to provide source rpms to compile into a standard Linux distro.
The new Linux filesystem "alternative" to ZFS will be btrfs, but because it "respects boundaries" it is unlikely to ever have anything like RAID-Z, which would require btrfs to "wrongly" assume responsibilities handled by the lower-level Linux LVM/mdadm.
I realize the risk of UREs increases with larger drives, but as far as I know this isn't exacerbated with RAID.
You are correct. For example with the WD Red drives I am using, they cite a URE of "<1 per 10^14 bits", which works out to be "less than one per 11.3 TB" (how much less?). At first glance, that doesn't seem like a big deal. That's just a little bit-rot, right? Well, drives are sector-based, so let's say you hit an average of one 4K sector with a URE per 11.3 TB.
In normal RAID-5 operation, the system will be able to correct for that error. However, what happens when you've dropped a drive and are attempting to restripe the array onto a spare/replacement? You need to be able to read all the data on all the remaining drives in order to reconstruct the lost drive. In your case, it appears that would be a total read of 16 TB. It has to read all sectors on the remaining four 4 TB drives because standard, non-RAID-Z arrays are below the filesystem, so the array has no idea whether you are actually using 15 MB or 15 TB of the space your RAID-5 array. While reading in the 16 TB, your remaining HDDs will hit an average of one URE, and now your RAID rebuild has failed by definition. Again, though, who cares if you lose just one stripe in the array per RAID-5 rebuild?
Well, it depends on how your RAID system treats it. If it is programmed to halt the rebuild and mark it as failed, then you are left hoping you can backup all the data and nuke/recreate the array before another drive is lost. At the very least, that would be a significant inconvenience compared to the expected "plug in new drive and wait for rebuild".
Hopefully this 20TB RAID will last like 5 years, and I'll move the data to a new storage system using some better technology that's cheaper, faster, and safer.
I recommend you consider RAID-Z when you are weighing alternatives (haha, obvious plug is obvious). RAID-Z is single parity and so tolerates a single drive failure like RAID-5, RAID-Z2 is double parity like RAID-6, and RAID-Z3 is triple parity.
There are a lot of benefits that RAID-Z offers that can only be achieved by pushing the redundancy considerations into the filesystem, such as only having to read the data in your array that you've actually used when you are attempting to recover from a lost drive (rather than the whole disk). It's also demonstrably safer than RAID-5/6 equivalents, offers simple snapshotting of live volumes to make in-place time machine type backups, the ability to do the equivalent of chkdsk/fsck while the volume is online and accepting RW I/O, allows the addition of SSD cache devices to automatically accelerate the pool, etc. The principal drawbacks today are hardware requirements (performance is best if you have 1GB RAM per TB of disk and a fast, 64-bit CPU) and OS support (though FreeNAS makes RAID-Z effectively painless already and has lots of cool features).
All this isn't to bash RAID-5. I had a hardware-based 4x80GB RAID-5 at home that served me well for over a decade. Then again, when I recently booted up that server to finally offload the data I discovered it had dropped a drive and another refused to enumerate, so the array was toast.
Even double-parity RAID isn't worth it with consumer drives. You'll find out you lose multiple disks due to point failures too easily. RAID-Z2 or bust IMO.
Right, I decided to select the WD Red drives with TLER even though I am implementing RAID-Z2 and it isn't clear that TLER is necessary for ZFS. Anything to avoid resilvering a 20 TB encrypted array...
As a unexpected bonus, these WD Red drives run far cooler than my WD RE4-GP (which seems ironic, given the marketing).
Anyway, my point is that RAID-5 isn't the level of data integrity assurance it once was, even ignoring the RAID-5 write hole risk that is inherent to the RAID-5 and RAID-6 concept.
(rampant layering violation ftw!)