I wouldn't want to use this drive unless the encrypted data can be read in a raw mode so that the encryption can be verified so that an audit can be performed.
Just my thought. I have seen so many storage encryptions with more or less severe flaws. I have yet to see one that was completely flawless. Even if they didn't intend to put in a back door they for sure have introduced some weakness. I am confident that given the complete specification of how it is implemented, I could point out at least one minor weakness. I'm not saying that storage encryption in hardware is bad, but you have to see it mainly as a performance optimization, and there should always exist an identical implementation in software. Doing it in the drive itself does have the advantage that you can tweak the logical and physical sector sizes, which allows using the few extra bytes needed for random IV and such. You can work around the limitation even in a software implementation of the encryption, but that comes at a cost in performance. Almost every storage encryption in existence was designed for performance and does not use that extra storage for additional security. The one encryption I have seen doing this right happens to have a few other flaws.
Could using these in a RAID-5 configuration lead to a weakness due to the XOR stripes?
Since the disks would be encrypted with independent keys, such a vulnurability is highly unlikely. Actually such a raid setup does have an advantage. Whenever you replace a disk with a new one, you get your old data reencrypted under a new key. So no problem in returning defective disks to the vendor. The problem is key management. The simple, but not most secure, way of doing it is to just encrypt the key of every disk using the same password. A much better way would be to secret share the keys across all the disks (and reshare them when a disk is replaced). If implemented that way somebody who got all the disks leaving your raid would be unable to decrypt it even if they knew your password, since they would only get one share of each version of the sharing. And once they get the first sharing, the previous one would be destroyed. The "problem" with that approach is, that it cannot be implemented directly on the drive, since something have to do the secret sharing computations. So that would have to be implemented either in the OS or a raid controller. And if encryption is done by the disk itself, it is not certain that it would give you access to read and write some areas without encryption, which is what you would need to do your own key management.
The only "choice" is at the quantum level, and I'm not sure you can actually define that as a choice since you don't actually choose anything.
You can't really be sure about that. Sure physics says the outcomes of certain quantum mechanical events are random, but how can you really be sure they are random? But frankly I don't care much for studies claiming we have no free will. We have a free will, and claiming otherwise is just a bad excuse for doing things we know are wrong. Before I will even consider what their study really says, I'd like them to write down a definition of what free will means, then we'll see if that definition is something I can agree with.
So what if it takes another 20 years to build a working combat robot? There will surely still be a war in the Middle East it can take part in. (It would be fantastic if there were no more wars going on in 20 years, it is just not very likely).
many people when building arrays use identical drives, this is good for performance but bad for data protection.
Sounds like two urban legends to me. Sure if you buy two disks from the same vendor, they could be part of a bad batch. To avoid losing data because of that do an initial stress test of your disks before you start relying on them to store valuable data. If one fails during stress test get it replaced and keep stress testing until you are able to complete the stress test without failures. After that it would be highly unlikely that your raid experience two simultaneously drive failures unless it is caused by an external event such as a power spike, a short circuit, the cabinet got kicked, or something else. Even if drives from the safe batch were going to have similar life expectations, they are not going to fail within seconds of each other. And the second one to fail just has to survive long enough for data to be copied to the hot spare.
There still is a reason why you might see one disk fail and another disk fail before the raid recovery finishes. If one of your disks have a bad sector, it could go unnoticed for a long time, if you never read it. If the bad sector happened to be a parity sector in a raid-5, it is quite likely that it is not going to be read until you actually need it for recovery. So what happens if you have a bad sector on one disk, and another disk in your raid fails? In some raid implementations, the recovery hits the bad sector and marks that disk as failed, now your raid have two failed disks and recovery fails.
To protect against this you need a few things. First of all the raid must never give up on recovery because of bad sectors, even if a disk has bad sectors and should be replaced, the raid must still try reading other sectors from that disk, if that is the only way to recover data. And even if some sector is unrecoverable, it should just report that as a bad sector to the higher layer, and keep recovering everything else.
Above is of course not sufficient to prevent a bit of data loss in the event of some bad sectors and one total disk failure. Periodically reading the raw disks to watch out for bad sectors may help, and your raid system might be able to do this automatically. But even better is to have a bit more redundancy. Have three mirrors if you use raid-1, and use raid-6 instead of raid-5. In those cases even with bad sectors spread across all your disks and one total disk failure, you are unlikely to lose data.
You could use raid-1 with three mirrors for things you write frequently and raid-6 for things you don't write that frequent. I was thinking about setting up a file system with data on raid-6 and journal on raid-1, but dropped the idea again because the access patterns in a journal are actually not that bad for raid-6.
Those performance problems from mixing drives from different vendors, I have never observed, even though I have done raids that way. One advantage you get from using drives from different vendors is, that it improves your chances of being able to find a drive matching in size once you need to replace one. But if you are going to be using larger drives when you replace one in your raid, then that doesn't matter anyway.
Your point about one bad connection slowing everything down is right on the spot. Actually it is much worse than you make it sound. This would have a huge potential for DoS attacks. Just fake lost packets and have your peer slow down. Start doing this to a busy webserver and watch the outcome. And imagine the situation where I am backing up several GB between two machines in my home, while I am browsing some webpages from one of the machines at the same time. Should my backup over a 1Gbps link slow down to match the 1.5Mbps I have to the internet?
It quickly became obvious to me, that the author of that article was way off. When I came across the term "contracted peak bitrates" I realized what the problem was. I have never seen a contract between an ISP and a customer that stated a peak and an average bitrate. They usually just state one bitrate (in each direction). What is the user supposed to consider their allowed average bitrate to be, if not the bitrate specified in the contract? If the ISP really consider that to be just a peak, and the average to be somewhat lower, they should explicitly state the average rate in the contract, and enforce it. The ISP shouldn't filter based on protocol, enforcing an average bitrate should be simpler anyway. If the ISPs would implement this bandwidth management correctly, then TCP would take care of the rest. (They might need to support ECN as well in order to give users the full capacity they are entitled to, otherwise some percentage of packets destined for that user would have to be dropped after they already used capacity).
Throttling individual users (and not protocols) is the way to go. From an ISPs point of view I don't think the definition of what constitutes a user is any problem, each customer they sign a contract with is a user. Put an average bandwidth in the contract, and throttle the link to that. If they want to be nice, they only throttle when necessary, such that outside of peak hours customers can upload and download at will.
It seems the main reason for ISPs not to state the average bandwidth in their contracts is, that they want to change it without telling their customers. When the number of customers in some area increases, the average allowed bandwidth drops, and the ISP does not tell the customers about this.
Whenever people talk about 99.999 uptime for a service delivered over the internet I laugh in their faces.
I work for a company that aims at that. It does mean that we have engineers on call 24x7 to fix things if they break. Some of us even have two separate internet connections from home, just so we can always log in and fix problems. And the cost of that is more than justified. The easier it is to switch to a competitor, the more it matters how reliable your systems are.
Linux administrators will only try this as a last resort (and it almost never works).
I think that about 80% of the time I will know beforehand if rebooting a Linux system is going to solve a particular problem. But even if I'm convinced that a reboot would solve the problem, I usually spend some time looking for a solution that does not involve rebooting. There are multiple reasons why I look for other solutions. Sometimes a reboot is inconvenient because of all the programs that have to be shut down and started again. If I find a solution that does not involve rebooting, I will usually have saved some time on the long term, because next time it happens, I will be back to my work right away. And finally it helps me understand the problem, so maybe I will even be able to prevent it in the future.
If something used to work and suddenly stopped working, chances are, a reboot will solve it. Obvious exceptions are that the problem could be caused by faulty hardware or a full disk. Those are usually easy to spot.
You probably bought higher speed drives than the bottom of the line.
I have usually been going for the lowest price per GB. I think that usually means I pay about three times as much as the price of the cheapest drives, but I get more than three times the capacity. Speed was never a criteria I considered when bying drives.
When I run disk benchmarks, I don't use a filesystem.
Sure, that is the only proper way to test the speed of the drive itself.
Since 2000, you've been able to buy drives that can hit over 80MB per second.
Are you so sure about that? The machine I bought in 1999 could do 12MB/s (when everything was configured optimally). The first time I ever saw a drive above 65MB/s was in 2006. Sure I have never been using really high end drives. But I doubt there could be that much difference in raw throughput.
You can also still buy SATA drives that have trouble hitting 30MB/sec. (3.5" drives. At 2.5" you can easily buy drives that have a hard time with 25MB/sec. You can also get 2.5" server drives that smoke...)
I have never seen a 3.5" SATA drive that did less than 50MB/s.
If you are losing 50% of your speed from a filesystem... Well.. You're not. It's not even worth contemplating.
It sure does depend on file system and access patterns. Sequential reads and writes of large files should come close to raw disk speed. But I have not done any carefull measuring of that.
The average low-end desktop drive isn't going to give you more than 30MB/sec.
The IDE drives I bought in 2004 were capable of 55MB/s. The SATA drives I bought last year are capable of 75MB/s (I even managed to get 80MB/s over eSATA once). If you copy across a 1Gbit/s link, the hard disk is still going to be the bottleneck in many cases. But if you only get 30MB/s it is because you use a file system which is not capable of utilizing the disk's full potential.
The good thing about having the crypto performed in the enclosure is, that you can perform this kind of analysis. Had the same "encryption" been implemented directly on the disk or in a usb stick, it might not have been noticed, that it was so weak. My take on this is to never trust the crypto performed by such an enclosure unless there is a software implementation doing the exact same thing, and that one has been carefully inspected. The point of doing the encryption in hardware is performance, it does not add any additional security.
It's the TCP connection between two peers that Comcast is attacking, not the connection between the peer and the tracker. Using UDP for the latter doesn't solve anything.
If an improved protocol prevents one kind of attacks, Comcast might look into other kinds of attacks. Protecting against potential attacks that have not been performed yet, does make sense.
Nope, all you need is remote access to a local user account via ssh or something. Many users use weak passwords.
I recently disabled password based ssh logins on all my machines. Not because my passwords are weak, but because the steady stream of failing login attempts makes me paranoid. Now you first have to get to my key before you can even start trying to brute force the password.
Oh sure, you're recompiling the kernel of your production environments from home.
I have had to upgrade kernels on production systems located on a different continent. In such a case it doesn't really matter if I do it from home or from the office.
the first time I talked about RPGs he thought I was referring to rocket-propelled grenades.
What else could you be referring to? I couldn't think of any other meaning. I'm not from Asia, I happen to have lived most of my life in Denmark. And the only reason I have ever heard that abbreviation is the games we were playing back in high school.
Google will actually let you search for Died in a * accident. If you do so you can see what words people put in there. Right now the fourth result is actually "Died in a blogging accident" (right after three car accidents). I have used that to find out what might be the missing word in other sentences like Grab your * and double click or Either you are with us or you are with the *. Even more interesting if combined with the - operator to filter out the obvious possibilities.
I think everybody have changed their mind about multiple things, but don't actually remember what they thought about it a year ago. There may even be areas where we would be choked to find out what we were thinking a year ago. Sometimes experience can change your opinion faster than you would have thought possible without even noticing the change.
If you tried to sell a stateless filter as a "firewall" today, you'd be laughed out of the market.
That depends on which market you are aiming for. If you are aiming for buzzword compliance, then it is of course a no-go. But if you are aiming for people who know what they are doing and wants to run a large reliable production system, then it is a different matter. A stateless firewall is simpler. And being simpler means that it will most likely also have less bugs. Less bugs means you are less likely to have security vulnurabilities. And stateless means it is easier to have redundancy. Have two firewalls next to each other, and if one fails just route packets through the other.
I belive you. As a kid I was living in a residential area that was right next to an industrial area. There were no direct roads connecting the residential area to to the industrial area, the only road in and out of the residential area was in the opposite direction. Frequently there would be trucks with a destination in the industrial area trying to make a shortcut through the residential area. And they would always end up around where I lived and ask for directions. There were signs at the entrance to the residential area clearly stating that there was no other way out.
Oh and when I read this story the first thing on my mind was the recently proposed theory, that people would happily drive onto a highway in the wrong direction, if that was what the GPS told them to do. I don't know if there is any evidence to support this theory. But actually I think in some cases using GPS navigation can actually also improve safety, you just have to keep using your brain. A GPS is no replacement for a brain. The city I live in has so many one way streets, that you have little chance of finding the right way without GPS navigation. And if you constantly are unsure where you have to go, and have to change your mind in the very last second over and over again, you are not likely to be driving as safely as possible. In that case using a GPS and looking at the signs would be a very good combination.
If on the other side you have two hash function you have a good chance that you figure out that one of your hash functions isn't secure any more and thus have enough time to replace it with something new
Don't expect a better warning just because you are using two different hash functions. With md5 you did get the warning a few years ago. Still most uses of md5 are secure. If you were to be using two, there is no guarantee that you are going to be told about the first break, if somebody want to attack the combination, they might decide to keep silent until they have a working attack. It all boils down to the intentions of whoever finds the weakness. Besides, if you wanted to pay the price in terms of CPU usage, you could go for a 256 bit hash.
Slashdot Mirror
So what if it takes another 20 years to build a working combat robot? There will surely still be a war in the Middle East it can take part in. (It would be fantastic if there were no more wars going on in 20 years, it is just not very likely).
There still is a reason why you might see one disk fail and another disk fail before the raid recovery finishes. If one of your disks have a bad sector, it could go unnoticed for a long time, if you never read it. If the bad sector happened to be a parity sector in a raid-5, it is quite likely that it is not going to be read until you actually need it for recovery. So what happens if you have a bad sector on one disk, and another disk in your raid fails? In some raid implementations, the recovery hits the bad sector and marks that disk as failed, now your raid have two failed disks and recovery fails.
To protect against this you need a few things. First of all the raid must never give up on recovery because of bad sectors, even if a disk has bad sectors and should be replaced, the raid must still try reading other sectors from that disk, if that is the only way to recover data. And even if some sector is unrecoverable, it should just report that as a bad sector to the higher layer, and keep recovering everything else.
Above is of course not sufficient to prevent a bit of data loss in the event of some bad sectors and one total disk failure. Periodically reading the raw disks to watch out for bad sectors may help, and your raid system might be able to do this automatically. But even better is to have a bit more redundancy. Have three mirrors if you use raid-1, and use raid-6 instead of raid-5. In those cases even with bad sectors spread across all your disks and one total disk failure, you are unlikely to lose data.
You could use raid-1 with three mirrors for things you write frequently and raid-6 for things you don't write that frequent. I was thinking about setting up a file system with data on raid-6 and journal on raid-1, but dropped the idea again because the access patterns in a journal are actually not that bad for raid-6.
Those performance problems from mixing drives from different vendors, I have never observed, even though I have done raids that way. One advantage you get from using drives from different vendors is, that it improves your chances of being able to find a drive matching in size once you need to replace one. But if you are going to be using larger drives when you replace one in your raid, then that doesn't matter anyway.
Your point about one bad connection slowing everything down is right on the spot. Actually it is much worse than you make it sound. This would have a huge potential for DoS attacks. Just fake lost packets and have your peer slow down. Start doing this to a busy webserver and watch the outcome. And imagine the situation where I am backing up several GB between two machines in my home, while I am browsing some webpages from one of the machines at the same time. Should my backup over a 1Gbps link slow down to match the 1.5Mbps I have to the internet?
It quickly became obvious to me, that the author of that article was way off. When I came across the term "contracted peak bitrates" I realized what the problem was. I have never seen a contract between an ISP and a customer that stated a peak and an average bitrate. They usually just state one bitrate (in each direction). What is the user supposed to consider their allowed average bitrate to be, if not the bitrate specified in the contract? If the ISP really consider that to be just a peak, and the average to be somewhat lower, they should explicitly state the average rate in the contract, and enforce it. The ISP shouldn't filter based on protocol, enforcing an average bitrate should be simpler anyway. If the ISPs would implement this bandwidth management correctly, then TCP would take care of the rest. (They might need to support ECN as well in order to give users the full capacity they are entitled to, otherwise some percentage of packets destined for that user would have to be dropped after they already used capacity).
Throttling individual users (and not protocols) is the way to go. From an ISPs point of view I don't think the definition of what constitutes a user is any problem, each customer they sign a contract with is a user. Put an average bandwidth in the contract, and throttle the link to that. If they want to be nice, they only throttle when necessary, such that outside of peak hours customers can upload and download at will.
It seems the main reason for ISPs not to state the average bandwidth in their contracts is, that they want to change it without telling their customers. When the number of customers in some area increases, the average allowed bandwidth drops, and the ISP does not tell the customers about this.
If something used to work and suddenly stopped working, chances are, a reboot will solve it. Obvious exceptions are that the problem could be caused by faulty hardware or a full disk. Those are usually easy to spot.
Sure, that is the only proper way to test the speed of the drive itself.
Are you so sure about that? The machine I bought in 1999 could do 12MB/s (when everything was configured optimally). The first time I ever saw a drive above 65MB/s was in 2006. Sure I have never been using really high end drives. But I doubt there could be that much difference in raw throughput.
I have never seen a 3.5" SATA drive that did less than 50MB/s.
It sure does depend on file system and access patterns. Sequential reads and writes of large files should come close to raw disk speed. But I have not done any carefull measuring of that.
The good thing about having the crypto performed in the enclosure is, that you can perform this kind of analysis. Had the same "encryption" been implemented directly on the disk or in a usb stick, it might not have been noticed, that it was so weak. My take on this is to never trust the crypto performed by such an enclosure unless there is a software implementation doing the exact same thing, and that one has been carefully inspected. The point of doing the encryption in hardware is performance, it does not add any additional security.
Google will actually let you search for Died in a * accident. If you do so you can see what words people put in there. Right now the fourth result is actually "Died in a blogging accident" (right after three car accidents). I have used that to find out what might be the missing word in other sentences like Grab your * and double click or Either you are with us or you are with the *. Even more interesting if combined with the - operator to filter out the obvious possibilities.
I think everybody have changed their mind about multiple things, but don't actually remember what they thought about it a year ago. There may even be areas where we would be choked to find out what we were thinking a year ago. Sometimes experience can change your opinion faster than you would have thought possible without even noticing the change.
Oh and when I read this story the first thing on my mind was the recently proposed theory, that people would happily drive onto a highway in the wrong direction, if that was what the GPS told them to do. I don't know if there is any evidence to support this theory. But actually I think in some cases using GPS navigation can actually also improve safety, you just have to keep using your brain. A GPS is no replacement for a brain. The city I live in has so many one way streets, that you have little chance of finding the right way without GPS navigation. And if you constantly are unsure where you have to go, and have to change your mind in the very last second over and over again, you are not likely to be driving as safely as possible. In that case using a GPS and looking at the signs would be a very good combination.