This seems like it's only compatible with *nix OSes.
In Unix terms you would say, that these encryptions works on the block device layer. But that doesn't mean it cannot be done in other operating systems. AmigaOS had a similar concept, and I'm sure Windows has one as well. I don't know what it is called in Windows, but it is bound to exist because it is so tightly related to the way storage devices work. And there already exist multiple disk encryptions for Windows working on this layer.
That is exactly why my prefered solution for on-the-fly hard disk encryption is TrueCrypt.
TrueCrypt is vulnerable to watermarking attacks. Some time ago I created a watermarked file to demonstrate this weakness. If you put this file on a file system encrypted with TrueCrypt, some easily recognizable patterns will show up in the encrypted container. You simply take each pair of neighbor sectors in the encryption and XOR them with each other. When you reach the place where this file is located, the result is easily distinguishable from random.
Would you be willing to go to all that effort just to modify some article to say, "OMG GNAA RULZ! LOL!"?
I wouldn't. But then again I don't have any incentive to vandalise Wikipedia, so I'm not the person you should be asking. I'm afraid there does exist a few persons that are both intelligent and willing to use quite some time on performing vandalism. Luckily I think they are rare.
And yes, you neeed to track the IP addresses to make sure the same guy doesn't try to read the article five times to approve his own changes.
But if NAT or a proxy is involved different legitimate users may come from the same IP. And if somebody wants to perform vandalism, it doesn't take much to read the page five times tunneling through five different hosts. I could easilly access the site from 40 different IP adresses from a handfull of different networks. (And that is counting only those to which I have legitimate access).
About the only process that you can't completely restart manually (AFAIK) is "init."
No problem. You can tell init to reexecute itself in which case it will send state over a pipe from the old version to the new version. On the systems I use it is done by using tellinit u
Have you ever actually had a serious problem on a unix system from having two versions of a library in memory?
I have experienced problems, but not any serious ones. If an executable is started while the upgrade is progressing, it may end up having mapped incompatible versions of the different files in the package. But running two different processes where one use only the old version and another use only the new version never caused any problems.
The article seems to focus only on Windows. To get good performance from hyperthreading, the scheduler has to be aware of situations that could lead to decreased performance and avoid them. So is this a problem with the Windows scheduler being unable to deal with hyperthreading or is hyperthreading really broken? How is hyperthreading performance on other operating systems?
Another question one needs to ask is, how is performance on single and dual CPU systems? Getting good performance on a dual CPU HT system (which means four logical CPUs) is more complicated and thus requires more sophisticated algorithms in the scheduler.
Applications are most likely not to be blamed for the decreased performance. Such hardware differences should be dealt with by the kernel. Occationally the scheduler should keep one thread idle whenever that leads to the best performance. Only when there is a performance benefit should both threads be used at the same time.
Try playing an mp3 on Red Hat Linux 7.3 or anything newer than that. It doesn't work. (Ogg/vorbis OTOH does work.) So saying mp3 works everywhere is not true.
What is the best patent-free hasher (with an opensource impl) ?
I think there are open source implementations of all good hash functions.
SHA-1 ?
Probably not the best choice, there seems to be a litle doubt about how secure SHA1 is. It may soon be broken. I think there is one called SHA-384, which may be a better choice for you.
Of course, it must be short enough (it must be fit in an URL).
How long can you accept the hash to be? If the length of the string is important, then hexadecimal encoding is not the best choice. If you use base64 you can reduce a 384 bit hash from 96 to 64 characters. If 64 is too much, you will need a hash of less than 384 bits. It is supposed to be secure to just truncate the string to what length you find acceptable. But of course the shorter the hash is, the faster a collision can be found by brute force. AFAIR SHA-384 is in fact just SHA-512 truncated to 384 bits. And since it seems the length of the hash is more important to you than the CPU time spent, then a hash where the internal state is larger than the final result may be the best choice.
I slapped-on code-signing to see if it would work, and using Microsoft's Certificate Server as a CA, to generate a key to sign the Macro template, the Macro template now takes (on average) about 1000 times longer to open than it did before.
So a lot of other stuff is going on, and you don't really know what part of it is spending the time. Signatures means asymmetric cryptography is needed, and that is a lot slower than the symmetric primitives. So as long as you are only signing small amounts of data the signature itself will require more time than the SHA1 hashing.
What is the probability to have two files with the same MD5 and the same length?
All the MD5 collisions which have been produced so far had the same length. It is much more difficult to produce a collisions with two strings of different length. Which means your extra check is essentially worthless as far as security is concerned. But of course knowing the length of a file would tell you if that mismatching MD5 sum was caused by an incomplete download. And of course if the length doesn't match, you wouldn't need to spend time computing the MD5 sum.
For my application, SHA-1 incurred a HUGE performance penalty. (factor of 1000).
Sounds a litle unlikely, SHA1 is not that slow. I did some timing on my computers, and SHA1 speed was in the range 8MB/s to 70MB/s (K6 and Athlon CPUs). Now if you could do something a thousand times faster without SHA1, it would have to be at least 8GB/s. Even my fastest computer would have a hard time doing memcpy at that speed.
By running from source rather than binary, I knew there would be no surprises if I ever had to modify the source code
I agree that is a good point. Somehow I see this more as a question about how much trust you have in the vendor rather than a question about what kind of license is used for the software. Can one trust the vendor did not accidentially or deliberately supply mismatched versions. And can one trust the source, which is available today will also be available once you need it. By building everything yourself and keeping the source you need that less trust in the vendor. I actually see this as one of the good arguments for using source based distributions. Though I must admit, I don't do that myself. I think the procedures used for building rpm packages avoid the accedential mismatches. And I do try to keep all relevant SRPMS in case I'm going to need them. And if I were to worry about deliberate mismatches, I'd have a lot of other worries as well.
but I'd rather not have web sites or people gathering data about me be able to count on that IP always being a single person
In principle I have a dynamic IP address, but it hasn't changed in 14 months. And the reason it changed at that time simply was that I hadn't used the connection for five months. So how much does that dynamic IP address really hide my identity? Not that IPv6 will change that. Sure with IPv6 I can switch between the addresses in the assigned block, but it doesn't take much to figure out that two addresses are in the same block. If I really don't want to be tracked I use an ssh port forwarding through another host, but I don't recall when I last saw a need for that.
It just takes a bit of time for HW vendors to see that they should be doing the HW and let someone else write the drivers.
It would be great if hardware vendors would focus on making the best possible hardware, which implies it should be documented such that you can make the most use of it. In that case no doubt somebody will write Linux drivers for it. But unfortunately most customers can't tell the difference between good hardware and bad hardware. That means today it is possible to sell crappy hardware as long as you accompany it with software, that can hide some of it. And since that seems to be cheaper to produce, we will keep seeing such products until customers learn to see through it.
I would say it in a litle more diplomatic way: A binary only driver would take away the very reason I'm using Linux. I started using Linux because I needed something cheap and Unix compatible (and I'm probably not the only person). But after having used an open source operating system, I'm no longer willing to give that up. Price is no longer an issue for me, but it is hard to imagine an acceptable license which would not imply I could get the OS for free. Anybody who will put up with a closed source driver doesn't understand and value the real advantages of Linux, and they may as well use some closed source Unix system.
Because telnet was enabled by default,... Autorun can be disabled,
Are you trying to compare telnet and autorun? Telnet is not nearly as insecure by design as autorun. If telnet is enabled and the user doesn't do anything about it, it will just sit there idle doing no harm. Autorun OTOH will act autonomously once a CD is inserted. Autorun is insecure by design.
Red Hat use to do it
Yes, even Red Hat makes mistakes. I'm not sure if it was by default configured as insecurely as it was the case on Windows. As soon as I found out about this feature's existence on Red Hat Linux, i started uninstalling it on all my machines. (Yes, you can actually do rpm -e autorun). The autorun in Red Hat Linux was something running under KDE and Gnome. As long as you were not logged into one of those environments, there would be no autorun. Loging in as root in a VT was safe.
but early Linux distros had no firewalls by default
No matter what problem you are trying to solve, there is always a better solution than a firewall.
didn't ask you to use a non-root account.
Red Hat Linux 6.0 warned me when loging in as root.
Gone are the days when telnet was started by default.
Having telnet open is in itself not a major problem. But of course if you use it, you will send passwords in clertext. Like any other software, it must be kept updated. I don't remember exactly when Red Hat started making updates easilly available.
If this autorun executed as a regular user
It would still be a security problem, but not as bad as it is now.
You do know that rootkits started on UNIX and have plagued Linux for some time now.
Plague is an exaggeration. You can write rootkits for any OS. The major difference is that Windows has a security hole, that will allow any CD to easilly install software without the user's knowledge.
What this rootkit does to Windows could be done to Linux as well, and it would have the same negative effects on the system. Between Linux 2.4 and 2.6 it was made more difficult to modify the system call table for the exact same reasons Microsoft made it more difficult when moving from 32 bit to 64 bit.
But eventhough you could write the rootkit for Linux, it does not install just because you insert the CD. And as long as the rootkit is just on the CD, it does not influence on your ripping.
M$ will give MA full OpenDocument support. But don't go thinking that the version of Office 12 you get with your Dell or on the shelf at Best Buy will have it.
Once people start to realize that the support exists, don't you think there will be more requests? If they are really going to support it for this one contract, it means that it is available for those who will pay. And probably it is also soon going to be a pirated version with the support. And according to the claims we hear again and again, those two groups account for the majority of MS Office users.
I can promise you, that once I have seen credible evidence for the support, I'm going to tell MS Office users to upgrade to a version with ODF support. The price is going to be their problem, I can offer them a free alternative, if they find MS Office too expensive.
I don't think there has ever been a document format more complex than ascii text that hasn't changed.
Ascii text has also changed. Until about 3 years ago almost every system I used, was using iso-8859-1 encoding. The only exception was MSDOS, which used something nonstandard. But now UTF-8 encoding is poping up everywhere. And even besides the character encoding, I know about three different ways to encode a line break (four if you consider the fact that one of the three was complicated enough for multiple people to get it wrong in the same way). I have even come across a.tex file using a mixture of all three ways to encode a line break. Reading that file wasn't easy.
I don't know if that particular survey was posted on slashdot. However another story about someone having much more success with running KLEZ on WINE was posted on slashdot. Unfortunately none of the mirrors of the article works anymore. But maybe it can be found on google if you use the right keywords.
I took a quick look on that patch, and on a first look it looks good. I'm surely going to try that out some day, because it is a feature I have often missed.
Isn't the entire Linux kernel written in C?
Almost. There are some low level stuff which need to be done in assembler. But except from that it is all C.
Right, but that was the simplifying assumption I started with - that you lose the drive, the bad guys get it, and that's it.
This depends on what the bad guys are able to do with the encrypted drive. Are they able to read data that has been overwritten once? Probably they aren't, but if they manage to do so just once, there will be an information leak. Are they able to replace the firmware and read remapped sectors (if any) on the disk? This is more likely and will also cause an information leak.
Do you create backups of your encrypted disk? Are the backups encrypted?
if your laptop turns up in the lost and found, the first thing you do when you get it back is wipe the disk and restore from backup. Do you restore from an encrypted backup? If you backup by simply copying the encrypted version and restore by copying it back, then you are in trouble if you lose a disk twice. So you would be required to reencrypt when backing up data, and reencrypt when restoring.
If your encrypted container is not a physical disk partition, but rather a file in a file system, then you also have to worry about the host file system remapping data.
However, I still can't see a reason to use CBC - I'd rather use CTR with a real nonce and MAC.
If you start each sector being written with a new initial counter value then CTR may be safe. The problem with CTR is that reusing the counter is worse than reusing an IV in a CBC mode encryption. But using CTR the right way may actually be secure.
Slashdot Mirror
This seems like it's only compatible with *nix OSes.
In Unix terms you would say, that these encryptions works on the block device layer. But that doesn't mean it cannot be done in other operating systems. AmigaOS had a similar concept, and I'm sure Windows has one as well. I don't know what it is called in Windows, but it is bound to exist because it is so tightly related to the way storage devices work. And there already exist multiple disk encryptions for Windows working on this layer.
That is exactly why my prefered solution for on-the-fly hard disk encryption is TrueCrypt.
TrueCrypt is vulnerable to watermarking attacks. Some time ago I created a watermarked file to demonstrate this weakness. If you put this file on a file system encrypted with TrueCrypt, some easily recognizable patterns will show up in the encrypted container. You simply take each pair of neighbor sectors in the encryption and XOR them with each other. When you reach the place where this file is located, the result is easily distinguishable from random.
Would you be willing to go to all that effort just to modify some article to say, "OMG GNAA RULZ! LOL!"?
I wouldn't. But then again I don't have any incentive to vandalise Wikipedia, so I'm not the person you should be asking. I'm afraid there does exist a few persons that are both intelligent and willing to use quite some time on performing vandalism. Luckily I think they are rare.
And yes, you neeed to track the IP addresses to make sure the same guy doesn't try to read the article five times to approve his own changes.
But if NAT or a proxy is involved different legitimate users may come from the same IP. And if somebody wants to perform vandalism, it doesn't take much to read the page five times tunneling through five different hosts. I could easilly access the site from 40 different IP adresses from a handfull of different networks. (And that is counting only those to which I have legitimate access).
About the only process that you can't completely restart manually (AFAIK) is "init."
No problem. You can tell init to reexecute itself in which case it will send state over a pipe from the old version to the new version. On the systems I use it is done by using tellinit u
Have you ever actually had a serious problem on a unix system from having two versions of a library in memory?
I have experienced problems, but not any serious ones. If an executable is started while the upgrade is progressing, it may end up having mapped incompatible versions of the different files in the package. But running two different processes where one use only the old version and another use only the new version never caused any problems.
The article seems to focus only on Windows. To get good performance from hyperthreading, the scheduler has to be aware of situations that could lead to decreased performance and avoid them. So is this a problem with the Windows scheduler being unable to deal with hyperthreading or is hyperthreading really broken? How is hyperthreading performance on other operating systems?
Another question one needs to ask is, how is performance on single and dual CPU systems? Getting good performance on a dual CPU HT system (which means four logical CPUs) is more complicated and thus requires more sophisticated algorithms in the scheduler.
Applications are most likely not to be blamed for the decreased performance. Such hardware differences should be dealt with by the kernel. Occationally the scheduler should keep one thread idle whenever that leads to the best performance. Only when there is a performance benefit should both threads be used at the same time.
I choose mp3 because it works everywhere.
Try playing an mp3 on Red Hat Linux 7.3 or anything newer than that. It doesn't work. (Ogg/vorbis OTOH does work.) So saying mp3 works everywhere is not true.
What is the best patent-free hasher (with an opensource impl) ?
I think there are open source implementations of all good hash functions.
SHA-1 ?
Probably not the best choice, there seems to be a litle doubt about how secure SHA1 is. It may soon be broken. I think there is one called SHA-384, which may be a better choice for you.
Of course, it must be short enough (it must be fit in an URL).
How long can you accept the hash to be? If the length of the string is important, then hexadecimal encoding is not the best choice. If you use base64 you can reduce a 384 bit hash from 96 to 64 characters. If 64 is too much, you will need a hash of less than 384 bits. It is supposed to be secure to just truncate the string to what length you find acceptable. But of course the shorter the hash is, the faster a collision can be found by brute force. AFAIR SHA-384 is in fact just SHA-512 truncated to 384 bits. And since it seems the length of the hash is more important to you than the CPU time spent, then a hash where the internal state is larger than the final result may be the best choice.
I slapped-on code-signing to see if it would work, and using Microsoft's Certificate Server as a CA, to generate a key to sign the Macro template, the Macro template now takes (on average) about 1000 times longer to open than it did before.
So a lot of other stuff is going on, and you don't really know what part of it is spending the time. Signatures means asymmetric cryptography is needed, and that is a lot slower than the symmetric primitives. So as long as you are only signing small amounts of data the signature itself will require more time than the SHA1 hashing.
What is the probability to have two files with the same MD5 and the same length?
All the MD5 collisions which have been produced so far had the same length. It is much more difficult to produce a collisions with two strings of different length. Which means your extra check is essentially worthless as far as security is concerned. But of course knowing the length of a file would tell you if that mismatching MD5 sum was caused by an incomplete download. And of course if the length doesn't match, you wouldn't need to spend time computing the MD5 sum.
For my application, SHA-1 incurred a HUGE performance penalty. (factor of 1000).
Sounds a litle unlikely, SHA1 is not that slow. I did some timing on my computers, and SHA1 speed was in the range 8MB/s to 70MB/s (K6 and Athlon CPUs). Now if you could do something a thousand times faster without SHA1, it would have to be at least 8GB/s. Even my fastest computer would have a hard time doing memcpy at that speed.
By running from source rather than binary, I knew there would be no surprises if I ever had to modify the source code
I agree that is a good point. Somehow I see this more as a question about how much trust you have in the vendor rather than a question about what kind of license is used for the software. Can one trust the vendor did not accidentially or deliberately supply mismatched versions. And can one trust the source, which is available today will also be available once you need it. By building everything yourself and keeping the source you need that less trust in the vendor. I actually see this as one of the good arguments for using source based distributions. Though I must admit, I don't do that myself. I think the procedures used for building rpm packages avoid the accedential mismatches. And I do try to keep all relevant SRPMS in case I'm going to need them. And if I were to worry about deliberate mismatches, I'd have a lot of other worries as well.
but I'd rather not have web sites or people gathering data about me be able to count on that IP always being a single person
In principle I have a dynamic IP address, but it hasn't changed in 14 months. And the reason it changed at that time simply was that I hadn't used the connection for five months. So how much does that dynamic IP address really hide my identity? Not that IPv6 will change that. Sure with IPv6 I can switch between the addresses in the assigned block, but it doesn't take much to figure out that two addresses are in the same block. If I really don't want to be tracked I use an ssh port forwarding through another host, but I don't recall when I last saw a need for that.
If the price scales with availability, that means they should charge me $.00004 a month on IPv6 for an extra IP.
You got those calculations wrong. For that price you should not get just one but 1208925819614629174706176 IPv6 addresses.
It just takes a bit of time for HW vendors to see that they should be doing the HW and let someone else write the drivers.
It would be great if hardware vendors would focus on making the best possible hardware, which implies it should be documented such that you can make the most use of it. In that case no doubt somebody will write Linux drivers for it. But unfortunately most customers can't tell the difference between good hardware and bad hardware. That means today it is possible to sell crappy hardware as long as you accompany it with software, that can hide some of it. And since that seems to be cheaper to produce, we will keep seeing such products until customers learn to see through it.
then you don't really deserve Linux
I would say it in a litle more diplomatic way: A binary only driver would take away the very reason I'm using Linux. I started using Linux because I needed something cheap and Unix compatible (and I'm probably not the only person). But after having used an open source operating system, I'm no longer willing to give that up. Price is no longer an issue for me, but it is hard to imagine an acceptable license which would not imply I could get the OS for free. Anybody who will put up with a closed source driver doesn't understand and value the real advantages of Linux, and they may as well use some closed source Unix system.
Try further back.
Back then I was using AmigaOS.
Because telnet was enabled by default,... Autorun can be disabled,
Are you trying to compare telnet and autorun? Telnet is not nearly as insecure by design as autorun. If telnet is enabled and the user doesn't do anything about it, it will just sit there idle doing no harm. Autorun OTOH will act autonomously once a CD is inserted. Autorun is insecure by design.
Red Hat use to do it
Yes, even Red Hat makes mistakes. I'm not sure if it was by default configured as insecurely as it was the case on Windows. As soon as I found out about this feature's existence on Red Hat Linux, i started uninstalling it on all my machines. (Yes, you can actually do rpm -e autorun). The autorun in Red Hat Linux was something running under KDE and Gnome. As long as you were not logged into one of those environments, there would be no autorun. Loging in as root in a VT was safe.
but early Linux distros had no firewalls by default
No matter what problem you are trying to solve, there is always a better solution than a firewall.
didn't ask you to use a non-root account.
Red Hat Linux 6.0 warned me when loging in as root.
Gone are the days when telnet was started by default.
Having telnet open is in itself not a major problem. But of course if you use it, you will send passwords in clertext. Like any other software, it must be kept updated. I don't remember exactly when Red Hat started making updates easilly available.
If this autorun executed as a regular user
It would still be a security problem, but not as bad as it is now.
You do know that rootkits started on UNIX and have plagued Linux for some time now.
Plague is an exaggeration. You can write rootkits for any OS. The major difference is that Windows has a security hole, that will allow any CD to easilly install software without the user's knowledge.
What this rootkit does to Windows could be done to Linux as well, and it would have the same negative effects on the system. Between Linux 2.4 and 2.6 it was made more difficult to modify the system call table for the exact same reasons Microsoft made it more difficult when moving from 32 bit to 64 bit.
But eventhough you could write the rootkit for Linux, it does not install just because you insert the CD. And as long as the rootkit is just on the CD, it does not influence on your ripping.
M$ will give MA full OpenDocument support. But don't go thinking that the version of Office 12 you get with your Dell or on the shelf at Best Buy will have it.
Once people start to realize that the support exists, don't you think there will be more requests? If they are really going to support it for this one contract, it means that it is available for those who will pay. And probably it is also soon going to be a pirated version with the support. And according to the claims we hear again and again, those two groups account for the majority of MS Office users.
I can promise you, that once I have seen credible evidence for the support, I'm going to tell MS Office users to upgrade to a version with ODF support. The price is going to be their problem, I can offer them a free alternative, if they find MS Office too expensive.
I don't think there has ever been a document format more complex than ascii text that hasn't changed.
.tex file using a mixture of all three ways to encode a line break. Reading that file wasn't easy.
Ascii text has also changed. Until about 3 years ago almost every system I used, was using iso-8859-1 encoding. The only exception was MSDOS, which used something nonstandard. But now UTF-8 encoding is poping up everywhere. And even besides the character encoding, I know about three different ways to encode a line break (four if you consider the fact that one of the three was complicated enough for multiple people to get it wrong in the same way). I have even come across a
I think this was posted on /. before.
I don't know if that particular survey was posted on slashdot. However another story about someone having much more success with running KLEZ on WINE was posted on slashdot. Unfortunately none of the mirrors of the article works anymore. But maybe it can be found on google if you use the right keywords.
Try this patch.
I took a quick look on that patch, and on a first look it looks good. I'm surely going to try that out some day, because it is a feature I have often missed.
Isn't the entire Linux kernel written in C?
Almost. There are some low level stuff which need to be done in assembler. But except from that it is all C.
Right, but that was the simplifying assumption I started with - that you lose the drive, the bad guys get it, and that's it.
This depends on what the bad guys are able to do with the encrypted drive. Are they able to read data that has been overwritten once? Probably they aren't, but if they manage to do so just once, there will be an information leak. Are they able to replace the firmware and read remapped sectors (if any) on the disk? This is more likely and will also cause an information leak.
Do you create backups of your encrypted disk? Are the backups encrypted?
if your laptop turns up in the lost and found, the first thing you do when you get it back is wipe the disk and restore from backup.
Do you restore from an encrypted backup? If you backup by simply copying the encrypted version and restore by copying it back, then you are in trouble if you lose a disk twice. So you would be required to reencrypt when backing up data, and reencrypt when restoring.
If your encrypted container is not a physical disk partition, but rather a file in a file system, then you also have to worry about the host file system remapping data.
However, I still can't see a reason to use CBC - I'd rather use CTR with a real nonce and MAC.
If you start each sector being written with a new initial counter value then CTR may be safe. The problem with CTR is that reusing the counter is worse than reusing an IV in a CBC mode encryption. But using CTR the right way may actually be secure.