Slashdot Mirror
Ask Slashdot: Simplifying Encryption and Backup?
New submitter FuzzNugget writes "A recent catastrophic hard drive failure has caused me to ponder whether the trade-off between security and convenience with software-based OTFE is worthwhile. My setup involves an encrypted Windows installation with TrueCrypt's pre-boot authentication, in addition to having data stored in a number of TrueCrypt file containers. While it is nice to have some amount of confidence that my data is safe from prying eyes in the case of loss or theft of my laptop, this setup poses a number of significant inconveniences." Read on below; FuzzNugget lists some problems with this set-up, and seeks advice on a simpler system for backing up while keeping things locked down.
FuzzNugget continues: "1. Backup images of the encrypted operating system can only be restored to the original hard drive (ie.: the drive that has failed). So, recovery from this failure requires the time-consuming process of re-installing the OS, re-installing my software and re-encrypting it. Upgrading the hard drive where both the old and new drives are still functional is not much better as it requires decryption, copying the partition(s) and re-encryption.
2. With the data being stored in large file containers, each around 100-200GB. It can be come quite burdensome to deal with these huge files all the time. It's also a particularly volatile situation, as the file container is functionally useless if it's not completely intact.
3. As much as I'd like to use this situation as an opportunity to upgrade to an SSD, use with OTFE is said to pose risks of data leaks, cause decreased performance and premature failure due to excessive write operations.
So, with that, I'm open to suggestions for alternatives. Do you use encryption for your hard drive(s)? What's your setup like and how manageable is it?"
2. With the data being stored in large file containers, each around 100-200GB. It can be come quite burdensome to deal with these huge files all the time. It's also a particularly volatile situation, as the file container is functionally useless if it's not completely intact.
3. As much as I'd like to use this situation as an opportunity to upgrade to an SSD, use with OTFE is said to pose risks of data leaks, cause decreased performance and premature failure due to excessive write operations.
So, with that, I'm open to suggestions for alternatives. Do you use encryption for your hard drive(s)? What's your setup like and how manageable is it?"
Aka: you are doing it wrong. First think of backup: you have a machine, and you copy its contents to another drive. Ok. Easy. Now take a breath, and use OTFE for the original hard disk, and now add OTFE for the external drive/media. There. The backup has NOTHING to do with encryption. If you have forced yourself into a backup solution which requires encryption integration to the point that it only restores to a specific hardware, you are failing hard time, precisely for the reason backups are for when you don't have the original hardware.
Again, separate backup from encryption. I mean, next you will want an integrated internet/remote backup and you will cry us a river? Compartimentalize each function and then you can mix them freely.
I use encryption across all my desktops and laptops. On my laptops I just use dmcrypt/cryptsetup and encrypt the whole disk running ubuntu.
For storage I use my fileserver which is 1x500gb encrypted with dmcrypt for the OS and for the "storage" of the fileserver I have redundancy against failure:
LVM with 2x 1TB sata disks. The LVM has both physical volumes as seperate "mirror" slices (encrypt 1 disk, add a mirror disk). The total usable storage is around 790 GB but I already had one disk fail and I could simply "mount" my data without one disk being present & rebuild the LVM mirror using a new disk! Secure & reliable!
The only issue I have not been able to solve in this setup is if/when one disk fails, your data is only available read-only because the lvm-mirror is only "partial" and physical volumes are missing. If anybody knows a solution for that, please comment. This was just a temporary issue though, as soon as a new disk was added and the mirror rebuilt, all was back to normal.
Then you take snapshots for a full-system backup. This may not work for your use. Perhaps try a Seagate encrypted hard drive.
I use encryption across all my desktops and laptops. On my laptops I just use dmcrypt/cryptsetup and encrypt the whole disk running ubuntu. For storage I use my fileserver which is 1x500gb encrypted with dmcrypt for the OS and for the "storage" of the fileserver I have redundancy against failure: LVM with 2x 1TB sata disks. The LVM has both physical volumes as seperate "mirror" slices (encrypt 1 disk, add a mirror disk). The total usable storage is around 790 GB but I already had one disk fail and I could simply "mount" my data without one disk being present & rebuild the LVM mirror using a new disk! Secure & reliable! The only issue I have not been able to solve in this setup is if/when one disk fails, your data is only available read-only because the lvm-mirror is only "partial" and physical volumes are missing. If anybody knows a solution for that, please comment. This was just a temporary issue though, as soon as a new disk was added and the mirror rebuilt, all was back to normal.
On OS X, you can easily create bootable images of drives using programs like SuperDuper! and Carbon Copy Cloner (stupid names). You can encrypt these files, put them on various types of drives and OS X can boot off of USB and FireWire drives which makes backups and restores pretty easy.
OS X also allows for encrypted sparseimages (folders) that can be stored on Dropbox or similar. Between the two concepts, I avoid the hassle of whole drive encryption and just worry about encrypting some of my data.
I would think that Windows would have similar functionality - mostly the ability to create bootable backup drives - is this not so?
Faster! Faster! Faster would be better!
aside: "OFTE" seems to stand for "On The Fly Encryption" - an initialism I hadn't heard used by IT folks before ... but anyway....
Why aren't you backing up your files from one encrypted volume to another, at the file level? It sounds like you're doing block level backups of your container files. Do you not trust your backup computer to have those volumes open and decrypted at backup time? Dealing with block-level diffs isn't an easy way to approach the problem, but you could look at mirroring a copy-on-write filesystem, or a dedicated backup application that does its own block diffs and maps for incrementals.
I use LUKS on linux for my backups, and then the backup drives go offsite. But the backup computer is allowed to access the files while the backup is running - which isn't a problem for the risks I'm trying to defend against. If you can't trust your backup computer, another approach is to run Windows as a VM and handle your backups with linux, which has a lower intrusion rate.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
From your description you seem to do imaging of whole disks instead of backup of files.
1. Why can't you restore the still encrypted image on a new (same size or bigger) disk? As long as you image the whole disk (including the truecrypt boot loader) and not only partitions it should work. I've done it successfully from a disk with some bad sectors to a new disk with dd-rescue. If after restoring you want to resize the partitions to use the whole bigger drive you probably have to decrypt before resizing.
2. Are you using TC containers on an already encrypted drive? Why?
3. not sure, but I think if you use the ssd with full disk encryption from the start remapped sectors will always contain encrypted data. No idea about performance and endurance.
I'm using LUKS encryption and LVM2 on my Linux Desktop and there are no problems.
I don't see the point to encrypt the system partition because there is no private data on it. I just encrypt my home partition.
Backup and restore I have multiple possibilities: just use dd and copy the whole partition, use rsync or rsync-backup to backup the files. To store my backups I have created a cheap software RAID10 with external USB hard disks: https://www.anr-institute.com/projects/projects/raid-10-usb-2-5zoll-extern/wiki
With the RAID I have some security of the data in case of driver failure and I can just add more disks if I need more space.
If I have a new computer I can just install a new os (takes about 20 minutes) and copy the home partition.
http://www.mueller-public.de - My site http://www.anr-institute.com/ - Advanced Natural Research Institute
OP says "the file container is functionally useless if it's not completely intact".
This is wrong. If there's an error in a container, it just affects the 1 file that the error is on.
Every other file in the container is ok.
> "Do you use encryption for your hard drive(s)? What's your setup like and how manageable is it?"
I have two backup drives, placed in different locations so that, even if my house burns down, I still have the data. I encrypt my data. My data is not tied to a specific hard drive. It's just a bunch of files in a TrueCrypt container (either encrypting the whole drive or an encrypted file container). I can go in and access/move/delete/rename any files when I need to - from any computer the backup drive is attached to, as long as I have TrueCrypt running and have punched in the password.
"Backup images of the encrypted operating system can only be restored to the original hard drive (ie.: the drive that has failed)." What? That seems complicated if you lose your hard drive. Treat your encrypted backup as a bunch of files.
1) Not all illegal stuff is immoral stuff.
2) If it's personal, private stuff, don't encrypt? That's just dumb.
3) Junk premise and answer.
4) Also, company data that can't get into the open.
ans to 2 is why? private is private and provided you use proper security on access controls all is hunky dory
I would encrypt all easily stolen/lost devices like laptops, tablets and smart phones.
What do you mean by "proper security on access controls"? Do you mean encrypting individual files or folders depending on their contents (e.g. if you store your tax returns on your computer)? Out of convenience and avoiding the hassle of figuring out exactly which files I need to encrypt, I can just encrypt the whole thing and be done with it.
Can't say I've any experience, but the kind of things you mention seem to be a potential factor for any encryption technique.
The primary problem being wear-leveling which the drive will manage through the trim command. By using Trim the drive knows which blocks are unused and can logically remap blocks on write to ensure even usage across the drive.
If this isn't enabled, then your SSD life will suffer due to uneven wear on the drive - this isn't OFTE causing increasing writes, it's excessive writes to a single area of the disk caused by not letting the drive manage wear leveling.
If it is enabled, then data isn't overwritten, so someone maybe able to access blocks of data (possibly from before when encryption was applied), and in the worst case the "root" block containing encryption key information protected by previously compromised credentials, which wasn't overwritten when you updated credentials - thus allowing an attacker another attack point. These are also possible for not OTFE setups. Also being enabled some sort of profiling of disk usage could be made which could help uncover hidden volumes etc.
Truecrypt seems to pass trim commands through, so using on an SSD compromises security for a couple of specific attacks, do you care about those attacks?
In fact truecrypt have documentation on these issues:
http://www.truecrypt.org/docs/?s=wear-leveling
http://www.truecrypt.org/docs/?s=trim-operation
Here's the thing, encryption and backups are two separate things. I once didn't encrypt, nor did I backup. I then accidentally deleted some very important things. Whoops. I then started backing up (and taking greater care with the command line). I then got paranoid and started encrypting stuff using the built in encrypting stuff that comes with Ubuntu (and Debian and similar). Backups were still going to an unencrypted external HD though. So then I started using the built in encryption thing for that too.
And then I started using DejaDup (GUI front end to Duplicity) instead of my home rolled rsync based script. And it does backups the correct way.
So, my suggestions:
1) Use a Linux based OS, such as Ubuntu. Encryption comes free. If you have some stuff that needs M$ Windoze you can run it in a VM.
2) Forget about your OS and programs. Your data is number 1!
3) Don't backup huge encrypted containers. Mount them, and then backup the contents (to another encrypted location).
HELP MY ACCOUNT HAS BEEN HACKED BY AN ILLIBERAL ART STUDENT SET TO DESTROY THE INTERWEBZ!
private is private and provided you use proper security on access controls all is hunky dory
If your data is stored unencrypted, and your physical security can't guarantee to prevent hostile access to storage devices, then you don't really have any access control at all.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
I think your answers are wrong.
1) All the more reason to encrypt stuff, although it would take more than that.
2) Protect your privacy in the case of hardware getting stolen.
3) Because you can is a very good reason (see #2).
I'm looking into adding this extra layer of protection to laptops and external disks for our customers as an option.
home
...that it almost becomes a work of art. I want to just sit and admire it and try and tease out the nuances of idiocy and subtle details or inanity that lurk within the depths of its stupidity, in hopes that I'll reach some new plateau of understanding as I gain insight into the essential nature of the moron of the species.
alas, time is short, so I'll have to return another time to bask in the aura of this commentator's ignorance.
No data, important or confidential data, resides on any system drive, including smartphones. It gets written to a thumbdrive. Stuff like music and video is just backed up the usual way with the system up and running. The thumbdrives reside in a safe, along with the wallet and other hard to replace stuff. And I rotate the stuff out to another site. It all just became a habit as I found things that worked.
He doesn't say what OS he is using. If he is not using a mac I would reccomend a combination of full disk encyption on the local machine and use crashplan (java application) to back up an incremental set of encrypted backups. Crashplan works very well and is very reliable in my experience. (It's only problem is the bloat java program tend to do when they have been running for a long time.)
If he is using Mac OS then since 10.7 it is possible to manage encrypted disk backup most easily with the tools apple provides built in to the OS. The way it works is that you use Full disk encryption. After you boot the disk is readable by the OS. The OS then runs the backup system (Time Machine). If you use time machine you can set it to encrypt the backups. These backups can be done to a remote OSX drive and still use encryption regardless of whether the remote drive itself is Full Disk Encrypted. (That is Time machine manages the backup as an encrypted disk image).
The FDE on the Mac is accelerated with special decryption Intel Chip ops so there's no measurable speed decrease even when using an SSD, thus it does not need a special hardware encryption disk. It behaves just like a non-encrypted disk from the point of view of every program trying to access it.
That is to say FDE is preferable to the old style of OSX encryption that used encrypted disk images. With those the problems you list were all manifest (no incremental backups, catastrophic loss of all data from image corruption, and brittel behaviour of apps that expect their paths to be valid at all time regardless of the mount state of the image).
Some drink at the fountain of knowledge. Others just gargle.
On Windows, I prefer to use Acronis software for imaging and TrueCrypt for encryption.
Since the TrueCrypt operations happen at a low level that's transparent to Windows and other applications that interact with the disk, once I enter the pre-boot password for TrueCrypt and load Windows I can interact with the disk as if it were not encrypted: by making images with Acronis after Windows has booted, Acronis sees the disk as a standard NTFS drive. I can save the image of the unencrypted contents of the disk to some sort of secure backup media.
The backup media may be encrypted on its own, or I could use the encryption mechanisms built into Acronis to protect the image files. If I were use Acronis bootable media and try imaging the disk, I'd only get an image of the encrypted data -- by booting into Windows first I can make an image of the unencrypted contents of the disk.
If the encrypted drive were to ever fail I could write the image back to a new drive sans encryption. This also allows me more flexibility in regards to resizing the filesystem to new disks: since I took the image of the unencrypted contents of the disk I can resize the filesystem to a new disk. If I had encrypted the raw disk itself then I would not have this option. After restoring, I can then encrypt-in-place using TrueCrypt to secure the new drive.
As for the encrypted containers, mount them and back up their contents.
My solution is to only encrypt the data, and then only encrypt the data that needs encryption.
I partition the hard drive into system and "user" disks, then make sure that I always save data/do projects on the user disk. That reduces the encryption/backup load immensely. No need to make a backup of the installed programs, or the system executables, or my installed libraries, or browser plugins, or anything like that.
I do monthly backups, but for each project I have a "work" abbreviation that changes directory to the right place and sets everything up for me. (Ie - I type "AIWork" as a command and it cd's to the right directory, adds things to the PATH and LIBS vars, starts emacs, and spawns a remote data display server. Another command "WebWork" is similar, but with different actions.)
Each of these calls a backup routine that makes a copy of the working directory as a first step. Before AIWork is complete, everything in that directory is copied to a disk on another machine. Hard disk failures sometimes happen for electrical reasons, so you should always make copies to a machine with different electrics. (The backup routine knows not to copy non-critical file types, such as .o files)
The backups are file copy operations - if I mess up a file, I only need to navigate to the saved version and grab this morning's copy.
For secret-decoder-ring work I have a TrueCrypt partition in a file that's 1GB long - plenty of space for source files and written documents, but small enough to make a backup of the partition file itself on any day I choose to work on such a project. A little harder to recover trashed files (I have to unencrypt the saved backup before copying things out), but still secure. (Note: I increased this to 2GB just recently. Time marches on!)
Another advantage of this is that the encrypted things are not as prominent in my system. A border crossing ape can ask me to boot my system and log in, and a cursory scan won't show anything unusual. He would have to find the TrueCrypt partition file, recognize it for what it is, and ask me to boot it up. That's assuming that it's even there; it's so small I never carry it physically across the border.
I know this is elaborate, but most of it is done for convenience. There's probably more elegant solutions people will recommend, some open source one-size-fits-all cloud-based workspace management system I'm not using, but it's simple and it works for me. Also, like a 1960's Chevrolet, it's easy to repair and maintain.
Intelligence agencies do NOT want you using Truecrypt. So Slashdot obliges with a carefully constructed attack against Truecrypt that is designed to encourage betas to seriously consider the commercial options that always contain back-doors.
You see the same thing in nonsense reports that tell you intelligence agencies have the ability to recover properly erased files, or files from smashed hard-drives. Slashdot frequently promotes stories suggesting that smashing platters or properly erasing files is a waste of time. Each story is carefully created to lower the likelihood of people in general using proper security protocols.
What do hard-drive failures and encryption security have to do with one another? Absolutely NOTHING. Why would someone wish you to conflate the two things in your mind?
Simple bit errors in Truecrypt volumes do not destroy access to all the encrypted files, but I can understand why certain people have an interest in telling you so. All forms of file storage, encrypted or not, are vulnerable to hardware failures in pretty much the same way. A catastrophic failure will make you wish you had used proper back-up protocols, regardless of file security. Indeed, back-up and encryption have nothing to do with one another, and encryption certain doesn't compromise back-up methods.
"I used Truecrypt, and when something went wrong, I lost all my files". Read this sentence. The read again. Then imagine someone saying it at the beginning of an advertisement. Why does the ad start this way? What is the ad trying to get you to think? Then, perhaps, you might want to Google a bloke named Edward Bernays.
Once again, every commercial security system has back-doors, and therefore the so-called encryption provided this way isn't worth a damn. Rock-solid encryption algorithms are in the public domain, and no, the NSA does NOT have secret UFO technology allowing them to break such encryption. By law, every single security vendor that operates in the West or does business in the West has to give NSA people full co-operation to allow intelligence agencies ways to bypass security offered by their products.
Slashdot does not daily carry stories attacking North Korea, Syria and Iran by accident. Remember, this is supposed to be a 'nerd' tech site, and yet one might think, form the content, that it was one of Rupert Murdoch's media outlets instead from the sickening political propaganda. Every story promoted here is thus suspect, if you have even one functioning brain-cell. You must always ask "why is this story chosen to be promoted?"
Well-written, amusing, and insightful.
You deserve mod points but alas, today I have none.
A few more words about Crashplan.
Crashplan markets itself as a competitor to things like Mosy and other purveyors of managed remote backup. But Crashplan is distintly different than all these others in a way that is unbeatable. Namely, you don't have to use their archives to store your data. With crashplan you can target any disk as backup storage. This could be an external disk connected by USB 3.0 or one over at your freinds house (they run crashplan too), or you can use crashplans servers. They sell the app not the service if just want to use it with your own disks or a freinds.
The difference here is what happens when you need to restore. With any other service (like Mosy) you are hosed. How the heck are you going to recover a terrabyte from the remote storage to your local disk over the internet????? Not going to happen. FOr a fee Mosy will burn DVDs and mail them to you. But that assumes you know what date you want the back up for. If you are trying to recover from some slow disk corruption or a trojan you want to inspect the backups first to find the latest possible date before the corruption started, then you want to add back the newer files you can salvage. That's not going to happen with the DVDs you have sent to you.
But crash plan is different. You just drive across town to your freinds house and pick up the drive. Mount it locally and find all the files you need for the backup. Just like what you would like to have! perfect.
If crashplan would just solve their Java memory management issues it would be perfect. when you launch it it starts off with 100MB but a week later it's up to a gigabyte of memory use. Fortunately it seems the Virtual Memory manager is able to page out most of this when it's not active, but java programs are such out of control pigs.
Some drink at the fountain of knowledge. Others just gargle.
He doesn't say what OS he is using.
Yes, he does: "My setup involves an encrypted Windows installation ".
Being on OSX I've been using the built-in FDE for a while and haven't noticed any performance issues on both spinning and solid-state drives. For backup I use CrashPlan extensively. It has free P2P backups and you can pay a very reasonable rate for remote backup services. You can pick your own encryption key so that only you can actually decrypt the data. The other nice thing about the P2P option is you can partner with all your geeky friends and provide distributed backups for each other for free (ignoring bandwidth and storage)
I've written a script that backs up files to AWS Glacier. Files are encrypted using GPG (keys are separately encrypted and pushed to S3), file names are hashed and then the optionally compressed files are uploaded to Glacier. Still in alpha though.
I use BitLocker on all my drives mostly for giggles, since all my important data that I don't really want to share is stored on True Crypt Volumes, The Volumes are either stored on Dropbox if I need access from multiple machines, or backed up using Crash plan along with the rest of my data, which by the way is stored on separate hard drives from my system, so that incase of system failure...
7zip is nice because it quietly adds encryption (unlike xz).
tar cvf - (directory_path) | 7za a -si -mx=9 -pPASSWORD directory.tar.7z
7za x -so -pPASSWORD directory.tar.7z | tar xpf -
You are thinking of doing this on Windows, so beware that tar will not preserve NTFS ACLs. You can use cygwin tar if you want, but I find that the mingw tar works all right too.
If you really want to use flash media, make sure it's SLC, rated for 100,000 write cycles. If you use cheaper MLC media, media failures begin at only 5,000 writes.
Duplicty allows for scripted backups with the archives being encrypted by GPG and therefore can be restored to any drive, so long as you know the password.
I use it for all my Windows encryption stuff. Not tied to hardware, can encrypt partitions as files or partitions in situ, you can even make hidden encrypted volumes for plausible deniability's sake. Oh, and it's open source. Enough said.
This stuff is handled perfectly in the Truecrypt FAQ isn't it? RTFM
- You can create backups of TC containers, it'll actually be much faster to create a differential or incremental backup of the container since only 1 file needs to be read (cache hits will be more efficient) compared vs millions of little individual files (each costing IOPS on both sides to initiate the backup as well as compare and store it).
- 1 (or multiple) bits falling over in an encrypted file system does not cause the entire system to go corrupt. In some cases, the encryption will be more safe as internal checksums may be able to detect and/or auto-recover the errors.
- Disaster recovery does not require you to restore a TC volume to the same disk. That is a Windows-specific limitation. You can't restore MS Windows to a different system and expect it to boot because of it's DRM and driver model (Mac, Unix or Linux systems do not have this limitation). You can still open the backed-up TC volume or even open the disk TrueCrypt volume in another machine.
Custom electronics and digital signage for your business: www.evcircuits.com
Do you live in an underground bunker, with automated blast doors and multi-layer security? I doubt it. Does anybody really care enough to defeat such measures to get into your house? I suspect you're like the rest of us, with standard locks and maybe an alarm system or a dog, or both. That is sufficient to deter all but the most determined criminals. And if anyone is determined enough, your extra security won't stop them anyway.
Your data isn't that different. Nobody is really after your data, at least not to the point of being willing to spend serious money and time getting into your system. The real threats are things like malware (which won't even be slowed down by your encrypted drive), or somebody snooping around on your hard drive after stealing your laptop (when actually they are more likely to want to just sell it).
Common sense is the best protection for most of us. Don't save passwords in an unencrypted file. Use a non-trivial password to log on to your system. Hang on to your stuff. You get the idea.
That is extremely bizarre and I guarantee you that 99.9% of people who encrypt their disks, don't have that problem. I suspect you're either wrong about the limitation, or you are using some kind of weird crippleware. If the latter, drop it now and go to something normal like dm-crypt or whatever.
Things just get weirder. Ok, I think I get what you're doing now. You're backing up entire block devices, rather than files. Most of us would back up a filesystem (which involves decrypting the block device, interpreting it as a file system, the backup system backing up each file, and in turn re-encrypting it for backup purposes). Our approach, I admit, would be slower. But simpler. And fixes the problem you're talking about.
You'd probably have the same problem if you weren't encrypting. Do you backup machines which don't use crypto, as full partition backups, rather than as files?
If that's what's going on, stop it.
Do you know for sure what exactly is illegal in different countries? Some people travel internationally. Maybe some of your completely legal stuff is illegal in some other country you're travelling to.
That's the main reason for encryption. If someone steals your laptop, losing the hardware is bad enough. You don't want them also to have access to your private data. And no, your OS level security cannot protect your files in that case, because all one has to do is to boot a live system from CD or DVD (using an external drive, if necessary) and use that to access the files. Since your installed OS isn't even started, it cannot prevent that access. Or alternatively, remove the hard drive and put it in a different computer.
The Tao of math: The numbers you can count are not the real numbers.
I have all of my drives on all of my computers fully encrypted at the system/partition with Truecrypt.
For each drive I also have an identical encrypted backup drive which is stored in a drawer and never used unless I'm actively backing up its partner. When I want to backup a given drive, I pull out its backup partner, plop it in my Thermaltake drive dock, and run SyncToy. When I'm done, the backup drive goes back in the drawer.
It may seem a little cumbersome, since I have to actually physically touch drives in order to do it, but SyncToy makes it really quick and easy since it's only doing incremental backups. I have a rotating schedule of backups so that each drive is backed up at least once every couple of weeks. I also backup the most critical files on my laptop to another server over my local network once a week with SyncToy, so there are always 3 encrypted copies of my most important and frequently-used files.
There are a lot of similar and tangentially related responses, so I'll make this one post instead of responding inline to each one.
SETUP
My partitions look like this in gparted:
Note that TrueCrypt replaces whatever existing bootloader is on the drive with its own so it can run pre-boot authentication to decrypt the OS.
Yes, I realize that I don't store any data on the OS/software partition, but there are still system caches, logs, databases and other potential data leaks to consider. For example, the encryption keys for the file containers get stored in the hiberfile. Without encrypting the OS, it would be fairly easy for an attacker to access the keys to the encrypted file containers. With the OS encrypted, this is not an issue.
BACKUP
I use TBU IFL (self contained bootable imaging software) to backup the OS partition in it's raw, encrypted form. Yes, this causes it to backup empty space and it takes quite a bit longer to run. Differential backups do work and are a smaller filesize than the original full backup, but they still take just as long.
From the research I've been doing on TrueCrypt's forums, it appears the reason that I can only restore these backups on the original drive is due an insufficiency in TrueCrypt's rescue system.
What I've found is that I can use Windows-based imaging software to backup the OS partition in it's decrypted form, restore the image to a new drive and re-encrypt it. It's a bit of a process, but it's still faster and easier than running the full TrueCrypt decryption process, copying the partition to the new drive and re-encrypting. I could use the imaging software's own encryption scheme to protect the image if I want.
When I backup my data, I do exactly what many have suggested here: back it up at the file level by using synchronization software to periodically compare and copy changed files (from inside the TrueCrypt volumes that I've already mounted) to an external drive which contains mounted TrueCrypt volumes of identical size.
FOLLOW-UP
So, to rephrase and answer some of my questions:
1: Non-Portable OS Backups
Solution in this context: use Windows-based imaging software to backup the system volume in its decrypted form. Use the encryption feature built into the imaging software to protect it.
2: Volatility of Data Containers
The problem is not backing up data inside the containers. The problem is that they are volatile. Any data corruption in the area of the partition where a container resides renders that container functionally useless and all contained files inaccessible. Basically, adding a layer of security also adds a layer of instability, but I don't currently see a better way of acheiving that security.
3: SSDs
Still not sure what to think about SSDs, all of my research has yielded mixed results from polar extremes of the spectrum.
Get a Mac. Turn full disk encryption via Filevault2 on. Backup using Time Machine with an encrypted backup drive. The encryption is invisible except that you have to enter the password from time to time.
You might want to have a look at Duplicati - that's what I ended up using after I spent a while looking into how to do backup securely. It'll handle scheduling, partials (i.e. diffs, if you want), compression, encryption of the result, and finally upload to a whole range of different cloud providers (or a local directory, of course). It's free, and available for Windows, OSX, and Linux.
Enable BitLocker on your drives. When you connect a drive for backing up, bitlocker it then use any backup mechanism (win backup or file history in 8) problem solved. Everything's encrypted and backed up.
If you ever expect the data in there to be useful to others then don't bother to encrypt your backups, it's an accident waiting to happen. Rely on physical security of the backup media instead.
Even if it's you own stuff, do you really want to mess around at 4am with a recovery procedure you can't quite remember that is written down somewhere you can't get to?
I backup website data to a local NAS. I don't allow external access to the NAS. Access must come from within the LAN. Second, I use linux, and use rsync to make backups. It works very well across networks, will try to recover data from bad drives (I've recovered data from a drive that would only reliably work for the first 5 minutes after turning it on), and makes excellent full archives, and subsequent snapshots. I don't encrypt, although that's an option I could explore in a hundred different ways.
yes it's one of those tradeoffs isn't it. I could make my files and backups more secure but if I get hit by a bus my wife will likely lose access to our family photos, tax returns and other files due to the complexity of the setup.
Sorry, I could have been more clear about the crux of the matter.
I *do* have multiple onsite and offsite backups which I update them regularly and religiously (I did have to spend two days reconstituting some data as my backup software had failed 5 days prior to this drive failure and not warned me ... but that issue has been resolved and is completely external to the matter at hand)
How often, if at all, were you doing restore tests? And how complete were your restore tests? Were you doing test restores of individual files or bare metal restore tests?
In the free world the media isn't government run; the government is media run.
I can't speak to the security of SSDs.
There are two reasons why it may reduce the lifespan: 1) no TRIM support. Here is a Q/A which confirms this for LUKS on Linux, I doubt Truecrypt have TRIM support either. http://superuser.com/questions/124310/does-luks-encryption-affect-trim-ssd-and-linux . TRIM is relatively new, and while most filesystems do now support it, you're not losing out on much performance. An alternative is to leave a percentage (e.g. 10 %) of the drive completely unused, as an unformatted partition (NOT encrypted). This gives the drive's wear levelling algorithm some more room to work with. Drives do of course already have some such space which is not visible to the user.
2) Encrypted data are not compressible. Some SSDs use compression to get better write bandwidth and to reduce the number of writes. You shouldn't worry too much about this. You get what the hardware can support. The same thing would happen if you only stored H.264 videos, which also can't be compressed much, so the manufacturer has to allow for this.
Some programs have their own notion of what a "user" is and store all users' data in a central location OUTSIDE of where Windows stores its user profiles.
This MAY wind up being on the C:\ drive.
"Temporary" copies of user data may also wind up on the Windows system drive, Windows Boot drive, or even the C:\ drive (yes, even if you "Boot" from "D:" some old programs have C:\ hard-coded into them, sigh).
And of course, pagefile.sys, hiberfil.sys, and similar files are by default also on the Windows system drive or Windows Boot drive.
There are no doubt many other one-off cases to watch out for as well. The moral of the story: Know your software and what it is really doing behind the scenes or be prepared to be surprised.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
That's true, and if you are worried enough to encrypt your user data partition, you should do your system partition as well to make sure everything is being properly encrypted.
This post was to address the fact that that it isn't difficult to separate the users directory from the C: drive. Software throwing files all over the place isn't new, but thankfully most are following recommend procedure which puts everything in the user's directory.
You can even do it on a per-user or even per-subdirectory basis by using NTFS mount points, but that can be a bit more confusing if you forget what's mounted where.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
... the first thing you need to ask yourself, is WHY do I need to encrypt data?
1) Illegal stuff 2) Private stuff 3) Just because you can
ans to 1 is don't do it - you will be caught eventually ans to 2 is why? private is private and provided you use proper security on access controls all is hunky dory ans to 3) paranoia
Your burrying your head in the sand. Everybody should have concern about point #2 you listed. If you dont believe that you do any private stuff then you are not using online banking or sending email. As a minimum, these are things you should try to protect. 99% of your data from an unecrypted windows or linux system can easily be accessed. Your word processor documents and email messages and wifi passwords are accessible to anyone who steals your desktop or finds the laptop you lost. I have done this as an excercise. I am not a "hacker" and do not have desire to steal from others, but it is easy for me to get all this information from your system. So a hacker would easily know how to do it and then exploit that information to make life difficult or exensive for you.
So if you use a laptop and you do not use encryption... then it would be good for you to actually be a bit more paranoid then you current are.
At my place of business, I was able to easily show how both windows and linux emails and documents could be read off of anyones work computer. This was eye opening for our managers, because we dealt regularily with customers private information and shared private information between other business owners. Encryption is important in a business settings, but it is probably most important on portable devices like laptops, which are far more likely to be lost or stolen. Because I assure you it takes no effort at all to clearly see everything you ever put into it without having to break any user passwords.
I run full disk encryption using luks on linux. Full hard disk encryption with luks is easy to install during linux installation... just two extra steps from the install menu. I install my operating system onto an SD card (although USB would work too). In this way it is very easy for me to do full system backups that are also encrypted. And a full backup costs roughtly $20 as SD and USB is cheap. For mass storage, I mount either encrypted or unencrypted hard drives. Usually, I do not encrypted the mass storage devices, because this primarily contains pictures and video that do not have great value to theives and have little chance of being used for identity theft. For more private videos and pictures, I can always store these on an encrypted partition if desired.
By running my main system off of an SD card, a full backup is simple. I can simply plug in a fresh SD card and call my backup script which creates a new encrypted partition and copies all my data. Then if my computer is lost of stolen, I just plug it in and go. There is no down time of reinstalling a new operating system and transfering backups to update it. Plus since the backup is full encrypted itself, I have no concerns about someone stealing the backup either.
I find my current solution simple and cost effective.
Here is something to keep in mind. This assumes you are using truecrypt on windows.
1. Fat is not a very reliable filesystem, however its needed for trucrypted stenography features, therefore: /f on the drive. If not, you then run rstudio again on the mounted drive, and recover to another locatioin. In the worst case scenario, you need at least twice the size of the container in free space on an unrelated/new drive.
2. Create the hidden volume using NTFS filesystem and the normal volume using FAT/exFAT.
3. Always use a container file.
4. Use the "Backup Volume Header" feature in Truecrypt to backup the encryption key info from the container file. Save this to flash drive or something not on the same machine.
5. Using RStudio (Disclaimer: I dont work for them, I am just a customer than successfully recovered a truecrypt volume). http://www.r-studio.com/
6. With RStudio you recover the entire container file to the replacement drive. You ask rstudio to recover the entire file even if there are read errors, you get it to fill those in with nulls.
7. This will take about 24 hours for a 1tb drive, so sit back and wait.
8. Once you have the container file on the new drive, attempt to mount it. It will mount if the unless the volume header section is damaged. If it is damaged, use truecryt to restore the volume header that you backed up.
9. Once it is mounted, if windows can see the drive and files on it, you are lucky and all is well. You can then run chkdsk
10. Note that the price of encryption is you need to backup the entire volume, or you need enough space to attempt to recover a damaged volume. Having everything encryted is nice, but it comes at a price. You need to determine the appropriate trade off. Remember that if you are using the stenography feature, you are already willing to loose your data if you are compromised as the moment you mount the outer volume without protection enabled, you risk corrupting the data on the inner volume. This is the price you pay for plausable deniability. Of course dont forget that is the someone really wants it, and they dont believe your fake outer volume is real, they gonna beat and torture you anyway. Obligatory XKCD: http://xkcd.com/538/
Also Note:
Ideas on the outer volume also is to use a tool to kill the timestamps of all the files that you place there. This is recommened, because updating the files in the outer volume after the inner volume is made is a difficult and tricky operation. If the attacker mounts the outer volume and sees are the files are like 5years old, they gonna smell a rat. If they see all random timestamps, you say you ran the tool as recommened practise. Examples of such a tool is timestomp, however that doesnt work on FAT, so you need to find another one, I just cant remember the name right now.
BitLocker
http://windows.microsoft.com/en-us/windows7/products/features/bitlocker