Slashdot Mirror
How Do You Store Your Personal Photos?
mxhf writes "I just came back from a four-week vacation to Mexico. This is the country for Aztecs and Maya Ruins and we visited plenty of them. Needless to say we took thousands of pictures with two cameras. Having arrived back home I realize that my hard-disk does not have enough space left to hold the additional 16GB that I collected on the other side of the globe. Now, my hard disk already is 250GB. I work exclusively on a laptop and do not want to change this. I know that there are larger disks today. But I figured that the time has come to finally move my image collection from my laptop to somewhere else. But where should I go? So, how do you store your photo collections? And how do you keep backups? These are obviously images that I want to keep for my life. So the need to survive fires, burglaries, etc. I think the amount of data I have rules online storage out. Should I just get two USB disks and leave one at a reasonably save location? I think this must be a common problem today. And yes — before you ask — I do know that the first thing to do is to go through your collection and dump what is not worth keeping."
In your case, since it sounds like you don't create that much data, you'd probably be fine picking up a couple of portable USB drives (2.5" drive, powered over USB = tiny). For consumer use, the Samsung Goflex 1TB (the 2.5" version) is around $100, widely available, and works great in my experience. Buy two. Use one as your master repository, one as a backup of that, and keep the second in a water-proof container (hint: try rubbermaid containers, they're waterproof and cost about $4), locked in an inexpensive fire safe, safety deposit box, or at a nearby friend's or relative's house. If you aren't needing to store more than 64GB of material then you could substitute "thumb drive" or "CF/SD card and reader" for portable USB drive ... solid state media will be 'safer' for long-term storage but obviously afford less space-per-dollar.
A better option, but beyond what you wanted is a SAN/NAS. Drobo makes some decent products, and I currently have a DroboFS at my home, loaded with 2TB drives. This gives me a little over 7TB of RAID storage to backup all my footage, images, documents, and so forth. It's network addressable, so any of the several machines in my house (both Mac and Windows) can access it. The total cost (Drobo + drives) was around $1100 or $1200 iirc. The downside to the FS is that its max transfer speed is around 20MB/sec, but they do offer other models with transfer speeds that are better suited to live editing — I only use the FS for backup, I have 4TB [in the machine I am posting from now] dedicated to live editing. The Drobo is nice, imo, because it's a consumer-oriented appliance (with RAID built in) that can take any SATA drive, will allow you to mix and match drive capacities on the fly, and they offer 'Time Machine' style automated backups (along with other apps) if you want that sort of thing. Beyond the Drobo, I also do separate backups to portable drives and keep them offsite (as I mentioned above), just as an extra level of paranoia in case my house burns down. If you are really paranoid or into safety, LTO would be a better way to go for this.
Actually, given how little data you (the original poster) might need to backup, an old LTO machine bought on craigslist (LTO 1 will do 100GB, 2 does 200GB) might be the solution. The tapes are relatively cheap, and the format is both open and reverse-compatible for a few generations (so when your LTO 1 craigslist machine dies you can buy an LTO 2 or 3 machine from the same venue and still access your content (and then migrate it forward to LTO 2 or 3)).
I am Jack's complete lack of surprise.
with no backups. Faulty power supply fried it last year. Yeah, I keep regular backups now.
I definitely wouldn't rule online backup out. Unlimited backup providers like Crashplan, Carbonite, etc. certainly provide a service that can be very useful.
This type of question comes up a lot. How do I store for the long term?
Simple answer. Have it spinning on disk (or flash, or SSD, or...) and live accessable, plus an off-site backup.
Any off-line media will at some point be unreadable. Keep it accessable & live, and migrate it each time you upgrade your system.
Sure, I've got a few 5.25" floppies around, but how to read them? Keep it spinning & live.
It's the safest, most secure and private place on the internet I can think of.
I made an app! Shoutium
That's the simple solution. Then have that backup to the web via something like Crashplan.
In addition to external hard drives, I use DVD-RAMs, which are supposed to survive a few decades.
Get a couple of NAS drives. Have your laptop run backups between the two devices in case 1 drive fails, or just run 1 device with RAID 1/5. Burn Blu-ray backups every 6 months or so, throw them in a fireproof safe or safe deposit box. Or take a separate USB drive to do the backups and throw that in the safe. If you're running Windows and the NAS is available as a windows share, you can run the free SyncToy app to do incremental backups.
Buy a larger internal hard drive for your laptop...that will solve your "immedeate access" needs.
If you're really serious about actual back up:
1. Buy a 1 TB external hard drive. Copy all of your pictures on there, then put the hard drive in a safe deposit box. This will be your "iron-clad" backup, one which you only update after major trips such as the one you came back from.
2. Buy a second 1 TB external hard drive that you keep at home. This will be your "primary" backup, one that gets updated every time you have new pictures.
3. For extra protection, buy a crap-ton of DVD-Rs, and burn all your photos on them.
Living With a Nerd
I do know that the first thing to do is to go through your collection and dump what is not worthwhile keeping."
Keep doing it ... get it down to like 20-30 pictures.
Face. Book.
If brevity is the soul of wit, then how does one explain Twitter?
As a part time pro photographer, I'll let you on a little secret. You rarely need more than 5 MP of data. Downsample all your images to 2500px on the long side, with the appropriate aspect ratio, save as 98% JPG and enjoy. Unless you are going to print 30x40-inch high gloss roll off prints, or crop massively, your 12-15 MP camera is really chewing up disk space for no good reason.
...that leaves external hard drives. So buy a couple, back up from one to the other, and keep one somewhere else.
I put the best of the best of my pictures up on Flickr pro account, but that only works out to a couple dozen a month on average at most.
Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it ;)
* Torvalds, Linus (1996-07-20). Post to linux.dev.kernel newsgroup. Retrieved on 2006-08-28.
I do not move them of my computer, I use a backup service called backblaze (www.backblaze.com) that gives you unlimited storage and continuously backup for 5$ / month, hassle free and cheap.
http://www.intellipool.se/ - Intellipool Network Monitor
You'll never ever look at the vast majority of them. If you don't have time to look through them and only keep the good ones why not just delete them.
Come on, Mexico is _not_ in the other side of the globe from US...
I upload them to 4chan, from where they will be stored on a multitude of /b/tards' harddrives forever.
Get an external drive to keep a local copy, and upload everything to flickr.
Bam. done.
1. Get Website
2. Install http://gallery.menalto.com/
3. Store Photos.
4. ????
5. Profit!
$25 / year. Easy to use. Easy to share. My 70 y/o dad harasses me on a regular basis when we'll post new photos. Currently have 10k+ pics online. Back up of our Flickr is Carbonite. -K
God is good all the time! -K
If your data is important to you, you keep it on a radied device that you back up to another device, that is also backed up to a remote location.
It is well worth the $100/year to shell out for an online webspace to store your photos if you want to keep them for life.
10 gb is nothing, just setup a background process to sync and limit it's upload bandwidth, and it'll do it over a few days/weeks, no matter how big your file is.
That way even if your external dies, or gets stolen, you have that ace in the hole.
Peace of mind, especially for valuable memories, is worth the money, plus it has the added benefit of giving you a way to share photos with friends/family easily. Plus any other things you want to do with some webspace.
The reason i recommend buying a full webspace somewhere rather than dedicated backup utilities is because you can normally get more storage/cheaper, and have a little better direct control over your data, with the added convenience of access through http!
You never realize how much manually made unmanaged "linked" lists suck, till you have src.link.link.link.link...
5 2TB drives in RaidZ2 (Yes, it's not optimal, but it'll take 2x failures.
2 external 2x2TB enclosures with the drives mirrored. Rotated off site every week. When an enclosure comes home I scrub it make sure I didn't break anything and then for the next week everything is synced nightly.
If I'm not shooting any photos, then I really don't rotate stuff.
I'm not quite at a TB of photos, but shooting 8GB at a time does start to add up. Last resort nearly everything is on Facebook. They do allow photos up to 2000 pixels / side. It's not a lossless backup, but if it means having children's photos vs not, it's better than nothing.
Sadly NOTHING has happened. No drive failures, nothing. I haven't been able to test any of it out other than when I upgraded to 2TB drives from 1.5TB drives with a ...
zpool replace tank
I like to keep them in picasa. I trust the cloud (especially for a company like google) much more than my own management of a couple hard drives.
Plus, I like the service (its interface, being able to download the original, easy sharing, transparent sync to my phone, etc..).
The big downside is not being able to download entire albums in one download (maybe there are 3rd party apps that do that), and the fact that you can't upload videos unless you are using the windows client (I usually just use the web).
Funny enough, I was just thinking about this insofar as my backup disk died, while the main disk in the machine is still running fine. I've listened to enough TWiTs and the like to know about Drobo and checked out the site. I like that the size can be increased over time (up to whatever limit the firmware supports in the enclosure). I was thinking I could also justify it by getting the version that sits on the network as a NAS and use it for all my Time Machine backups, etc.
I also have a separate external disk (not a Drobo or NAS or anything fancy) that I do an overnight copy of all the important files using rsync with the disk plugged directly into the Firewire 800 port, then I take the disk with me to my folks house and let it sit there. After a week or two I bring it home and the whole process repeats.
I've also got a private vpn to a Linux machine I set up, but even though I did a full update on it for backup, rsync takes forever (many many hours) to determine what files need to be updated/added, and the machine gets pretty bogged down. Still working on a good solution for automatic offsite backups...
I'd be interested to know what others think of the Drobo before plunking down the $$$ for one.
Every single one? Why? Everything you feel you must have for life is another thing you'll be "paying" interest on for the rest of your life, in the time and money spent managing it. When you die, will anyone want to continue saving these thousands of photos from a single trip, or even have time to look through them?
Simply, just use silver halide films (35mm or bigger) and keep them in a fireproof safe. They will last pretty much forever, and you will always have the technology to view them...aka your own eyes.
"And yes — before you ask — I do know that the first thing to do is to go through your collection and dump what is not worthwhile keeping."
Well let's start with that then, as it is totally false.
You have a mere 16GB of data. You can fit that on a Flash drive and leave it somewhere in Death Valley to be found 500 years hence if you wish. You could mail one to every continent on earth for a pittance.
Yes it's good to sort through images looking for thing that are worth more effort editing and sharing now, but you should never assume you know now what pictures will most interest you 10, 20, or 50 years later!
As for the base of your question, I use an external hard drive, which I clone to another external hard drive, which in turn I keep another clone of in an offsite location backed up about once a month.
However I am strongly considering other backup based web services that would let my backups by updated more in real time and more geopgraphically dispersed than my current solution, so you might want to look into that. It involves a monthly fee though, so if you can't pay there goes your backup... such services often have a way to "prime" them with a hard drive you mail in, so you don't have to transfer many hundreds (or thousands) of GB of data to them over your capped internet connection.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
I have a NAS (RAID 5) array that I keep the photos on. Every once in awhile I'll burn the photos directory to two DVDs. One is kept locally just in case of a failure of the NAS. The other goes to family/friends house in case of a disaster (fire, etc).
Most of the pictures we have of the kids since they were born are all digital, so I don't want to take any chances with loosing them.
Unix is user friendly, it's just selective about who its friends are.
back up frequently, and try to keep one off-site if possible. Online solutions are also fine- they just take while to send the data up the first time around.
The price of a couple of thumb drives is insignificant compared to the price of your vacation and the memories associated with the photos. I buy a thumb drive for every vacation and store them this way, in a medium sized fire proof safe. Yes, thumb drive storage is more expensive than hard drive storage, but I trust their longevity more than I would a hard drive.
For me I keep two local copies. One on my internal HDD and another on my external HDD which is for backup exculsively. And one remote copy on my flickr pro account. I think my photos are pretty safe this way.
There aint no pancake so thin it doesn't have two sides.
You either take way too many pictures. Are you really going to look at thousands of pictures of ruins? Hardly. However, since that's not the kind of advice you asked for, I'd suggest an external HD. It's cheaper than a similarly sized pen drive (1.5Tb ~$80).
I bought a ReadyNAS bare and added 2 2TB drives in a mirror. I have a 1.5 TB USB drive which also plugs into the NAS and I rsync them periodically.
I store my photos on the terabyte hard drive I have dedicated to archive storage. In my case it's in the machine that acts as the general-purpose server for my home network, but you can buy dedicated storage boxes from companies like NetGear if all you want is a file server.
External services? Why in the world would I want my stored photos to be at the mercy of a free service deciding to close up shop? And it certainly isn't financially feasible to keep paying a monthly fee for storage.
>I do know that the first thing to do is to go through your collection
>and dump what is not worthwhile keeping.
I'm not sure why you say this in such an authoritative tone, but this is a great example of something you *shouldn't* do. There's nothing to say that a shot that you're not particularly fond of today will remain so forever. This is especially true when you shoot in RAW, since there's so much to work with and techniques you can learn to salvage a so-so picture.
Moreover, your attitude of 'I have a laptop with an internal disk and don't wish to change' is a terrible one to have. If your data's security is important to you, you'll need to expand your horizons quite a bit.
That's hardly a backup. What if you miss a payment on the hosting bill because you're in the hospital or something? Your files = gone for good.
There's no -1 for "I don't get it."
Zip them up and call the file something like "HD Celebrity sex tapes collection" and upload it to your favorite bit torrent site.
I call it my free foolproof backup solution.
Then you can just download your pictures any time you need them.
If I were God, wouldn't I protect my churches from acts of me?
Compress all your images inside a .RAR archive (you'll understand why in a moment), then make a torrent available under a name similar to "Hundreds of thousands of pictures of naked celebrities".
The .RAR archive is to prevent people from looking at the individual filenames and checking only a few pictures to confirm if it's really naked celebrities.
You came back from Mexico alive?!
I just upgraded my wireless network to allow for a shared hard drive. I have several laptops at home, but I was running out of space quickly with all the photos and music that I own. I had some portable hard drives that I would move photos to, but keeping that plugged in at all times was a pain. Now with a wireless external storage, I just keep what need right now on the local drive, and the rest is shared. When I am at home, I have access to everything. Right now it's 500GB, but that will expand to a few TB soon. I will also be looking for online backup once I am a bit more organized.
I use the United States Postal Service, in business since 1775. Basically what you'll want to do is:
Many years from now, after your home has been looted and your laptop and backup drive have been stolen, or gone up in flames, or destroyed by an act of God, your treasured photos will arrive, hand-delivered by a dedicated, part-time USPS contractor.
As the subject as for me. I get my photos from my camera onto my laptop, and then semi-regularly sync them with Time Machine to an external drive of 500 GB. Automatically starts backing up as I connect the drive, which is important to me -- if it's not convenient enough, I'm not going to use it.
Unfortunately, my laptop's drive is "only" 250 GB large and it's one day going to fill up. It's actually taken surprisingly long despite me photographing mostly in 12 MP RAW, but when it do happen, I guess I'll move to a larger hard drive as my primary drive and (surprise!) a larger backup drive. I'll of course still keep my old drives - no point in needlessly throwing away redundancy even if they don't offer complete redundancy.
I think that'll do for me for now. Won't help much against burglary or extreme fires where I won't have time to get out and carry a backup drive with me, but I guess that's my limit then. If I were to go further, I think I'd have looked at online storage despite the storage needs. It's getting cheap today with Amazon S3 and all the services that make use of that as a back-end. You may also wish to look at Google Docs. Stores any file format (including encrypted file archives *hint*), 200 GB there is $50/yr ( http://docs.google.com/support/bin/answer.py?answer=39567 ).
Beware: In C++, your friends can see your privates!
I use three hard drives in my main computer. One small drive for the OS and installed applications, a second large drive to store my media (1 TB is sufficient for me), and a third drive to hold backups. Differential backups are automatically made for WIP data on a nightly schedule, everything else is automatically done on a weekly schedule.
Every few years I pull the hard drive and wrap it in some bubble wrap, package it into a cardboard box with the date on the outside and give it to my parents to store in their attic as a fallback.
The total cost of this operation comes down to about $100 every three years.
I keep three backups at all times, all on 1TB hard drives.
1 is kept running on my home server (the equivalent of on your laptop, I suppose)
1 is kept offline, but nearby
1 is kept in my car
Every month or so we sync all the new photos onto all the disks.
I don't have a truly off site location available to me, but I figure that the car is probably going to survive a house fire and the house should survive a car crash.
My plan is not meteor proof and should not be construed as such.
I print them out in Cauzin Softstrip format on archival paper. It's the only way to be certain that my blurry thumb will be preserved for my grateful descendants.
Consumer laptops and their components are engineered and manufactured based on priorities that the market dictates. Were I in front of you, I might well pick up your laptop and smack you in the head with it: You are NOT using a device optimized for mass storage.
Your laptop's design priorities are: 1. low price, and 2. small size. In that order.*
Notice that I didn't mention reliability, speed, or storage capacity.
In addition to the laptop computer that you already have, I recommend that you get a desktop computer (ridiculously cheap desktop computers normally outperform most laptops and are much more reliable). It's a realistic, grown-up thing to do.
* Unless your laptop is a Toshiba, which are sometimes
no lighter or smaller than desktop tower computers.
6 strips of 6 per page
Is it just me or is the rest of the world getting just as trigger-happy as the Japs (assuming that the OP isn't :})? 16GB of data over 28 days at a resolution of say, 4MB per picture, is about 150 snaps every flippin' day. That sounds more like work than a vacation to me.
Reminds me of this chap.
In any case, I rely on Picasa for low-res storage, NAS for general storage and an external HDD for back-up of essential files which I sync every now and then and store at a remote location.
... and it wants it's Ask Slashdot question back. 250GB of data? But seriously:
1. Buy USB or SATA HDD dock (3.5"). Probably USB3 is what you want, and get some sort of card so that your laptop can connect at USB3 speeds.
2. Buy several internal 3.5" HDDs. Browse newegg and pick the best $/GB that is as reliable as you need it to be (judge by % of low star reviews). They make them in TB these days.
3. Find a good priced local store and buy it there if you want to minimize risk of it being damaged in shipping, or buy on newegg if you want.
4. Back your 250+GB of photos to at least 2 of these internal HDDs, one of which should be somewhere else that is safe. It will cost you a couple hundred bucks, or 5% of a somewhat decent digital SLR with lenses, flash, tripod, bag etc.
5. Profit.
If I have seen further it is by stealing the Intellectual Property of giants.
2.5" USB-attached hard drives that only require a small USB cable and no other power cable are cheap and large now. Then use FreeFileSync to make sure that the same information is on two or three of those hard drives.
and it wants its grammar back. Doh.
If I have seen further it is by stealing the Intellectual Property of giants.
I keep all of my photos on a file server that automatically backs up my files every night to a USB drive and have another that gets written to monthly. These two stay in the house and are in case the server dies. I also have a USB powered drive in a safe deposit box that I update yearly. Then I have two other USB powered drives that I rotate through the safe deposit box monthly. I'm also kicking around the idea of keeping a drive at a friends house and backing up through the internet. A few years ago I was backing up my family photos manually and got a little lazy for about 6 months and lost a drive with many of the pictures from my daughter was 4. That sucked, so I decided that redundancy and automation were the way to go. You may want to check out some of the internet sites that do on line backups.
... particularly those that predate ubiquitous image capturing (I can't in good conscience call it photography), I just don't take many personal photos. A few each time to document the event, but not enough to warrant a question about how to store all that stuff.
Instead, I prefer to *live* the moments, seeing them personally through my own eyes, rather than experience them through the camera viewfinder and then later via images. My epiphany came in the hospital when I was faced with the choice of documenting the birth of my daughter with a video camera plastered to my face, or putting the fucking thing down and living the experience myself. You can probably guess from my choice of words which option I chose. So I'm left with my own imperfect memory of the event rather than a memory as seen through the viewfinder and replayable later.
Your precious personal photos and videos are like the dreaded vacation movies/slideshows back when people did that kind of thing. Odds are you will never look at your archive of photos very much - if you did, you wouldn't be experiencing new things, you'd just be reviewing your old experiences over and over again.
So stop worrying about your "precious" photos and just go out and experience some new things. Pay attention while you're doing so, and you can tell stories later about the wonderfullness of it all.
IMHO, this is much better than compulsive photo-documentation.
But I don't expect many to agree. Shiny gadgets have captured our souls, and I'm afraid they may be lost forever.
Seriously, are all those thousands of vacation pictures worth storing?
Or could the feeling of the vacation be summarised by less than a hundred pictures? What about less than fifty? Less than 30? 20?
We really are behaving like mad magpies, hoarding this data as if it really were the memories of the event (well, if one takes multiple thousands of pictures then one may actually have spent the whole vacation behind the camera instead of enjoying the experience. See "experiencing self vs remembering self" http://sheshtawy.wordpress.com/2010/03/01/experience-self-vs-remembering-self-experience-vs-memory/ )
I've recently taken to culling my selection of pictures which I actively back up; selecting only a dozen or so images from each month. That still results in less than 150 images a year. This selection gets backed up both on multiple media here at home as well as backed up online. The other thousands of pictures are saved only at home, on an external drive (external USB drive connected to an Airport Extreme) an on my laptop's internal drive. These extra images just don't require the safety of an off-site backup. They're just not that important!
And nobody will care about the 500 pictures of an Aztec pyramid in a couple of years. Even if you and a loved one are in the pictures it will end up that there are two or three pics which are great, the rest serve only to bore housegests senseless when subjected to the torture of a thousand picture slideshow of places they haven't been and people they don't know...
When I think to my childhood I actually remember large parts of it, especially extremely good or bad events. This is independent of whether pictures exist from that event. Where pictures exist, they tend to colour my memory, and in many cases change it (events which I KNOW weren't fully positive, but the single picture from the event shows something enjoyable happening and everyone smiling).
Pictures LIE, and they change how you remember. Taking them also changes how you experience life. Live a little.
OH the HORROR! What should I DO? Please slashdot, help me solve this difficult problem!!! I need a team of NERDS for this!
Seriously, with HD prices at under $100 for 1.5 TB, who gives a flying fuck? If you don't know how to plug in a USB drive you should be shot.
Seven puppies were harmed during the making of this post.
One component of a solution might be to make lower-resolution versions of the pictures (e.g. use ImageMagick to do a batch scale-down) and store those in multiple copies. You could probably reduce the size by 90% and still have perfectly useable pictures. Obviously lower-resolution is not as good as the original, but it would make it more practical to make many backups, and keep them in many places. Lower-resolution is better than nothing. (This is not meant to replace backup of the originals - just make it possible to make a lot more, and as a result a lot better chance of something surviving.) BTW - Make sure you also backup the captions for the pictures - you are going to do that, right? I've just spent the last few months going through my father's old slides, trying to figure them out. BTW2 - I made a bunch of backups on CDs about 10 years ago. I went through them a few months ago - about 75% were unreadable.
Flickr Pro imposes no storage limit, and you can keep your photos private if you choose to. You can also organize your photos in many ways, thus you can show them to your friends if you'd like to. Granted, I don't know any way of downloading many photos at once (ie. restore the backup), but I've never been interested in doing it either. Check this.
1. Get an external hard drive. I'm sure there are many comments available for good solutions for this
2. Get some of those digital photo frames, load up some SD cards/USB sticks and keep them around the house. You could have one photo frame for one holiday, another frame for a special birthday or other event, etc.
This way, you have your main back up in the form of a hard drive somewhere, and a visual back-up/reminder of the events for which you took photos for/of to remember.
I'm happy with my Apple iDisk which is included in a mobile me subscription. I use it along with storage on my computer and an external backup drive. The standard subscription has 20GB of storage, which is about right for photos, but you can get more with different subscription options. Yes, there are probably cheaper options from other companies out there that are equivalent, too, but these sorts of services are good because they give you your "off site" backup. Personally, I like Mobile Me because it has lots of services I find useful, including easy ways to distribute photos on your Mobile Me web site (if you choose). It's good for a busy guy like me who doesn't have time to fiddle with computers after work--it's easy for the family to use, too!
http://en.wikipedia.org/wiki/IDisk
I am a semipro photographer. One raw picture is >20M, and I tend to take between 500 and 2000 pictures for an event.
I keep all pictures. All of them. With the usual exceptions of the black ones or a very blurry ceiling.
My computer is also a laptop. I removed the useless DVD drive to host a second hard drive, only for the pictures. That gives me 750gb for pics.
I also have a 2TB external hard drive, and a general backup 4TB drive.
The workflow I use is as follows:
- I put all my pictures on my computer.
- Once transferred, I plug and copy all the new pictures on my 2TB, never removing anything from there, only adding.
- I then process the pictures, adjust them, do whatever needs to be done. I sort them in 3 buckets (deleted, meh, good).
- I copy the working copies for the good ones to the 2TB also.
- I delete the deleted/meh from my laptop, only keeping the good ones.
- I do a general incremental rsync backup of my laptop to my 4TB.
For me that's enough protection, I always have my "good" pictures with me on my laptop, and have access to everything else on my dump drive.
For fires and burglars, I also have a second encrypted 2TB at work. I can safely recreate everything else from that part...
So far it has served me well, and I haven't lost anything. I've been burned badly in the past after crashing a HD while doing a backup, and having 6 HDD failing me in the same year (yeah, lan partys will do that to your gear) so I am very anal about my data.
Save photos to a RAID-enabled NAS, and for offsite backups use some form of online storage.
I'm a SmugMug user and SmugMug provides EXCELLENT value - you can even store RAW images with SmugVault if you have a Premium account (If you don't shoot in RAW, Standard will be fine for you most likely.) If you don't shoot RAW, Flickr might be a good alternative.
As to, "And yes — before you ask — I do know that the first thing to do is to go through your collection and dump what is not worthwhile keeping."
NO. The definition of "worthwhile" can change. I do exposure/focus weeding initially (An initial run in digiKam on import for the most obvious, then a second weeding during RAW conversion in ufraw), but after that I never delete anything.
retrorocket.o not found, launch anyway?
Long term idea: use FUSE to create a directory structure where every directory has error correction run on the contents. From the user view, just a normal directory From the backup view, backup will back up date and ECC files. In that way the backup can handle limited corruption. Also this will help detect/correct another problem I have: Over the years, I find that some files get corrupted and I don't notice it. Trouble is, all the backups eventually get overwritten. And when you literally have tens of thousands of pictures, its impossible to go through them all on a regular basis. So what I hope to do with ECC over FUSE is to run periodic ECC validation scripts to detect and correct these problems.
PS: does anyone know of a utility that will go through and validate JPG or ELIF files? Or any video files?
Might not be good for 250GB at once, but after a trip, I burn the photos to a set of DVDs. One set to my work DVD case, one set kept at home.
Check out The DAM Book and dpBestflow.
What I do. First, I use Lightroom to manage everything which lets me categorize and rate.
I keep one hard drive at home on my desktop. I keep another at my office attached to that desktop. I use unison to keep the two in sync. So I have offsite backup.
On my laptop, I keep two sets of photos. First is the recent stuff I've taken that I'm "working" on. The second set is only stuff I rank above a certain level and processed by lightroom into JPGs of reduced size (still bigger than the laptop screen). So I've got my whole collection that I can show to people at any time, but not the original full resolution RAW files. The laptop is kept up to date with the originals also with unison.
I upgrade the size of the offsite and onsite hard drives as needed. They are currently 1 TB each. And I have no need to take an external drive with me anywhere.
Be careful with fire safes. They are generally designed and rated for paper, not electronic media, and will get too hot for electronics to survive. Be sure the safe you get is rated for electronic media. Also such electronic media rated safes I've seen are really designed for disaster not security, a claw hammer can probably open them. If you are just storing your family photos this is probably a plus.
I use Gallery, an open source PHP one, because I've not been motivated to find one that fits my needs more perfectly. It's not terrible, though finding specific pictures is hard. If I had infinite time, money, and motivation to spend my time on maintaining pictures (which, arguably, is a better expenditure of my time than a lot of my hobbies), these are the features I would want:
I'm a what is called a serious amateur photographer. I probably take around 25,000 pics a year and keep maybe 3% of them. I always have two copies of everything I value on separate media--at the moment, DVDs and an external hard drive, but I expect that to change in the future as storage technology evolves. In addition, all of my best work is stored as high quality JPGs on an online photo site. I consider this a minimal scheme. If I were a pro, I would have additional off-site copies of everything, but if my house burns down, my pics will be the least of my worries. How much you invest (time and money) should be a function of how important the photos are to you. Another thing to consider is how you will find photos years from now, especially if you accumulate thousands of them. Besides a solid backup scheme you should consider investing (time and maybe money) in some catalog software that will help you locate photos by keyword, location, date, whatever.
...and it wants its cliche back.
All photos are uploaded to the server into my Gallery app. The gallery is rsync'd to an external USB HDD 2x per day. I have an external USB HDD locked away in my safe deposit box that I update every quarter or so.
For primary storage, I keep mine on a NAS device that currently has four drives split into two RAID0 pairs. When space on the NAS gets low I install two new, larger drives, copy the data across and remove the now defunct pair. Given that you have a laptop you'd probably want to substitute a portable DAS system instead of my NAS for convenience's sake, and only keep the latest and favourite images on the laptop's internal HDD.
I then do incremental backups (you don't really need any other method for media collections) to pairs of USB drives, one of which is always off-site. Whenever I add new data I backup to the on-site drive, then take it to my sister's, swap it for the off-site drive, come home and re-run the backup. That means I have a quick way of access/recovery for local file loss and also a means of recovery from more disasterous scenarios.
UNIX? They're not even circumcised! Savages!
Get a few of your friends in different cities to all agree to be one anothers' hot-backups. Each of you buys a NAS box. You backup your files to the NAS boxes of two of your friends in different cities. Your site holds the backups of two friends... Hell, encrypt them in case someone's house gets burgled... Personally, I have a friend with dedicated bandwidth supplied by his employer. I store a small Mac Mini over there. I also have a small vhost on the other side of the continent that I pay $150/year for which is also my mail server and DNS and backup for the Mac Mini.
External harddrive. Second harddrive in different location and/or a flickr account so you don't have it only at home.
Now, seriously? I have photography as a hobby too, and in the very best case about one in ten shots are actually worth saving, and that proportion drops the more shots you take. Most pictures end up being crap - that's not an indictment, just a fact. Most shots Adams or Bresson took ended up as crap too. You have perhaps two hundred pictures, tops, that you or anybody else would care about.
Chances are you're never ever going to look at most of those thousands of shots ever again, and your kids will simply throw it all away unseen in the far future when they're cleaning out your belongings. Problem is, if you haven't edited the collection and thrown away all the thousands of duds, they will end up throwing away those good, important images along with the rest.
So, best approach: go through and delete all the crap, all the duplicates, all the technical duds. Then delete all the ho-hum images. Aim for, say, two hundred images to save, or better, fifty. No matter how eventful a vacation, you don't have more than fifty great places or events to record. And the more pictures you keep, the more you dilute the impact of the good images.
Trust the Computer. The Computer is your friend.
The RAWs from my camera are already 25-35MB each. A quality-9 jpeg post-production final is usually in the 7-10MB range. I archive both so I'm sitting on about ~40MB per picture in archive storage. Needless to say it adds up pretty quickly.
I keep the archive on a 2TB drive which is backed up as part of my normal backup, meaning it gets backed up to another machine on the LAN and from there to a colocated off-site box. So I have three physically separated copies. I also usually wind up uploading the jpegs to my paid account on flickr for distribution and for 'just in case'. And even though it isn't the RAWs it's still what is most important to me... a viewable picture.
I would caution against having a single copy with no backup, even if it is on a RAIDed system. That's a recipe for complete data loss.
On my laptop I have around a ~120G SSD to spool-off RAWs taken from the camera's 16G card when I'm on the road. I try to do that every day just in case the card fails (which has happened to me), or in case the camera or laptop gets stolen. I only erase the cards at the last possible moment in order to retain local backups as long as possible.
I usually don't try to do post-production on the laptop. Post-production with lightroom is fairly straight-forward though if you spend some time taking the original picture properly Canon's free software works just fine too. Frankly it takes actually printing the pictures out to get all the parameters correct, even good screens don't really match printed output. I don't like to 'play' with my pictures like other people, I like them to be as original as possible so most of lightroom's features go unused.
-Matt
For my personal photos I have a RAID-1 NAS setup. I wasn't going for speed, so two lowspeed 2-TB drives is a cheap option. Since RAID isn't a backup. I also make two DVD (and more recently BD-R) backups of the photos as well, keeping one copy in my offsite vault (my Mom's house).
Given the main idea of this article I am interrested to know is there a standard deffinition or classification of levels of backups?
If not, I propose one.... something to this effect.
Level 0 - No backup, file is stored locally on single machine on single hard drive
Level 1 - File is stored on single machine with RAID
Level 2 - File is stored in original location as well as on single backup drive or thumbdrive (not offsite)
Level 3 - File is stored in its original location, a second storage device such as portable Hard drive, and then a third copy off site
No backup solution can be called a true backup if it does not include an offsite backup and the best offsite backup is one some distance away from where you store the originals. If you leave a copy of the files at your nighbors house and your house burns down then you're okay, but if a tornado blows both your hosues down then your screwed. Get a safe deposite box at the bank or something... Right now I have the original files on my PC, a second copy on an external drive I keep in a fireproof lockbox I update once a month or so, and then a third copy on another external drive I carry from work to home with me from time to time. There is always a possibility that one day when I bring my drive home from work and have all the drives in the same location disaster could strike.
You really need to ask yourself, "how much is my data worth."
I'm in the same predicament. I shoot video and still images. Video will always be a beast to store, and I ended up buying external drives and storing them there. More recently I've been using laptop drives in externals (mostly Seagate 5400s in Rosewill cases). You can get pre-made externals, but the internal drives are not swappable.
Anyhoo, you asked about still photography...
I shoot in RAW generally, and have a couple Canons and miscellaneous Point and Shoots, plus cellphone snapshots... Every month is about 20G more of data. I shoot a couple times a month on the DSLR and fill up a 4G CF card. Many cameras support an small-RAW mode or similar compressed mode (it's still RAW, but smaller file size).
For the initial storage I have a fileserver with a single 1TB disk. After I load them into the PC, I immediately copy them to the staging area on the 1TB disk. I then prune the junk and crappy images of my thumb and shoe and other stuff that's definitely not worth keeping. This usually shaves off 1G of the 4G. Then I sync the folders. I use rsync with a --delete option under Linux, and Windows and MacOS have similar tools. This gives me two copies.
Next is tagging and making the contact sheet... Tagging can be as simple as a README in the folder or as complex as using an image archiving program. fSpot or similar can do the tagging and contact sheeting. This is important if you have lots of images.. Some of the pro software will even manage your EXIF and geotags.
Once this is done, I copy the really good images to a mirrored storage drive. I have 500G on this, but it's old. This storage grows maybe 5G or so each month.
Then I delete from my laptop.
What you do next depends on how attached you are to your pictures.
You can backup your images to an external drive. At 20G a month, you will fill a 320G drive in about a year. That's about $90. You may want two though, one for onsite, one for offsite that you rotate away.
You can also store in the mystic cloud. Google charges something like $20/20G/year. So that's about $100/year (a little cheaper than an external drive but a pain to sync).
There are other options, but this is what works for me.
I use TWO Vosonic 320Gb USB devices. Look them up on Ebay. They have card slots and LCD Screen so they can copy your camera media out in the field.
Back home, I do my editing on a MacBook Pro and store the photos (in RAW + JPEG) on a QNAP 4 drive NAS configured in a Raid 1+0 setup. Another copy is held on a couple of 1TB USB Drives.
I shoot with a Nikon D700 or D2x and now a D3s. My last long trip (23days) resulted in 11,576 shots. At an average of 14Mb per shot (RAW + JPEG) it is easy to see that a lot of storage is going to be used on such a trip.
I'm a Professional Travel Photographer.
From D-Link, http://www.dlink.com/products/?pid=666:
The availability of four different hard drive modes (Standard, JBOD, RAID 0, and RAID 1) allows you to choose the configuration best suited for your needs. Standard mode creates two separately accessible hard drives. JBOD combines both hard drives into one for maximum space efficiency. RAID 0 combines all drives in a ‘striped’ configuration, splitting data evenly across the hard disk drives to provide the highest performance, while RAID 1 causes the drives to mirror each other, providing maximum protection. If one drive fails while configured as RAID 1, the unaffected drive will continue to function as a single drive until the failed drive is replaced.
I have a few of these in my AV setup. One is Raid 1, for stuff that I can't
replace, pictures, home movies, important docs, etc. The others are JBOD.
Accessible via UPnP for your AV setup
Not expensive for what it does...
Google Products listing: http://goo.gl/4KT6c
Last I checked (a year ago) it would accept up to
2x 1.5TB drives. And I started with two spare 250's,
when I got my first one, and JBOD'd them into a
half TB. Which was really nice.
[ If anyone else is considering it... if you have
spare SATA's laying around, this thing is GREAT! ]
Oh yeah, fast too on the ethernet, giga ethernet.
-AI
For me, it is far better to grasp the Universe as it really is than to persist in delusion
About $200 at Costco will get you a nice 2TB ioSafe USB Drive. Throw in some freeware backup software and you've got two of the three issues covered for disaster. Secure it somehow and the third of theft is covered with a nice tidy package. http://www.iosafe.com/products-solo-overview
I'm a big fan of at-home network storage- similar to what other posters have written about.
It sounds like 2TB should be fine for you, and you can get an external 2TB hard drive to backup to and put in a fireproof box.
If you set up a VPN router, you could access anything at home or away.
It sounds like you wanted a more direct notebook - based solution.
I haven't used them myself, but it sounds like 1TB notebook internal drives are available, e.g.:
http://www.engadget.com/2009/07/27/wd-ships-industrys-first-2-5-inch-1tb-hard-drive/
and soon, 2 TB should be available.
Even if you have some limitations on your boot disk, you should be able to run 500GB on your primary internal drive, and many notebooks will support a second hard drive. You might be happy with this approach - you can carry all your photos on your notebook, keep a backup on another drive on your notebook, and leave a 2TB at home for backup purposes should anything bad happen to your system or it gets stolen.
Also, you might want to do a post later to let us all know what decision you ultimately come to and how you like it.
Good luck-
Sam
External firewire drive duh. If you are that much into digital photography I assume you use picture management software, possibly? Many don't support pictures on a NAS, or are simply too slow when dealing with large amounts of huge RAW files. As for backup, you can keep pictures on the NAS, it's just if you prefer to edit and manage your photos then you will need to use an external drive.
I have about 50 GB (yes GB) of photos. My solution isn't elegant, but it has saved my ass. 1) DL pictures to my primary HD (I do not delete from camera) 2) When I get a chance I backup to a second internal HD 3) Then after that I backup the secondary HD to an external HD 4) Then I will wipe the memory card in the camera A few times a year I burn the whole collection off to DVD's and store them away. I would like to have a permanent off-site backup at some point. I imagine it will be another HD in my work PC and dump stuff there.
Fear Is the Only God
Mod me redundant as this has been said already, but this is my personal strategy:
1. Cull photos. If you do serial shots take time before the 1st backup to delete the duplicates. Also, be strict to remove all shots that are not perfect technically, unless their composition or motive is really outstanding.
2. Tag photos. No point in having an archive of 20000 pictures if you can't find them.
3. Back up. As said elsewhere, external disks are cheap. Make sure you have multiple backup disks, and store them physically separate.
4. Use online services. A Flickr pro account is $25/year and allows you unlimited uploads. Backup everything to Flickr as private pictures in the original resolution, then share what you want via guest passes. It may take hours or days to upload a batch of photos, just let the computer do it overnight.
Sorry I couldn't resist ^_^
Dude! Cmon! Thousands of photos? How many of them are good? Do you think you will want to go though all of them. This is what I hate about digital cameras, people just take milions of shitty pictures never bothering to sort them out and keep just the "good" ones, I think if you take a hard look at your collection you will not even need a new drive, and if you still do, external 1TB drives are soooo cheap, so buy two of them and keep them mirrored and there is your backup. Online storage is also very cheap...
ReadyNAS Duo with 2x2TB drives, stores everything for backup, photos, videos, music, time machine. Love it.
Run a (Linux Apache MySQL PHP) Server with the appropriate disks (RAID1 Minimum + LVM) and install Menalto Gallery.
I had good experience with it.
To upload files to it, there's various methods available (http://codex.gallery2.org/Other_Clients) but the easiest for those on windows would be the Publishing Wizard. You need to merge your gallery's registry file into windows and voila!
I like Gallery because of of its simplistic album approach and photos can be resized automatically either thumbnails or mini-size for previews and still have full-size available for download. There's automatic watermark too.
No matter what, you want at least 2 copies (although 3 is what I'm doing). My suggestion I use:
;-)
1) If you don't have enough disk space, and you cannot upgrade the internal drive, get a USB IDE drive to put these files on. 2) Find a friend/family member who will let you rsync to and come to some agreement where you give them a hard drive you'll use as a place to backup to. On the drive you give them, rsync your files to it first so you have a copy so the first rsync doesn't take a ton of time and bandwidth. From their, you can setup a scheduled event tailored to whatever is most reasonable to both of you (i.e. throttle the bandwidth to make it less of a strain on the connection; stop and resume the rsync during specific hours/days, etc) If you are concerned with them accessing your files, install truecrypt, make a virtual encypted disk, put your files in it, and call it a day. None of this requires much in the way of advanced computer knowledge and could be able to be worked out in a day of light pursuit and some luck finding someone friendly to answer any questions
3) Find another friend/family member, make them another copy of your files to keep. This can be in whatever is the most cost/time effective method you are comfortable with. I just buy any plain SATA drive that matched my needs and go with that, generally I go for something at least 2 or 3 times the size of my data at the time, to allow for some growth. Then, swing by there house once or twice a year to do an update, or maybe after any significant picture shooting extravaganza/vacation.
The key is not having your eggs all in the same basket. You'll always have a complete and current copy at home, you'll have a second copy that is a week to a month behind being rsync'd, and you'll have a third copy getting updated once or twice a year on a drive sitting in someones drawer. Works great for me, and is pretty cheap for the redundancy.
I do something similar to this. Redundancy is the key.
Disk space is cheap & online storage for photos is cheap. I never delete a single photo.
1) 1 copy on my own PC. I may have gigs and gigs of photos, but I also have gigs and gigs of free space on my local system.
2) 1 local copy on a NAS that automatically syncs over every night.
3) 1 online copy using mozy - its like $5 / month. Again - the syncing happens automatically within hours of the original photos being placed on my home PC.
4) 1 copy on DVD in a safety deposit box. This is the only non-automated portion of my backup routine. Every couple of months I backup up all my photos, bring them to my bank where I have a safety deposit box and swap out the old copies. I was paying for a box anyways (although I needed to upgrade to a slightly larger one to hold the DVDs).
Ultimately, photos rank up there among the most important things in my life. I would be devastated to loose some of them. Redundancy is your friend. I may spend a few dollars a month doing this, but I am pretty sure that unless my house burns down on the same day that there is a worldwide financial crisis causing mozy and my bank to shut their doors, I should be ok.
Word game?
The fallacy of digital photography is that a price of a single image is essentially 0. This caused an exponential explosion of digital photos, most of which have neither artistic nor informational merits. In fact, I dare say that 99% of photos taken currently are not even viewed by their own authors except perhaps for a brief preview on camera LCD and, may be, once more during transfer. Chances of viewing most of these photos later on go down from there.
The best way to store these photos, if you do decide you need to keep them, is to sort them and delete 9 out of 10 (or 99 out of a 100). That'll answer a question of where to store them pretty well. You and others are also more likely to have some interest in looking at these, presumably better photos, because they are not buried in a heap of bit garbage.
They are all so cheap, you might as well get 2 of them just in case one goes belly up with your data (which seems a rather unlikely event in any case).
When it fills up just get another one. Though I doubt the grandkids will have a USB port to hook it up.
For justice, we must go to Don Corleone
... Keep it accessable & live, and migrate it each time you upgrade your system. Sure, I've got a few 5.25" floppies around, but how to read them? Keep it spinning & live.
I don't think you have to keep the old hardware spinning for more than a generation or two. I have a DVD with my backups (src and doc, not OS or apps which I could just reinstall). The current DVD includes older CD based backups. The CD backups include older QIC-80 tape based backups. The QIC-80 tapes include older diskette based backups. Next year I expect it will all be on a blue ray.
I use Apple's iLife suite — specifically iPhoto — to manage external FireWire drives. I know it sounds like I'm just another Apple fanboy, but it really is a great photo management suite.
"Adventure? Excitement? A Jedi craves not these things."
Of course not. For my 2TB NAS, I rotate back ups to a bunch of 1TB hot swappable drives in a USB enclosure formatted XFS. A rule of thumb for personal use is "plan to spend as much on backups as you did on your computer (over the lifetime of the computer)."
Which is better than the standard at work, which is, "plan to spend as much on a years' worth of backups as you did on the computer." For some of the larger disk arrays it's more like "plan to spend as much on a single full backup as you did on the computer." And that's for internal backup. Off site commercial backup is four to ten times as expensive as that.
Support SETI@home
you heard me.
I did a setup where I backed up the photos from my computer to an external hard drive. So I had two copies of my photos.
I just did a simple file copy using xcopy for any files on the source that were different from the backup. Mistake.
My desktop crashed once and some of the photos got corrupt on my desktop. I didn't notice this and months later I found them corrupted on my desktop. No problem I thought I'll pull my backup out.
Xcopy had copied the corrupt files over the backup ones because they looked different. :(
I wrote my own backup script in Perl that would only backup new files and report an error of all files that had the same filename on my desktop and usb drive that were different (flagging corruption). This has saved me from the downside of hard disks getting corrupted and losing a backup.
Also recordable DVD and CD have a shelf-life of around 3-5 years before errors get on them. I lost some photos this way too. So if you backup to DVD-R you have to reburn all the backups every few years and this can take a lot of disks. Blu-rays that you burn are suppose to last longer but can still be affected by this.
Your best option is to go with an online provider that you pay monthly. For a fee you can send them a hard drive of the initial 200GB+ of data and then upload via the Internet after that. There are many that provide multiple data centers around the world to secure your data. Note however that many have a 30 day or less limit of history of files you can view that you deleted. So if you delete a photo on your computer the online backup will eventually be removed too.
Another option to go cheaper would be to use something like CrashPlan which lets you do free backups to a friend's computer (or pay for their online storage or do both). Just make sure the friend is at least 1 state away to minimize fire/flood/etc damage in a single city. You can even do an initial "seed" of the GB of data to a usb hard disk and send it to the friend so it doesn't take so long for the first backup. I use this method now for 30GB of data.
1) Print them all out.
2) Post them all to your Face Book page.
Really? You don't know how to back up your data?
USB Drives, Flash Drives, DVD's, BluRay's, On-line backup.
Visit the Arcade Restoration Workshop @ http://www.arcaderestoration.com
My house burned down. You could barely recognize my computer hardware afterwards. Years of digital photos gone forever. They were all backed up to other drives that also burned. I now backup all my photos online, but plan to purchase a fireproof hard drive.
Unless you are working a lot with audio, video, or hi-res scientific measurements, or you actually pay for the amount of traffic you use instead of flatrate, online backup combined with local backup is the safest.
I recommend Duplicity. It is free software, it's reliable, it encrypts everything by default, it supports a number of communication protocols. It's of course incremental, so even if it will take a long time for the first transfer, later synchronisations should be snappier. Put a cheap file server in a friends house, pay for space in some web hotel, or use one of those fancy storage systems at Tahoe or Amazon.
From the parent "These are obviously images that I want to keep for my life. So the need to survive fires, burglaries, etc. I think the amount of data I have rules online storage out." These two things are mutually exclusive. To avoid fires and burglaries the *only* practical answer is offsite (i.e. online) storage. So, if you won't use online storage, and you can't take the miniscule risk of being burgled, you're hosed.
I looked into godaddy unlimited and they said that 10TB is beyond what they deem unlimited. I don't know what dictionary they are using but to unlimited means just that. anyhow, for something like 6 - 8 bucks a month you can get your own website with at least 2TB (godaddy's definition of unlimited) and store them there. That way they are in the cloud but under your control. You can encrypt them if you're nervous.
a) buy a bigger HD for your laptop, so you can get them off your camera.
b) buy a NAS for your home. Doesn't even have to be an expensive one. Something with Mirrored 2Tb drives is an awful lot of photos. Quite a few of these you can get online, so if you're away from home you can trickle back backups if you're feeling paranoid.
c) off-site backup - OK, your house is unlikely to burn down, but just to be safe and all that. Not that expensive. No offence, but 90% of the photos you could lose, so if you want to keep to a budget, just backup the ones you really couldn't live without.
Actually, storage fails very rarely and most online backup services trade (and charge) on the ability to immediately restore. How about just burning to a decent disk and mailing to a family member (ask them nicely to put them in a pile somewhere and return them should a meteorite hit your house).
A new virtual drive with online backup as well. There is not pricing model listed yet, but it looks neat.
I would love a cloud photo sharing service that is easy to use for family photos. Something like facebook photos but not in facebook. Something that allows albums, tagging, comments,access control, etc.. Any suggestions?
It seems that most people here have assumed that it should be digital storage. For example, flash drives or USB hard drives that can be stored in a safe place. These are certainly a good idea, and you should do this. However, there is one thing you should also do: Print photos. Nothing is quite as durable as photographic paper - especially when you treat it right, by which I mean keeping photos in somewhere away from moisture and heat variation.
Keeping images in paper form does require a device to view them. Devices and interfaces change, so keeping things in the digital space means moving when interfaces change. Also formats change, such as on disk formats. However, paper requires no maintenance; you can store it more or less indefinitely and you will never have any trouble "reading" the data. Sure paper does not have the same information density, so you might have to be more choosy about which ones to print, but this will only increase the quality of your collection.
How about an external Hard Drive Enclosure and a couple of 1 TB drives! All you have to do is swap them out after the backup is complete and keep one in a safety deposit box or GrandMa's house, or where ever you feel safe storing it, the other stays in the enclosure and does regular backups till time to swap them around again! The interval when you do this is up to you. Cheaper than a NAS or two dedicated external drives and only take a couple of minutes with a screw driver to swap the drives around!
I've been organizing my digital photos for almost 10 years now and have used a few different approaches. My primary computer is a laptop, which is no match for the hundreds of gigs worth of photos that I've taken over the years. I pretty much only have room on my laptop for photos from the past 1-2 years. I also have a home PC which has a 1 TB internal drive (which I added for less than $100). That PC is hooked up to my home network and has all my photos from 2001 onward. I usually take photos when I'm on the road, which is also when I have my laptop, so they always go there first. Here's my approach:
* Occasionally dump photos from camera to laptop when my memory card fills up
* Keep everything from the past year or so organized on my laptop
* When I'm home and connected to my network I use a free program called SyncBack (http://www.2brightsparks.com/downloads.html) to synchronize what's on my laptop to the PC. SyncBack is great and is pretty powerful for a free program.
* Every so often I back up the photos on the home PC to a DVD
So basically, memory card overflows to laptop, laptop to home PC, and then that to DVD for archiving. Although I'm not sure I trust the DVDs for real long term storage, so I may make new ones some day.
For online storage & sharing albums with friends I use Phanfare (http://www.phanfare.com). I upgraded their unlimited storage which I think is maybe $80 or $90 a year. Their slide shows are great, because you can upload songs/mp3s directly from your home PC / iTunes to the slideshows.
It all works pretty well for me.
I gotta have more cowbell.
I use SpiderOak
https://spideroak.com/
Installs on all major OSs, reasonably priced. Not amazing, but good enough and the $20 a month I'm paying for 200GB is not an issue compared to losing all the pics of the kids growing up.
.: Max Romantschuk
I was in the same situation you're in now. I was dealing with more than just photos, but since it's all just data, that doesn't really matter. I bought two USB hard drives and used them as offsite backups. I just took mine to work and left them in a desk drawer, but you can scale up/down from there depending on how much the data is worth to you. Other possible options might be a safety deposit box at your bank, or a relative's house in another state. Like I said, it all depends on how much not losing the data is worth to you, and how paranoid you are.
Remember that if your primary copy is not going to reside on your computer, then you need three USB drives (master at home, and two backups). ALWAYS keep a copy of the data offsite. You don't want to end up in the situation where you bring your backup drive home to sync it up and that happens to be the day your house burns down. If you're really paranoid, you might want to look at storing your backup copies at separate offsite locations.
There are various utilities that you can use to periodically sync the backup copies to the master, but I found that SyncToy works pretty well. It's a MS PowerToy and is free, although it is Windows only.
I replace all of my USB drives every 2 or 3 years. My rationale behind this is that all storage eventually dies, so by replacing the disks periodically I feel like I'm lowering my chances of a failed disk. If you do it frequently enough, you can sell the replaced drives on eBay while they still have some value and offset some of the associated cost. Alternatively, you could try using an online backup service, but relying on someone else to store my data makes me really nervous. Especially so when you consider that if they happen to mysteriously lose your wedding photos, you really don't have any recourse other than cancelling the service. I don't know about you, but for me it's just not worth the risk.
When uploading to photo sharing sites - beware!
I just finished moving my photo collection OUT of the cloud and I have to say, getting my 33,000+ photos BACK from Flickr (which is relatively open, as cloud photo services go) was not an easy task.
Cloud photo storage is plagued by compression and data loss (picasa), by warrantless unrecoverable deletion (Flickr - of a paid account! and obviously - Facebook) and other reliability/survivability problems.
Personally, as an avid photographer, I can't sleep soundly unless my photos are backed up in at least three places, one of the offsite. I accomplish this using a local mirrored drive, and the great cloud backup service - crashplan.
A mirrored drive would be tricky in your case, but you could use a USB hard drive connected to a family member/friends always-on computer. Back up to that using either the crashplan client (which is free for such uses, and works great) or rsync, syncback or any other homebrew solution. Pair that with a cloud backup service, and you should be fine.
Most importantly, never relay on the cloud as your single backup strategy - the internet is full of horror stories of people who THOUGHT they had everything backed up in the cloud... a USB drive sitting at a friends place is much easier to verify.
https://iosafe.com/
http://addonics.com/
http://www.caloptic.com/store/index.php?main_page=index&cPath=4 (Bridge Boards)
http://www.zfsbuild.com/
http://www.nexenta.org/
http://sourceforge.net/projects/freenas/
http://www.freebsd.org/
http://www.illumos.org/
http://openindiana.org/
For backup: if you want effective offsite backup, you need two backup drives (in addition to the presumably external drive you get to hold the primary copy of the data). One backup drive is onsite, and gets updated frequently (like every time you add new photos to it), and the other is offsite, and is updated only when you get a chance to go to the offsite location (possibly by simply swapping it with the onsite backup drive). If you have an offsite location that you can visit very frequently (like a workplace), you might be able to get by with just one backup drive, but that may take more effort, and requires that the primary copy of the data be in the same location as the only backup on a regular basis. The other thing that's important is a good organizational system for your photos. I currently use Aperture, which isn't bad if you have a Mac; Lightroom is supposed to be pretty good as well. I'm sure that there are consumer grade options that are also decent -- just make sure that they can handle the size of library you need (i.e. check the reviews). Make sure that it stores the images in some transparent format (i.e. an ordinary directory structure) or incremental backups will be impossible -- in some cases (like Aperture) you just have to set the options correctly to do this.
.. to asian ladyboys. Now I have thousands of backups all over the place.
Only save the photos that you will actually take the time to see in a few years time. Delete the rest.
The only time I actually delete a photo is if it's completely and totally worthless (like a completely blurry picture of my foot, or something, from when I'm walking across a field.)
Like I said, you never know what will have value to you in 50 years - perhaps you are then into abstracts, or that ends up being the only photo of a shoe that in retrospect you treasured above all others!
Delete nothing!
"There is more worth loving than we have strength to love." - Brian Jay Stanley
This is not a joke. Online sevices charge a minimal amount to get a 4x6 photo printed on photopaper that has been printed with light and then developed (ie a real photo, instead of an inkjet print that will quickly fade if you haven't dropped large money on archival ink and paper). Stick these photos in a closet. In 50 years it won't matter if JPG is still around, let alone if anything can read RAW from your now antique dSLR. They can still burn, but flood damage just gives 'em that old fashioned crinkle. You don't have to worry about losing them in a transfer gone wrong, formats, or even electricity. Your photos of the Myans will still be viewable after the zombie apocolpyse.
For the bits just get a cheap USB hard drive and store it somewhere like many other have said, but for photos, nothing has the track record of developed photo paper.
Print them and put them in an old-fashioned album. That way the will only degrade in an analogous way, where you can at least see something even if a bit discolored. Albums don't crash and the photos don't become illegible. The only real risk is fire or flood.
Maybe they didn't take a ton of photos for the memories, but because they are hobby photographers? Having new and different environments to take pictures of is one of my favorite aspects of travel. What makes a trip enjoyable for you is not necessarily the same for everyone.
Isn't enough that I ruined a pony, making a gift for you?
The beauty of digital photography is that you can fit a life's worth of digital photo's on a single hard drive that somebody else can toss in the garbage without looking at it. Much better than those bulky albums people used to have.
Actually the chance of getting your electronic things confiscated exists when entering the glorious nation of United States of America as well. Yeah, who's the Free Nation(TM) now?
What time is it/will be over there? Check with my iPhone app!
We have a rather expensive drive that writes on special DVD media. It does not use dye - it literally punches holes in the media with a laser. This means the discs are not subject to degradation of the dye layer or thermal stress like ordinary write-once optical media.
Estimated life is 1000 years.
Check out the web page attached to this posting. Or look at the web site for Cranberry as they are using the same technology.
I have thought long and hard about this problem. The result - a project I call "BOAR". Big Object ARchive... or something like that. Quite seriously, I think there is a huge gap in data security between backups programs and version control, and BOAR is my attempt to fill that gap. People keep saying that they are using ZFS or Dropbox or whatever and seem to be quite content, but I think they are missing the point. You also need a container for your precious information, that allows you to verify and ensure that your data is consistent and complete. BOAR aims to be that container. I quote from the project front page:
"BOAR aims to be the perfect way to make sure your most important digital information, like pictures, movies and documents, are stored safely.
* BOAR prevents data loss due to human or machine error
* BOAR makes it possible for you to restore any or all of your files from any point in time.
* BOAR makes it easy to maintain verified backups of your data, including file history.
* BOAR will make it much more likely for your digital heirlooms to reach your grandchildren some day.
If you are familiar with vcs software such as Subversion, you might think of boar as "version control for large binary files". But keep reading, because there is more to it."
Please check it out at google code: http://code.google.com/p/boar/
I keep the master copy in iPhoto, with the library stored on a high-quality LaCie Starck drive. A backup copy is made automatically by Time Machine (Mac OS built-in backup) onto a Drobo, which automatically makes 2 copies to guard against disk failure. An offsite backup is made automatically by BackBlaze, which is $5 per month.
So I only have to work with the photos themselves in iPhoto, I don't even have to touch the files, and there are always 3 backup copies made automatically by the Mac. I can even step back to previous versions with Time Machine. Very convenient.
Buy yourself a Thermaltake BlacX external hard drive dock for $44. Now you can get a 2 TB hard drive for $80 (with rebate) and store all your data for 4 cents per gigabyte.
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
This is a bit off-topic in that it doesn't relate to storing photos, but it does relate to merging collections of photos from the same event (several different cameras from a family vacation, wedding, etc). When you have multiple cameras, you typically run into the issue of different time zones, blatantly incorrect timestamps, and differences of several minutes which might make for chronologically sorting the entire collection a bit awkward.
If possible, plan. Have everybody take the same photo at least once, ideally of a clock that includes the seconds. This gives you a reference. Now you need only determine the difference between the timestamps and then adjust them. I wrote a shell script called timefix to do that for a vacation a few years ago. It can be aided by my timecalc script so as to take inputs like 15:43 instead of just seconds. Use find and/or xargs if you have too many inputs. After this, you can view all photos by timestamp regardless of which camera was used.
Use my userscript to add story images to Slashdot. There's no going back.
Upload them to Facebook (but don't bother encrypting them http://yro.slashdot.org/story/11/01/20/2028252/Facebook-Images-To-Get-Expiration-Date). They'll never disappear from the cloud (even if you want them to).
I'm a fairly serious amateur photographer (took well over 10,000 photos last year all around the globe)... and here's what I do.
1. Use a Mac with Time Machine and an external HD (if that's not an option I'm sure you can find a backup solution for windows that is similar). With Time Machine I always have at least one backup of every photo that hits my HD. Further, this also backs up my Lightroom database files, which if you are shooting in RAW are almost just as important!
2. Backup offsite. This is another external HD that I bring home periodically and plug in and do a manual copy to it. You can keep it at work, at a friend's house, at a family member's house... wherever.
3. Backup to the web. For me this is Flickr. I pay the yearly fee (don't remember what it is) and it gives you unlimited uploads and storage of the full size images. All of my finished and worked up photos go here. It's not a complete and perfect backup (like using a web backup service would give me because I can't upload the RAW files to Flickr).... but it gives a huge leg up on that "My house just burned down therefor I lost every photo" scenario.
Just my $0.02
just putting them on view-master reels? Just as they say, "View-Master® 3D products open up an exciting world of fun and adventure. You'll feel like you're there!" :p
If they are worth looking at, I like to look at them.
Another nice thing to do is make postcards and send them to other people.
As much as I like geeking out... I tend to view my home solutions as needing to be utilitarian and cheap. I simply have a WD 1TB external HDD and use it in conjunction with the $10/mo. cloud back-up created by Elephant Drive. It's one of the few cloud products that allows external HDD back-up. This allows me to dump all the photos on the WD HDD and FORGET about it. Elephant Drive does an excellent job and allows you to create jobs. All this keeps the photos off whatever machine I'm using and I don't worry about theft, fire or flood because it's riding around in the sky somewhere under Zues' Thundering Asshole. I don't have to worry about RAIDed internal HDDs (doesn't solve physical damage), setting up a VPN and rsync (not always reliable from) to some remote machine at work... and I don't have to sit around all weekend long trying to support whatever solution I've come up with. PS. I'm not paid to endorse Elephant Drive. They just kick the shit out of Carbonite and for that I love them.
What's this crap doing on /.?
I find you annoying.
Whats your issue with taking pictures, perhaps he likes to do that.. then you go off on some tangent on something you fucking dreamed up, probably while stoned. WTF do "subtitle sunglasses" have to do with the best way to store photos?
Fuck off.
And yes — before you ask — I do know that the first thing to do is to go through your collection and dump what is not worthwhile keeping."
The guy who runs www.lightroomkillertips.com wisely stated that going through your collection is easier to do a month or 3 after dumping the photos to computer. He said something about an emotional attachment to the photos you just took, and I find this to very often be true. I've went through yesteryear photos and realized I didn't need all those angles of that same shot, just a couple.
1: Dump the photos to computer.
2: Look through, Delete what you know you don't want.
3: Make prints of what you do want.
4: Wait a few months, then go back through, and do the real Cleanup.
Nope. Not good enough.
- On the trip use a USB hard drive image tank or netbook for backup. Back up to multiple drives even while on your trip. If you have multiple drives with you, erasing and reusing cards is no big deal.
- When you get home you should have 3 copies. 1 main online accessible copy. 1 backup local copy powered off. 1 off site. Ideally you'd use 3 different kinds of media, but since the only large convenient media is hard disk buy different brands (no one wants to copy terrabytes around 1 DVD or Bluray at a time!)
- A 4th copy at a second off site location for irreplaceables like wedding photos
- Monitor hard disk temps via SMART. Don't leave this step out. I learn this when I found my WD Elements drives were cooking in the summer at 75 degrees celcius when doing entire drive copies. I managed to bring temps down to mid 40s by spacing the drives out a little more and using a room fan to circulate air
- If you really care about your images not changing over time you'll keep checksum files (md5sum should be fine). Too many pieces of software will alter the EXIF data on even read-only files.
- Plan to transfer onto new media every 3 to 5 years depending on usage. Do not wipe old media. 5 year old hard disks aren't worth much. Your photos are.
These posts express my own personal views, not those of my employer
You've really asked about two issues, how do I manage that many photos, and what do I do about backups?
Backups, IMO, is the easier one. What I do, get two identical hard drive enclosures, each with a suitably large drive in them (these days, look at 2Tb + each). Aim for a powered one (so you can use larger drives), preferably with eSata connectivity (for speed). Then mirror your photos and whatever else you want to back up regularly onto the drives. Then, every week or so swap drives, and take the now disconnected drive to somewhere off site (your work, parent's house, safety deposit box, etc). Doing the mirroring via an automatic scheduled task is better. Now you're covered for most risks, and if your house is on fire and you've got time to grab something on the way out, you grab the currently connected backup drive.
Now, how do you manage a large connection of photos, possibly stored across multiple machines? There's commercial solutions, with a pretty hefty price tag, but not much out there with distributed capabilities in the open source world. At least, not that I know of. For myself, I've kludged up something using f-spot as a base, and using Mercurial to track the photo database, but it's messy. And now, f-spot in Ubuntu 10.10 has become a pile of flaming crap, so I'm going to have to try the same approach in Shotwell.
SD cards are dirt cheap and the native media for cameras. 2GB $5. Fill one up, upload it's contents to a NAS then toss it into a firesafe with a label. If you don't care for the SD option, back the images up on a Bluray or DVD and toss that into a firesafe. I think technically the lifespan of an SD card is more, but in either case I'd refresh your backups every few years for peace of mind.
Two of my imaginary friends reproduced once
When my wife started doing video editing a while ago, I got her a desktop computer with a large harddrive, and I signed her up for the $5/mo Mozy plan. It's a pretty reasonable price, and we've already restored a couple videos she somehow managed to delete accidentally.
You don't need a powerful computer for photos, but it sounds like you do need more disk space. If you don't want to replace your laptop HD, you could get a large external USB HD.
Something else that I do, since I am the one that manages the photos (as opposed to the videos) is I put the photos on my computer and manage them with Picasa. I upload all the latest at a reduced resolution to Picasa Web Albums to share with family, deleting older albums when I get close to the 1GB limit (who wants a lifetime of photos to be public anyway?). And I occasionally sync all my photos over to my wife's computer, so that they get backed up to Mozy too.
1. Relentlessly delete crap. No point in keeping five versions of the same photo around. No point in keeping bad photos around (blurry, overexposed, etc). No point in keeping photos you'll never want to see again. This gets rid of at least 90% of photos right away.
2. Import images into Aperture. One project per month, albums within the project for events and subprojects.
3. During the year, periodically rsync the library to RAID5 array (that's in addition to TimeMachine backups every 10 days)
4. At the end of the year, move originals to an external FireWire 800 hard drive. This is pretty nifty, and I don't know if anything other than Aperture supports this feature - you still can see the photos, but at the lower resolution. Once you connect the external drive, the full versions automagically become available. Once images are copied off the hard drive, the external drive and the Aperture library gets rsync'd off to RAID5 again.\
5. An encrypted version of the full yearly backup sits on the shelf at a friend's house, in the form of a 2TB USB hard drive.
6. When viewing old pictures, delete crap that you did not delete the first time around due to not being sure it's crap.
I've got all my pictures stored on a 8 year old samsung external harddrive that hisses and coughs. I've been meaning to get a new one, but that will have to wait a few months - or years - when I have more to spend. I've got my pictures stored on two laptops as well (one of which is starting to hiss and cough like the external). Just don't burn anything to DVD or CD... that never ends well.
You protect your photos the same way that you protect the rest of your data: you back it up. Backup strategy is determined by the state of cheap storage hardware.
There is no cheap hardware that will reliably archive your data, unattended, for 20 years. So avoid any strategy that involves the concept of "archive." Instead, leverage this useful property of disk drives: if you can successfully write some data to the drive, then you will likely be able to read data from any location on the same disk drive. If you do so on the same day, at least. With this principle in mind, you will need to place a copy of ALL of your data onto a disk drive that you use at least occasionally. As you use the drive, you are passively testing it.
First, upgrade your main computer disk drive. It must be big enough to hold ALL of the data that you wish to keep forever. Don't cheat yourself: if you want to keep certain photos, movies, music, etc. forever, it must all fit on the disk drive that you use every day.
Your main disk drive may fail, or you may alter its contents accidentally. Therefore, you need backup. This would be an external disk drive, your choice of USB, firewire, or eSATA.
Your external disk drive may fail, or the cheap backup software that you use might do something stupid to that disk. Therefore, you need two external disk drives, not one. You swap them each time you back up. Your goal is to have two external disk drives, both of which contain copies of all of your data.
Your house might burn down, so you need to cache a third external disk drive away from your house. Your job site might be the best location for this.
You can't depend on any of your disk drives if you don't test them. So, you need to periodically rotate your three external disk drives. Me, I take an external disk drive to work one morning a month. In the evening, I return with a different disk drive and start using it for backup.
Since you are taking disk drives out of the (relative) safety of your house, you should think about encrypting your external disks.
If you use this strategy, then you are protected as follows:
- If your main, internal disk drive craps out, you have backups to recover from.
- If one of your two in-use backup drives craps out, you will find out within a few days, when you try to back up to it.
- If your house burns down, you have a copy of your data off-site.
- If your offsite disk drive craps out, you will find out eventually, when you rotate it back to your house to use as a backup disk.
Conclusion: this is the minimum bill of materials for securing your data against single faults:
- A big disk drive that you use for all of your day-to-day activities.
- three external disk drives.
- backup software, preferably with reasonable encryption.
- a willingness to promptly replace any of the above items that fails.
A photo doesn't exist unless it exists in at least 3 places. You need _at_least_ a local copy and two backups, one of the backups being off site. rotate the backups at some interval you are comfortable with, bring the off site one home and the home one off site - so that all 3 sources are not in the same place at any one time.
Hard drives are CHEAP CHEAP CHEAP these days.
I use cygwin on my work pc to rsync to my NAS at home as my off-site backup. My home NAS is a $100 Buffalo linkstation (1TB). The trash-folder option on the NAS (samba configuration) has been handy to find deleted files. This setup requires a ssh hole on my home router/modem, but with the proper configuration that is not a large risk. I also run a https hole too to give me access to my files from any browser. I would recommend a single drive NAS, and one that doesn't cost more than $150. Most can be configured for rsync and lighttpd very easily.
I suppose so. If this guy shot 15GB in a vacation, and vacationed every year for 50 years, he'd only have 750GB. This of course ignores that his camera will be higher resolution in 50 years, but that is probably offset by greater hard disk storage densities (yeah, like the hard drive is going to be obsoleted anytime soon... ha). Still, the question becomes one of being able to make use of all that data, rather than it becoming a chore to maintain because you don't want to delete it, but never use it either. Maybe we'll have better data mining tools in the future that make it more accessible via various means.
I know that many people have problems to keep their photo collections down to a managable size. I always recommend to first somehow mark photos for deletion and then delete them later. Why? Because deleting a photo is something nobody likes to do -- you *may* want to look at it again, so you don't delete it outright and later you never come around to go through all your photos and delete those you don't need.
So use some photo managing app and flag those photos as soon as you see them. If you use some app that has this one to five stars thing, use a one star rating for photos you think you could delete (because "no star" could also mean you haven't rated them at all yet). Later then it's easy to just delete all photos flagged this way.
Everything else is futile. Keep your photo collection small. Do not try to delete photos immediatly because you very probably won't do it anyway. Flag them for deletion. A year later or so you will have no problem at all to wipe them away then.
I did that and I've an additional backup of it (among other things) via http://duplicity.nongnu.org/ (excellent software by the way)
Additionally I only keep pictures I have reviewed and decided were worth keeping.
Million of people make some gigs of pictures in 2 weeks during holidays and stash them, and never look at them - they couldn't it's just too many.
I prefer keeping 20 to 50 shots that are worthwhile.
Likewise with videos. (i rarely keep any video to be honest, i'm not very good at that)
FWIW, I use DigiKam to organize my pictures. It's nice. I keep the originals on my laptop. After I catalogue them with DigiKam, I copy them to the RAID on my network. Every night, the RAID rsyncs with another RAID that I keep at a friend's house. This gives me redundancy at home in case I delete a picture by mistake and an off-site backup in case of theft, fire, or flood. I use a pair of Netgear ReadyNAS NVXes.
Dude... if I had (more) pix that my dad took, I would
look thru them. Or pix that my mom took during the
war, or after the war... or during the occupation, so
yeah... just because you can't conceive of yourself
wanting to look at images someone else took, does
not preclude others of the desire.
AND, right as I was about to hit send... this thought
popped in my head as a footnote; Technology is
SOOO rapid right now, what you may consider to
be a kludge at "interest" in time and money spent
might be a trivial issue, of a holographic cube, that
costs $50 in a decade and has a write speed in the
Gbps.
Cause a decade ago, that 1TB raid server I have,
cost over $20,000 and was SLOW! But this only cost
me $200 and is capable of streaming 1080p to my
media PC while having data written to it.
In fact, after buying another 1TB the other day for
$50, I realized that I wasn't going to bother with
"trashing old/large files" as a task anymore... it's
not worth my time. Spring cleaning because the
drive is full? "Cleaning" hard drives manually?
Why? Of course I'll run some scripts every now
and then to clean temp files, cache files, etc but
if the drive gets full, that time spent is more than
halfway to a whole 'nother drive in cost/value.
I also considered that with my current setup, I
may break the mirror every...
3 months? 4? 6? And save that drive as a very
convenient backup of my current OS's states,
data and programs that are installed. Freezing in
time a drive that I can just pop back in the box
and boot back in time. Wayback machine meet
the home PC =) Much easier than a backup
and more convenient when you need it.
Um, just in case there isn't a patent on that
concept, here is my public statement of IP.
5) profit!!
-AI
For me, it is far better to grasp the Universe as it really is than to persist in delusion
My photos are imported into iPhoto, backed up by Time Machine to a ReadyNAS NV+ within an hour, then overnight also backed up offsite to a Time Capsule sitting at my parents house in another state. The overnight backup works by mounting my local time machine backup disk image, mounting the remote time capsule via Back to my Mac (because I put my credentials into my parents Time Capsule), and then rsync -av from the latest backup to the time capsule. Back to my Mac takes care of creating the encrypted tunnel for me, so it's also secure.
Two separate computers in the same house with a photo directory on each set up with automatic two way sync between those computers every night.
Plus a remote computer about 10 miles away with one way backup every night.
Drop a pic or directory of pics into the synced directory on either of the home computers and boom next day photos are stored in 3 separate locations.
Fast simple and hands off.
Prints. Good, old fashioned prints. The stuff that REALLY matters. Baby pics,kids first Xmas. Grad pics etc. This is stuff you want around for future gens to look at. Make several copies. Hand em out to family. Standards change, people get bored making backups/updating to new formats and standards. Can you imagine where you would be if all your photos were on 5.25" floppies? Optical drives will 'prolly be gone in 15 or 20 years. Even if your cd/dvd will still work at that point.
By all means, keep copies spinning. Keep some on portable storage. But make prints too!
Try Carbonite, it's great, you get unlimited storage space. I have over 11,000 photos backed up there and counting. Of course it's not free, but you can't put a price on peace of mind!
250gig? Are you serious? Welcome to 2011. I've got 6 TERAbytes of storage on my network... and that's not all that big. You don't need external storage or anything, you can get a terabyte drive for under $60. Buy 2 or more. Then have them in separate computers on your network and each computer up to the other with the various free backup software packages out there. I use http://www.gfi.com/ but there are plenty of others. This solution will not protect you from a house fire, you'll need a safe or off-site for that.
No affiliation, just a satisfied customer for many years. Best way to view and store your pictures. $40 / year for unlimited uploads. The above link will get you $5 off for the first year. I also use an external disk for backing up my RAW files.
i have four old desktops i came across years ago. all old p3 systems. i threw linux on one, setup gallery, and samba, and stored all our family photos there.
i keep the photos stored as images in the gallery application, and in their original format on a samba share the wife and kids can access.
another box has the same setup, and rsync's anything new nightly. that box sits upstairs, away from the original box, and is inaccessible to anyone else but me.
a third box, again setup the same way, is at a friendly location with a high-speed connection. weekly the second server sends everything up to that third box.
the fourth box is spare parts for the other three.
this is not the best solution, but one i came up with given my budget. after several years of using this, they all now have sata/pci cards, and 500GB-1TB drives.
this is just a placeholder till i send back my real sig from the future.
Pretty simple solution is to get a NAS like the Synology box (http://www.synology.com) and then set up some sort of offsite backup as well. Just remember that pictures are really not a replaceable sort of thing. If I lost pictures of my kids or of places I've been there is no way I could ever re-create them. If I lose my resume, or my tax forms, or even my music collection, who cares? Pictures of my kids standing for the first time? I'll pay some extra money to have multiple copies in multiple locations. This is not the place to skimp.
Why do you need so many pictures?
What the world needs is another wannabe photographer. Gbs of pics you will never ever look at and neither will your
grandkids because they are not going to care that much.
So do the world a favor and stop taking so many dam pictures.
I use Picasa's online storage. I believe you can purchase additional storage from them. They have a great program, a great viewer, and a great sync function. It's the only real option in my mind.
Guns don't kill people, "with glowing hearts" kills people.
Whatever I want to keep, I archive on hard disks, as of now. (Used to also use DVDs, but they're a pain and slow as hell.) I have several disks, some the portable 2.5 inch variety, and one large 2 TB external for a complete archive. (Lots of unnecessary additions to it...)
The problem is managing the risk of disk failure. You must ensure you have copies because your disks will eventually die or the data may become corrupt. You also have to keep up with new media technology and standards and transfer your archive(s) over as changes occur. It's also not smart to store all these disks at the same location, in case of fire, theft, the zombie outbreak.
My photos are, obviously, a subsection of the archive proper. Care and tend to what it vital to you; be willing to part with that which is not.
I store everything on my hard drive. Whenever I buy a new disk I move it to the new one.
In case of a drive failure I lose everything I can't recover from the drive.
At that point I simply start over collecting pictures.
When am I really going to look at several gigabytes worth of pictures of a beach, some historical buildings,
or whatever else I have pictures of.
Work related things are stored and backed up at work, so not my problem.
It's señor.
It's really funny how you talk about 'third-world countries taking your stuff for ransom' when most of the western world laughs while you get grouped by the TSA.
And about your comment how 'You have the best memory and image storage system between your ears.', well, that's really funny. If some day you need to rely on such perfect memory devices to prove you are innocent of something, you'll be wishing some camera would have captured it.
Dilbert RSS feed
Copy all the photos off the flash drive on to a NAS set up with RAID 5 (or a couple external HDDs) AND a local laptop/PC. Don't delete anything off the flash drive until you are absolutely certain that you have all the images there. Back up your laptop/PC to the cloud (i.e. Mozzy or SpiderOak) - I prefer spideroak as it is cross platform - Linux, Mac, Windows, but select a service that encrypts the data locally before uploading to the cloud. If you are truly paranoid, use two cloud services - just in case one of them goes under or is unavailable for some reason. That should do the trick :)
I am a semi-serious photographer - mainly because is is a tech fiddle where I can tell the wife new toys are 'for the family' and I have a longer budget leash.
Photo nerds are as addled as anybody on this forum and have pretty well developed best practices for this.
'In-the-field' backup. - use your laptop or a self-powered HD with built in Card reader and you can save your photos on a daily basis.
Once at home, a small NAS or RAID system will save you from losing data due to a HD crash
For true archival, by archive grade Optical media and keep them in Jewel Boxes (not plastic sleeves) and stash them somewhere else. (that is a whole 'nother can of worms here on /. ... you may prefer tape, or stone tablets depending on your opinion)
On-line services are OK, but maybe a bit slow for big data sets. I can come home from a sports event with a couple hundred new photos, and uploading all that can be slow.
I have 35GB of photos. I keep my "master" copy on my main HDD on my computer at home. I have a USB HDD plugged into it which is a generic backup device; I use pathsync to back up my photos there.
I can use my work hard drive for storage, so I also keep a copy on my work HDD as one off-site backup - I can connect to the office network via VPN, mount my work drive on my home PC and I can also use pathsync - though if I have many gigs of new photos I'll just copy them on my phone and bring them.
I also have a simple personal web hosting account at GoDaddy with some stupid amount of storage, so I just randomly FTP up copies of images when I remember.
One big concern I have is making sure the images aren't getting corrupted in transit or by disk failure, so I'm thinking about putting together a simple system to regularly verify them all via md5 to ensure there's not any changes that I'm not expecting. A side effect of this system is it would easily help me figure out exactly how many copies I have of all these images and where they are.
Real men don't make backs. They upload it via ftp and let the world mirror it.
I've been telling people for years, if you want to keep an image, you must have prints made.
Keeping data around for any significant length of time is hard and takes continual effort.
Keeping photos around, on the other hand, is so easy your great-grandmother could do it. It's as simple as forgetting about "that box in the closet".
So delete all the blurry, pointless, and redundant images and have the remaining 500mb printed up.
In 10 years or so, you'll be glad that you did.
Required reading for internet skeptics
What nerd doesn't have a RAID array? I've had one for my "collection" since the mid to late 90s. Slashdot is starting to turn into an online help desk... I mean c'mon... How do I store my photos?
I use a dock that allows me to just buy hard drives and I can swap them out any time I want/need to. The dock uses internal hard drives in case you are not familiar with them.
What kind of a cunt fails to make arrangements for his bills to be paid by direct debit or some other equivalent scheme so that his affairs can continue without his direct (and tedious) regular involvement? Rest assured, if you die your bills will be stopped by external agencies (and you won't have to care anyway) and if you live you will still be able to audit what the direct debit agency is processing from your account anyway.
RAID SAN -> TimeMachine on external USB -> offsite (backblaze.com), and I use Adobe Lightroom to organize the library. I have 43k pics so far (and counting).
The RAID is for protecting against single HDD failure for "live" data (e.g. data being worked in before the next TimeMachine snapshot runs), the USB drive snapshots the raid once an hour to protect against mistakes (deletions, etc), fs-corruption, SAN meltdown, and certain classes of malicious activity (virus that doesnt see the USB mounted, etc). The offsite/nearline backup is $50/yr and protects both other mechanisms against fire, theft, etc. Since its nearline (e.g. not mounted) its also impervious to virus infection, etc, and I can always see previous versions of the files if some corruption were accidentally backed up.
Yes, I am anal.
If you are archiving to optical discs, make sure that you use dvdisaster:
http://dvdisaster.net/
It allows you to utilize all of the unused (otherwise wasted!) space on a disc with distributed error-correcting data. It is free, cross-platform, and trivial to use. As an experiment, I burned a dvdisaster-padded CD-R and made a deep scratch on the surface with a key. Dvdisaster was able to recover the data without any trouble.
It's quite brilliant software!
At that size just stuff it on some DVD-R's. Of course you do have a large (2TB is cheap) external hard-drive to keep them live, right?
Now a better question would be to ask how to backup the 6+TB HTPC systems a lot of us own. Of course we use RAID for some protection but that's not a backup. I still have no answer to how to back these up, it's basically impossible without spending tens of thousands of dollars on some sort of large robotic tape system.
I setup a good size RAID array of 4 250 GB disks a couple of years ago. Personally, I like spinning storage for archives. It is cheap enough that you can add some redundancy. Is it as rock solid as a multi month tape archive that is regularly restore tested (you do test restores right?)? But... in terms of cost, both labor and $$... well... its cheap.
I recently upgraded the array. I saw 2 TB disks on sale, and grabed 4. Best part? The new disks after just 2 years are big enough to back the whole physical array volume (using linux LVM2) to 1 disk, and then build the new array. Simple and safe.
Then I put a VM on there and set it up to take backups of everything else. It is big enough now.
I figure that, with a monitor setup to alert me when the array loses a disk, and that is about as robust a solution as I need.
"I opened my eyes, and everything went dark again"
I see lots of advice on what storage media to use and how to configure, but how about the robustness of the file system? NTFS vs EXT2/3/4 vs other? How about using PAR2 to mitigate bit-rot? And what I really what to know is has anyone optimized a file system for archival storage? Maybe bake PAR2 into EXT4? Presumably this could all be done in addition to mirroring and RAID.
The world is made by those who show up for the job.
if you're serious about keeping your files, get a DROBO. it's a consumer-grade RAID, super easy to use, and provides redundancy.
Remember kids, if you're not paying for the service, YOU ARE THE PRODUCT THAT IS BEING SOLD.
I've been having the same problem, every time I offload a weekend's photos (just moved to New York City, spending my free time exploring) my hard drive gets near full. I've been deleting things, moving things to an 750 GB external drive and all that. It's getting to be a pain in the neck.
I've decided to drop down to 6 megapixel photos as well just so they're not so big.
Really, what works best for the photos is to get a box of crayons. Study each picture on the screen and using your hands and the crayons, copy it onto a sheet of paper, then store those in the closet.
(This is Macintosh based, so YMMV.)
I have all my photos (*many* years' worth, including scans from film -- over 20,000 files right now) on an old MacMini into which I transplanted a 1GB drive.
The Mini is backed up via Apple's TimeMachine to an TImeCapsule with a 2TB drive (the TC handles other machines, too).
And because I am paranoid, it is additionally backed up using CrashPlan to a computer in a different location (just in case my house containing both the MacMini and the TimeCapsule burns down).
Since I am *really* paranoid, I am considering paying for backup to a commercial "cloud" (which might be CrashPlan's), though my slow uplink speeds make this a somewhat dubious proposal for now.
first off, you took way too many photo's. You trying to remember the trip, or relive the trip?
Never ever use harddrives are backup. Never.
You might think that 1TB drive is good and all, till you go to plug it it and your computer doesn't recognise it. And I don't know about you, but I've been seeing that happen alot more with the 500+gb drives these days then I was seeing before.
another thing, and I've fell victim to this, and will again, is you do NOT need to keep everything.
You aren't ever going to go thru all the pictures you take, and since we took the cost of developing out, you take way too many.
Be seeing you...
It's like 25GB and pictures from my iphone is growing ~5-10 GB year. When I run out, I'll create another login.
It's kind of slow uploading unless you compress them as you upload - like 10X faster.
I upload the originals uncompressed. I use UNXUTILS for windows to call split.exe to split the MPGs into 50MB chunks. It doesn't play back movies anyway.
Then I can browse them on the phone!
http://www.dpbestflow.org/backup/backup-overview
I'd say get a Lacie Quadra 2TB (or other multi-interface external drive of your choice) as your working drive, a DroboS or FS (or other multi-drive array of your choice) for local backup, and choose an automated off-site backup ... Use a portable self-powered external 2.5" drive for backup of the raw files off the camera's memory card before you even start editing/organizing/color-correcting if you can as well...
It appears we really can store a lot of data on the head of a needle.
I use Wuala for this http://www.wuala.com.
Wuala is a p2p "cloud storage" solution. I share some cheap disk space from my desktop and a server (that's on 24/7), and get "free" storage in return. So far it has worked ok. I also have DVD backups, but i guess DVD will soon be obsolete.
It might not be a bad idea to think about something like Amazon S3 for the "offsite" part of your strategy. It is a lot of data, but the Amazon folks are flexible, and they will do an import/export operation from portable media: http://aws.amazon.com/s3/#importexport
Once you get the bulk of the data transferred, managing incremental backups is not so difficult. I use "S3 Backup" (http://www.maluke.com/software/s3-backup) because I wanted something simple, with data encrypted both in motion and at rest. The monthly cost from Amazon is ridiculously small.
At home, I'm duplicating my pictures on a dedicated disk mech on my desktop machine, plus a local media server. The offsite backup to S3 runs nightly.
The problem I see with manually backing up - to anything, or anywhere - is that you'll forget. It's inevitable. If you don't set up an automated process - and test retrieving your data occasionally - then you risk losing some substantial portion.
I've tried a few dozen different methods of making backups and nothing ever really works- if you want cheap, it won't be automated or simple. If you want simple, it won't be cheap. If you want automated, you might miss something.
While 16 gb of data seems like a lot there are days I shoot 16 gb before lunch- when mentoring HS students and filming events you can blow through 16gb very quickly. Now there are some great photo applications that offer you opportunities to back up everything, but I'm just really not into those as I don't trust them.
My suggestions, which work for me-
DIM (Digital Image Mover). Download, rename, renumber, date, and CRC check your files.
(I really wish.... it had a dual copy method).
Immediately burn a DVD backup. I've gotten lax here because I started shooting more... otherwise... ...insert another 500gb to 1tb drive into an external e-sata or usb3 reader and copy all files that you just downloaded (or run DIM again) to the external device.
Duplicate that a third time.
Disconnect both copies.
When one is full, add another drive and store off site.
I have about 10 different HDs in rotation and store images on my media server as well as the internal hardware RAID-5 system. Hard drives go to my Father-in-law's place for backup.
Raw images in folders. Date them something like
2010-01-23_Photos_In_Spain
So that they sort alphabetically. Keep them all in one folder, like Pictures01. Eventually, you may start another folder when that gets a bit unwieldly, like 50-100 GB or whatever limit you want. Maybe it coincides with a life change? Or maybe one bit folder per year?
Select out the good ones from a single folder. Copy them into something like goodpics_001_Spain_Day_1 and keep those folders all in a folder in Pictures01, like GoodPics
I upload good pics to share online. Having the good ones selected out saves time. Say you need to find something, look in goodpics first. Say yahoo stops photo online service, you have them ready for the next server.
Set up rsync scripts to upload / sync / copy folders / pics to a few sites. I triple back up my family photos. Occasionally keep snapshots, in case something goes awry.
I keep them in meticulously indexed binders, organized by year. All film and contact sheets are archival processed and stored in archival sleeves.
Two recent articles on PC Mech alerts had some information on "bit rot" and storage media: http://www.pcmech.com/article/how-to-avoid-bit-rot/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+pcmech%2Farticles+(PCMech.com) and http://www.pcmech.com/article/how-long-will-that-media-last/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+pcmech%2Farticles+(PCMech.com)
Use an old 100GB machine to donate 100GB to Wuala, then you get most of that back in guaranteed, online storage (encrypted and distributed as a bonus). Sweet. http://www.wuala.com/
I keep all my media (includes pictures, movies and music) stored on a RAID 5 array on my primary desktop computer. It's currently 3x1TB, which is about 1.8TB usable when formatted EXT4. Of this, about 150GB is pictures. Since RAID5 is fault tolerant, I'm protected from a single hard drive failure. In addition to this I back up my photos to DVDs. I keep the DVDs in a Case Logic case in my fire safe. In the future I plan on making a secondary backup, either to an external HD or a second set of DVDs, which I will keep at my parents' house in my Dad's fire safe. In your case, since the amount of data you're storing is relatively small, I'd pick up an external HD for primary backup, and backup to an online service as a final line of defense. If an online service costs on the order of $25 per year, it's really a no-brainer in terms of value. If you're like me, your pictures are the only truly valuable things on your computer, and they are of utmost importance.
and don't have any. I prefer to remember the world the way I wanted it to be, instead of the way it was.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
You could buy an external hard-drive and keep it in a small but fireproof safe. A Synology NAS-box is a good choice because it comes with (2) terabyte drives and you can access the drives through your web browser. You have to have an account to access the drive too. It is usb-supported and has an gigabyte-ethernet ports.
On my hard drive, rsynced to a NAS box (mirrored disks) and backed up nightly to Amazon S3 storage via JungleDisk. (Disclamer: I work for neither Amazon nor JungleDisk)
Have you ever considered piracy? You'd make a wonderful Dread Pirate Roberts.
First off, is this really an askslashdot subject?
I keep ALL the photos I shoot. Why? So I can review the bad shots and determine what I did wrong, and improve my technique. I focus on technique rather tham gear lust, even though all of my lenses are slow.
Just get an external hard drive to back up your photos. Also, consider archiving individual shoots to DVD-R, or if your budget allows for the media, BD-R.
Lastly, order yourself a 2TB or 1TB drive, and use clonezilla to move the system to the new drive.
The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
This is the 21st century, and companies like smugmug do offer unlimited storage for your JPEG and videos for 50-60$/year
However, if you have RAW files, then its kinda sucks. Not much hope.
My Aurora : http://www.youtube.com/watch?v=o91ZsGwJYyg
FB : https://www.facebook.com/TanveersPhotography
However, I store only JPEG. If your pics are in RAW format, how about Amazon?
Lets see. for 250GB disk space it costs 25$ approx. Well its expensive all righ/month
My Aurora : http://www.youtube.com/watch?v=o91ZsGwJYyg
FB : https://www.facebook.com/TanveersPhotography
I've had a lot of awesome experiences in my life. So many, in fact, that I don't remember them all--or all the details. So it is nice once in a while to go back through my photos and videos and remember/relive those experiences--sometimes with people with whom I shared the original experience. Doing this is itself a pleasurable experience, one that is not possible if I don't take and keep some photos.
I'm on board with the idea that taking photos can sometimes get in the way of the experience. But I think you've maybe taken it too far when you're chastising someone who just wants advice on how to hold onto photos he's already taken.
Build a man a fire, he's warm for one night. Set him on fire, and he's warm for the rest of his life.
Get a Pair of NAS setup.
Find a friend with the same issues.
Place one at each house
Store your photos on yours and replicate to other nas thru internet
your friend does the same.
I use a QNAP ($1100 - 4 1.5TB disks in a RAID5) unit right now and will be setting up with an associate of mine to have a unit stored at his location shortly.
The QNAP has the option to rsync to another unit built in so non techies can set it up sortof easily.
The QNAP also has media server app built in so I store my music on it as well and access it from anywhere in the house including the stereo/TV setup.
In the mean time I have 2GB external storage for the laptop (e-sata) that I clone to the QNAP (network - Raid 5)
Online service might work for the original poster but for those of us on the higher end of hobby photography their virtually unusable.
I tried carbonite, can't talk to network drives! and deathly slow if you have any serious quantity of data. I have near 300GB of just photos. One night casual shooting in a blues club and I can have 6-20GB of new data and that's not even shooting raw!
I absolutely agree that there is a very real loss of the fidelity of life when you are stuck behind the viewfinder trying to capture the moment instead of just living through it.
I read your post and found it very true from my personal experience trying to take pictures and document events of my life, what a loss that was. The other people who disagree with you, defend their tedium of work, joined like Siamese-twins grafted at the face to a camera device that is blocking and shielding the moments of life happening right in front of them.
The thing to do is to free yourself from the task of trying to document the precious moments of life and setup a free-running recording system so that you can participate and experience everything without having to worry or concentrate on the meta-task of capturing the moments.
Maybe soon we'll get stimsim technology to record our own experience in life without having to consciously do it.
Seriously, online storage is the way to go. Mozy and Carbonite both offer unlimited storage for $55/year. Both are incremental, so after that initial transfer backups are executed extremely fast. They're in the background, and you can set it to work only when the machine is idle, so you won't even notice it's there. I swear by online backup, personally. It's the cheapest and easiest solution for most people.
Truckin like the Doo-Dah man...
You're correct, my mistake. Mixed up my giant companies. Seagate is the correct brand. Toshiba also makes 1TB drives that are similar. Avoid the Western Digital 'passport' line of products though, the last year or two they have begun to ship with horrible embedded software that tries to autoload on your machine, and cannot be deleted because the software is embedded in the firmware of the controller.
$40/year, unlimited storage, unlimited uploads, will keep things private, and unlimited traffic for the photos you want to share. They use Amazon's S3 server system, which is the platinum standard in digital archival safety. Plus, they're cooler than Flickr. The only way this doesn't work is if your internet connection isn't pretty decent. Then, keeping up with the uploads could become a pain.
I upload pictures to my PC. Look each of them for about ten seconds and delete right away. This works well for me but it's not for everyone. It depends on whether you have or don't have photographic memory.
If enithin kan gow rong it whil. (Murfey)
Seriously, get CrashPlan, get additional internal or external drives for a couple of friends' or family members' (desktop) PCs and set everyone up with either CrashPlan (free, only every 24 hours) or CrashPlan+ ($2.50 monthly per PC with 10GB online storage, down to $1.50/month for 4 years or $5 monthly for unlimited online storage (down to $3/month for 4 years)). CrashPlan+ adds some nice features (better encryption, backup sets (e.g. mail, financial, docs go to many spots & online, photos go to fewer places), changes backed up every 15 minutes) but is by no means required.
Like any real backup solution you'll get to keep multiple versions if you make changes to files. You can also "seed" backups by creating them while you're at the friend's house because really, even over a local connection 200GB is going to take quite a while for the initial backup. CrashPlan has a lot of documentation of scenarios, there's a good chance one will match your situation and will cover issues you didn't even think of.
If you provide the drives and the setup, your friends will probably be just fine with doing this, particularly when you point out that they can back up to each others' PCs as well - everybody wins. Since you're providing everything, they should even be fine with you not making yourself available as a destination.
This addresses more of the possible things that cause you to lose data:
* OS or program goes insane, stomps everything (also includes accidental deletions or malware such as the occasional encryptor offering to sell keys back to you)
* Catastrophic hardware failure
* Catastrophic hardware disappearance
and frequently missed
* Catastrophic home disappearance (fire, flood, tornado, etc.)
The biggest drawbacks that I know of are increased network traffic (upload for you, upload and download for your friends) and the need to leave your laptop on at least occasionally so it can actually run the backup. Either or both of these may be non-issues or deal-killers depending on your personal situation.
fencepost
just a little off
I keep all my digital photos and video in a Perforce revision control repository. The free version of Perforce is enough for my needs (it limits the number of user accounts and client specs you can create).
Perforce handles large files with ease (I've added a fair number of 1 or 2 GB files) on commodity hardware. It allows me to check out a portion of the photos/video on a given computer so it's nice when a client is constrained in hard disk space. Since it's all version controlled, I can tell my wife and kids to edit or delete to their heart's content and I never have to worry about losing an original copy. And because of Perforce's support for lazy copies, I can move files around all I want without requiring more hard disk space.
Backups are a simple p4d stop, rsync to an external drive, p4d start. Do what you wish with the external drive, like store it in a safe or somewhere off-site.
Disclaimer: I don't work for Perforce, I just like their product and haven't found anything free that will do what it does (SVN is close, but no cigar).
Online backup is definitely worth a look. I use Mozy (http://mozy.com), it costs $4.95 per month and includes unlimited storage and a handy little app which handles all your backups for you. I have ~120gb of photos and important docs backed up online. Sure the initial backup takes a while but after that it only uploads new/changed files. Oh and it's all encrypted and you can use your own key. Also, if the thought of downloading all your data if you lose it scares you then they'll send it to you on DVDs.
You need need to take two things into account, reliability and redundancy. External hard drives are notoriously unreliable, cd-r and dvd's can degrade within one year. Your best bet is a NAS with at least 2 drives and two backup systems (local and off-site). I use a Linux server with rsnapshot for my local backups and Crashplan for external backups. Instead of a Linux server you can opt for a decent NAS from Synology or Qnap, I don't know if Crashplan supports these though.
The key to a good backup system is simplicity:
- Something that doesn't rely on proprietary backup software.
- Something that stores the backed up files 'as is' and doesn't do funky stuff like change the file names, rewrite them into unintelligible binary blobs or databases etc. This is related to the 'proprietary software' point above. For instance the built in backup tool in Windows is pretty good in most respects but I don't use it because it stores the files in such a way that getting them back without a copy of Windows is a PITA.
- Something that requires relatively minimal human interaction after the initial set up. The more tiresome a chore it is to run a backup, the less often you are going to do it.
So my system is quite basic:
1. On the systems in my house, various 'important' directories (e.g. photos) are mirrored to my NAS overnight. Just a plain old file copy that rewrites files changed or added since the previous backup, and also replicates deletions (people advise against this but I'm usually careful about what I delete). I use a variety of software to do this: plain old rsync does the trick on *nix, something like Karen's Replicator or Microsoft's SyncToy works well in Windows if you want a GUI. My wife uses ArrSync on Mac OS to do this, although you could just use rsync from the CLI of course. You can set these to run as tasks automatically (every day/week/whatever), or do it manually if you are disciplined enough. I do it manually each time I do any significant changes/additions (e.g. each time I offload new pics from the camera).
2. The NAS is a cheapo DLink DNS-323 enclosure. Performance isn't particularly fast or anything but it's cheap and reliable (I'll probably upgrade to a higher end Synology unit or something eventually though). It has two drives in RAID1. So this offers redundancy: it is very unlikely I will completely lose the data on the NAS short of it being physically stolen or destroyed. The few times I have done something stupid on my main machine, the backup on the NAS has saved me. But RAID in itself is not a backup of course. So...
3. Every so often I pull one of the hard drives in the NAS RAID 1 array (while it's powered off, of course). It goes in the back of the car and dumped at my parents' place 10 km across town. While there I collect the HDD I left there the previous time, and when I get home, give it a quick format, and whack it back into the NAS. The RAID volume rebuilds itself and that is that.
Naturally this method has its flaws: it's not particularly automated, and I don't keep multiple revisions/versions of my backup, meaning that if I deleted a file a long time ago and need it back, I'm outta luck (i.e. I've done a full PC -> NAS -> parents' place cycle). But it works well enough for my needs. One thing I might start doing though is to keep hashes of important files along with the backups to protect from corruption over time (no point having multiple backups of files if the files themselves are corrupt!)
My photography archive is approximately 100GB in size. I keep it safe in the following way:
1. If you care at all about keeping the fruits of your photography labours safe, I cannot recommend highly enough Peter Krogh's "Digital Asset Management for Photographers, 2e". The bucket concept is from there. See http://www.thedambook.com/
Janie took my gun...
I use Mozy's Mozy@Home - $5-ish a month for unlimited storage and it also syncs the backup to a portable drive as well.
Plently of other features as well..
What I chose is to make 2 back-up drives, one at home, the other in a safe deposit box. I swap 'em back and forth. As your needs change you simply get larger drives. vamnam
nothing happens.
These are obviously images that I want to keep for my life.
You wont go through the pain of going through a thousand pictures more than once
I did some online research a few months ago and found BuddyBackup. It lets you store your backups on your friend's disks (and vice-versa). It's free (as in beer) and by the feature list it looks like the company behind it knows backups.
I just installed Gallery3 on an EC2 instance with the S3 plugin for storage. Works great (with a few bugs, but could be fixed easily). I believe there are also plugins for Gallery3 that allow you to sync photos with Facebook and other services.
There is a fine line between being a cultivated citizen and being someone else's crop. - A. J. Patrick Liszkie
For me the workflow is like this:
Copy to laptop harddrive if on the road, to external drive at home.
I used to back up to DVD, but it is too bothersome. So I have a Synology NAS for local copies.
I also have a USB dock for raw SATA disks, so I can replace them. That is for easy access backup. Stored at home.
Finally I have just started using Crashplan. That is online backup. Covers the situation where my laptop and media at home is stolen/burned etc.
I was thinking about carrying a backup disk to work. But there is a large risk that you do not remember to shuffle the 2 disks. The Crashplan backup software will allow for this though. Syncing only changes to each of you external backup disks. I might do that with the raw disks I have.
Never delete from the memory card. Copy stuff you like, keep the rest on the card. When it fills up, buy a new one. Instant backup.
Keep your photos (final JPEGs) on two external drives from two different batches (e.g., different manufacturers/dates) in two physically different places to avoid fire/theft and the high correlation between failures in single batches. I keep one drive at my work office and one at home. Then upload your photos to your family website. Most hosting provider provide "unlimited" storage for files. I currently have 100GB of photos uploaded to my provider. Note that the first upload will take a long time, but you can use rsync thereafter to make it manageable.
This ones easy, smugmug.com. You can store your entire collection including the original raw images. Nothing's better.
I can't believe nobody uses checksums...
For raw files, I add checksums (an md5sums text file in every image folder) right after importing the images from the flash card.
Jpegs tend to get changes in the exif data, so I add the checksums for those after the editing phase.
This makes it easy to verify the backups, since the md5 files get backed up, too.
In case of a corrupted file, I can hopefully find a good copy om my nearline backup HD, my NAS hidden in a bookshelf, or an offsite USB disk.
I have similar issues as you. I discovered that the workflow I use matters as much as the technology.
Here's what I do:
1) I download all images into a "downloads_DATE" folder with subdirectories called "print", "enlarge", "raw", and "video". I do NOT let anything rename the files so that the filenames are unique (for later searches of full resolution and raw images)
2) I use Irfanview to sort the pictures into exceptional quality stuff (enlarge) and general images to keep (print). What is left over in the folder is "disposable" but generally kept anyway
3) I copy the newly organized folder to an external HD. This is (ideally) backed up to a second external HD that I keep in a fireproof safe (I've been slacking on this).
4) Now that I have a backup, I delete all files not in "print" or "enlarge" as well as raw files I do not need at the moment. Eventually, "print" will go away as I run out of space on my laptop, leaving only the best pictures from that batch
5) The pictures I really care about get sent to my own gmail account and several of my relatives with good descriptions. After many moves and machine changes, I find that the stuff I sent to myself is more accessible than anything else (I haven't run out of space in 5 years...) Once in a blue moon, I print some of the pics and let people who care to do such things put them in photo albums. Other people's albums may be the only backup some people end up with given the uncertainty of life...
Flicker is indeed saves full resolution images, but getting them all back in case of hard drive failure is quite an ordeal. It took me over a week, and a python script to do it - I wouldn't recommend it to the faint of heart (see comment below).
Flickr is nice for sharing, but don't relay on it as a backup.
Quite. So you're one of those irritating wankers who insist on looking at the most fantastic stuff through a 2.5 inch tft screen? Thanks for making all my holidays just a little bit more annoying than they need to be.
I keep all my photos in one place, categorized by theme and then by date (e.g. photos/family/2010-02-birthday-mom) :)
Before putting photos there, I go through them and remove blurry or duplicate ones. I have big HDD, but not so big to keep junk photos
For backup, I use CrashPlan. It has free backup to external HDDs or to other HDD in your PC, or even remotelly to other PC. I also subscribed to their unlimited online backup, and now I have multiple different type of backups, synced automatically.
My approach is fairly simple. I have a pair of cheap dual disk NAS boxes, one in my shed and the other in the house. The larger one backs up to the smaller one and both have the redundancy of RAID. I'm fairly happy with that, they should survive a fire without me feeling the need to dash into the burning building to save them.
I then have a HTPC (Acer Aspire Revo running XBMC) connected to my main TV which allows photo browsing. I've set up scripts that automatically file the photos based on their EXIF date (/yyyy/mm/dd)
However, something I also do, by virtue of my wife 'ordering' photos, is get them printed. She goes through the vast numbers of photos that we rarely look, cherry picks the best ones which I sometimes crop or adjust and sent off to be professionally printed. Our house is littered with photo books, albums and frames. Sure, if the house burns down they're gone but that's pretty normal. My mother has a collection of photos with burnt edges from her childhood.
Ain't I just an insensitive racist jerk?! Nobody in the real-world would ever act like that! Just another stupid paranoid American buffoon who is too stupid and too scared to travel farther than Disneyland!!
BIGOTRY DETECTOR OVERLOAD
DOES NOT COMPUTE
Best Buy is selling off the 1TB USB 2.0 Toshiba HDDs for $86.00 because the USB 3.0 models have arrived for $129 or so. I got everyone with a digital camera one for christmas, and 2 for myself.
I was going to buy (pricing) an enclosure to use my HDDs that I replaced with SDDs to reuse them, but they wanted $60 for the empty box so I figured the 1TB HDD was worth the $26, and bought myself one, and nearly a month later 4 more for Xmas. Best Buy had 8 - 10 on the shelf every time I went in there (way to often) so they likely have one left for you, if you hurry.
The smaller sizes had not been replaced with USB 3.0 and were not on clearance as of my last lunch break:) I would imagine they are next as USB 3.0 works its way down the line.
NABBE-JARC
Cheers!
I work on a laptop so, I use a fast external SATA hard drive as the work copy. Once a week, I turn on my raid 1 NAS and I launch a rsync backup, then turn it off.
I will soon buy a second nas to have a second copy in case of a NAS malfunction, and another external drive for monthly out of site backup.
Stupidity is the root of all evil.
As a dive instructor, travelling is just part of my job. I do take a lot of underwater pictures and of course, I don't want to lose those valuable pics, specially when you're not sure you'll encounter some subjects again. Of course, when working in remote areas, the internet connexions can be quite a hassle. This is what works for me:
I upgraded my laptop HDD to allow me to store all the original pics in RAW format. while on the road. I also have an 2.5" 640GB HDD for local backup.
- I store the original pics in a folder of my laptop
- I use a synchronization tool to make sure this is also backed up on my external HDD
- I import everything in Aperture (MAC) (Windows users could use Lightroom) ans I start sorting and retouching, keeping about 10-20%
- When done sorting/retouching, I backup my Aperture database on my external HDD. That allows to keep the original files and all the modification I may have done.
- I use Mozy (online backup solution with unlimited storage for an annual fee) to back up my Aperture database.
That way, I can make sure the best pics are saved online. If my laptop and HDD burn or get stolen, at least, I still have the 10-20% good "must keep" pics. If the rest get lost, It's not a really big deal. The best and essential is safe. That way, I can also save the amount of internet bandwidth I need. A few Gigs take forever to upload over a 3G connexion. Of course, If you have a decent DSL internet access, you could store the whole thing online and not just a subset.
The hardest step is to decide which pics are worth keeping or not, but as far as I know, nobody takes only good pictures. You'll have 3-4 times the same subject with different composition or lighting, so keep the best one. Your friends and family will be happy if they only have to watch the 100 best pics you took from your trip. If your slideshow is too long, you'll lose their attention quite fast anyway. In my experience, if the job is done well, you NEVER have to go and search within the 80-90% you discarded. Digital photography is great, but the process doesn't stop after pressing the shutter button. It should be a shoot, sort, retouch (if necessary), backup workflow.
If we are just talking about photos, there are even more options. A Flickr Pro subscription allows unlimited photos for $25/yr with (optional) sharing of photos.
Until Yahoo kills the service and deletes all your data.
But I'm sure they'd never kill Flickr the way they killed
Yahoo Photos and
Yahoo Video
and deleted everyone's data, right?
Yahoo happily deleted dead people's personal webpages which cost them nothing in terms of bandwidth or processing power.
Most of pages were lame but they were still someone's page.
Well, cheap backup with years of durability is tape. They will kill me for that but, ask any company why they still use it.
The issue could be, the price. Tape guys moved to enterprise level with enterprise features/interfaces and the price. I really don't know if some kind of semi-pro solution exists.
Obviously, you shouldn't store tapes in same location. A personal safe at bank etc.
If we are speaking about "I got backup anyway" peace of mind kind of offline backup solution, it is tape.
I personally use smugmug.com for photo backups. I am a serious amateur photographer and my current pix number is approaching 15000. Smugmug offered the best deal for the money - about $40.00 a year for unlimited storage. Their privacy is very granular. You can share one pix or a whole gallery or stay completely private. You can arrange your photos in a way that suits you. Keyword tags are supported. You can upload from most any photo organizer or send your pixs in via email. They can send your photos on cd to you if you decide you don't want their services. They offer professional photographer packages as well. I chose them over flickr because of their privacy controls and unlimited storage. At 15000 photos, I'm already over the flickr limits. I don't work for them - just a happy customer.
I just put all my photo's in a gigantic encrypted file labeled "WikiLeaks Insurance" on the Piratebay. For some reason, people seem to seed it.
My wife and I have, indeed, looked back at almost all the video and pictures we've taken on vacation multiple times. Maybe you just go to boring places.
It seems to me that the combination of a large external drive and online storage is easy and safe. The crashplan backup software lets you perform backup of a set of data to multiple destinations, including any connected drive/folder, online storage and sending to friends' computers. Thus you can be a couple of friends (3 should be more than enough) who team up to have 3 copies of each others' backup sets.
That way the backup is performed quickly to the external drive, while it trickles to the online repository and to your friends, depending on your broadband speed. Get some international friends and you're ridiculously secure from any disaster. I imagine crashplan soon will have "virtual friends" which are backup sites in the siberian tundra or something, why not?
Simon
Although so many people have posted similar methods I just thought I would share mine.
I take the cards and dump them to my local HDD.
I then take that and burn it to a DVD, and label it and put it with the rest of the source DVDs. This includes everything and is labeled with a date.
Once I have that, I then go through and toss the worthless photos.
After that, I make another round through to make sure I only have things I really want to use. I then import this in to Picasa.
From here, I will find and edit only a few of them and upload them to Flickr/Facebook/etc. for display.
I also have online backup to back up the remaining collection from the HDD to somewhere. There are many, many cheap ways of doing this. I make mine cheaper by not backing up everything online since I prune it before it gets to that point. You could use Mozy, Crash Plan, or even just pay Google for more Picasa web storage.
If I wanted a more secure backup that included everything, I would probably burn duplicates of the DVD media and put one in a fireproof safe or outside storage. Alternatively to that, I would at a relative’s/friends/co-host location, setup a box, and use rsync or Crash Plan’s software (both free options) to have things backup automatically offsite to HDD.
I only mention Crash Plan because they at least allow you to "seed" your initial storage by sending you a drive to fill and mail back, so you don't hit a bandwidth cap trying to get things started.
Your options for this are so numerous that it all really depends on your own comfort and financial levels.
I am storing my images locally on a USB drive attached to my desktop and with a copy on my NAS server that does RSync to an offsite (Internet based) backup service I pay for.
This NAS server is setup so it does not allow deletion and the RSync is also setup to not propagate deletions on to the backup.
Workflow for backup is handled by Adobe Lightroom where Import have option to copy RAW files to 2 places.
I have had to restore from net once, before I added the NAS server into the mix.
git-annex basically takes your files, moves them into the .git structure and replaces your files with soft-links. You can define remotes, push/pull to/from there, etc.
The beauty is that you are not actually checking any files into git. So you are free to have whatever subset of your files you want on any given machine.
Add to that that you can simply tell git-annex that any file matching a given glob (think *.jpg) needs to be in existence at least n times within your git-annex cloud and you can't delete stuff by chance, no matter what you do.
Two caveats, though:
1) It does not really handle files that change very well. But with images, that should not be a problem
2) The original ctime is not applied to the soft-links, nor does git-annex seem to store that kind of data. Which sucks when you want to extract files permanently.
PS: Thousands of pics and a mere 16 GB? Be happy you don't own a dSLR with RAW support, I guess :p
PPS: Thanks for asking, I will go through the whole thread with a comb. Highly interesting, to me.
Corporate has sent out, several times, a warning not to bring onto the premises any device that could potentially be connected to the corporate secured network.
In addition, they have implemented a policy of laying off people annually (great for morale). The people are immediately escorted out of the building; you can't even collect your personal stuff from your desk, they'll ship it to you.
I find this whole approach Draconian. Because of this I won't keep a backup drive at work; I got a safe deposit box.
I use a usb connected terabyte drive for media (pics, songs, etc.). I have purchased a subscription to Mozy and back it all up offsite. Every year, I purchase a new drive, copy the stuff over to the new one, wipe and sell the old one. Drives that size are cheap (and as my media collection grows, drive size has grown at least as fast for about the same cost), so I don't mind the annual cost of the backup nor the the annual cost of the new drive with zero hours on it.
Before you design for reuse, make sure to design it for use.
I store mine in several places. The primary store is on the main desktop system. These are then backed up to a portable drive which is kept locked in a fireproof/water-proof safety box when not in use. I also take the portable drive on occasion to work and backup the photos to a drive there as well so that I have the data in more than one location.
I've got 12 years of digital photos that number in the the tens of thousands. Yes, I'm ruthless when it comes to tossing the losers. I'm also consistent in rating the photos. Those with three or more stars I would like to think that my kids would keep. The truth is that I don't think that will happen. I feel that the only photos that may be retained are the ones I had printed in several books (I use Blurb, but there are others out there).
In the mean time I would like to make better use of the photos. I have a digital frame at work which is great. I'm thinking of getting a WD-TV setup for home. But one thing I would like that I've never seen is a photo slide show screen saver that is ratings aware. Anybody know of one of these?
At home I use desktops and my family photo and video collection are backed up across 3 hard drives (in 2 different PC's) ~daily via a scheduled back-up script.
I have just archived everything to date to BD-R. Something I intend to update every 6-12 months. I also intend to leave a second copy set with a relative in case of fire.
When travelling with laptop I keep wherever possible all video/photos on thier SD/CF cards from camcorder/camera and back up nightly to laptop HD
I use a B3 Linux-based personal server from Excito (1TB Wi-Fi model) and backup the photos stored on it to rsync.net and an external hard drive using rsync. Simple and effective. Disclaimer: I work for Excito.
I picked up a Dlink DNS-343 NAS with a quad bay for a few hundred. It works great....and then I burn photos to disk twice a year and store all the disks. I would definitely look into a Dlink or Netgear consumer nas for your photos.
First what not to do:
Don't invest in RAID-5 or RAID-6 arrays unless you can afford a high quality controller and SAS drives. Cheaper RAID arrays often have significant problems rebuilding. They're also expensive and large. You're also gambling that you choose a hard drive model that doesn't have any serious systemic problems. Choose poorly and you're risking total data loss. Besides, they key desired feature here is mirroring and one doesn't need RAID just for mirroring.
Forget about recordable DVDs or BluRay discs. The media capacity is small and thus cumbersome to mange. The recording dyes can degrade surprisingly fast, again varying greatly with manufacturer or lot. There's lots of research on this and you can download software to measure correctable errors before the discs become unreadable. I've had some discs become unreadable within 6 months. Some discs test with significant error rates immediately after being burned, before even leaving the drive tray. Basically if you're not testing error rates of your dye-based media, the joke's on you for buying them.
Tape drives are inconvenient for random access. Longevity could be better or worse than hard drives. Without hard data one can't recommend tape drives. Like everything else, longevity probably varies quite a bit between media brands and drives/recorders.
Internet backup services aren't going to scale depending on how much you need to access.
Here is what I do to manage about 3 TB of family photos and video, including about 0.5 TB of encrypted (truecrypt) content. Its not a perfect solution. However its cheap and easy to set up and maintain.
1) I have several "master" directory trees on my main computer. It runs Linux currently, but windows could work too. There are several drives and partitions, all mounted in a master "media" directory tree. There are subdirectories organized by year for which the media was generated. This helps with locating media of interest.
2) I have a large number of 1 to 2 TB external USB 2.0 hard drives from different manufactures (mostly Seagate and WD branded external drives from Sam's or Costco), each one clearly labeled. Some have one partition. Some have two partitions with the second partition being a truecrypt partition. The external drives are grouped together such that each group is a complete backup.
3) For each drive I have a unique shell script that calls the 'rsync' command.
4) I keep one group of drives at my house (the local backup), a second group in a bank safety deposit box and a third group at a family member's house. So that's the original plus three backups.
5) I sync the local backup about once or twice per week, or after significant media generation. This only takes a few minutes and is fully automated.
6) Once every month or two I will do a local sync, swap the local with one of the other groups then do another sync.
7) I periodically reformat the drives and check the SMART data.
My photos etc are backed up periodically to an external drive and kept in a bank vault. That gives me more peace of mind than these other offline solutions and it is cheaper, somewhere around 25$/yr for the vault and cost of whatever drive I use. Under what scenario other than drive failure or something during the drives time outside the vault could data loss occur?
What I do is split my photos for current ones (allocating about 50 GB on my local disk), and Archived photos (the rest of the 100s of GB). The archived photos get stored in the Cloud on (eva) Drive (http://www.evacloud.com/evaDrive.php). That way, I can quickly edit RAW files and such for current photos, and get high-speed access to the rest anytime I want. The (eva) client software lets you cache however many GB you want as well, which makes it basically like working on them locally with the unlimited capacity and stability of being in the cloud.
which will give you a RAID like system with software. Either use any known Linux distribution or install e.g. FreeBSD on a server. Then set up a network drive in your preferred networking protocol et voila: you have yourself a cheap extendable RAID.
People who are suggesting Hard Drives clearly haven't tried to load information from an old IDE drive. The interface for internal media changes quicker than external - you can still buy 3.5" drives, and only for about $10. A technology usable since 1987 - 20 year storage without media transfer is amazing! Today you'd use DVDs: DVD's provide 4.7GB at a time, which for photos is fine. One just has to be careful of WHICH DVD's - the discount brand will degrade in about 10 years. Buy some "Medical grade DVDs" (online), and use them. Its not a cheap as buying a 1TB HD, but its portable, and more reliable.
I have one "master media" folder on my primary PC. I put all photos and videos on it organized by year and month. I back it up with Carbonite. As for your 250 gb hard drive, it sounds like it is time to upgrade. Easy as that.
Have the best of both worlds: use a Pogoplug! www.pogoplug.com
You can copy your files straight to your external USB drives, plug the drives into a Pogoplug, and then access them from anywhere with Internet. It shows up on your laptop as just another hard drive. I use a Pogoplug as my primary storage device now.
In addition to backing up photos on a DNS-323, I also have an external hard drive in a safe. This one in particular: http://www.sentrysafe.com/Products/278/QE5541_FIRE-SAFE
This safe features fire, water, and theft protection, and has an external USB port that connects an outside computer to a hard drive contained within. I use a Patriot Gearbox: http://www.newegg.com/Product/Product.aspx?Item=N82E16822219005&cm_re=patriot_gearbox-_-22-219-005-_-Product
This allows me to access the hard drive contained within the safe from anywhere on my network. Very convenient.
Thanks for the content
http://www.hiwaar.org/vb/forum.php
Of course your number one problem is simply finding enough reliable storage/backup to hold everything. I'd suggest you look at iomega's storage appliances (no working relationship with them, but have used them). They are inexpensive, easy to use, support multiple protocols, and offer lots of RAID protected space.
But there really another aspect to storing huge volumes of digital photos for the long term that I haven't seen discussed in any of these threads; which concerns digital asset management generally. How do you index, search and retrieve these things over the long term? How do you do things like "show me all pictures of grandma."? A sequential scan becomes awfully tedious, and eventually impossible as first hand knowledge of such collections fades. How will your kids, grandkids etc. accomplish these kinds of tasks? It's extra work to maintain a properly curated collection, but if you don't do it, in the long run the collection becomes almost worthless.
I use 2 external, 1TB hard drives. All of my photos are backed up onto the first hard drive. That hard drive is then backed up to the second hard drive which is stored "off site." This way if one hard drive dies or is stolen/destroyed, we still have the other one.
My sci-fi novel, Ghost Thief, is now available from Amazon.com.
Calling complete strangers annoying in print, mocking their creativity, their intoxicational preference and then telling them to "fuck off" is neither polite behaviour nor the path to a long and healthy life, amigo.
'Subtitle Sunglasses' refers to an exciting and innovative user-interface concept for an essential technology whose elements are just now beginning to come into focus. 'Subtitle Sunglasses', when they arrive, will combine nanoengineering, bio-motion power generation, molecular electronic circuitry, Sapir/Chomsky proto-linguistic theory, independent-speaker speech recognition, 'heads up' bio-luminiscent microdisplays into such a world-changing technology that will make image storage questions look as primitive as toroid-donut-based RAM memory does today.
Today's currency is not dollars or Euros, it's creative ideas. I give you a fantastic idea for free and the best that you can offer in return is a hearty "fuck off"?
I suggest, honored sir, that you are in the wrong industry. Might I suggest getting a few more tattoos and becoming a excrement-scraper at the elephant house at your citie's zoo?
The obvious omission is gamma correction and color correction. I'd want to do that somehow, without losing JPEG quality. I don't want to crop.
My photos are a megabyte each on average. If you got gigabytes back from Mexico I guess you use RAW. I'd probably convert most of those to JPEG (but be careful to keep EXIF data).
There are three basic ways to get your personal electronics stolen by a person in uniform at a border crossing.
One is by a simple corrupt thug who uses the customs office as his own private little WalMart. These creatures are most commonly found in Africa and the outlying former Soviet republics. They resell your property for a profit. They can be bribed if the value of the bribe is equal to the amount of effort that it would take for them to resell your property locally.
Second is the paranoid secret policeman who thinks every foreigner is a spy. Found in fascist police states (like Burma and Belorus) and backward religious tribal areas (like Pakistan). Often will be satisfied with the destruction of the data if they comprehend the technology. Otherwise they just smash the equipment so no one can use it.
Third is the clueless corporate 'security personel' found in the USA mostly. Given a wide mandate, nearly unlimited violence authorization, and extremely vague guidelines about what to actually be looking out for, they are very arbitrary in what they conficate. They never destroy or resell what they take, and would never consider themselves thieves, only employees. They can best be thrwarted by having a senior corporate-executive set of clothes and personal appearance. Having a corporate logo on your media and equipment (not a brand name, but a marker that shows that the item is corporate property, not personal) often offers good protection. People in the USA are very hesitant to confront anyone whom they preceive as being a corporate superior to them, even when they have broad but poorly defined ability to do so.
So yes you are right about the USA, but I was referring primarily to the African model in my scenario. I had assumed that you would have been aware of that also. I don't need you to remind me about the stupid Americans, (nor do I need your assuming that I am one of them).
Thank you.
Sorry to imply that I mocking your vacation activies and memories. I simply wished to imply that I found the original poster a touch obsessive towards photography.
I've been many places, some boring and some more interesting. In the past 20 years, my city of a million people has hosted over 250000 people from other countries throughout the globe.
I don't 'go' anywhere anymore to be in a foreign country. I just sit at the bus stop and the world comes to me. The man on the end was a former South Vietnamese officer who sold guns and supplies to the Viet Cong and used the money to bring his family to the USA. The little Chinese woman sitting on the bench was a Red Guard when she was fifteen. Beating up her teachers for being 'capitalist running dogs' and giving too many pop Algebra quizes. The little girl by the newspaper stand is an Iraqi Christian whose entire family was blown up last year with a car bomb. Her mother's body absorbed all the shrapnel that would have killed her. The other woman on the bench is from a village in Central Mexico where live hasn't changed since before the Conquistadors. She doesn't speak English: or Spanish; only an ancient pre-Columbian Aztec language known only by people living within 50 km of the mountain that her village is on. How she got to 'El Norte', she doesn't remember.
Canon DSLR captures images on SD card. Plug camera into Macbook, import into Lightroom. Macbook's Time Machine makes wireless backups to the 500GB hard drive hidden in the closet (in case of burglary, but doesn't do anything for fire). I then purge all the lame pics (most of them) and then sync my Lightroom libraries across my iMac and my Win7 PC libraries for redundancy.
The great pics I upload to Flickr, and the rest I burn to a DVD and keep the DVD at work.
Not the greatest solution, but the key for my workflow is I didn't have to purchase anything in addition just for my photography. Everything is already being used for other purposes as well. People trying to price how much money per picture external storage would cost are leaving out the fact that most of us are already using that external storage anyway for other stuff on our computers.
Those are some interesting stories. One time, I actually got quasi-kidnapped by a Belizean taxi driver and forced to go to a Nicaraguan whorehouse owned by a Chinese man.
I'm not mocking you--that actually happened. Couldn't happen in the US, though. ;)
I did data recovery and data archiving a bit in the past, here are some things to be aware of:
:)
1. Optical discs will go bad over time, either the plastic will become opaque or they will oxidize between the layers, you can look up "bit rot" for more information but be warned that term has been abused a bit and applied to other things.
2. If you really want to use optical discs be aware that there is a huge difference in quality between different manufacturers. Also many name brands will use discs from different manufacturers so if you walk into an office supply store and buy the store brand discs they may have been manufactured in a different part of the world by a different company than the ones you got in the same store a week earlier. Some websites such as videohelp.com have done reviews of burned disc readability after time has passed, the short answer is for single layer dvdr discs use dvd+r discs made by Taiyo Yuden (now often sold under the JVC name). For dual-layer discs it's a little more muddied but Verbatim is usually a safe choice.
3. Data stored on flash memory (ssd or usb flash drives) will degrade over time, this is because the data is stored in cells that are either charged or discharged to represent a 1 or 0 (charged is zero for whatever reason). Over time electrons will escape these cells making it harder to distinguish between a one or zero, providing power to the drive will not recharge these cells, you have to actually re-write the data. It is not clear how much of an issue this is because the drives have not been around long enough, the estimates are anywhere from a couple years to a decade. I suspect there is wide variation between different qualities of flash memory since this is true of other reliability metrics. This isn't a huge deal for most people but I would say don't put the only copy of your documents on a usb flash drive, throw it in a drawer, and count on being able to read it perfectly in 20 years.
4. Fire safes are generally designed to protect against paper inside the safe igniting, optical discs and other forms of digital storage may be destroyed at far lower temperatures. I prefer safe deposit boxes at a bank. Obviously this is less convenient than being in your place of residence but they have the advantage of being more physically secure, climate controlled, and off-site. Prices and sizes available vary widely at different banks so call a couple in your area. I have seen as low as $20/year for a 3"x5"x36" in my area.
Summary advice:
If it's a small amount of personal data (tax documents, personal projects, emails) stick it in an encrypted archive if you care about it being encrypted (7zip is an easy to use, cross-platform, open source, well vetted option). Then put it on a couple different forms of media, such as a spinning magnetic drive in an anti-static bag, and an optical disc. Then store these off site somewhere such as at a friend or relative's house, or in a bank safe deposit box. And also stick a copy online somewhere such as on your google documents account or a dropbox account, this is an especially good option if you have encrypted it first.
If it's a large amount of data like full disk backups or a huge photo archive that are very important to you or your business stick it on a spinning magnetic hard drive, put it in an anti-static bag, and put it in a bank safe deposit box. Spinning magnetic drives are very stable if stored in a temperature/humidity controlled environment, more so than optical or flash memory. They are also still the king when it comes to dollar per megabyte (a good quality 2TB sata drive can be had for $80-$100 right now) and sata ports are likely to be common on all motherboards for at least another decade.
One last thing which will seem obvious, label every backup drive/disc/whatever, even if it's just a post-it note. You will not remember exactly what it is 5 years down the road.
happy archiving
Your "security personnel" are actually people working for the FBI, presumably also under orders of their boss to violate people's fourth amendment rights as set forth in your sacred constitution(tm), so the corruption goes all the way to the top. Although to throw a racist blanket statement like you do, that's the way it works in "Africa" as well isn't it.
American exceptionalism: we think we are exceptional! Sorry America, you're as shit as everyone else.
What time is it/will be over there? Check with my iPhone app!
I store a primary copy in a Pictures folder of an old computer that I keep running as a "file server". I keep a second copy on a NAS I keep in my basement. Regular pruning occurs off and on, and we may even go back several months or years ago to delete stuff we don't think needs to be kept any more. I use rsync in Cygwin to copy everything from the PC to the NAS, and I keep a backup folder with a date-time stamp of each rsync for everything that gets deleted/changed, and I keep more than 6 months worth of those change directories just in case we change or minds or it was an accident. To be especially safe, I regularly rsync a copy to a PC that my in-laws keep running, several states away from us. For this, I simply added a little USB powered disk to their PC, so I don't use up any of their PC storage. That way, if our house goes up in flames, taking our PC and NAS with it, we have a perfectly up to date off site copy of all of our precious memories.
I recently discovered Wuala where you can trade up to 100gb with "the cloud".
printf($randomline(sigs.txt) \n "-- "$randomline(authors.txt));
-- myself
There are plenty of easy ways to find the additional local capacity, but in terms of backups, IMHO any practice that requires you to manually perform tasks is setting you up for failure. You'll forget to put that backup disk at your mother in laws or you'll carry a few weeks of extra risk because you've been busy or any number of other reasons. And as for keeping backups at home, there's the risk of burglary, fire, flood, four horses of the apocalypse etc, etc.
There are some great online backup services these days that take care of the whole thing for you. Point it at your data, define a backup schedule and let it run. SugarSync gets some good feedback. Personally, I've found Mozy very good and for the sake of $5 per month for unlimited storage, I reckon it's a bargain. Here's my setup: http://troy.hn/bhP4F9
In terms of network and speed, even from Australia (typically slower connection to US based services), I pushed up over 100GB in about 4 days recently. A combination of fast, cheap bandwidth, unlimited storage and a reasonable rate of data collection makes this perfect for the scenario you describe.
Microsoft MVP - Developer Security
Suppose I have too much data for my internal drive, so I dump some of it to the external drive and remove it from the internal drive.
From this point until the external drive is backed up that a drive crash will result in lost information.
Using RAID1 for the primary external drive closes that window. The "backup" is another system (larger and preferably also RAID1) that contains periodic snapshots going back in time. (Ideally a combination of incremental and full.) That way both systems are protected against drive failure as well as accidental/malicious deletion of information.
The "backup" could be offsite, or there could be a third system (preferably also RAID1) at an offsite location.
Prints fade, have smaller dynamic range, and contain less information than the digital file.
Sure, having prints around is nice, but you want to keep the digital negatives as well.
Get two pogoplug devices each with say 2TB USB drives. Put one pogoplug in your house and put the other in a relatives house. If both pogoplug devices are registered on the same account, you can have the pogoplugs auto sync with one another. The pogoplug drive on the device in your house can be setup as a shared network drive. So just drag and drop your photos and the pogoplug will take care of auto syncing to the remote device.
First get a bigger HD for your laptop. Next, invest in your own backup solution, 1 or 2 TB external drive or RAID - preferred. Online backup is fine but I personally don't like having my pictures online. If you really want to save them for life start moving the more important ones to DVD. Mechanical or magnetic media can and will fail over time.
I use a mixed strategy for backup.
My regular machine is a laptop with 340GB hard drive, with a copy of this data. I have a 1TB USB hard drive (Seagate, bought at Walmart) where I copy everything to (backup 1).
I also have a web site, which I use as on-line storage. It is not the primary reason for getting the web site, but I do have "unlimited storage space" (my ISP is 1 & 1) and I access it via FTP. That costs me $7/month. So for that sum, I have a web site and on-line storage (backup 2).
I also have a desktop with a 1TB drive where I occasionally copy the data too (from the USB drive) (backup 3).
I find several advantages to that strategy. My data is always accessible from anywhere (through the ftp site) and having a local copy on the USB drive makes it easy to view the data from another machine if the need arises. The USB drive is faster (at least not slower) than most SAN solutions, at least those priced for the home market, at a fraction of the cost.
I also have a copy of everything on a desktop (500GB local hard drive) which I do not use very much, but that is nice to see pictures with because of the 22" HDTV monitor.
Finally, for the day-to-day documents that I work on, I use Dropbox. That allows me to work from different machines without having to carry a flash drive. Of course, I do not use Dropbox for long term storage, as the price is quite a bit higher than my ISP, so once documents are archived, I move them to the ftp site. I have a free 2GB Dropbox account, and I must admit that it works remarkably well.
I admit that this strategy would not scale well if I had 10 times as much data, particularly maintaining the backups up to date is a mostly manual solution at this time, so it only works for me because I do not edit older data. I simply add new documents to the repository. Unfortunately, I have not been able to find a cheap (I would like free, but I would not mind paying a reasonable price for it) incremental backup software that will work across all these storage methods. My ISP does not provide me with a shell account, so for automating, I am limited to what I can do via cgi or php scripts. At this point, I think I will have to write the software myself.
For offsite backup I recommend CrashPlan: $50/year, unlimited space for one computer (or you can mount network drives so it thinks it's one computer), or $100/year for 10 computers. Files you delete are never deleted from CrashPlan's online backups. Works on Linux, too.
You can also just use their software to backup on local devices or to friends'/family's computers, and the software itself is free.
Another nice thing about it is that it does integrity checking all throughout the process, and over time, so that data degradation can be avoided. rsync and rdiff-backup, duplicity, etc. are great, but they don't protect against media degradation.
"Those who consume the bulk of goods are those who make them. We must never forget this secret of our prosperity."
Around the house I have a personal desktop, a work laptop, and my wife's personal laptop, and wanted to keep them in sync, for some datasets (like home folders, photos, software repositories, etc). In addition, there is a backup external drive for the desktop where I want it to sync nightly, and various external drives I use personally (for myself and wife), which also stores backups of some locations. Occasionally I go visit my parents in another city and bring my personal external drive (750gb) so sync my photos into an external drive my dad has there, since he has enough space. This is my off-site storage. I did some testing with various sync programs, and found SyncToy didn't handle more than 1-to-1 sync gracefully. It definitely didn't handle 2-way sync between multiple computers well. rsync would work, but obviously not as non-UNIX user friendly. Finally settled on AllwaySync, which I've used now for about 3 years very successfully. I paid a 3 system license for it, and haven't regretted it. It has been updated scores of times since I bought it, and in fact handles a few online drop-box type systems. Additionally, it just started handling sync-to-zip type function, so I can sync my home folder to a home.zip file, and it will just find the differences and re-set the zip accordingly (which handles setting the password on the ZIP, so use that with a small USB pen i've got; not as good as if it did RAR, since the filenames etc aren't encrypted, but better than nothing). Oh ya, can also handle sync over ftp, for another offsite alternative. Also has a standalone app version you can run off of a USB in case you don't want to install the app on a system. Simple and graceful, that's what I use.
pretty easy
naked photos of the ex girlfriends get raided, stored in a secret gmail, and pushed to rsync.net in another continent.
the rest gets copied to a second hard drive every 6 months.
Seriously, it's the one format that you know will keep. Go buy a decent HP inkjet printer, if you use their premium photo paper and ink they are independently rated for 200+ years of archival life. Seriously, no digital format is going to hold up to that kind of life expectancy. I'm not saying you shouldn't keep them digitally, just realize that anyone "inheriting" your digital archive is a lot less interested in sorting through it than the physical copies you keep someplace safe.
I ask to a pro service for take a copy of my Hard disk. All my pictures became corrupted before they take de copy. Each picture looks broken, cuted, with colored bands crossing over de images....
what happened here?
How to repair it? (i only lost 23 pictures like this, i'd like to save)
cccccc
Damn, that's a useful device! Amazon carries this one and the slightly smaller QE4531 if you want faster shipping than the month wait on SentrySafe's website.
One thing I've found is magnetic media is not permanent and I've found on-line storage to be risky as I often hear someone complaining about photos they lost when some outfit closed. We really don't know how long optical media will last but *generally* it's a long time. So even with the most reputable there is at least some risk. HOWEVER using an on-line, reputable back up might be best for a small data set like yours. I've had photos stolen even from the chains in the old days so I'm not real fond of on-line or the so-called cloud. You do have every thing on your computer backed up? Right? (I'd bet against it) You can use DVDs, but they do require care in storage, but are more likely to protect your data integrity. Keep two identical sets and if you view the backups often keep another set for viewing. People who keep small data sets (photos) on their computer, or even locally on separate media inevitably end up losing photos unless they develop a very good naming convention AND indexing system. I have over 35,000 scans from the film era, and several times that shot in digital. The total represents nearly 60 years and two generations of photography. I keep them on a computer, backed up to a second computer across the network and on DVDs. Actually there are 5 computers that back up each other across a network. I find as I go back though the older images on disk I have to refresh about 5 a month although I may go for months at a time without having to refresh any. JPGs are the format that usually ends up corrupted. I don't recall having lost any tiffs. So it becomes a problem of choice. All forms of storage for the individual carry some element of risk. Few back up as they should, then complain when the HD fails. HDs don't often fail, but IIRC I've lost 5 over the years. However data on HDs does become corrupt far more often although most of the time it's the operator who corrupts the data due to mistakes. So.. 1. Develop a good naming system 2. develop a good back up system with at least two copies kept in different location. 3. Choose a good archiving method either personal or on-line that is different than your back-up.