Slashdot Mirror
Ask Slashdot: What To Do After Digitizing VHS Tapes?
An anonymous reader writes Now that I've spent close to a month digitizing a desk drawer's worth of VHS tapes, deinterlacing and postprocessing the originals to minimize years of tape decay, and compressing everything down to H.264, I've found myself with a hard drive full of loosely organized videos. They'll get picked up by my existing monthly backup, but I feel like I haven't gained much in the way of redundancy, as I thought I would. Instead of having tapes slowly degrade, I'm now open to losing entire movies at once, should both of my drives go bad. Does anyone maintain a library, and if so, what would they recommend? Is having them duplicated on two drives (one of which is spun down for all but one day of the month) a good-enough long term strategy? Should I look into additionally backing up to optical discs or flash drives, building out a better (RAIDed) backup machine, or even keeping the original tapes around despite them having been digitized?
Upload them to youtube! The internet never forgets, apparently.
Keep a copy in a different building to protect against fire/flood/theft/etc.
In other words: throw lots of money at the problem regardless of whether or not the solution is even vaguely appropriate.
A Pirate and a Puritan look the same on a balance sheet.
If you're concerned with keeping them for a long long time, consider using flash, magnetic, and optical storage. Blue-Ray burners are pretty cheap now days, as are big multiple terabyte drives.
Don't see the point. You can get jerry maguire on blueray.
Not sure what you should do about local backups, but I think you should consider off-site storage in additional to whatever local backup plan you come up with. If you trust cloud services (or the videos are not too sensitive), you can sign up for something like CrashPlan which offers unlimited backup to the cloud. It is a pain transferring that much data back and forth from the internet, but the idea here would be that the cloud backup is your "last resort" in case something happened locally.
Enough with backups. Now for the fun part: set up a Plex server and have it catalog your videos. Then, stream them to any device that you own. Instant enjoyment and easy of access for all those old videos.
... Tape to Tape
If you're looking purely at longevity of storage and reliability, Tape backup is still the way to go.
Baring that, spread backup copies around, and make sure to keep redundant copies on several flash drives?
This space for rent. All reasonable inquiries will be entertained at proprietors discretion.
Put them on YouTube, DropBox and so on.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
Literally anything can get posted as an Ask Slashdot these days. This one isn't as bad as some, but when did Ask Slashdot become a substitute for a basic Google search. It's not like you're not gonna get zero hits for "long term video tape backup"
I archive my home movies in three ways:
This way if anything were to happen to one copy, there's always other locations to get a copy of it.
You're going to hear this more than once, so I'll get it out of the way for my part: Raid is not a back up.
don't know how much space you're talking about but I have a local backup and then an offsite backup. If the whole city gets taken out, I'll lose my data but I have other problems that concern me more.
It's trivial and native in most modern Linux distros.
That said, data redundancy is not a backup policy. Even your current solution of mirroring on a monthly basis is not a real backup. What if somebody stole your computer? Where would you be, then?
There are a lot of backup-to-cloud solutions, but I, personally, can't sleep at night knowing that my data is just bouncing around out there, somewhere, on somebody's servers, (allegedly) encrypted or otherwise, and is probably available to me, assuming I don't mind the download time. No, get an external hard drive, and store it offsite: take it to work; stick it in a storage locker or deposit box; something. That protects you from theft, fire, and most natural disasters, without opening you up to data spillage that might compromise more than your home movies.
For under $1k, you can get a 6-drive multi terabyte fileserver from Netgear. Hard drives are cheap enough that you could also do off site backup with a relative.
optical discs
No. Optical is shit, and will degrade long before the purported lifespan given by manufacturers. No, spending money will not help here.
flash drives
Better option.
building out a better (RAIDed) backup machine
An even better option.
So, uh, what happens if your house burns down? Had to ask, because you're clearly not serious (in terms of angry storage nerd serious) about your data. ;)
In truth, you're probably fine simply by keeping duplicate copies on separate devices (eg, your oft-powered-down hard drive). I would suggest looking into flash storage though, if only because it's more reliable than a normal hard drive (assuming you're not writing like craaaaazy) and tends to be a hell of a lot smaller. Physical storage footprint can be important, too!
Originals (non-recompressed): Outside the house and inside the hose in some HDDs.
Finals (h.264): Outside the house, inside the house and on your local server.
Always 3 copies of anything "actual"
Then the roaches can watch them after WWIII.
Some later minidisc players had a data storage option, in my experience mini-disks are easier to get hold in quantity than other MO discs, and they look cute. You're probably more likely to find an antique minidisc player in 50 years too.
A 64GB flash drive is under $25 on Amazon. You can copy the files on to one and leave it at a relative's house.
Depending on size, you could look into storing online. Amazon Glacier can act as a low-cost long-term storage @ 1 cent/GB.
Alternatively, grab 1-2 externals and rotate off-site. Would allow controlled consistent cost, but could be subject loss based on update/rotation. External Raid1 device could mitigate the chance but are more costly.
Could roll something like a WD my cloud, and have it sync to a raid 1 fixed NAS.
http://www.wdc.com/en/products/products.aspx?id=1180
Lasts thousands of years, just ask the egyptians. Or recognize that it is not worth the expense.
excitingthingstodo.blogspot.com
When my son were little I used to shoot video of them as toddlers, growing up.. building snow forts, sledding, soccer.about 15 years on tape. from that point I was an early adopter of digital video back when the 1348 bus came out. All my family video is stored on a Plex server. Plex works well.
I also have the data backed up 2 different ways. Once to a ESATA drive. I use the black widow ESATA dock. Second way is with a usb western digital passport drive.
If they are naked vids of you and your GF, the Internet will gladly archive millions of copies.
Just uplad everything, and let the netowrk be your backup.
-><- no
Two cloned drives. One for one site (i.e. in your house, readily available). One off-site (anywhere - safe deposit box, parents house, you name it).
I had a similar problem as you and I found what I consider the best solution for home users. There are many options where you can store you data in the cloud for a yearly fee but it does get expensive as you pass the 200GB mark. What I use is Crashplan. Their software allows you to backup to their cloud or to a friends storage. So if you know someone else with the same problem you can have them allocate some space on their machine for you and you do the same for them. The backups are protected so he won't be able to play with your data. This is all at no charge.
I personally use the Enterprise version which allows you to host your own server. I have my reasons for doing this but most users will be happy with the free home version.
After the digital backup, I then transferred them to film.
Then, I put filmed video onto a reel of similar films.
Done!
Then, I rented a salt mine in Utah to store those films.
Yes, I really love my porn collection.
Exactly. That's what I do and eventually it works.
Analog Data fairs better when it isn't touched, every read could damage it a bit, and copying a copy of a copy using analog methods will degrade its content.
For digital data, it wants to be moved around.
The more you copy and move digital data the better it is.
Raid Disks makes sure you have a couple of copies.
You post it the cloud and it will last longer.
You could get fancy and have a backup method that copies your data from one drive to an other. When it fails you swap the drive out with a new one.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
You didn't say how much space was needed, but many people have next to nothing on their machines. Copy whatever is appropriate onto as many family/friends machines as possible. Also 1TB drives are very cheap, store copies offsite with those.
Whatever you do, keep it as simple as possible (e.g. no RAID). Reevaluate things every 5-10 years to be sure whatever medium you use isn't becoming obsolete/unreadable.
My pics, digitized from family photos and digital camera sources are stored on my mac with time machine backup, also synched to my NAS which in itself a RAID with a back up NAS which is also a RAID synced with the first NAS and finally they are all offsite synched with an encrypted back up service. In the middle somewhere (between mac and NAS), I have a manually activated sync that I run once in a while which I can inspect the result. This avoids the human error of accidental deletion trickling through the system (been there done that). What are the chances I would lose em now? In short, I wouldn't call RAID a backup, but a redundancy. Can still loose all the RAID or house burn down or the NAS is stolen. The offsite and/or duplicate NAS is a true back up. The trick is to have it always live and moving. Sitting in hardware on a shelf never used or disks that can degrade will slap you in the face later when you really need to recover from it (been there done that).
Git Annex ( http://git-annex.branchable.co... ) (if you're a geek) is the perfect answer to maintaining multiple copies of digital data.
I have two external disks on alternating cadences of backup. At any given time, one or both of them are in a desk drawer at work (while I work, I keep both there, and take home the one that needs to be run that night).
Cloud for me is impractical as the price structure is pretty steep at these capacities. Even if it wasn't, my bandwidth is inadequate for the task. Offsite backup to my desk drawer is adequate.
You can encrypt the backups if you are concerned about the privacy of such a setup (the desk drawer locks, but the employer has keys).
XML is like violence. If it doesn't solve the problem, use more.
Better to keep a backup on Earth and on Mars, just in case.
Get free satoshi (Bitcoin) and Dogecoins
Would this not be a pretty good use of Amazon Glacier? Tape like backup without the tape like hardware? The billing model is a bit confusing but it's essentially $0.01/gb/month until you need it back and then you pay to retrieve.
Amazon Glacier is for long term storage of lots of data. While they only provide an API, it is easier to code to and 3rd party interfaces exist should you want a GUI.
It is also dirt cheap. My current bill is less than $1 per month. You'll pay more to access the data should you need to but storage is priced reasonably.
tape is safe.
oh wait... nevemind
ZFS is designed to combat bit rot on media. FreeNAS will do the trick
Well, here is what I do for photos:
1 local 'main' copy stored locally on RAID storage (RAID15/6, whatever, just have some redundancy)
1 Local CrashPlan copy to portable drive that gets run occasionally, then stored offsite in a safe deposit box at the bank
1 Remote CrashPlan copy to the Cloud (Or use the friend option if you want, not practical for me since I'm doing TB+)
1 Additional local copy stored on UnRAID for quick recovery access if the above have issues etc.
But, one thing you also have to consider is possible loss/corruption of data at the file level. Do you store MD5's, SHA1's or such of each file to see if they changed? Do you KNOW your files are intact?
My rule of thumb on this is that anything you truly want to keep should be on at least three devices, kept in at least two places.
My personal system has a USB drive hanging off a Mac Mini as the master. A second, larger capacity USB drive maintains a Time Machine backup of the master. My offsite is a remote machine that I can VPN to from the master, and I user CronoSync to periodically copy the master over. If either drive on the master fails, I'll pretty much know the same day, and I'll simply walk over to Best Buy to buy a new USB hard drive and recover the failed drive (this has happened once). Both USB drives failing at the same time is highly unlikely - it would take a disaster (hurricane, house burned down, meteor, etc.) Which could happen, but unless the remote drive ALSO failed at the same time, I'm still covered. And since I'm using USB drives, I have zero issues using heterogenous hardware, so I'm not locked into a One True Vendor approach (Master is FAT32 formatted in the highly unlikely case I ever want to switch to Windows....)
Do NOT use RAID for this! Yes, RAID can protect you from drive failure. But it CAN'T protect you from inadvertent deletes - if you tell a RAID1 set to delete a file, it's gone from both disks. RAID is NOT a backup solution!
LTO5 it's the most reliable long term backup solution for your converted videos. (yes tape still kick ass). About cloud sites, think about some wwiii in the future.
http://en.wikipedia.org/wiki/Archival_Disc
If you really want to save your data:
Step 1: make a ZFS array and save your data there.
Step 2: copy the data to single hard-drives and store them in a different location then home.
Step 3: upload a copy to some online 'cloud storage' provider.
Use checksums/md5 hashes to determine data integrity.
Based on your budget pick any of the above 3. If you are paranoid, do all 3.
"The price good men pay for indifference to public affairs is to be ruled by evil men." ~Plato (427-347 BC)
What To Do After Digitizing VHS Tapes?
Might as well just lie down and wait for death. You're never going to top this!
Alternative pop culture reference:
There's plenty of things you can do to pass the time: hitch up your pants, air whittle, make friends with a Chinese man...
systemd is Roko's Basilisk.
I copy the pictures/videos from our phones/cameras/etc. to a drive at home. I then use a portable drive to take them to another location. At the other location I transfer them to an external hard drive (I keep everything in folders by year/month/event such as 1995/August/Yellowstone). Every night this drive is backed up using Carbon Copy Clone to another drive. This drive is also backed up to a third drive with Time Machine. The drive is also backed up to CrashPlan each day. I also sort out the best pictures from each month to a fourth drive which is backed up to MediaFire. So, in the end most of our photos/videos end up on about seven drives in four separate locations. Most of it happens automatically. Currently I have nearly 1TB of data backed up in this manner. My next step is to start burning Blu-Rays and putting the backups into a safety deposit box.
stop asking stupid questions about vhs to /. every few days
that works... rsync to offsite nas or friends server. Possibly even bittorrent sync
Silence is a state of mime.
I mean really, are you ever going to want to crank up cousin bettys 8th birthday party from 1992?
Nobody EVER watches their wedding video, their birth video, their kids birthday videos, that school band recital.
Nobody ever watches those ever again. Stop kidding yourself.
Just throw away the damn tapes.
Also all those old newspapers.
In addition to my "online" storage pools, I keep a three-way ZFS mirror on encrypted hard drives of everything that is irreplacable. If you have more than fits on one disk, maybe use a 1+0 or 1+5, even, setup. This story does depend upon you being able to connect two complete copies of your backup to one machine at a time, so again, if it's really a lot of data, external RAID enclosures may be in your future. In any case, one of the full copies lives off-site and every month I grab one of the two at home, swap it with the off-site one, allow it to come up to date with the third, and then push that month's backups to both. Rinse, repeat, like clockwork.
Anarchy$ dd if=/dev/random of=~/.signature bs=120 count=1
A few options for backup (I used all of these for my collection):
* Buy a bunch of USB external HD's, put the movies on them, and send them off to various members of your family (they'll love to have them too)!
* Upload everything to a Private YouTube account, then give your family access to them
* Backup to a Hard Drive, and place it offsite
* Carbonite or Dropbox
So my Ask Slashdot post about "Should I have bacon or sausage for breakfast" is rejected, but this gets through?
External hard drives are cheap. I Have one in the safe deposit box at the bank, one at home, and I rotate them every couple of months. I just do a copy of /Users (on my Mac, /home on Linux, not sure what the WIndows equivalent is) to the hard drive.
Best Slashdot Co
analog media is cleverly resistant to flaky format changes, bit rot, drive death, one too many cycles of use on a flash drive, and system obsolence. as long as there are any tinkerers around, there is a way to pull off one more pass of an analog media item.
if this is supposed to be a new economy, how come they still want my old fashioned money?
You will never watch them again anyway. Most meticulously saved data is never accessed again.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
just don't drop the box...
2 servers are set up raidz2 with 4 disks per server. So about 6-7 TiB of actual storage space.
The servers do dns, mysql, and smb via plugins and a jail.
the primary backs up to the secondary every evening.
All the TVs in the house are really xbmc clients connecting to the SMB shares and mysql.
The most expensive part of it is the 8ea 4T HDDs. ;).
Unless you have 10 people in your house watching different TVs at the same time, you can use real low end computers.
disks are $150 ($120 if you get externals on sale from huevonuevo & open the box). Excellent computer for this is a Dell poweredge T20 ($300).
These T20s have ECC RAM (you want this)
Anyhow 8*150 + 2*300 + a hundred bucks for misc. cables, bootable memory stick, maybe a switch...
Under 2 grand for the whole mess. Put one in your basement and one in your attic. Then you are protected from a flood or a tornado--but not both together.
If your house burns down, though, you're hosed
Upgrade plan is to "destroy" (that's the command...) the zpool in the secondary then change it from raidz2 with 4 disks to raidz2 with 6 disks.
let rsync do its thing, then swap the usb keys with the embedded OS.
Repeat with the old primary which will now be the secondary.
Already tried this once; works no problem. At any given moment I'm tolerant to at least 2 disk failures.
"Reality is that which, when you stop believing in it, doesn't go away." - Philip K. Dick
I have a set of digitized film videos of my family growing up. I burned copies on BluRay disks, have copies on 2 hard disks, and have them backed up on the cloud as part of a unlimited cloud backup subscription. Also, my dad and brothers also have copies.
Is there a better solution to lots of large backups?
There are two types of people in the world: Those who crave closure
Try to understand the impermanence of data and let them go. You are just going to torrent the blue ray rips next time you want to watch them anyways.
Take any of them that contains nude selfies and upload them to the cloud unencrypted then take the rest encrypt with RSA 8912 and upload them to the cloud.
If the tapes are home videos or if you value the "off the air" artifacts like commercials, keep the tapes as long as you keep the copies. If possible keep the tapes or at least one copy offsite in a climate-controlled place.
If it's old TV shows and you just made copies so you wouldn't have to pay $BIG_MONEY for DVDs then junk the tapes.
I'm facing a similar situation but without the tapes: RetroTV is airing almost all existing "classic" Doctor Who. Deciding how many copies to keep and in what format is a chore. Unlike you I'm not too concerned about offsite backup - I hear that in a few years these will all be available for purchase in BBC-authorized versions. In the meantime I'm sure thousands of fellow Whovians are recording these episodes to their own personal recording devices.
Edit them together, dude. Into multiple iMovies ;-)
Is having them duplicated on two drives (one of which is spun down for all but one day of the month) a good-enough long term strategy?
I am going to say no. Nothing says that spun down drive won't go dead and if it is not a complete mirror of everything on the active storage set then you're still prone to losing data.
Should I look into additionally backing up to optical discs or flash drives, building out a better (RAIDed) backup machine, or even keeping the original tapes around despite them having been digitized?
The original tapes will do you no good. Consumer grade VHS tapes will begin to physically degrade in 20-30 years unless stored in an envirnment like the bottom of a an abondoned salt mine. What happens is the magnetic material on the plastic tape will start to delaminate and fleck off over time. I worked digitizing video tapes for years and have had to clean machines and get heads replaced because people handed us VHS tapes that were almost 20 years old and the stuff came off all over inside the machine or scratched the hell out of the heads.
I would suggest to use a Blu-ray writer if you can (~50GB per disc) or a tape drive if you have the need and can afford it. Buy brand name discs (or tapes) from major manufacturers. Don't skimp on your backup data inegrity! I used CDs and DVDs to backup to for years and can still pull data off ones that are going on 20 years old without trouble. Store them in a cabinet somewhere out of the light and don't leave them exposed to fluorescent or sun light for any length of time and they should last. It is also important that the cabinet not be somewhere that isn't environmentally controlled to within the storage specifications for the media. If you like to keep your house outside that range store them in a bank vault security deposit box. Better that they're offsite anyway.
Write the files to a tape drive.
Any medium will ultimately fail, over long enough spans of time.
Further, just the transcribing process itself has chances of introducing errors.
Personally:
- back them up to the cloud. That's about the closest thing you're going to get to "permanent" storage, as you're outsourcing your (individual) chance of hardware failure to some online entity that (at least allegedly) backs up things redundantly across multiple methods, and/or
- just stop being OCD about it. At a certain point, trying to 'preserve' things forever just becomes silly. If you have the only unique recording of some substantial historical event, that's one thing. If it's your child's first steps, understand that while that might be important to you and maybe even to them, nobody else cares about it. Really. While losing it would be sad, it wouldn't be tragic. After all, there are billions of person-years of lives that have vanished, unrecorded, and life goes on.
-Styopa
For pro video editing - which is to say lots of content that frequently changes - tape backup still makes sense. There's still no better way to archive large amounts of data, although 2.5 TB (IIRC) tape size for the latest LTO has fallen behind, and the next gen isn't due for likely a year.
But the one-time cost for tape drives is pretty steep. If you're going to use many tapes each month, it's worth it. Heck, I'd say even at 10 TB of new data a month being archived, there's no better way. But for, say, 10 TB of fixed data that just needs to be archived once, it's overkill.
Buy a few HDDs, keep their shipping containers, make a backup and ship them to a friend in a different state. Repeat yearly. That's the economical way. Eventually it will all fit on a single drive, after all (aren't there leading edge 10 TB drives already?), and so you're looking at ~$10/month long term (heck, no matter how much fixed data, eventually it will fit on one drive and cost about that much).
Socialism: a lie told by totalitarians and believed by fools.
You say that, but how many analog VHS tapes have we been able to read from ancient Babylon?
Is it just my observation, or are there way too many stupid people in the world?
MD5 sum all your video files, saving the checksum of known-good files. The two sets of checksums in your master and backup should match, and re-calculating the checksums should always match. If checksums fail on the backup or master drive, then replace it - especially the master, because if you have a damaged master and subsequently run a backup, then both copies will be corrupt.
I store all mine on a dedicated home NAS. Raid 5+1, over 4 2tb drives. That way I have drive failure redundancy. Anything truly important I store backups on an additional external drive that can be grabbed quickly in an emergency.
The more the better. Just check what you're going to give them *before*, thank you :-)
I had bought a digital video camera long ago when they first started recording digitally. I felt compelled to rip the videos to hard drive for "permanent storage"... But sooner or later I realized that the best way to store this stuff for long term chance of success is actually back on the tapes. I suggest you now take your digitized videos, borrow a camera (if you don't have one) from a buddy that has a camera that stores miniDV, and store those videos on tape. Then put em in the closet - so if your hard drives ever fail (which they will some day), you'll have the tapes with which to recover the memories from the good 'old days. I had thought about going to DVD, bluray, or whatever, but when you think about it - it's really cold storage - the best device suited to the task is tape.
I keep a working copy on my NAS, but I back them on on Blu-Ray discs now. I end up burning two copies, and keep one in my desk at work.
I also generate a SHA1 hash file on the discs, to make sure the integrity of all those bits. I also try to check every 5 years or so to make sure the media is still good.
--
SLam
Stone tablets from Babylon are still readable because analog degrades gracefully.
Want real long-term storage? Write them to analog tape again.
Metal punch tape - chosen by the military to survive nuclear apocalypse and still be human readable without tools. Much denser than cuneiform, too.
Socialism: a lie told by totalitarians and believed by fools.
unlimited backups for your videos and other stuff. just cough up the money, its a few bucks a month. if its not worth $50-$100 a year to save them along with all your other data, is it really worth saving them?
At any time, a proper setup involves maintaining a minimum of three copies of any important data:
1) The copy you use.
2) Your local backup.
3) Your off-site backup.
How you choose to implement those can vary. For instance, if you have the cash, I think most of us would agree that maintaining separate RAID arrays for your in-use and local backups would be ideal. The reason you'd keep them separate is because of the all-important mantra: RAID is not the same as having a backup (you don't seem to be under this misconception, but it bears repeating, nonetheless). RAID can protect against certain forms of hard drive failure, meaning that you wouldn't even need to resort to using your backups in the case of those sorts of failure, but it does nothing to protect against your data being corrupted by the file system or deleted by an accidental action on your part.
If you don't have the money for RAID, you could start out by just putting your in-use and local backup copies on separate hard drives (which it sounds like you're already doing), the first of which backs up to the second. That'll work most of the time and in most cases, but it means that hard drive failures will be more of a threat and an inconvenience, since you'll have to be more reliant on your other copies being intact, given that you'll be suspending your use of the damaged copy while you replace the drive and restore the data to it.
In addition to your local copies, you should have an off-site backup in a location that is geographically removed from you, that way if natural disaster does its worst, you don't lose your data. CrashPlan is the one I use and is a good place to start, since it offers multiple options for backing up off-site, including a free option where you and a friend provide off-site backups for each other. Their for-pay options are reasonable in price (though they have more than doubled since I joined a few years back), offer unlimited storage, and provide the ability to set your own encryption key (i.e. keeps them from being able to pry into your data if they're served with a warrant).
So, at a minimum: a drive for your in-use copy, a drive for a local backup, and CrashPlan backups to a friend, all of which would only cost you as much as the hard drives involved.
Ideally, however, you'd also do something to protect against corrupted data or accidental deletions on your part, which means storing multiple versions of your backups, and doing so both locally and off-site. CrashPlan subscriptions all provide full versioning of anything you backup in perpetuity, so if your data becomes backed up in an incorrect state, you can rollback to a previous version easily. Even so, you should still have versioning stored locally in some form or fashion, that way you're not dependent on CrashPlan always being around and always working. If you're a Mac user, Time Machine can serve this purpose (it should be in addition to any other local backups mentioned above), and you can even backup your Time Machine data off-site if the off-site backup system you choose doesn't offer built-in versioning like CrashPlan does. I'm sure others can make some recommendations for Windows and Linux alternatives to Time Machine.
And yes, you should keep the tapes around, if only so that you can demonstrate ownership should any legal questions come up. But once you verify that the copies you've made are all correct and working, you can probably box them up and put them in an out-of-the-way spot in the attic where you'll never have to bother with them again.
So, I did this with all of my movies, VHS, DVD, BluRay ect.. I ended up with a very large library of video on my computer. In my computer I used an Adaptec 5805 RAID controller with backup battery and 4 3TB WD Red HDD's to store all the data in a RAID 5. I am also running a PLEX Server to organize and add meta data to all of the video files as well as serve them to the HTPC and Roku 3 in my house. As for backup I purchased an account with Crash Plan.
With all of this I get a nice organized library for all my video files with meta data. The ability to stream them around my house and to my phone on the road. Data redundancy both local via the RAID 5 and remotely through crash plan.
The initial backup took a really long time. (Almost 2 weeks.) Even with my 25mbps up stream so you may want to pop for the seeded backup option if your internet isn't very fast but it is limited to 1TB so it wouldn't have saved me too much time. I've been satisfied so far but I've been lucky and haven't needed the backup yet. Though I guess it's like insurance I pay for it but I don't ever want to use it.
You imply a backup but your current setup does not provide it, Peter Krogh had is succinctly in his 3-2-1 rule see: http://dpbestflow.org/node/262... at bottom
In summary, as well as your two local copies you need an offsite backup, possibly from a trustworthy cloud vendor. ;-)
This all depends though on if the vital media is really worth preserving. If they really are, historic documents that should be preserved for all time you should think about investing in some analog archive storage, as well as the digital to forgo the risks of technology drift overcoming your ability to update the format as new systems replace old formats i,e. Some archive quality 35mm B&W colour separation movie film with integral optical sound recording. Thus can be expensive though for your average family movie, but just think what it will be like in a millenium when yours is the only home movie left in existence
The 3-2-1 Rule
The simplest way to remember how to back up your images (ed: or any media) safely is to use the 3-2-1 rule.
We recommend keeping 3 copies of any important file (a primary and two backups)
We recommend having the files on 2 different media types (such as hard drive and optical media), to protect against different types of hazards.*
1 copy should be stored offsite (or at least offline).
*While 3-2-1 storage is the ideal arrangement, it's not always possible. A second media type, for instance, is impractical for many people in the ingestion or working file stage. In these cases, many people make do with hard-drive-only copies of their data. Best practices, however, still require 3 copies and some physical separation between the copies.
I would argue that the raid is useless. Better to use the excess drive capacity for rsnapshot external with off site backup.
If theft or fire takes out your place, then that data is safe. Such an event would still be traumatic, but at least the data would be intact.
XML is like violence. If it doesn't solve the problem, use more.
You are interested in long-term reliable storage of this data, no?
After digitizing your videos, use uuencode and print the resulting files on acid-free paper.
I also digitized our family VHS tapes and other old stuff.
I keep one copy on my home computer/server for watching and using.
I keep one copy on a USB hard drive in a fire-proof safe in my basement.
I keep one copy on a USB drive in the trunk of my car.
And I copied the data to my dad and siblings hard drives too.
https://www.facebook.com/digitizeicm -- Show your support for the digitization of the Iron County Miner newspaper archiv
You know you never watch those old wedding/graduation/bar-mitzvah/ritual-sacrifice tapes anyhow. Live in the now. Your brain is the best camcorder.
I store my offsite backups at my office. To do this effectively, I use three backup HDDs. One sits in a SATA dock at home, and mirrors my data every hour. The other two are at the office. Every so often, I take one of the HDDs home and stick it in the dock so that it updates to the latest version of my data, then I bring it back to the office. Next time, I take the other HDD home. This ensures that one of the HDDs is always offsite, and all three of the HDDs are never in the same place.
The obvious downside to this is that I have to remember to carry my HDDs back and forth. I haven't done it for a few months now. I suppose that an automated and encrypted rsync solution would be superior, but I honestly don't really care about my data very much.
The things I care about a lot I copy to a hard drive once a year or so and place them in my safety deposit box at the bank. Some things I also re-copy to DVD's.
It's affordable and easy to access long-term storage. It's also about as fire-proof as you can get, and I suspect that it would be about as EMP proof as you can reasonably expect outside of a mountain. There is near zero chance of flooding where my bank is, but that may be a concern in other locations.
For my office data, I have an external HDD that uses rsnapshot to create incremental snapshots every hour, day, week, and month. The server data is also mirrored to each desktop in the office, and my laptop, daily. For offisite backups (other than my laptop), I use duplicity to backup to Amazon S3, which costs about $3 per month. I realize that there are some security issues with this setup.
snapraid will verify your data against checksums and fix it if it detects errors. Combine this with multiple copies to avoid loss and bit rot. How you find what you want to watch is a completely separate issue. I'd suggest throwing out 90% and keeping only the best 10% to make your life easier.
I assume these are all hardcore porn or snuff flicks or something else worth digitizing. And not shit 80s movies or your daughter's dance recital.
Copyright (c) 1990 - 2014 Dice. All rights reserved. Use of this comment is subject to certain Terms and Conditions.
Store them on dvc tapes.
The underlying question has absolutely nothing to do with video, digitized from VHS or not. The question is, "How can I securely back up a shitload of data that I don't want to lose?"
Now that you've forgotten about the video aspect and just think of it as bytes, the problem is reduced to one that's been solved a zillion times over. Google "offsite backup".
Personally, I use RAID on my home NAS, and rsync the really important stuff daily to an encrypted 1.5TB drive sitting on my desk at work. If you don't have the bandwidth you could do essentially the same thing by carrying your external backup drive back and forth to your office (or friend's house, or safe-deposit box) weekly or monthly. Have a couple backup drives and just rotate them.
Chelloveck
I give up on debugging. From now on, SIGSEGV is a feature.
Burn them to M-Disc. As long as there is a DVD player somewhere, no worries. M-Disc doesn't degrade like magnetic media or dye-based optical media.
I once had a RAID5 array with 4 disks on my home computer. Two disks were connected to the motherboard, and two were connected to a SATA PCI card because the MoBo didn't have enough SATA slots. One day, the PCI card had a little hiccup, and two of the 4 disks got out of sync. The array was toast. Note that my RAID5 array contributed to this failure -- it would not have happened if I had not been running RAID (and if I hadn't made a poor configuration choice). Fortunately, I had a backup.
RAID is great for protecting mission-critical systems from HDD failure when uptime is a major concern -- but it can also cause more problems than it solves. Now, my business server uses RAID but my home computer does not.
This maybe too late, but here are the (finnish?) rules of homeporn:
1) Don't make homeporn
2) If you make homeporn keep it hidden
3) If you can't keep it hidden, don't digitize it
4) If you have to digitize it, don' take it to work
5) If you have to take it to work, don't fucking put in a public share
The only problem with punch tapes is that we've ALREADY lost the readers for them, without even having a minor apocalypse. I guess if people kept backing up to punchtape though, we'd still have workable units that could plug into this decade's computing devices.
The guy said he had some old VHS tapes lying around.
He didn't say anything about having any delusions about being the owner or even an employee of the likes of D2 or ILM.
A Pirate and a Puritan look the same on a balance sheet.
...you can never be too sure, lest they may survive an erase of the digital remastered copy.
So he should use tape to back up his tape? (Yo dawg...)
If your files are ever deleted, you can just use 'undelete'.
I wish Slashdot would check the last decade of Ask Slashdot to see if their question has already been answered two or three times over.
(challenge: ruined)
Sadly backups aren't one-size-fits-all as the right solution for one person may not right for another. But here are some of my experiences which may help in deciding what to do: Backup up to optical media is okay, but after a few years the discs may be very hard to read and some data loss may occur. I don't use optical media for backups anymore. A RAID array will protect you from a single drive (dual if RAID 6) drive failure, but won't protect you from accidents, viruses, fire, theft, or malicious intents of others. A local offline backup will protect you from drive failure, viruses and accidents, but may not protect you from fire, theft, or malicious intents and won't have the absolute latest edition of the data. A remote backup will protect you from accidents, fire, and theft but not necessarily from malicious intents. A remote offline backup will protect you from a lot, but the data may not always be up-to-date. The cloud can be expensive, unreliable, or unsafe. (And for those who quote flat-rate unlimited backup offerings, not everyone can make use of those offerings otherwise the solution wouldn't make fiscal sense. You also don't know how well they will protect your data and Murphy knows that your local copy will die around the same time that your 99.99% reliable cloud provider loses half of your data.) (Note that I consider 'offline' not just to be unmounted, but actually physically disconnected from everything.) Personally, I have a RAID array for local safekeeping against hardware failures and I have and offline storage system at a relative's house which I update a few times a year to protect against environmental, accidental, virus, and other problems. For really important things I get, between when I put on the main array and when I do my offline Sync, I'll typically keep the original media around, just in case. There are some holes in my personal backup system, but it's good enough for my needs and doesn't cost that much.
Buy a few HDDs, keep their shipping containers, make a backup and ship them to a friend in a different state.
Or just put them in a safety deposit box at a local bank.
It doesn't mean much now, it's built for the future.
They're both still vulnerable to supernovae. You should have at least one backup in another galaxy.
Fun fact, the real reason for the Voyager mission was someone wanted a permanent backup of William Shatner singing "Lucy in the Sky With Diamonds". You didn't ever see the back of that record they included with Voyager, did you... now you know why.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
As the original aim was to record the memories of the event, the best back up is to create new memories as soon as possible. Sit your kids in front of the monitor and subject ^H^H^H^H^H^H^H show them the videos. Provide plenty of glucose and caffeine so that they back up the memories as accurately as possible. Verbal annotations will help too.
If you want extra back-up diversity, invite members of your close family, and then extended family, work, church, sewing club, etc., to multiple showings. Produce fill-in-the-blanks question sheets about key events and run competitions to spot hard-to-see or difficult-to-catch occurrences. This will ensure multiple imprints of the video material in the neuro-cellular structures of the viewer's heads. Indeed, additional augmented memories may be generated by this whole exercise that can be shared later. And if you record those events on up-to-date video recording media, you will have a useful meta-recording to digest and disseminate further!
Analog to digital. But yeah Don't let that get in the way of a brainless meme.
I use M-Disk and keep an archive copy here at home and one in a safe deposit box. They don't lose data over time and are really tough. I keep my genealogy records on them too. I also have a cloud backup but it sounds like you have too many movies for that to be practical.
For me... I've converted all my AV contents to mp4 format and have them automatically backed up to iCloud....
I do have a decent amount of storage on my reasonably priced service.... I feel confident that my contents are safe with Apple.
.is it on Dropbox, Apple's iCloud, company XYZ's cloud, or ???
Yes. I hope.
Just buy a bunch of appropriately sized external drives, set them up with encrypted file systems (and make sure whoever may need it has the password), copy all videos to each drive, and get some friends and family to store them for you. Put all your other important content such as family photos etc. on the same drives (if you are going through all this hassle anyways, might as well save more data in the process). Redistribute updated copies in a few years (to back up new content, and to protect against ageing drives). In order for this scheme to fail, each drive has to either fail, get lost, or otherwise kept from you due to a scheming ex-friend.
But seriously ... what is the value of this data, really. Is anyone ever going to watch this stuff; will they notice if it goes missing. There is no need to over-engineer a system for nice-to-have data. If you have 2-3 independent offsite copies, that is already approaching that limit. Assuming this video material is probably never going to be used or watched, the best scheme is likely some system which provides a feeling that the data is safe - and then one can go on for the rest of one's life ignoring it exists. In this case, the subjective feeling of data safety is more important than actual data safety.
I may be wrong, your tapes may be really important ... but if they were, why would they be a random organization of clips sitting in a drawer, with no copies in the first place. This does not sound like something that requires a complex setup.
As a photographer with over 100k clicks on three bodies, I have just over 2 TB of my own photos on a dedicated drive. I have a two step backup system:
1) With a USB "drive toaster", perodically back up my files to a raw drive, mark it with a sharpie, and put it on the shelf in a different part of the house.
2) About twice a year, ghost my primary storage to a brand new drive, install the new drive in place of the old drive, mark the old drive with a sharpie, drive it over to a friend's house and put it in his fire safe. This serves as my hardware refresh and disaster recovery.
The older drives from previous backups in his safe are repurposed for music/movie storage, or used to rebuild PCs for other family members.
Were I really serious, I'd also mirror my primary storage, but at some point you have to say "this is good enough". Besides, any photos I've published exist on various websites, and I can always fetch copies from there in an emergency.
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
You need to keep the original tapes if you plan on keeping the digital copy as well or else you would be breaking copyright unless you alternatively have purchase receipts for each of the movies. As for backups, I've read before that spinning drives up/down will kill them sooner than keeping them on 24/7. To keep a safe backup, make 2-3 copies of the files on separate Blu-Ray discs and store them in a safe deposit box or relative's home. Use archival grade discs and record at the slowest rate supported wit full error correction and CRC files.
Of course there is also a restore operation :-)
I need beta testers and someone to build and try it on POSIX systems other than Debian. Currently there is no Windows version, which would require a bit of work, given the difference in filesystem layout.
On the website there is a contact email and I'd be super glad to have some feedback on it.
"What is worth saving forever? Thanks to M-DISC, the permanent storage solution, you don't have to decide. This new standard of storage engraves your information into a patented rock-like layer that has been proven to last 1,000 years and is resistant to temperature, light, and humidity:"
'nuf said.
Ditto.. been there
As mentioned elsewhere RAID is not wht your looking for in near term storage SnapRAID is built for this problem
http://snapraid.sourceforge.net/
If you want it to last beyond your life time, look into the M-Disc, many DVD burners support it, essentially is a Rock-like etching system that uses Lasers to "Engrave" the data onto a stable media good for about a 1,000 years.. if your writing the Bible it might be useful after that.. but otherwise even your great great ancestors will cease to care by then.. they are more likely to show up on your porch or recall you from the dead for a first hand account.. via some spooky resurrection program in a simulation 21st Century world.. ala Caprica
http://www.engadget.com/2011/08/15/m-disc-holds-your-data-forever-we-go-hands-on-for-a-few-minut/
lasts basically forever...
Nah, sell your homemade porn tapes in a tag sale. No need to digitize and restore them yourself. Someone else will get off doing that, post the results online, and soon they'll be replicated not just "in the cloud", but redundantly all over the world. It won't matter, you'll never know as you'll never desire to watch them anyway. You might earn a couple bucks at the tag sale though. Use it to buy vibrator batteries.
Eventually it will all fit on a single drive, after all (aren't there leading edge 10 TB drives already?)
A "desk drawer's" worth of standard definition VHS content will fit on a usb stick these days, unless digitized by a moron.
RAID is not solution for anything, unless your problem is that you'd like to propagate drive errors to all copies.
Well, Assuming that you are a domestic consumer with personal video's the way you talk about it... Tape is prohibitively expensive, RAID with Hotswap is probably a better option over a parked drive. If you need disaster proofing, then something like IOSafe is worth investigating. Fire and Water rated storage. https://iosafe.com/ IOSafe also has USB drive and NAS options. I am not affiliated with them, but do use them.
crashplan unlimited storage, $5 per month. Will take a long time to upload depending on your internet speed, but once you get it all uploaded, you're done.
I have developed scripts that manage my backups. Because I'm always experimenting with computer systems and apps, I make 100% backups, every day, for every computer. Each computer makes it own scheduled backup, copies it to one central system, then shuts down.
In the wee hours, the central system (an old, low-power XP box) makes it own backup, and then copies ALL the backups for that day to an attached external 1TB drive.
The central external drive has a hierarchy of backups (e.g.: P:\Backup\Backup\Backup). When each computer makes its' backup, it starts a copying process. That process makes sure that any older backups for that specific, named system are pushed down in the queue, and the oldest one is discarded THEN, I copy this evening's backup to that drive.
I have three 1 TB drives: One is connected to the central system and hold "this weeks' backups" (depending on how often I decide to change it); the next drive is the one most-recently retired from service, held nearby in case I have to go back several days or a week to find something; the third drive is stored in a safe place, off-site, so even if my building burned down, I've still got a lot of backups I could use to restore new computers from scratch.
When last months' MS Windows Update fiasco struck, all I had to do was restore the C: partition on the affected machines from last night's backup, and I was back in business without a hitch.
Finally, the reason I wrote these scripts for commercial backup software is that if backups aren't completely automatic, they'll never get made, so you won't have the critical data to recover when you need it. I've been thinking about reprogramming the CMD scripts in another language, to commercialize it, because loss of critical business (or even personal videos, photos, etc.) data is still a problem for those who choose not to use up all their bandwidth on a "cloud" service (although that could be easily added). It may sound like overkill to some, but I nearly NEVER lose my O.S., configurations, apps or data.
Amen to that !
Depending on what the video are about and copyrights, and if you think that videos may be a part of history, you can upload it to the Internet Archive: https://archive.org/
Some library recommendations require the drives to never spin down. I think it was either the (US) Library of Congress or the Smithsonian Institute who had a page describing their methods. Mind moisture, temperature, electricity feed, duty cycle limits and bit error rates, and make replicas as a function of those. Be conservative with software updates touching kernels and file systems.
If you have a RAID system, I'm guessing it's RAID 1 or RAID 10 then you're off to a good start. Connect an external HDD and keep it connected and schedule a periodic backup system. Then Configure an off-site backup. Although Google doesn't provide full HDD backup, it does save important files for you using Google Drive for an extremely cheap price: http://www.cnet.com/how-to/how-to-use-google-drive-as-a-backup-tool/
This question is basically "What is the best backup solution?"
you'll have much less to archive.
The standard IT solution for this problem is to encode the data as DNA and inject it into a few dozen cockroaches, which you then drive to the nearest KFC and set free.
If you ever need to restore from backup, just put some twinkies in a bowl outside your door, and some copies of your data will be available to you by morning.
I don't care if it's 90,000 hectares. That lake was not my doing.
A cheap solution to avoid catastrophic loss of one of a kind, but not valuable, data/
Except I'm sure Youtube over time will continue to morph into something unrecognizable...
I mean, recently Youtube is inserting ads in the middle, you read right, in the middle of piece of music. Worse than radio. (I'm going back to paying for music, sometimes.)
You don't need a reader though, you just need some patient geeks. Because the media is human-readable, it's just a matter of effort to recover the data. If the military really did back up a synopsis of modern science and engineering, one could imagine a future monastery full of monks patiently transcribing the works to scrolls for wider distribution. Of course, one could also imagine them mindlessly transcribing the pattern of dots with no clue as to the meaning of the holy tape, but that's people for you.
Socialism: a lie told by totalitarians and believed by fools.
No. Digital content needs to be worked. Digital archives are a certain path to unreadable formats, corrupted files, failed electronics, etc. It's different with archiving paper/film/etc, where constant handling reduces lifespan and data decays in a "human friendly" way. USBs, harddrives, DVDs, all shitty archive material unless they are being constantly used (and thus checked) and copied and themselves backed up.
Even with a archive folder(s) on an active drive, every few years you need to check that the formats are still readable, and that the player/editor software still works on your current system and/or that newer player/editors play the older files. And periodically convert the data to newer formats (by all means keep the old to avoid lossy conversion to short lifespan formats.) And it all gets backed up with your normal backup regime, which itself is a system that gets periodically updated because it's in regular use.
Science is all about firing a drunk pig out of a cannon just to see what happens.
I burn CD or DVD, then I run dvdisaster on them. Once in a while, I burn a DVD of dvdisaster's checksum files, and run dvdisaster on it too.
Example:
- burn DVD with Indiana Jones all episodes, store "ij.checksum" on hard drive
- burn DVD with all Aliens, store "alien.checksum" on hard drive
- burn DVD with all Back-to-the-future's, store "bttf.checksum" on hard drive
- burn DVD with *.checksum, store "cks1.checksum" on hard drive
- continue with other files...
http://hardware.slashdot.org/s...
There are three kinds of falsehood: the first is a 'fib,' the second is a downright lie, and the third is statistics.
Put them on a seedbox in the BitTorrent cloud!
Look up "digital assett management video."
Fuck, if you don't know - delete them - you never really needed them!!
Compare the rated interior temperatures of any fire safe you might buy with the maximum storage temperature of any drives, tape, discs, etc. you might plan to use.
Any decent fire safe will keep your important papers below 451F, but a quick googling of “max hard drive storage temperature” suggests that’s a good 300 degrees above “safe” for drives. I suspect tape will melt or demagnetize a good bit below where paper burns as well.
If you’ve gone to the trouble of making a second offline copy, store it someplace other than your basement. Relative or friend’s house? Locked box in your desk at work? Plenty of safer options
Bank safe deposit boxes are not cheap.
I understand that Kodak makes some archival-quality DVD-R media which are for exactly this sort of purpose.
You could do a lot worse than to try that. You can store them in a cool, dry place and forget about them.
I think that you missed the OPs concern with redundancy.
Having your backup in the local bank is really going to suck if they've been flooded out by the same event that flooded you out. (Floods may be fluids other than water, such as lava and volcanic ash.)
I'd be much more wary of shipping them across US international borders, where they'd be liable to seizure. Possibly at state borders too. But in that case, taking them to Auntie Flo isn't going to be any protection either.
Birds are not dinosaur descendants;birds are dinosaurs, for all useful meanings of "birds", "are" and "dinosaurs"