Archiving Content from a PVR?

ayden asks: "Now that the universe has conspired to keep me unemployed for the foreseeable future, I'm taking the time to fill in the gaps in my Babylon 5 collection thanks to the SCI-FI channel. I'm frustrated the linear nature video tape and the problems associated with recording directly from broadcast to tape. It occurs to me that there has to be a better way. I've thought about using my ATI All In Wonder Radeon to record the program directly to my hard drive and editing the resulting file to remove commercials. Should I then record the file to video tape? Or would it be better to make a Video CD I could play back on my DVD player? Are there other options should I consider? How are people archiving shows from their Personal Video Recorders? What techniques are people using to accomplish these tasks?"

Comments

If you have a TiVo...

[damiangerous]

there's ExtractStream to pull the MPEG data off the hard drive. Don't ask about it in the TiVo forum though, video extraction is a banned topic.

Re:If you have a TiVo...

[dschuetz]

there's ExtractStream [9thtee.com] to pull the MPEG data off the hard drive

I thought (and I may be wrong here) that you needed to set some kind of system flag ahead of time to disable local encryption of the stream. So that anything you've already got on the box was non-extractable? (I'm unsure because I haven't had the time, lately, to watch these developments too closely...)

A related question: Has anyone managed to modify the myworld program on TiVo to allow it to, basically, function as a TV tuner with MPEG-2 output? I'm specifically considering trying to hook an ethernet port to my DirecTV tivo unit and use it as a streaming source for my home network...

Re:If you have a TiVo...

[affegott]

What about extracting right from the hard drive? That way you don't need to wait for the video to be transfered over the network.

You could record a bunch of stuff, shutdown and grab the files...

Re:If you have a TiVo...

[delus10n0]

The DirecTiVO stores encrypted streams, but a regular TiVO does not.

I don't know of anyone that's modified their DirecTiVO to do what you mentioned.. I think it would require quite a bit of work.

Re:If you have a TiVo...

[sup4hleet]

ExtractStream doesn't seem to work on Tivo3.0 and by the looks of the page hasn't been updated for a year. I tried it the other night on Tivo3.0-01-00 series 1, it couldn't even display the "Now Showing" list.

Re:If you have a TiVo...

[EABinGA]

ExtractStream doesn't seem to work on Tivo3.0 and by the looks of the page hasn't been updated for a year. I tried it the other night on Tivo3.0-01-00 series 1, it couldn't even display the "Now Showing" list.

I wonder if my tivo knows this. My 3.0 seems to be doing it. I did have to change one file in the tivoweb software to get "now showing".

DVD-R

[GusherJizmac]

Record it using your all-in-wonder, and then burn the eps onto a DVD. If you have a Mac and a superdirve, iDVD is supposed to be great. There are probably similar alternatives on Linux or Windoze.

Re:DVD-R

[=weezer=]

iDVD isn't quite great if you want to fully unlock what a DVD can do and hold. Burning with iDVD you're limited to 90 minutes of video, while on the other hand using software like DVD Studio Pro (though expensive) you'll be able to store the maximum (not sure what that would be). Since with a VCD or an SVCD you would be limited to around 30 minutes anyway, the added cost of DVD media/DVD-Recording drive (if he doesn't have one already) wouldn't really justify the difference. However, there's also the issue of (S)VCD compatibility(my DVD player doesn't play home-burnt VCD's, for example).

Alternative...

[bowronch]

For archiving my favorite shows or at least some of them, I just download the AVI's or MPG's from Direct Connect... Its much easier that way... Let somebody else do the hard work... I got the The Osbournes, Scrubs and Sopranos Season 3 this way...

I would think that Babylon 5 episodes are probably easy to come by on Direct Connect or Gnutella...

For archiving

[p7]

I would leave it in a high quality format on the computer. Maybe set up a removeable hard drive. Going to VCD or VHS you will end up losing some quality. Hard drives are cheap and will probably take up less space than the tapes or vcds. You would then also have the option to make the VCDs or tapes from your archival copy.

Re:For archiving

[reverius]

I disagree - a high quality VCD is as good as broadcast quality. And the ability to watch it on a DVD is a big plus. Only problem is, you're limited to the space on a CD - about 74 or 80 minutes. That's only enough for two one-hour episodes, and that's with commercials, credits, and opening scenes cut out.

Re:For archiving

[NoMoreNicksLeft]

I couldn't live without the opening scenes, or probably even the credits. But getting rid of the commercials is a good thing.

You seem to be confused...

[hackwrench]

He said:
I'm frustrated [by] the linear nature [of] video tape

Without those words, it can be hard to read.

Re:You seem to be confused...

[ayden]

Thank you for the clarification. That is what I meant to say.

babylon 5

[Satai]

I'm going to take the conventional approach and buy the DVDs this fall, as they will also include TONS of interviews and JMS commentary on both Signs and Portents and Chrysalis. The following seasons should, well, follow shortly after the first...

Re:babylon 5

[ayden]

Although I may do this some day, the cost of over 100 hours of DVD video makes the cost prohibitive while unemployed.

Thanks for the info, though.

Re:babylon 5

[Anonymous Coward]

Same here, I use Gnutella because the cost of 100 hours of music on CD makes the cost prohibitive while unemployed.

Great quality, only one D--A conversion

[derinax]

I've gotten quality that rivals ExtractStream from the TiVo by recording S-Video onto Mini-DV tapes through either a Sony DV Watchman or a Canon digital video camera.

Once on MiniDV, you can go firewire into your machine. From there, it goes onto DVD via mi iMac.

Easy, no weird cropping (like you have to do with ExtractStream), and the quality is top.

Re:Great quality, only one D--A conversion

[kableh]

And if you dont have a DV cam, try the Canopus ADVC-100. It runs $300US, but supports PAL/NTSC, A/V sync, and has composite and S-Video ins. I played around with it a bit to record shows off Adult Swim. The quality is phenomenal.

And not to plug them, but I got mine from DV411.

Re:Great quality, only one D--A conversion

[delus10n0]

Or if you want to remove a middle step, get a Sony DVAnalog converter. It will convert S-Video/Composite, as well as audio into a DV stream through firewire. Makes capturing digital satellite/etc. a snap. They run around $300 though.. so they're not cheap. But they're the best.

Re:Great quality, only one D--A conversion

[crisco]

Hey I've got a buddy thats looking to get a All-In-Wonder card to turn his PC into a kind of media hub, I wonder if this would work better for him?

From what I can see, it does everything but VGA out to TV? If his current card has a S-Video out (even the junk on lots of nvidia cards) he could overcome that issue. He's got a firewire connector built into his box (some Sony Vaio), that should handle digital IO. Anything I'm missing?

Re:Great quality, only one D--A conversion

[kableh]

The ADVC-100 is just a hardware DV codec. It has S-Video/composite and audio in/outs. I used it with Adobe Premiere on my PC, or iMovie on my friend's G4, to rip video as DV. It doesnt need drivers or anything, as it just makes an analog video source look like a DV cam to the computer.

If your buddy wants to make a media hub of sorts, he probably would be better off with the All-In-Wonder, especially on Windows. If he is keen on writing a ton of custom software for linux, then perhaps this device would work better. Recent kernels have support for raw IEEE1394 devices, as well as video devices, so some of the software is there already.

Hope this helps. Feel free to email me if you have any more questions.

Why?

[Anonymous Coward]

Is there anything on tv which is worth archiving? You are not going to watch Babylon 5 twice, or are you?

Re:Why?

[Louis_Wu]

Is there anything on tv which is worth archiving? You are not going to watch Babylon 5 twice, or are you?

No, of course not. Twice is not nearly enough.

First there was watching it as it came out. That took five years. (Well, more, if you count the wait between the pilot movie and the beginning of the first season.) Then I watched them again, as I taped them on Sci-Fi. It was during this round that I started recording episode numbers and titles, to make certain that I got all of the episodes. Then I watched each episode again, looking for good quotes to copy down. (I collect quotes. One of these days I'll organize them.) I haven't watched the entire series since the quote copy round of seasons, but I have watched individual episodes; usually I rememeber a quote, look it up in my records, start reading the surrounding quotes, remember why I like that episode so much, and then pull out the tape and watch it again.

Let's see, that's at least three times I watched each episode. Twice, no, that's not nearly enough.

Re:Why?

[Anonymous Coward]

Wow, that was a clever reply. The hallmark of a good troll is that you can hardly tell it from a genuinely stupid comment.

Re:Why?

[Anonymous Coward]

You are not going to watch Babylon 5 twice, or are you?

Not unless he's a masochist :o)

TiVo

[4of12]

We wanted to archive a few shows, too.

What's worked pretty well is just to buy a low end TiVo with a 20 GB drive. Then, stuff two 100 GB drives in the thing and reboot. No, it's not that easy, but detailed instructions on the procedure for doing this are available. I think there's even a CD with a specially built Linux distro that includes all the necessary tools for mucking with the special MFS filesystem they use on the Linux PPC that constitutes a TiVo.

I've thought it would be nice, though, if there were a way to connect the IDE interface of the TiVo to some uber-RAID of terabyte or so. My queries about this returned some disappointing information that IDE drives were somehow limited to no more than 128 GB, ruling out that possible solution.

The Ethernet connection to NAS sounds pretty good in a lot ways (I don't want loads of disks whirring in my media cabinet), but from what I last heard it sounded kind of iffy from the standpoint that TiVo's automated system upgrades could stomp on your local tweaks to use such an Ethernet card (I think 9thtee sells them.)

I suppose once could always go "off grid" with the TiVo and not use it to call up for the programming guide and automated software updates, but having the programming information is kind of useful. I was willing to fork over the bucks for the lifetime subscription for the guide. I don't mind supporting TiVo monetarily since they've been generally pretty mellow about hacking on their hardware.

Re:TiVo

[Anonymous Coward]

somehow limited to no more than 128 GB

ummm... I dont think you understand how RAID works. Here's a simplified intro: (note the simplified part, this is not a complete technical explanation)

You have 8 disks, all the same size, and all connected to a hardware RAID controller. When the OS reads/writes the disk, access is made via the RAID controller, not to the individual disks. In general, for an 8-disk array, one bit for each byte resides on each disk. This allows your storage to increase to 8*disk_size_of_one_disk, and I/O speed is increased by a factor of approximately 8. To the OS, all 8 disks look like one drive, because the RAID controller fools the OS into thinking it's the drive.

One major problem is risk of disk failure. You can add 8 more disks (16 total), again all the same size, which mirror the main disks continually. Should a failure occur, the second array takes over as the primary array, allowing time for the faulty disk to be installed and mirrored. This maintains good uptime for the server should a disk bite the dust.

Again, this isnt the complete description, if you want that go to google. This was just to give you a quick explanation on how RAID works.

Re:TiVo

[4of12]

mmm... I dont think you understand how RAID works.

Maybe not as well as others.

Loosely, I do understand the concept of RAID. I know there's even number qualifiers to describe the different variations of mirroring and striping strategies.

The problem, from what I understand, is addressing limits built into the specifications of the IDE interface. So, even if you have a 2 TB RAID system, if you interface it through an old IDE connector, you can't make it seem larger than a pre-set limit. Likewise, if the OS doesn't support the newer, better IDE interface, then you can't gain the size benefits of the RAID.

Maybe the limit is 137 GB (not 128 that I mentioned earlier), but newer IDE disks of 160 GB are out there.

I don't know if a TiVo can be made to support those larger drives in hardware (it uses a UDMA-66 connector) or software (they use some tested (i.e. old) version of Linux for PPC).

This site suggests that newer Linux kernels can take advantage of the ATA-6 specification that allows sizes greater than 137 GB.

Re:TiVo

[dougmc]

ATA/33. ATA/66 and ATA/100 are limited to 137 GB. ATA/133 extends the limit by at least a few orders of magnitude.

"Conspiracy"

[GCP]

You say the universe is conspiring to keep you unemployed, yet what you want to do with your time is fill in gaps in your Babylon 5 collection?

What's wrong with this picture? Could it be that YOU are part of the conspiracy? Perhaps, say, the leader?

Re:"Conspiracy"

[peeping_Thomist]

Mod this up!!

Re:"Conspiracy"

[ayden]

You say the universe is conspiring to keep you unemployed, yet what you want to do with your time is fill in gaps in your Babylon 5 collection?

What's wrong with this picture? Could it be that YOU are part of the conspiracy? Perhaps, say, the leader?

That's only *ONE* thing I'm doing with my time - and that's only 1 hour a day, four days a week.

I've been working hard to find a job as a UNIX SysAdmin in Boston, but the tech sector sucks and the recovery is slower here than in other parts of the country. My contact sheet for this week (required for unemployment in MA) is already over 32 lines this week. I've had several interviews already, but no job, yet. I'm also talking to someone about a contracting position tomorrow morning that came through the Boston Linux User's Group discussion list.

But then again, why am I defending myself against a troll?

Re:"Conspiracy"

[Anonymous Coward]

But then again, why am I defending myself against a troll?

Hmmmm... because you're unemployed and have nothing better to do?

Re:"Conspiracy"

[Anonymous Coward]

But then again, why am I defending myself against a troll?

Becaise if you don't, the troll will eat you!

Re:"Conspiracy"

[earlytime]

hey buddy relax,

it *was* funny. i've been unemployed before, and understand your pain.
but it's like when somebody owes you money and you're ready to get paid back. Every dime they spend better be spent wisely, or it should have been *your* dime.
Even if you only spent even 10 minutes a day 4 days a week on something as, ahem, *important* as your babylon 5 collection, somebody out there might have a different take on your "universe conspiring against you" theory of unemployment.

Easy with a ReplayTV 4000

[jtl]

ReplayPC (C, cross-platform, commandline, scriptable) and DVArchive (Java, cross-platform, user-friendly UI) can archive shows from ReplayTV 4000s and 4500s. DVArchive or ReplayServer can stream the archived content back, so you can watch on your tv, or you can burn VCDs or DVDs or whatnot.

OT: The Osbournes

[Anonymous Coward]

What is the appeal of The Osbournes? My lil' brother was watching that, and I was bored silly. I tried to get him to explain why he still watched the show, and the best answer he could give, was that Ozzy's feebleness was somehow funny. I guess it's like the Mr. McGoo cartoon or something.

Did my brother pretty much sum up the appeal of the show, or is there some other entertaining aspect that I haven't perceived?

Two solutions, I've tried both..

[delus10n0]

First and easiest solution is to just make a "media box" with an All-In-Wonder in it to record your shows to MPEG2. Then I used TMPEGEnc and other tools to make them into SVCD (or sometimes DiVX.) An hour long episode could be edited down to 40 minutes or so (with all the commercials out), and it would be pretty decent quality on a single 700meg CD as SVCD.

The second and more difficult solution is to get a Tivo, install TurboNet on it (a 100mbps NIC), and try to extract the streams off of the Tivo. I have had a lot of problems with this method, as most "long" shows (over an hour) won't extract properly, or give me other errors. The nice thing about this method though is that it's interlaced, and already in the proper resolution for TV. Plus, the hardware MPEG2 encoder of the TiVO is by far a million times better than the software one of the All-In-Wonder.

For more information on TiVO hacks, visit http://www.9thtee.com/ -- and for more information on video stuff in general I'd try http://www.vcdhelp.com and http://www.doom9.org

Re:Two solutions, I've tried both..

[cdh]

I haven't had too many problems pulling videos off my Tivo with my TivoNet card (hooked up to a wireless AP used in client mode to connect to the rest of my network). I used the boot CD and installed the Tivo Web stuff, then I use a program called TivoApp (yes, it's Windows, so shoot me). Works pretty well, well, at least it did until the 3.0 upgrade came out, now TivoWeb doesn't work at all :(

At least Tivo allowed the TivoNet (and TurboNet) to keep working in the 3.0 upgrade via different dial prefixes.

cdh

Re:Two solutions, I've tried both..

[delus10n0]

TivoApp will work again if you update your ShowList.tcl -- I can send you the updated version if you'd like.

TivoApp works on some shows, but ones that are long end up bombing out or being corrupt TyStreams. Haven't figured out a solution yet.

Re:Two solutions, I've tried both..

[cdh]

Ahh...I haven't seen an update yet. Thanks.

Now that you mention it, I've had varying degrees of success on long shows. I have all of the Formula 1 races this year recorded (approx 3 hours each) and I've been able to pull off all but 2. Wish there was some solution for that too...

Re:Two solutions, I've tried both..

[Anonymous Coward]

I heard your mom had her cervyx shatterd by BasharTeg and his mighty fire hose of justice.

Any thoughts ?

Re:Two solutions, I've tried both..

[Anonymous Coward]

One often posts as an AC.

Do you quarrel, sir?

Re:Two solutions, I've tried both..

[Anonymous Coward]

I DO NOT QUARREL !

Re:Two solutions, I've tried both..

[Telecommando]

I've been trying to do this with an All-In-Wonder for a couple of months. The video quality is very good, TMPEGEnc converts them into a format my DVD player accepts, but I haven't had much success getting rid on the commercials. Currently I can get 2 half hour programs (with commercials) on 1 SVCD disk. If I could get rid of the commercials I could get 3 to a disk.

I've used Ulead VideoStudio and VideoWave with mixed results. Both have a flashy interface but using them seems counter-intuitive. Anyone know of a better product or have any hints on how to better use what I've already got?

Re:Two solutions, I've tried both..

[delus10n0]

There's a few ways you can do this. My easiest solution was to set my All In Wonder to record to high-bitrate MPEG1, then use VirtualDub to load the MPEG and edit out the commercials, saving each chunk out to AVI (Using the HuffyUV lossless codec). This takes up a bit of hard drive space, but the quality is great. Then I'd load the AVI's into TMPEGEnc and encode them into an SVCD program.

I haven't dealt with SVCD in a while, though, as now I am ripping video right off of the TiVO and deinterlacing/processing it, then compressing it using DiVX. I've found I can fit a 30 minute show inside of around 250 megs and it still looks great.

Something else you might want to try it TMPEGENc's "cut" feature, where you can cut out portions of an MPEG2. It works 90% of the time (and the other 10% it bombs out), but it may work for you.

Hope these suggestions help, let me know if you need anything.

Re:Two solutions, I've tried both..

[Telecommando]

Thanks for the advice. I'll go find VirtualDub and try it this weekend.

Here's a good guide to extracting with TiVo

[bwillcox]

I used the information here to get started

http://www.30below.com/~zmerch/tivo/index.cfm

I use the aforementioned TiVoApp to pull the tystreams off of TiVo (running extractstream and sending the data to the PC over netcat). A raw tystream is best and avoids the problem with tyconvert in TiVoApp.

You need a new NowShowing.tcl to work with TiVo's 3.0 software. It's available in the TiVo hacking forum on dealdatabase.com

I then use a program called vsplit to split the tystream into m2v and m2a audio files. You can find this in the TiVo hacking forum on Dealdatabase.com. (Jdiner, you rock!)

This program leaves an audio offset which you use later. I have found vsplit works better than the zss program Merch uses.

Merch's guide tells you how to make SVCD's and is the best guide I have found to date.

The only problem is the amount of time required to encode the video. One episode of Enterprise (sans commercials) takes 6 hours on my 1 ghz Athlon.

-brent-

PVR use

[lsdino]

Here's what I'm doing:

I have a ReplayTV 4040. I figured I can get one of the lower models they offer because I can just network it, and save everything on my PC.

I've been using SwapDV which basically acts as a ReplayTV.

I can download shows from my ReplayTV to my computer, and it can serve shows back up to my ReplayTV. So I just use my ReplayTV, browse over to my computer, and can watch shows off of it.

I'm about to setup a 5 80gig HD IDE RAID-5 system to keep all my data on. That way I get some redundancy. The only thing I need now is some offsite backups!

Windows solution

[slaker]

My solution is Windows. Shoot me.

My setup is a nice, easy to cool Duron 950MHz with an ATI Rage VIVO, a JVC SVHS VCR for tuning/additional capture, and staggering amounts of hard disk space. Local storage on that box is maybe 360GB (three WD1200ABs), but the storage available to me is approaching 2TB. I use Windows 2000 Pro and a Philips Acoustic Edge to complete the picture.

I cap in highres MPEG1. Why MPEG1? Throw a high enough bitrate (four or five GBs an hour) at it and you can't tell it from MPEG-2 anyway, plus I can then use Virtualdub, probably the single greatest GPL'd software package available for Windows, to do all my editing and encoding work. I haven't found anything that works on MPEG-2 that is nearly so nice as vdub.

It works marvelously. I maybe lose five frames of video per hour captured. Editing out commercials takes only seconds... and then the edited video gets passed to one of my faster boxes for procesing down to a Divx .AVI file or shrinkage to VCD format.

Incidently, VCDs hold around 70 minutes of video, not 30, like someone above me said.

likko vdr2100

[Anonymous Coward]

i use a likko electronics vdr2100 to burn
straight to VCD from my tivo. works good I
record about 12 VCDs a week. no software,
no pc, nothing needed but cables to connect
from tivo to the vcd recorder.(which come
with the likko)

its not real cheap, i got mine for about $469
or something from www.lik-sang.com (hong kong).
I searched for hours and could not find a U.S.
reseller for it. there is also another popular
VCD recorder, i forgot the name/brand but
its from a company in texas, not quite as good
as the likko. Terrapin, thats the brand.

i used to record to my pc using qtvidcap from
the avifile package, straight to DiVx, got tired of that, and got the likko...

also picked up dreamcast vcd software, and
a few used dreamcasts as players, also got
a portable VCD player(also from lik-sang.com),
the size of a portable CD player ..works good.

I currently do this in standard and Hi Def TV

[The_Laughing_God]

I usually save VHS as 320x240 video compressed with the DivX4-fast motion codec, which only consumes 150-250MB/hour after removing commercials. A full seasons can be jammed in two roomier than average "standard" jewel cases (depending on the series). However since up to 560 hours of DivX4 video fits on a 80GB HD ($80 at Compugeeks.com) HDD is comparable to the cost of VHS tapes, and the CDs are mostly a hedge against HD failure.

A DVD-R recorder probably isn't an option if you're unemployed, but with a digital archive,you can copy a full season to a single DVD-R with no degradation when your finances rise or street prices fall.

I use a Pinnacle Studio Pro (rebadged as an IOmagic PVR. Cost: $30-$20 mail in rebate at Outpost.com when I bought my dad one in January) or a MyHD HDTV card (under $300 from Digital Connections)

1) save the program in MP2 format with the included application.

2) trim out the commercials (takes 2 minutes) with VirtualDub (a free open source Windows app). There are other similar free programs like AVIsynth D*scaler, etc, but I don't use them much.

3) VirtualDub encodes in your choice of codec and performs many other operations, like clean-up, enhancing, or resizing as it saves the file. The codecs, filters or plug-ins are available free from many archives (Google it) This processing/saving may take some time, depending on your CPU.

4) Periodically batch-burn CDs

A clean, well-encoded digital 320x240 video on a computer monitor can be better than a VHS tape. NTSC's 480 lines are actually two successive 240 line fields. The VHS standard can only resolve 400 of NTSC's 480 lines (two successive 200-line fields) but few VCRs can even resolve that much. And don't forget that the analog degradation of TV, VCR, and all connections, etc is cumulative.

The MyHD is more expensive, but it also turns your monitor into an HDTV - and have you priced *those* lately?. You won't want to keep the pure HDTV stream. It's compressed with MP2, but it's packetized and pretty sizable (2.5MB/sec or almost 2GB for an edited 1-hour episode). I resize to 320x240, but you might prefer to preserve the high def resolution (1080i) and make 1-2 episode VCDs in DivX