Ask Slashdot: Huge Digital Media Libraries

An anonymous reader writes "Like many slashdotters, I have several TB of digital media: music, books, movies, tv shows, games, comics, you name it. I've put it all in a few HDs, but handling it all has proven to be less than optimal. I'm covered when it comes to music, since [pretty much any music player/library manager] allows me to quickly find songs by interpreter, album, genre... For everything else, all I have is a series of hierarchical folder structures, but hierarchies have limitations. I can find Blade Runner easily, but what if I wanted all of Scott Ridley's films? Where is 'Good Omens', in the Terry Pratchett folder or in Neil Gaiman's? Furthermore, in a collection with hundreds of similar items, it would help to have some extra clues such as covers (for comic books) or synopsis for TV shows' episodes. Do you have any software to help you handle digital media libraries? Specialized software (say, something that only work for comics, something else for movies), or generic media libraries? Opensource alternatives are preferred, but commercial software is fine as well."

Comments

Nope

[Anrego]

I tend to just use directories of symlinks on the odd occasion where I want a logical collection of something. Kind of the hacked file system equivalent of a playlist. I can even put additional detail in the symlink name that I would leave out of my “main tree”. Generally though, a simple hierarchical structure has worked fine for me and my 6+ TB of media. If I anticipate wanting to search for something down the road, I also sometimes put it in the file name (indexed by slocate every night).

You are probably looking for a tagging/metadata tool but I think the problem with those is you have to obsessively tag/provide that metadata and they aren’t going to integrate with all your favourite viewers and such. It just seems more trouble than it’s worth to me, but with different levels of motivation and borderline OCD, it could work very well (and probably does for many).

That's alright

[sneilan]

You can admit it. Slashdot understands that you have a large *personal* media collection *AHEM*.

Data crow

[cmiu007]

Use data crow and make a container for every HDD. It works for music, movies(imdb details import) and software. http://www.datacrow.net/

Fedora Repository

[oneiros27]

Just use what the libraries use:

Fedora

What you're looking for in general is either a repository (if you want it to manage the files) or a catalog (if you want it to just track info about the files). A catalog might also be called a 'registry' when dealing with sciences archives, where the term 'catalog' is used for something else.

For more options, see any of the following lists on wikipedia:

http://en.wikipedia.org/wiki/Category:Free_institutional_repository_software
http://en.wikipedia.org/wiki/Category:Digital_library_software
http://en.wikipedia.org/wiki/List_of_next-generation_library_catalogs

Calibre, open source ebook manager

[metrometro]

Used iTunes? It's like that for books but less bloated. Syncs to many devices, and can scrape RSS feeds from magazines, build them into EPUBs and sync them to an ereader, like a text-based podcast. This works surprisingly well, superior in some ways to reading the same material on the Web.

And it's FLOSS.

http://calibre-ebook.com/about

purge

[anyaristow]

How much of it do you really re-watch? You'll spend the rest of your life transferring it from medium to medium. Is it worth it?

Re:purge

[Hatta]

How much of it do you really re-watch?

Probably only 10%. But how am I to know which 10% I'm going to want in the future?

You'll spend the rest of your life transferring it from medium to medium.

Sure, It'll take an hour or so to rsync onto a new array every 5 years or so. I'm going to migrate data anyway, so the question is whether I want to migrate 100GB of data or 4TB. Since rsync can run in the background, there's essentially no cost to migrating a large array.

Is it worth it?

Definitely! Having everything I want locally saves me time searching the internet for it. When I decide I want to use something, I don't have to wait for it to download. The effort it takes to maintain is minimal, so why not?

iTunes

[aclarke]

I'm likely going to be flamed and modded into purgatory for this, but I use iTunes for most of this-at least, for music and videos. Some PDFs are starting to go in there if I want access to them on the go on my iPhone or iPad.

I understand that Apple's universe isn't perfect, but for me it all works together pretty nicely. I replaced my high-maintenance, increasingly noisy, power-hungry media PC with a second-generation Apple TV. This works great except that it won't play many video formats. Therefore, I've had to go through the obnoxious step of using VideoDrive to transcode videos into an Apple-approved format. However, it's not the end of the world.

Otherwise, I guess everyone's different, but personally I want to spend my time doing fun stuff like riding my bicycle or spending time with my family, not categorizing my "vast media collection". I guess I'm just getting old, but iTunes does a good enough job, with less work than any DIY system I've successfully maintained in the past.

Re:iTunes

[Anonymous Coward]

This is what I've done as well.

We have pretty much sold our soul to Apple just for the convenience of not worrying about pulling stuff when we need it. We create a significant volume of recordings of interviews, meetings and design brainstorming discussions during software development. We upload these to iTunes to keep them organized. We use Home Share from a (reasonably old) Mac Mini so everyone in the office can get them when they need it. Additionally, just bought Apple TV's for two conference rooms so we can use AirPlay to play back presentations to the room we are in.
One more step though, we use IDentify from Justin Pulsipher to edit tags on the videos and podcasts. This means that if we ever need to rebuild the library (which we've had to do a couple of times), we don't need to tag items again. The tags are embedded in the file and will work irrespective of which file they are in.

Re:iTunes

[Mister Whirly]

You realize that most "fireproof" safes are designed so paper will not burn - at 451 degrees Fahrenheit. If you read the fine print, the temperatures in those safes can hit well over 250 degrees, which I am sure your DVDs would not survive. Unless you bought a Fireproof safe specifically designed to protect DVDs, keeping them in there is probably a colossal waste of space and only provides a "feel good" solution that will not actually protect them from damage. I know becasue a friend of mine had a fireproof safe with computer backup DVDs in it that all melted into one big lump when he had a fire.

P.S. I can save you the time/money by letting you know all of Glenn Beck's books are absolute crap. No need to buy them in the first place - unless you like to be frightened by really dumb stuff.

Re:Scott Ridley

[tgd]

Well, now we know why he couldn't find them ...

Re:XBMC ?

[Compaqt]

I was going to say this, but you already have, so I'll just expand on it.

XBMC is great for organizing media. It has some neat features:
-looks really nice, suitable for a living/theater room, not geeky
-movies, pictures, sound
-IMDB integration
-scripts (do anything)
-contributed lists of Internet TV stations
-support for IR remote controls and universal remotes
-remote playback (playing computer being separate from the storage computer)

http://xbmc.org/

One thing it's not really designed for is to record TV. For that, use MythTV.

MediaTomb

[gregthebunny]

I use MediaTomb for my digital media library. It manages all my music, videos, and photos and is quite extensible through scripting if you are familiar with JavaScript. Then I use XBMC or my PS3 as the front-end to MediaTomb. I'm currently managing over 1 TB of data without issue. I cannot speak for other media, such as books, as all my books are still in dead tree format.

re: iTunes (another option)

[King_TJ]

I certainly won't "flame" you for this suggestion. As maligned as the iTunes software is, I think its ability to index media and very quickly retrieve it by a number of different fields is pretty darn good -- especially for a program you can download free of charge for both Windows and the Mac.

As a Mac user myself, I started using another free program to manage my movies and saved TV shows though. I really like Plex (www.plexapp.com) for the purpose. It doesn't have the restrictions on playable video formats that iTunes has, and has a great UI to serve as your media center via a remote control.

I believe the latest update to Plex added some interesting, if slightly obscure, functionality -- like the ability to search the subtitles of your movies for specific strings, too.

Re:Oh really?

[ledow]

I own the David Attenborough Life Collection. It's 24 DVDs. Even assuming they are the single-layer type, that would be:

112.8 Gb. Roughly.

Now say I own, say, several boxsets of comedies, series, documentaries, a few dozen movies, maybe even a couple of dozen free promotional DVD's with full-feature films on them that are given out when the film stops selling.

Now, *NOT* including anything I've recorded from TV / Movie channels for my own consumption, not including any home videos, not including ANY Blu-Rays, etc. I can *EASILY* fill terabytes of data without even blinking an eyelid.

Hey, I could probably fill a terabyte or two with DVD images of cartoons (proper children's cartoons) and stuff I watched when I was younger (I have the complete set of Dangermouse, Batfink, etc.) and that's hardly something I go out and buy every day and keep buying. A terabyte, or even half a dozen terabytes, is NOTHING. It's just when you have to copy it all into a single place, like this guy is doing, that it appears to be a lot.

It's just that he's obsessively backing it up and/or converting it to free formats so that he can just browse from a media library, like the ones he desires, so it's all on one hard disk (or more likely RAID). It's not "abnormal". It's nowhere near "evidence of piracy". It's just a media collection stored on disk instead of the original DVD's.

Re:Are you with one of the **AA's?

[jedidiah]

No. "technically" DVD and BR rips are fair use.

Your attempt to hand Apple a stranglehold on the future on a silver platter is not appreciated.