Slashdot Mirror


Best Practices for a Lossless Music Archive?

Sparagmei asks: "I'm a big music fan, and I like listening to the music I own on various pieces of digital gear. Right now, my library's at about 20,000 tracks, ripped from CDs to MP3 at 256kbps (enough that I can't tell the difference on my low-end playback gear). However, with the MP3 judgement rippling through the world, I'm interested in perhaps moving to a different compression standard. Before I do that, I'd like to ask a question: what lossless format would you recommend for making a digital 'master library' that could be (relatively) easily down-sampled to a compressed format?" Important factors would be true losslessness, filesize (smaller than PCM WAV would be nice), embedded metadata (ID3v2-like), existence of automated ripper software, and (to a lesser extent) an open-source implementation of such software. Widespread playback implementation of the lossless codec is not an issue for me; the lossless library would likely be burned to archival DVD media and stored after being down-sampling with the chosen compressor. The reason I ask is this: I've got a 20,000-track re-ripping job ahead of me. I'd like to do that just once, lossless, so that years from now, when I decide to jump from Vorbis to 'komprezzor_2039_1337' or whatever, I don't need to drag out the old plastic discs. Thanks!"

29 of 176 comments (clear)

  1. FLAC. by Whiney+Mac+Fanboy · · Score: 5, Informative

    The Free Lossless Audio Codec sounds perfect for you.

    (all answers below are quoted from wikipedia's FLAC page).

    Important factors would be true losslessness,

    A digital recording (such as a CD) encoded to FLAC maintains the quality of the audio perfectly.

    filesize (smaller than PCM WAV would be nice),

    Audio sources encoded to FLAC are typically reduced in size 40 to 50 percent.

    embedded metadata (ID3v2-like),

    with support for tagging, cover art and fast seeking.

    existence of automated ripper software,

    Yup, lots.

    and (to a lesser extent) an open-source implementation of such software.

    See above.

    Widespread playback implementation of the lossless codec is not an issue for me

    Well, bad luck, you're going to get it anyway :-) Both hardware and software support.

    Also, consider SHN, (although it seems superceded). I'll also mentiuon wave pack - because it uses an interesting approach (splitting the file into a small lossy standalone & a lost bits diff). don't bother with Apple's lossless format - it's going nowhere.

    --
    There are shills on slashdot. Apparently, I'm one of them.
    1. Re:FLAC. by Anonymous Coward · · Score: 5, Interesting

      I've built this kind of system for the record company i work for and our digital distribution needs.
      I've used FLAC - it just works. Also I like the Application Metadata blocks you can put into the FLAC files. I use this to store the full logging information from cdparanoia. It allows me to perform a quality analysis of the rip and look for jitter, skips etc. If i find a certain pattern which leads to audible artifacts I can just go back through the archive of tracks and perform an automated analysis of anything else which mught show the same problem.

      Because of the amount of metadata which we need to store for business reasons (P&C, ISRC, barcodes, etc) I have developed an XML based format for entering the info - you wouldn't need this on a personal system I don't suppose.

      For work it's great because I can encode to AAC/MP3/WMA for retailers. At home i use it to export to Ogg because we have an iAudio player, but it's trivial to export to MP3 or AAC instead if we got another device.

      I store all the files in a flat system - each track has a unique ID generated when it was ripped - when I export out to the encoded versions I use the tags to create a Artist/Album/Track hierachy which again can be changed at at time fairly trivially.

      Periodically rsync the exports out to my gf's machine and i've got the collection whenever I want it :)

    2. Re:FLAC. by 19thNervousBreakdown · · Score: 3, Interesting

      Just one thing... FLAC does not compress to 40-50%. More like 60(rare)-70-80%. That being said, no lossless codec does better than 60% occasionally. There's no point chasing a couple percent, even when we're talking about hundreds of gigs, because if you're archiving this how much would it suck if you went to recover this years from now, Windows XP and Vista was no longer available, Monkey's Audio went out of business in 2008 and never made a Vista version, which is the last "audio path" that's compatible with Windows '84. IOW, you're fucked.

      What do I do?

      1. Rip twice fully with highest CDParanoia settings and drive offset corrected. Use a high-end drive like Plextor that doesn't allow unreported errors.
      2. Compare rips, with diff. One bit difference, and it's discarded.
      3. Same procedure with cdrdao to get the TOC. Don't rip with cdrdao, you need to edit the code to get highest paranoia settings and support drive offset. Did that once, couldn't apply my patch to the new version automatically, screw it, use both.
      4. Convert TOC to CUE to add to the FLAC.
      5. Encode to flac, embed the CUE (just in case, we still keep both TOC & CUE).

      Actually, I started splitting my flacs with SHNsplit and putting in Vorbis tags, but if you're going to archive and never play the list is the way to go.

      --
      <xml><I><am><so><damn>Web 2.0</damn></so></am></I></xml>
    3. Re:FLAC. by WilliamTS99 · · Score: 3, Informative

      I also recommend FLAC, that is what I am ripping all of my CDs to. When you can get a 500gig HDD for under $140,it is worth every cent and more to do what you are doing. I have an Ubuntu system set up for network storage so I can play my FLACs with my two Cowon iAudio (Highly Recommended) portable media players(support FLAC natively), streamed to my stereo through the Xbox media center( XBMC , streamed to my computer, my daughters computer, or transcoded for other players. In my opinion FLAC is 100% the way to go.

  2. Me too by Overzeetop · · Score: 2, Interesting

    I made the mistake of ripping my small collection to MP3 (some to MP3pro...please stop laughing) the first time. I made it about 1/2 way though the 300 or so CDs I have before I realized that I wan't happy with the format (could hear artifacts) and knew I wanted a lossless that I could transcode to the format du jour.

    I went with FLAC, and ripped 'em all. I'm using media monkey as a filing system, and am transcoding as necessary for portable apps. I'm without media server at the moment, so I can't help with streaming and such, though I'm going to be interested to see what others are doing.

    --
    Is it just my observation, or are there way too many stupid people in the world?
  3. At the risk of asking a stupid question by Dogtanian · · Score: 4, Insightful

    Bearing in mind that you're not going to save that much file size using lossless storage, and that you already have an "archive" of CDs in a box in your basement (or wherever), is it really worth the hassle of creating another lossless copy that'll take up even more space?

    If you're planning on re-converting from these lossless copies, it sounds like you're going to be doing a *lot* of work based on some second-guessing of where you'll be in 5 years time; and things may have changed then.

    --
    "Slashdot - News and Chat Sites Deviant". (Click "homepage" link above for details).
    1. Re:At the risk of asking a stupid question by MMC+Monster · · Score: 4, Insightful

      Well, a cabinet of CDs in the basement may just convert to a few drives in a storage area network. Network storage is getting cheaper every moment. Besides, he will have random access to his entire music collection (presumably well tagged) in lossless format, which really can't be beat.

      --
      Help! I'm a slashdot refugee.
    2. Re:At the risk of asking a stupid question by Dogtanian · · Score: 2, Insightful

      Not much size = 50%. That may not be much to you, but it's still a reasonable amount. Yes, but from that point-of-view alone, you're still having to do a lot of work to get it. And it's not 50%; it's 150%, unless he's binning his original CDs.

      With foobar, you can take that collection - or any subset - and do a custom recode. It may take a day or two, but its totally automated. I dunno; it kind of smacks of the geek tendency to spend a day automating a procedure that would otherwise take 15 minutes or so, and then using it 3 or 4 times before they decide they want things done differently and abandoning it. Can't beat that logic ;-) Maybe I'm wrong, of course, but I'm just getting that vibe; I know, because I've sometimes done it myself.
      --
      "Slashdot - News and Chat Sites Deviant". (Click "homepage" link above for details).
    3. Re:At the risk of asking a stupid question by vakuona · · Score: 4, Funny

      This is the RIAA maths department calling.

      We have done the math for you. (We will charge you for this service)

      You have ripped 300 CDs and you admit sending music to friends. Well, 300 CDs at about 12 tracks per CD to on average 5 friends means you now owe us well, 5 * 300 * 12 * $0.99 so this comes up to $17,820.

      For the hard math, we will charge you $1,000.

      Please make out your check to RIAA-R-US.

      Thank you for your cooperation.

      RIAA

    4. Re:At the risk of asking a stupid question by Surt · · Score: 3, Funny

      You forgot the statutory damages of $750(http://arstechnica.com/news.ars/post/20061110 -8193.html) per song, so that's really:

      5*300*12*$751 = $13,518,000

      Plus two thousand now since they had to do the math twice.

      --
      "Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
  4. Ape by mobby_6kl · · Score: 2, Interesting

    Since FLAC was already mentioned, I'll just suggest you try Monkey's Audio. It's lossless, usually compresses better than FLAC, the source is available (not sure what license though), supports tags, and basically does everything you want. It's probably not as widespread as FLAC, but that shouldn't be a problem in your case.

    1. Re:Ape by TeknoHog · · Score: 4, Informative

      Ape compresses a few % better than Flac at the expense of much more CPU usage. When compressing lots of CDs at once, the difference is significant. Flac was designed to be light on resources to facilitate portable devices, but it helps with modern computers as well.

      Also, Ape is not free software, despite the availability of source for certain versions. It's only officially released for Windows.

      So, in line with most others, I'd recommend Flac, but you might also look into WavPack as it also seems to be free software.

      --
      Escher was the first MC and Giger invented the HR department.
    2. Re:Ape by omeomi · · Score: 2, Insightful

      The grandparent said it wasn't "free". He didn't specify which "free" he was talking about. Since it is "free" under at least one definition of the word free, his statement is incorrect. While one who knows a bit about FOSS could assume that they meant free as in speech, they didn't explicitly *say* that. And since the majority of the population means "free as in beer" when they're talking about whether software is free, people who insist on claiming that software isn't free if it's not free as in speech are only making a conversation more confusing in ways that have no bearing on the main point as a whole.

    3. Re:Ape by Surt · · Score: 4, Funny

      You can get all the free beer you want, and it's just a sex change operation away!

      --
      "Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
  5. As everyone's said, use FLAC. by eddy · · Score: 2, Insightful

    FLAC doesn't compress the absolute best of the alternatives, but it's 'good enough' and is widely supported, even directly on some portable devices. You won't wake up one day to find out that FLAC support has all but disappeared because the original developer lost interest (since the source is out there, unlike many alternatives). You will also be able to trivially transcode FLAC to Vorbis with meta-data intact, and do it FAST. (not a unique property, but well supported with FLAC/oggenc2).

    --
    Belief is the currency of delusion.
  6. My advice by adam1101 · · Score: 4, Funny

    Don't use FLAK or some other monkey sound formats. Go with a real standard, like Windows Lossless, or
    Apple Lossless, they just sound sooo much better. But you do need high end Bose equipment and gold
    plated Monster cables to really bring out the warmer and fuller mid-range and the increased bass response.

  7. Hydrogen Audio by eddy · · Score: 3, Informative

    Forgot to mention that you'll probably want to go to hydrogenaudio instead of /. for these sorts of questions.

    --
    Belief is the currency of delusion.
  8. Re:FLAC by Anonymous Coward · · Score: 3, Informative
  9. Re:Mod parent troll by Anonymous Coward · · Score: 5, Funny

    He couldn't see the sarcasm because his monitor isn't using monster cables.

  10. Re:My advice - use PKZIP for DOS by Anonymous Coward · · Score: 4, Funny

    Yes, I agree with the parent. Use those most excellent formats for storing your music or else use PKZIP for MS-DOS and then copy the ZIP file to a IOMEGA ZIP drive for safekeeping

  11. Let me just say...Woooooosh! by Overzeetop · · Score: 2, Informative

    Everybody sit down. Did you see the line about Monster cables and Bose?

    Parent was trying to be funny. He wasn't, but don't go off the deep end.

    Overrated is more appropriate, though Troll probably will be the choice of the Mac fanbois for the Apple Lossless dig he included.

    --
    Is it just my observation, or are there way too many stupid people in the world?
  12. You've already lost, you just don't know it by Gothmolly · · Score: 2, Insightful

    Use of the term "best practices" indicates that you've already assumed that whatever you're doing is wrong, and that whatever someone else tells you to do is right.
    "Best practices" is to IT what the "zero tolerance" concept is to schools - no questioning, no thought required, simply doing whatever the current meme dictates.

    --
    I want to delete my account but Slashdot doesn't allow it.
  13. Re:Hardware based FLAC players? by DavidTC · · Score: 2, Insightful

    Anyone know of any other great sounding devices that rip, convert, and burn?

    Dozens of manufacturers have a device that can do that. I believe they're called 'computers'.

    --
    If corporations are people, aren't stockholders guilty of slavery?
  14. what's the point? by coaxial · · Score: 2, Insightful

    Seriously. What's the point? What are you trying to accomplish? You can't hear the difference at 256kps lossy versus lossless, so why waste you're life converting your already lossess music archive from CD form to harddrive form? It's not like you're going to be transcoding all that often, if at all. I encoded my entire CD collect 7 years ago as 320kbps constant MP3. I'm thinking about re-encoding it, because there's no point in having the files that big. 256kbps variable mp3 is would probably still be more than enough. And if you think that someday you can transcode to something higher than 320kbps, I doubt you'll be able to tell the difference if that ever happens, because your hearing naturally gets worse over time.

    Futhermore, if you think Alcatel-Lucent v Microsoft is going to change anything, you're delusional. MP3 is going to stay. Just like how LZW patent did nothing to GIF. No one is going to abandon MP3, because the public isn't going to buy a device that can't play their MP3 collection. Nothing will change, and FLAC and Ogg will remain forever an asterisk.

    1. Re:what's the point? by Ant+P. · · Score: 2, Insightful

      You can't hear the difference at 256kps lossy versus lossless, so why waste you're life converting your already lossess music archive from CD form to harddrive form


      Maybe because at some point he'd like to upgrade from the low-end gear he said made the difference inaudible? Even on $30 headphones the difference between MP3 and lossless is clear as day.
      Or maybe because he wants a backup copy to rebuild from in case the CDs get scratched? Not everyone has an audio CD collection made entirely of titanium and diamond discs.
    2. Re:what's the point? by tmasssey · · Score: 2, Insightful

      Seriously. What's the point? What are you trying to accomplish? You can't hear the difference at 256kps lossy versus lossless, so why waste you're life converting your already lossess music archive from CD form to harddrive form?

      There are so many thing wrong with this statement...

      1) Hearing artifacts at 256kbps. I will agree that even with decent speakers you may not be able to hear them. But with good gear, it's *noticable*. I used to have a pair of B&W DM602-S3 speakers as my mains and MP3's were fine. Then I upgraded to the 704's. All of a sudden I had to throw away my entire collection of MP3's: the artifacts just slapped you in the head.

      2) Why have your media on hard drive? Why *not* have your media on hard drive? And if you're going to go through the work of ripping them, why not rip them *once* (to a lossless format) and be done with it? To save a few gig of hard drive space? Let's assume 1,000 CD's. Is that enough? An audio CD holds Seeing as a 400GB hard drive is going to cost less than $200 ($400 if you mirror them), what is the point of "saving space"? Especially when you figure that those 1,000 albums cost probably $10,000 - $20,000? And what did your time cost to rip 1,000 albums? At $5/hour and 10 minutes per album, you spent over $800 just to do the ripping! So why in the world would you want to do it *twice*, just so you can save a little bit of storage?

      Of course, there's also the very real possibility that IHBT... :)

  15. Re:Pioneer PD-F1009 by KevinKnSC · · Score: 2, Insightful

    I have an old 400 disc CD changer from Sony, and as cool as the thing was at the time, I haven't used it to listen to music in years. When you can store that much music completely uncompressed on a $150 hard drive, it just doesn't really make sense anymore.

    Not to mention how you can't take it with you, like you can with an MP3 player, etc.

  16. Re:Conversion question by mahonri5 · · Score: 2, Informative

    Oggify was designed to handle flac to mp3 or ogg conversion in bulk, while preserving the tags. I've been happy with it (not that I'm biased or anything), and so have a number of other people. It's open source, free, and in Perl.
    You should try it.

  17. Re:no point using linux by Anonymous+Drunkard · · Score: 2, Informative

    I believe it is only possible to have the same rip twice under windows using EAC.


    Please read the documentation for EAC. You may be surprised to find that EAC was written because the author wanted a Windows port for the Linux program 'cdparanoia'. EAC's entire goal is to mimic the pre-existing Linux program 'cdparanoia' in its bit-for-bit redundant checking and matching.



    There are decent frontends for cdparanoia if you don't want to run it via command line. GRip happens to be excellent. But please check what you are saying before you say it; few things are more annoying than having someone announce "Oh you can't do X in Linux, you can only use Windows program Y" when, in fact, Windows program Y was developed to give Windows users the chance to use a superior program that was already established in Linux.