RIAA Tracking Songs by MD5 Hashes

aSiTiC writes "Apparently RIAA has obtained some technical experts in their prosecution of file swappers. Currently they are tracking traded mp3 files from the Napster network by matching MD5 hashes. This seems quite interesting but I was under the assumption that identical hashes could be created with identical rips and id3v2 tagging. Now may be the time to update your illegal mp3 file MD5 hash sums."

Comments

gee?

[Comsn]

The RIAA, the trade group for the largest record labels, said it also found other hidden evidence inside the woman's music files suggesting the songs were recorded by other people and distributed across the Internet.

ya think? and here i thought it was the magical mp3 fairy who put mp3s on my hd...

Re:gee?

[nearlygod]

About this interpretation of Fair Use: I agree that downloading mp3's of CDs that you have purchased should be fair use. I am in a similar situation. A couple of years ago I lost 90% of my CD collection in an apartment fire. I had about 20 of these CDs ripped at the time and since then, I have downloaded many of the others to replace what I had paid for. In some cases, I re-purchased the CD because I wanted to have an original for some of my favorite artists but I didn't mind the mp3 mastered replacements for many of the CDs. Would this fall under Fair Use? I would think that it does since the RIAA seems to think that we are only purchasing a license to listen to the music. However, if I had to present the original CDs to a judge to prove that I do/did own the physical CD, I would be SOL.

Re:gee?

[arth1]

This wouldn't, though, be a defense for the central problem that she made all of these MP3s available for download by millions of anonymous strangers without the consent of the copyright holders.

Unless she had an OC-48 or two going into her home, she didn't make the files available for download by *millions* of strangers. When the resource is limited, the magnitude of the crime is likewise limited. If you offer a stolen watch on the streets of New York, you can't be charged with trying to sell it to MILLIONS of people, cause there's only one watch. Likewise, in this case there's only enough bandwidth for a certain number of potential downloads, and speaking of millions here is plain misleading.
If the people who downloaded files from her spread them further, that's THEIR crime and not hers, much as the guy who sold a stolen watch won't be found guilty for the watch buyer illegaly selling it to someone else.

And in this case, it's even less severe, as it's not a theft, but a copyright violation.

Regards,
--
*Art

Re:gee?

[3terrabyte]

Excellent point. The "magic number" system the RIAA uses is astounding. 52X burners count as 3 cd burners? $750 to $150,000 damages PER song is crazy.

I thought I remembered seeing something about how you have to have a certain $$ amount before getting a felony. $2000? ANyway, they then said each song was worth about $200. I think it was something like $20 per song, times 10 people. 10 people being the gestimate of people you magically distributed it to, because obviously more than one person can download a song from you. Anyway, 10 songs and you're a felon.

Anyway, these numbers don't add up. The RIAA likes to paint a screen of terror by saying that your one song you shared, can then be shared exponentially after that. Sure, it's true. You share it to 2 people. They share it to 2. By the end of the day, 1,000,000 people have it. But why would you be responsible for the 2nd thru 20th level of distribution? You only gave it to 2 people. And if it's "worth" $1 on iTunes, why isn't the damage $1 per song per download?

It's this magic number system the RIAA counts by that causes them to sue 4 students for 47 billion dollars. It would have taken the RIAA 5 years of GROSS profits to hit 47 billion dollars. How can a search engine running for a couple months on a campus amount to 5 years of GROSS profits?? It doesn't...make...sense.. you must acquit.

Re:gee?

[Zigg]

Different drives, with the same disc, and identical software, certainly do give different results. Just tested. Identical versions of cdparanoia live on both systems.

I also ran lame with default settings (makes a 128K CBR) on both WAVs and got different sums there as well.

No tags involved.

What happen if

[Anonymous Coward]

you just normalize or edit the begining or the end of the song? Does the MD5 Hashes still works?

Re:What happen if

[l1gunman]

Any modification, to ANY bit of the file covered by the hash, will change the MD5 hash (that's how hashes work). If you assume the hash includes the ID3 tag info, then simply editing the info (putting something in the notes field, for example) would change the hash.

On the other hand, if I were the RIAA attempting to identify common files in this way, I might be inclined to exclude the ID3 tag from the MD5 computation since it is so easily modified.

Any changes to the actual content, though, will ripple into the MD5 computation.

Short answer: "normalizing" the file for volume, or even chopping off a few seconds of trailing silence with something like CoolEdit will certainly change the hash and make it distinct from whatever their baseline hash value is.

Re:What happen if

[1u3hr]

Short answer: "normalizing" the file for volume, or even chopping off a few seconds of trailing silence with something like CoolEdit will certainly change the hash

If that's all you want to do, much better not to use Cooledit, which has to expand and recompress the file to MP3. Use something like MP3Trim which can chop off any given number of MP3 frames, or normalise the volume, by operating on the MP3 directly. Much much faster, and no expand/recompress quality loss.

MD5 Cannot stand up in court.

[Organized Konfusion]

The md5 hashing algorithm has been proven to contain flaws allowing two files to produce identical md5 sums.

Re:MD5 Cannot stand up in court.

[Urkki]

A bit of clarification is in order I think.

First of all it's very clear that two files can give same MD5 checksums. After all, MD5 is only 16 bytes (2^128 different possible). So if you have just 17 byte files (2^136 different possible), it's clear that on average every MD5 sum matches to 256 of all possible files.

It's just damn unlikely to get 2 files with same MD5, and if you wanted to brute force it, you would have to try average 2^64 different files before you found one with identical MD5 to another file. And this would take a long time (actually not that terribly long, a few years at most, and it parallelizes perfectly).

The page you link to implies that it's possible to "easily" fabricate a file that produces a given check sum, so instead of months of processing time, only days or hours would be needed to get a MD5 hash collision.

So all P2P users / software makers need to do to circumvent this, is to agree on a specific MD5 sum, then patch every file so that they produce this same MD5 sum :)

Of course the obivious solution for RIAA would be to use a more secure hash algorithm, with more bits. Unbroken algorithm with enough bits can't be faked, as it would take more than age of the universe to brute force it.

Though the basic problem with this RIAA method remains. If you rip with same software from identical CD digitally, and there are not bit errors at ay point, then you should end up with identical file, and therefore identical hash no matter how secure the algorithm is...

MD5 Hash

[fruey]

This seems quite interesting but I was under the assumption that identical hashes could be created with identical rips and id3v2 tagging.

The only way for two files to have the same MD5 hash is for them to both be encoded with the same encoder, from the same WAV file, with the same bitrate and all advanced options, and to have exactly the same ID3 information, the same filesize, and to be identical to the last bit.

Otherwise, the MD5 will be nothing like the same, for two perfectly identical songs where one has a spelling error in one field of the ID3 tag. I imagine for any one song, there are many many different MD5sums out there, although perhaps one or another good quality version would exists on hundreds of different PCs...

Md5 hashes are also used for....

[shione]

hmm Isn't that how k-sig, built into Kazaa Lite K++, works, by tracking MD5 hashes so ppl get exactly the file they want.

Changing MD5 hashes on songs to avoid RIAA would also lessen the effectiveness of K-SIG. Trading hashes of know working files was one of the ways ppl on P2p avoided downloading those fake RIAA files.

Pity the RIAA

[heironymouscoward]

They are really fighting a losing battle.

Exchanging music is not about piracy, it is about exchanging culture, just like when my grandfather leant me some old Jazz records and said, "here, you might like this".

Today culture moves at the speed of light and the RIAA believes it has the right to tax this movement. It cannot succeed except by destroying the Internet.

I'm starting to believe, watching this debate evolve over many years, that the file traders are right, for the wrong reasons.

Human culture depends on exchange of ideas and information, and music and films are a large part of this in today's world. No album, no movie scene, no written text is a personal creation, they are all taken from the pool of common culture, modified, and redistributed.

Seeking all means to do this faster than ever - and ignoring the barriers, such as "ownership", that stand in the way - is the prerrogative of today's world. We simply can't put the genie back into the bottle and start exchanging pieces of paper and vinyl discs again.

The debate is huge, but the results already seem clear: any laws designed to stop the process from continuing will be further and further ignored until they are seen by a majority of people to be useless vestiges of a material-obsessed past.

HOWTO: Encrypted partition

[geeveees]

modprobe loop
modprobe cryptoloop
modprobe aes

losetup -e aes /dev/loop0 /dev/hdb1
(input password)

mke2fs -j /dev/loop0

mount -t ext3 /dev/loop0 /home/kombat/pr0n

enjoy!

Where does it say MD5?

[eddy]

Are we sure they're actually using MD5? The article doesn't even contain the string "md5" that I can see. It mentions hashes though, but there's something called Robust Hashing which can be used to identify, or at least, compare content in a "fuzzy" way.

Easy

[sprouty76]

Just take a random id3 field that you don't use for anything, and fill it with a random number. You can probably write a srcipt in a few seconds. Bingo, different md5.

The only problem is that a lot of file sharing software uses the fact that 2 files (from different sources) have the same hash in order to swarm the download from multiple sources. If everybody goes around intentionally making their mp3s have different hashes, swarming basically won't work anymore.

Give up

[Rutje]

Ok guys.. let's all give it up. Let's delete all our MP3's and start buying CD's now. The RIAA has clearly won!
Hail to the king!

Re:Or Perhaps...

[perly-king-69]

Ummm, I paid for a CD the other day but I want to listen to it on my MP3 player. The CD is copy protected. I run linux. The only way I can listen to it via mp3 is to, yup, download an 'illegal' mp3! Whoever thought that up was a fscking genius.

MD5 sums and different encoders

[Psyborgue]

Pretty much no rip is identical.

First step: the *.wav is ripped. Using libcdparanoia, which i personally perfer, i find slight variation in size depending on the machine and cdrom drive i rip them on.
Second step: encoding on different machines, with different encoders, using different algorythms, using different levels of floating point precision, on different architectures etc... produces vastly different files.
Third step: sharing. Oftentimes an mp3 is downloaded 99.8% before the connection is broken. You keep the mp3 becuase mp3 is a sequential file format and you only lose a second or two of music. The rest of the file is intact.

Their md5 searching scheme could be circumvented quite easily by changing a comment in the id3 but they could get around that by cutting out the id3 part of the file when they make their md5sum.
The downside to this is that if you are searching for music on something like gnutella by the ***sum, the content would differ and you would not get as many results. Gnutella would not download from multiple sources becuase the file would not have the same signature.
Whatever the case, it is clear that some form of file obfuscation is now needed for safety online. Or we can wait for freenet to mature.

Re:Now what?

[utlemming]

No, we need to create a honeypot farm. You remember that article way back when on Slashdot? It described how to implenent a whole farm. Then we strictly prohibit scanning of the networks for MD5 checksums. Since RIAA is using bots, they won't read the warning and fire off the subeona. When you get a subeona, then you slam them with a computer crime lawsuit. See, you can still get rich from RIAA. But how do you get illegal MD5 check sums with out possesing the files? If you wanna screw with RIAA you have to be damned sure that you right.

From the Napster Network??

[re-Verse]

From the NAPSTER network??? This is worse than i thought - it appears the RIAA has built a Time Machine! Next they will be going further back than napster andprosecuting free-thinking pilgrims who would share their newspapers.

Yikes.

Re:MD5-hashes

[nolife]

I just did some consecutive rips of an audio track and compared the md5 checksums.

I did the same song three times. The first two times, all things were equal including all settings. The MD5 checksums were the same.

I swapped out my DVD/CD player for a different model. Reripped the track on the same computer with the same exact settings and the MD5 was different.

I am using Exact Audio Copy in secure mode and Lame for the encoding. The ID tags were recieved the first time and the same tags used for all three attempts (EAC remembers the disk).

I'm sure I could try many things like changing the read speed, comparing the wav files and not just the resulting mp3 etc.. but I do not have the time for more analysis.

A failure to comunicate

[MarkusQ]

There is an interesting pattern here:

• Some one comments that the IP laws have not kept up with technolgical and social change, and that they are now impeding the cultural goals they origonally served. They may have made sense when we were limited to exchaging physical objects, but they don't make sense now.

And the responses are allong the lines of:

• But it's the law.
• I hope the RIAA gets you.
• Then I suppose an idiot like you won't mind if I take your stuff!

The respondents are completely missing the point. To see this, imagine what the discussion might have looked like if it had happened way back when:

• The rule about not eating X hasn't kept up with the times. It made sense when we didn't know about the parasites, but now that we know how to clean and cook them it doesn't makes sense.

I suspect the responses would have been along the lines of:

• But it's the law.
• I hope the gods get you.
• Then I suppose an idiot like you won't mind eating dog poop!

Every time I see this played out, my response is, "Gee, IP law really is dying, isn't it?", with the same sort of awe I had watching little bits of sand wash downstream at the bottom of the grand canyon.

-- MarkusQ

What nobody seemed to notice.

[Awptimus Prime]

The MD5 thing isn't for tracking the same song ripped by different people. The thread on this, so far, has left me scratching my head as to why folks feel the need to restate that encoding an mp3 with different settings/software will result in a different md5. Right, this is slashdot and we all know this already.

The reason for md5 matching is so they can nail someone as the 'origin' of the ripped song, then hold them liable for all the copies of a matching md5 on P2P networks. It would be more a demonstration of "look how much damage one copy did to us!".

Lost in a Fire?

[medscaper]

A couple of years ago I lost 90% of my CD collection in an apartment fire. I had about 20 of these CDs ripped at the time and since then, I have downloaded many of the others to replace what I had paid for.

Just out of curiosity...Did you have insurance? Did they write you a check for the CDs you lost in the fire? I doubt it, but if it had happened, would still feel you had already "paid for" the CDs, and simply thumb your nose at the RIAA and Big Insurance and download the files, as you'd already "paid for" them?

I promise, I'm not begging to be flamebait. I'm really curious.

Where does the line get drawn between physical property and intellectual property, and what rights do you have if you HAD purchased it, but it's gone now? I mean, I can't go to the lot and get another car because mine is destroyed in a fire. Of course, I could go take a picture of it...but I could do that anyway.

I'm curious.

RIAA Taxes

[brj]

Don't we already pay a small tax to the recording industry every time we buy blank audio CDs (but not data CDs)? I'd like to see some lawyer fight a case claiming that a P2P user has already paid the RIAA and is therefore exempt from their lawsuits when downloading the music and burning it to an audio CD. That would be an interesting lawsuit.