Slashdot Mirror

Wrong.

You just don't know how to use p2p effectively. What most people learn to do is combine p2p with a trustable source (of file "fingerprints"). You've never really been able to trust random search results. e.g. Searching for "paris hilton sex tape" turns up a LOT of renamed fakes, unless you know who to trust for verified file hashes.

Now, eventually trust networks will be built into the p2p protocols, but until then there are central and not-so-central websites that serve as very trustworthy indexes of p2p content.

A few of the more popular ones:

• BitTorrent: NovaSearch, torrentlinks
• eDonkey: ShareReactor, FileNexus
• Kazaa: FastTrackmovies -CLOSED - never really used kazaa much myself.
• Gnutella: Bitzi

--

FastTrackMovies.com is good if you are using Kazaalite or other "clones". Click on the sig2dat file, download the exact film/tv series you want!

It's about fucking time they did this. I was thinking of this myself about a week ago....I was attempting to download the movie "Identity" which only took less than an hour using Kazaa with my cable modem.

I couldn't find the real one at fasttrackmovies.com or their forum, so I had to guess at which one would be the real one. I tried twice and they were both fakes.

Man, was I pissed. I started thinking about the waste of bandwidth that had just occured and wished these fuckers would get nailed for spam.
I finally got it off of Bit Torrent, but how long before these motherless sons of bitches start poisoning the BT files as well?

It's about fucking time they did this. I was thinking of this myself about a week ago....I was attempting to download the movie "Identity" which only took less than an hour using Kazaa with my cable modem.

I couldn't find the real one at fasttrackmovies.com or their forum, so I had to guess at which one would be the real one. I tried twice and they were both fakes.

Man, was I pissed. I started thinking about the waste of bandwidth that had just occured and wished these fuckers would get nailed for spam.
I finally got it off of Bit Torrent, but how long before these motherless sons of bitches start poisoning the BT files as well?

And I say it's bollox that Kazza has the most content.

I guess it depends on what you're searching for. I had already installed eMule but rarely use it, as Kazaa generally has everything I search for.

For instance, looking for "Family Guy" I find only 10 files matching on eMule; only 8 of these are full episodes (>200 MB; the other two are 64 MB (a compressed episode perhaps?) and 1 MB).

On Kazaa, there are 54 full episodes (>200 MB).

My sister wanted to see "Strangers With Candy". I was able to get all the episodes from Kazaa (30 of them; not all were 200 MB, some were more compressed, down to 70-80 MB). However, eMule doesn't have a single episode (or match of any type).

Sometimes when my ReplayTV doesn't record something, I look for it on the net. In these cases I generally only find it using BitTorrent, as there's a big TV taping community sharing files through BT. Google for "BitTorrent TV" and you'll find it.

I downloaded the Bitzi software (BitCollider), but it can only tell me about files that I've already downloaded. And I have sig2dat but it's used by sites like FastTrackMovies -- they give a link, you click on it, and sig2dat converts the link into a search/download in Kazaa. I don't see a way for it to tell me whether a certain file which shows up from a search is valid or not. (If you know, please respond! Thanks.)

I was shocked to find out you were right about the original Harry Potter movie, but the second one (Chamber of Secrets) is already available on DVD-rip via FastTrackMovies (a very good site btw!)

So, ok these guys have essentially done what FastTrackMovies has done and hashed each file. Hunky dory. So, people implement this and think "no one can trade my files, cause we know what they look like (and have the hash), so we can block it."

Now, Joe Pirate simply .zips or .tars the music or movie.

Exactly how would they then block the .zipped asset from being traded? I know it won't compress the MP3, but it will change the fingerprint.

Methinks WinZip is the Sharpie for this expensive DRM.

I've figured out a way this can happen and is happening on p2p just now. Here's the sequence:

1. Troll user or industry contractor downloads a common file that is genuine. LOTR on DivX for example.
2. File gets renamed to something else.
3. New user comes along, sees the fake and starts to download it. important note: the file on your machine is always named the same as the original you first selected to download
4. Each copy of the file (including the properly named ones) becomes a valid hash-compatible alternate source. So even if the fake-providing user goes off-line, there will still be sources.
5. Here's the key part: Your partial downloads are shared. Other users see you with the fake file, even though you don't even know it's fake yet. They start to download it from you, with the same fake name. If it's a popular fake name, the effect snowballs from there.

Unless everyone deletes the fake file at the same time, it's going to be there forever. This works best for large files, so you'll see a lot of mp3 singles that need overburning to fit on one disk. A lot of users don't know what to expect as a filesize, so they can and are caught out by this.

I've been aware of this for sometime now. I didn't want to post it anywhere, in case it gives someone any ideas. This thread has kinda mucked that up though, so it doesn't matter anymore.

Now, the question is, what to do to avoid this issue:

1. Use WinMX and it's "Search alternates" feature, which will look for files with the same hash. You should remove the first line of the search (the filename) and re-click on "find", so you are getting back all files of the same checksum. If most of the names don't match what you have, it's usually a fake. Kazaa doesn't allow anything like this unfortunatly.
2. Install a tool like sig2dat which gives your system a new net service available through web pages. You get a link like "sig2dat://......" which, when clicked on it will create an empty partial download for Kazaa. Restart Kazaa and it will begin looking for that particular file. If you trust the web page, then you are happy. FastTrackMovies springs to mind.
3. Pay attention to file sizes.
4. Block IPs that are "nasty"

What really floats my boat is the evential outcome of this. The industry is shooting itself in the foot in this arms race between them and the world, that they cannot possibly hope to win.

Think of it this way. Soon, no one will trust filenames in p2p and the searches will become redunant. One of two things will happen: People will start remortgaging their homes again to buy CDs. Or, people will create better systems that allow ratings of files, like the sig2dat system.

This is fantastic for the p2p user. Not only do you know that you are getting the right file; you'll also have reviews and comments on it's quality and listings of other files you wouldn't have normally thought of searching for. Entire albums can be queued in one click (the question is, will Amazon sue?!? ;-)

What I envisage happening long term is p2p being more of a service on the PC, with little user interaction. To send someone a file, you send them a "link" to that file on the network, and your client seeks it out itself. Just like the birth of Napster, the record/movies industries choice of action (or inaction) will ultimately bite them. Evolution doesn't work well unless someone is hacking away at the weak links.

For Kazaa, etc. you can try out sig2dat, which works similarly to the MD5 checksums. You click on a sig2dat link and the program will generate a .dat file with a checksum in it, which Kazaa will use to search for the file.

It's not perfect though, since you have to shut down and restart the client to get it to see the generated .dat files. It's nice when using sites such as Fast Track Movies.

Pssst: http://www.fasttrackmovies.com may be able to help you there.... Have a read (and, hopefully contribute) to the forum as well..

Something like the sig2dat system that is used by very handy sites such as FastTrackMovies. Basically, sig2dat is a Windows program (although Linux variants are now available) which generates a checksum from a media file - which then can be posted on a website. This link uses the custom sig2dat:// protocol which, when clicked on, is picked up by sig2dat which uses the checksum to generate the necessary 'start download' file needed by the FastTrack clients (Kazaa, KazaaLite etc). FTM holds a list of these checksums (which are generated by the file's contents and the length of the file) for a large number of movies/tv programs - allowing you to click on one of the links and get the verified media with 99% certainty (ok, there is a chance that there could be another media file out there with exactly the same checksum, but I haven't encountered it yet).

So as I said, I do see this as one of the problems to be solved, although I feel it's of lesser importance. There are many ways of doing this. One of them is previewing - when downloading an audio or video file, when you're about 100k into it (100-200k if it's video), do a preview and see what you're getting. With this looping stuff you have to go farther than 100k however - preview one fourth to one third of the way into the audio files. Many Gnutella clients have a preview feature, as does Fasttrack (Kazaa).

Another method is to ban IP's and IP ranges spreading this. This is already being done - it's only a minor fix because they will always get around it, but it will help somewhat, they won't be able to have big servers spewing this stuff 24/7

The real way to fix this however is hashes. Which are already ubiquitous - they already exist and are known on Gnutella (Shareaza, Gnucleus, Morpheus, Bearshare, Limewire), Fasttrack (Kazaa) and Edonkey2000. On Gnutella (Shareaza) and Edonkey2000, you can click through or cut and paste these URI's (URLs) to files from web sites (or Usenet, IRC, e-mail, instant messengers, whatever) and start searching and downloading the files - for FastTrack (Kazaa), it is a little bit more time-consuming and complex, but worth it if you're going to be downloading a large file. The hash technology is already there, the key now is finding a trusted source for hashes which are both good and whose data is findable and downloadable on p2p networks, and for those sources to survive. I guess I'll detail how this is currently working with the various p2p networks, why not?

There are four major p2p networks - Gnutella, Fasttrack, Edonkey and Freenet. Freenet is a publishing network, the others are all file sharing networks, which is what we're concerned with. Gnutella and Fasttrack are the two largest networks. Edonkey2000 specializes somewhat in large files however, so if it's 100MB+ files you're after, Edonkey2000 is on par, and perhaps better in some ways currently, than Gnutella and FastTrack. Edonkey2000 and FastTrack are closed networks - closed source server/clients and closed protocol networks. Gnutella is open, the protocol is open, and robust open source server/clients like Gnutizen exist for it. This gives Gnutella advantages, such as a choice of multiple clients for virtually every platform, as well as other advantages. Of all the file sharing p2p networks, Gnutella is my favorite and I believe Gnutella is the future of p2p. I think competition amongst p2p networks is healthy however as every can steal everyone elses best features and innovations.

Gnutella files are hashed for HUGE with an implementation called sha1. You can read about the technical aspects here if you wish to. These hashes are useful for finding additional sources for found files so that one can resume downloads or download from multiple sources with integrity. Actually there's one caveat to that - if you are downloading from an honest client, it will tell you a truthful hash of it's data. A client could give a fake hash and then send other data - but you would have to directly download from the rogue. How clients deal with this is even more complex - Gnucleus downloads overlapping chunks - it downloads 1-2000 from one source and 1950-3950 from another - if 1950-2000 do not match from both sources, it marks both chunks as possibly bad. You can read more details about this in Gnutella documentation and discussion groups.

Aside from this usage, these hashes can be used externally as well. Currently, Shareaza, which is a pretty good servent (server/client), is the only one from which URI's (URL's) can be cut, paste, and clicked through to from the web/IRC/e-mail etc. I'm sure clients like Gnucleus will have this ability in the future. If you had Shareaza installed, you could click on a link like this - which is an, I believe uncopyrighted, Chomsky speech, Shareaza would launch (if you don't have it already) and would ask you if you want to download the file or cancel. If you select download it would connect to GnutellaNet, search for the file, and if it found a host which has the file and which has upload slots open, would start downloading it. Actually, the Slashdot "allowed HTML" filters are pulling some necessary characters out of the above link, so you can't click through on /., although you can on a normal HTML web page. I can't post an URL that you can cut and paste either since /. forces a line break after 40 characters or so, if /. didn't do this and the below was in one line, you could have cut and paste it into Shareaza, I'll show it here for an example, imagine this was all on one line for you to cut and paste, or better was just a link to cut. You can do this on any HTML page, it's just the Slashdot HTML parsing messing it up -

gnutella://sha1:HXHSJ6ATN3LQCCIOBGUEWV5FFCKP2KBL/N oam%20Chomsky%20-%20Audio%20Book%20-%20Noam%20Chom sky%20-%20At%20Johns%20Hopkins%20University.mp3/

I would give the above link a rank of "7", because the last time I searched for it, 7 people replied they had it. I have several hashes with a score of 80-90, meaning you're more likely to find or download them, but the above is the only one I have that I have enough confidence in that the data is uncopyrighted.

So now you have one link to a hash - where can you find trusted sources which tell you what hashes are ubiquitous, making it more likely you will find and be able to download them, are rated in terms of quality by multiple sources and so forth? Well for Gnutella, one source is Bitzi. You can search for data there, see what is the most reported, what things are ranked, see comments, see bit rates, file sizes, artists, titles and so forth. It is very cool. Most interaction is from Bitzi into Shareaza (the only Gnutella client that does this currently), but from within Shareaza if you find a file you can type "find Bitzi ticket" and see if the hash has been reported on already. One thing which I'm sure will soon be remedied is that Bitzi does not have direct clickthrough to Shareaza, I have to copy hashes to my clipboard, edit them to Shareaza format and paste them into Shareaza. I'm sure soon Shareaza and Bitzi will agree on a standard and remove this step so I can just click through. And soon Gnutella clients other than Shareaza will have this ability as well. Bitzi's data base is open to the public, you can read their open data policy on their web site, anyone is free to use the data as long as Bitzi is credited. Bitzi.com is the only large, good source of Gnutella hashes I know of. Edonkey2000 has had hashes for a while, and has several good, large sources for hashes such as Filenexus.com and Sharereactor.com. Since Gnutella is a larger network and it just implemented this ability, I'm sure it will have even more and larger sources in addition to Bitzi. And since Bitzi's database is open to all, if Bitzi goes down someone else can open the database up again somewhere else. I'm sure in the future, even the trusted rating system will become distributed.

Gnutella uses the sha1 hash, Edonkey2000 uses another, and Kazaa uses another. Web sites exist that centralize the hashes for these. I'm sure soon web sites will exist that coalesces and translates all of this. Gordon Mohr, who runs Bitzi, wants to see a universal p2p tag, magnet, which is agnostic about which p2p backend it is using. Why not? We can have a tag that we (more or less) trust, and can retrieve the data from Gnutella, FastTrack, Edonkey2000 or Freenet. It's a great idea.

I am less interested in other p2p networks than Gnutella but I'll discuss their hash and meta-data web sites a little. The most interesting one is Edonkey2000, which as I said, has come to specialize in large (100MB+) files, and which I have to admit is a pretty good way to download large files with some guarantee of integrity. There are two major meta data sites for Edonkey - Filenexus and Sharereactor. There are other sites as well. If you're looking for large files, they do a pretty good job currently.

Fasttrack (Kazaa) uses hashing, but the Kazaa client is not that friendly to this kind of thing. So Fasttrack/Kazaa is more of a pain in this respect than any of the others. Nonetheless, you can download a program called Sig2dat that helps you copy and paste FastTrack's UUhashes. The you can go to web sites that give meta data, rankings and so forth to these hashes. Kazaa/FastTrack is unfriendly to all of this so it is much more of a pain - you have to install files that help you do this (sig2dat), you have to restart Kazaa for every file you want to download in this fashion and so forth. With Kazaa, all of this is a hassle, it's much easier to do in Gnutella (Shareaza), Edonkey2000 and Freenet.

And lastly there is Freenet. Freenet has been using hashes since the beginning. Freenet is a publishing network, not a file sharing network. That is nomenclature - file can be and are shared on Freenet - from html pages to gifs and jpgs, to mp3's, to avi's, although Freenet is the last place you want to look for large files, Freenet's bailiwick is small files. Even a 4 meg mp3 on Freenet is harder to find and slower to download than any of the other 3 networks. Small files are the domain of Freenet - HTML pages and images. The Freenet protocol is more rich than the other protocols in many ways, thus you have more than just audio and video files going over it, you have third-party applications utilizing it, thus you have things like Fproxy (A world-wide web equivalent which runs over Freenet) and Frost and Freenet message board (Usenet equivalents - both for text and binaries). One benefit of Freenet is it's hard to crack down on people for publishing information - because no one knows who data is coming from or going to. This is not absolute, but it is much safer than the file sharing p2p networks in this respect. Also, people publish data, so that what you put out is stored somewhere other than your computer, and if your web site or shared file or whatnot is popular, it will be out there all the time without your node needing to be connected. Freenet also used a lot of signatures, encryption and so forth, so you already have a pretty solid trust mechanism and data integrity. It depends on what hash is used - KSK hashes are insecure, but SSK are signed. So with Freenet there are large upsides and downsides - the downsides are downloading is much slower, since you're downloading via intermediaries, not directly, and the larger the file, the slower the download and the harder it is to find a complete file. The upshot of Freenet is that there is less of a legal risk with regards to sharing/publishing data, data is signed by the publisher which greatly helps integrity, and also Freenet's protocol allows extensions other than file sharing with it's own internal network - web and Usenet like applications, and I'm sure there will be more in the future.