Slashdot Mirror

Wrong.

You just don't know how to use p2p effectively. What most people learn to do is combine p2p with a trustable source (of file "fingerprints"). You've never really been able to trust random search results. e.g. Searching for "paris hilton sex tape" turns up a LOT of renamed fakes, unless you know who to trust for verified file hashes.

Now, eventually trust networks will be built into the p2p protocols, but until then there are central and not-so-central websites that serve as very trustworthy indexes of p2p content.

A few of the more popular ones:

• BitTorrent: NovaSearch, torrentlinks
• eDonkey: ShareReactor, FileNexus
• Kazaa: FastTrackmovies -CLOSED - never really used kazaa much myself.
• Gnutella: Bitzi

--

Not only Sharereactor, but Filenexus too and a few other smaller ones, also for the Edonkey network.

I used to think they survived just because EDonkey had a lower profile compared to the big names like Kazaa, but EDonkey seems to be pretty widely known these days and the indexing sites are still going fine.

But... eMule use file hashes.

How can you possible pick a fake release?

Oh, it was a JOKE! ;-)

What this means for you: Say goodbye to KaZaA and other useful (meaning large, meaning commercially-supported) networks.

So what you're saying is less clients with banner ads, spyware and affiliate scheme tricks, but more clients like eMule and services like Filenexus? Sounds good to me. When do they get started?

This is why websites such as ShareReactor and FileNexus exist, so people can publish the correct checksums of their known-good files for others to download without needing to search for them.

This thought occurred to me last night while doing some kazaa downloading. Maybe a better P2p capture system would involve each client downloading 1 frame per movie, and sharing that with the world. The clients could assemble the movie from a distributed network, much like a frame server does in premiere.

eDonkey does something similar for the files that it downloads. It divides the file into 9mb chunks and when you have one complete chunk, that part is shared on the network. It works really nicely for large files like movies and ISOs. eDonkey also has this neat feature of having urls that stores the file's hash value. So if you share a file, and wan't people to download that particullar file, you just publish the url on a web page. There are whole sites devoted to edonkey links. ShareReactor and FileNexus are the two most common such sites. Check them out and see the power of this system! The eDonkey network is based on servers and anyone can set up one. The maker of eDonkey has now come up with a serverless P2P system called OverNet that is based on the same edonkey protocols for file transfers and link sharing

If you wan't to use edonkey, then get the open source eMule client. It is an edonkey clone with better features and it is open source. It has a lot of mods for various types of addon features. I personally use the eMule Plus MOD. It even has a web server that you can use to control the client remotely!

I personally use Kazaa Lite for mp3 and edonkey for other *cough* software and media. There are a few nice sites for your edonkey needs. :)

You're absolutley right. It's actually already done. Edonkey2000 already does this (site such as: ShareReactor and FileNexus have long lists of high-quality material, all of which is proven to be real. Also for other p2p systems like KaZa there are tools that can make hashes. Of course there are sites that list good hashes.

Not one link, eh? Plenty?

ShareReactor

FileNexus

Asia Movies

Jigle

Various sites specialised in files of certain languages (French, German), such as Spieleplanet

etc etc etc - just search for eDonkey links.

There are also IRC channels and uncounted web boards (similar to Asia Movies) dedicated to sharing ED checksums.

Oh, and about the checksums... look at a popular song, and see how many variations exist in file size at the same quality. Are you saying different files with different sizes won't have the same checksum?

So as I said, I do see this as one of the problems to be solved, although I feel it's of lesser importance. There are many ways of doing this. One of them is previewing - when downloading an audio or video file, when you're about 100k into it (100-200k if it's video), do a preview and see what you're getting. With this looping stuff you have to go farther than 100k however - preview one fourth to one third of the way into the audio files. Many Gnutella clients have a preview feature, as does Fasttrack (Kazaa).

Another method is to ban IP's and IP ranges spreading this. This is already being done - it's only a minor fix because they will always get around it, but it will help somewhat, they won't be able to have big servers spewing this stuff 24/7

The real way to fix this however is hashes. Which are already ubiquitous - they already exist and are known on Gnutella (Shareaza, Gnucleus, Morpheus, Bearshare, Limewire), Fasttrack (Kazaa) and Edonkey2000. On Gnutella (Shareaza) and Edonkey2000, you can click through or cut and paste these URI's (URLs) to files from web sites (or Usenet, IRC, e-mail, instant messengers, whatever) and start searching and downloading the files - for FastTrack (Kazaa), it is a little bit more time-consuming and complex, but worth it if you're going to be downloading a large file. The hash technology is already there, the key now is finding a trusted source for hashes which are both good and whose data is findable and downloadable on p2p networks, and for those sources to survive. I guess I'll detail how this is currently working with the various p2p networks, why not?

There are four major p2p networks - Gnutella, Fasttrack, Edonkey and Freenet. Freenet is a publishing network, the others are all file sharing networks, which is what we're concerned with. Gnutella and Fasttrack are the two largest networks. Edonkey2000 specializes somewhat in large files however, so if it's 100MB+ files you're after, Edonkey2000 is on par, and perhaps better in some ways currently, than Gnutella and FastTrack. Edonkey2000 and FastTrack are closed networks - closed source server/clients and closed protocol networks. Gnutella is open, the protocol is open, and robust open source server/clients like Gnutizen exist for it. This gives Gnutella advantages, such as a choice of multiple clients for virtually every platform, as well as other advantages. Of all the file sharing p2p networks, Gnutella is my favorite and I believe Gnutella is the future of p2p. I think competition amongst p2p networks is healthy however as every can steal everyone elses best features and innovations.

Gnutella files are hashed for HUGE with an implementation called sha1. You can read about the technical aspects here if you wish to. These hashes are useful for finding additional sources for found files so that one can resume downloads or download from multiple sources with integrity. Actually there's one caveat to that - if you are downloading from an honest client, it will tell you a truthful hash of it's data. A client could give a fake hash and then send other data - but you would have to directly download from the rogue. How clients deal with this is even more complex - Gnucleus downloads overlapping chunks - it downloads 1-2000 from one source and 1950-3950 from another - if 1950-2000 do not match from both sources, it marks both chunks as possibly bad. You can read more details about this in Gnutella documentation and discussion groups.

Aside from this usage, these hashes can be used externally as well. Currently, Shareaza, which is a pretty good servent (server/client), is the only one from which URI's (URL's) can be cut, paste, and clicked through to from the web/IRC/e-mail etc. I'm sure clients like Gnucleus will have this ability in the future. If you had Shareaza installed, you could click on a link like this - which is an, I believe uncopyrighted, Chomsky speech, Shareaza would launch (if you don't have it already) and would ask you if you want to download the file or cancel. If you select download it would connect to GnutellaNet, search for the file, and if it found a host which has the file and which has upload slots open, would start downloading it. Actually, the Slashdot "allowed HTML" filters are pulling some necessary characters out of the above link, so you can't click through on /., although you can on a normal HTML web page. I can't post an URL that you can cut and paste either since /. forces a line break after 40 characters or so, if /. didn't do this and the below was in one line, you could have cut and paste it into Shareaza, I'll show it here for an example, imagine this was all on one line for you to cut and paste, or better was just a link to cut. You can do this on any HTML page, it's just the Slashdot HTML parsing messing it up -

gnutella://sha1:HXHSJ6ATN3LQCCIOBGUEWV5FFCKP2KBL/N oam%20Chomsky%20-%20Audio%20Book%20-%20Noam%20Chom sky%20-%20At%20Johns%20Hopkins%20University.mp3/

I would give the above link a rank of "7", because the last time I searched for it, 7 people replied they had it. I have several hashes with a score of 80-90, meaning you're more likely to find or download them, but the above is the only one I have that I have enough confidence in that the data is uncopyrighted.

So now you have one link to a hash - where can you find trusted sources which tell you what hashes are ubiquitous, making it more likely you will find and be able to download them, are rated in terms of quality by multiple sources and so forth? Well for Gnutella, one source is Bitzi. You can search for data there, see what is the most reported, what things are ranked, see comments, see bit rates, file sizes, artists, titles and so forth. It is very cool. Most interaction is from Bitzi into Shareaza (the only Gnutella client that does this currently), but from within Shareaza if you find a file you can type "find Bitzi ticket" and see if the hash has been reported on already. One thing which I'm sure will soon be remedied is that Bitzi does not have direct clickthrough to Shareaza, I have to copy hashes to my clipboard, edit them to Shareaza format and paste them into Shareaza. I'm sure soon Shareaza and Bitzi will agree on a standard and remove this step so I can just click through. And soon Gnutella clients other than Shareaza will have this ability as well. Bitzi's data base is open to the public, you can read their open data policy on their web site, anyone is free to use the data as long as Bitzi is credited. Bitzi.com is the only large, good source of Gnutella hashes I know of. Edonkey2000 has had hashes for a while, and has several good, large sources for hashes such as Filenexus.com and Sharereactor.com. Since Gnutella is a larger network and it just implemented this ability, I'm sure it will have even more and larger sources in addition to Bitzi. And since Bitzi's database is open to all, if Bitzi goes down someone else can open the database up again somewhere else. I'm sure in the future, even the trusted rating system will become distributed.

Gnutella uses the sha1 hash, Edonkey2000 uses another, and Kazaa uses another. Web sites exist that centralize the hashes for these. I'm sure soon web sites will exist that coalesces and translates all of this. Gordon Mohr, who runs Bitzi, wants to see a universal p2p tag, magnet, which is agnostic about which p2p backend it is using. Why not? We can have a tag that we (more or less) trust, and can retrieve the data from Gnutella, FastTrack, Edonkey2000 or Freenet. It's a great idea.

I am less interested in other p2p networks than Gnutella but I'll discuss their hash and meta-data web sites a little. The most interesting one is Edonkey2000, which as I said, has come to specialize in large (100MB+) files, and which I have to admit is a pretty good way to download large files with some guarantee of integrity. There are two major meta data sites for Edonkey - Filenexus and Sharereactor. There are other sites as well. If you're looking for large files, they do a pretty good job currently.

Fasttrack (Kazaa) uses hashing, but the Kazaa client is not that friendly to this kind of thing. So Fasttrack/Kazaa is more of a pain in this respect than any of the others. Nonetheless, you can download a program called Sig2dat that helps you copy and paste FastTrack's UUhashes. The you can go to web sites that give meta data, rankings and so forth to these hashes. Kazaa/FastTrack is unfriendly to all of this so it is much more of a pain - you have to install files that help you do this (sig2dat), you have to restart Kazaa for every file you want to download in this fashion and so forth. With Kazaa, all of this is a hassle, it's much easier to do in Gnutella (Shareaza), Edonkey2000 and Freenet.

And lastly there is Freenet. Freenet has been using hashes since the beginning. Freenet is a publishing network, not a file sharing network. That is nomenclature - file can be and are shared on Freenet - from html pages to gifs and jpgs, to mp3's, to avi's, although Freenet is the last place you want to look for large files, Freenet's bailiwick is small files. Even a 4 meg mp3 on Freenet is harder to find and slower to download than any of the other 3 networks. Small files are the domain of Freenet - HTML pages and images. The Freenet protocol is more rich than the other protocols in many ways, thus you have more than just audio and video files going over it, you have third-party applications utilizing it, thus you have things like Fproxy (A world-wide web equivalent which runs over Freenet) and Frost and Freenet message board (Usenet equivalents - both for text and binaries). One benefit of Freenet is it's hard to crack down on people for publishing information - because no one knows who data is coming from or going to. This is not absolute, but it is much safer than the file sharing p2p networks in this respect. Also, people publish data, so that what you put out is stored somewhere other than your computer, and if your web site or shared file or whatnot is popular, it will be out there all the time without your node needing to be connected. Freenet also used a lot of signatures, encryption and so forth, so you already have a pretty solid trust mechanism and data integrity. It depends on what hash is used - KSK hashes are insecure, but SSK are signed. So with Freenet there are large upsides and downsides - the downsides are downloading is much slower, since you're downloading via intermediaries, not directly, and the larger the file, the slower the download and the harder it is to find a complete file. The upshot of Freenet is that there is less of a legal risk with regards to sharing/publishing data, data is signed by the publisher which greatly helps integrity, and also Freenet's protocol allows extensions other than file sharing with it's own internal network - web and Usenet like applications, and I'm sure there will be more in the future.

I hope you have edonkey installed. GET IT!

• LAME - the best MP3 encoder out there
• Exact Audio Copy - the best CD ripper, reads every sector at least two times to ensure maximum quality
• Standard MP3 Naming Scheme - So your MP3s are named consistantly
• Zeropaid - file sharing news and rumors
• Slyck - another news site, less community-oriented
• Blubster - A small but quickly growing MP3 community, very fast servant, good community
• EDKGuide - Once you master MP3s, the next step is DivX
• FileNexus - music, SVCDs, and DVD rips

Hope this helps!

If you're having problems finding valid files, try tracker sites like Gowenna/Sharereactor and Filenexus. You can also find dynamic serverlists.

eDonkey's developers are aware of the network's current strain and are implementing a new p2p method for the next version.