Peer-to-Peer Overview

An anonymous submitter sent in: "New Scientist has an interesting feature on peer to peer systems, taking a less copyright orientated approach, and going into some technical detail about how the various P2P systems work and compare to each other."

Comments

Re:Jews steal your food dollars

[s390]

You are one sick puppy. Seek help, please.

Too simplistic

[Sanity]

You seem to fall into the common trap of thinking that the different P2P architectures are just different approaches to doing the same thing. This isn't the case, there is actually very little in common between the various architectures as they generally have very different goals. For example, Napster and Gnutella are both designed to let people share their mp3s with other people, Freenet is designed to provide a secure forum for free speech, Seti@home is designed to combine people's spare cycles to find aliens etc. These systems are as different as chalk and cheese, just because many journalists think they fall under the P2P buzzword, doesn't mean that they have any more in common than any other software, nor that there is any more room for interoperability than there is with any other software that communicates via the Internet.

The claim that P2P would be great if only the systems would interoperate really doesn't bear much scrutiny, TCP/IP is often the full extent of what these systems have in common. This isn't a flaw, it is a simple fact.

--

Re:Too simplistic

[JohnsonWax]

Actually, it really isn't too simplistic.

Dave Winer is closing in on this with his just released Radio Userland product. It's a fairly generic product that uses XML, XML-RPC, SOAP, etc. to exchange information between clients and client-server. It doesn't much matter whether it's headlines, mp3 files, quicktime movies, news stories, html templates, whatever. It has the plug-in archtecture and a built-in database, scripting language, web client and server. I don't see any reason why it couldn't also work as a CPU sharing mechanism as with Seti@home. I could probably write such a plug-in for his product in a day.

Admin slashdot reader?

[lastninja]

I'm just making a wild guess here but the admin must have seen the /. article and pulled the cord or something.
-You'll never get me evil slashdotters, muawahahahahahahaha!

I've got the artical

[koshi]

As an actually subscriber to newscientist I have read the artical (in print).

It starts with a breirf history of P2P systems (focusing on the modern music sharing ones).
It then goes on to talk about Gnutellas technological downfalls. The main part of the artical is taken up with the method that by which freenet works

Its a good articale that talks quite a bit about the technologys and only breifly touches on copyright issues.

Freenet Inaccuracies

[retep]

Freenet is anonymous, it's archetecture automaticly does proxying and multiple layers at that. While the first node you connect to can figure out who you are they can't easilly determine if you initiated the request and any nodes down the further down the chain don't have a clue about who you are. Unlike Gnutella when a Freenet request is fufilled the requesting node doesn't directly connect to the node that has the data.

Secondly by creating many copies of popular data you are decreasing the amount of bandwidth used in exchange for increasing the amount of space used. Say you have a popular site in Europe that's getting lots of hits from the US. Without Freenet every time someone in the US wants the data their request has to go over congested international links, very wastefull that. With Freenet a copy of the data, probably many copies in fact, would soon get stored in the US reducing the load on those slow international links. Even though the data has to go through many nodes in between, and has multiple copies of it made, you still get a significant savings in bandwidth usage. And any copies made while en-route are likely to be used again by other people because those copies are by-nature "close" to the keyspace that node serves.

Ever heard of Akamia? They do essentially the same thing with their network to reduce the load on sites around the net.

Now currently Freenet doesn't do a very good job of this because it doesn't take network speeds into account when choosing nodes while routing messages. But it will be quite simple to add support for that and will be done after a few more releases.

Re:Freenet Inaccuracies

[the_quark]

Yes, it's true that there will be many copies, and many of them will be relatively "near" me on the Internet. However, it is not expected that all (or even most) data will be one hop away. Any data that is more than one hop away will end up being proxied (transferred) 1 time for each server. That's how anonymity is provided. If we're talking about really big files like MP3s, number one, that means that, as a Freenet node, I'm passing through a LOT of traffic to the people near me proxying through me. Number two, it means that, when I click "download," it's got to be actually downloaded and uploaded two or three times for the average case. If one of those links in the middle has a 56k (or reboots in the middle), it's gonna be a REALLY slow process.

Akamai caches, yes, and so does Freenet. But Akamai doesn't proxy your download through three other peers.

You then finish, "Freenet doesn't do a very good job of this because it doesn't take network speeds into acocount...But it will be quite simple to add support for that and will be done after a few more releases." Yes, and, if you do that, it'll become a backbone network, because all clients will "want" to connect to the few fast peers. If I'm the RIAA, I just DMCA notice all those fast peers' ISPs on the same day, and the network falls over.

Regardless of specific attacks, my point is that Freenet (or any P2P) is, by definition, going to be more complex (and hence more expensive) than a simple, centralized system, and will therefor only be used when the primary overriding concern is trying to hide who is distributing the content, not for people who want to put up family pictures.

Re:Freenet Inaccuracies

[retep]

Yes, it's true that there will be many copies, and many of them will be relatively "near" me on the Internet. However, it is not expected that all (or even most) data will be one hop away. Any data that is more than one hop away will end up being proxied (transferred) 1 time for each server. That's how anonymity is provided. If we're talking about really big files like MP3s, number one, that means that, as a Freenet node, I'm passing through a LOT of traffic to the people near me proxying through me.

You're forgetting just how powerfull the automatic adaptation of the routing is. Simulations have shown again and again that the number of hops for any data is kept *very* low, about 5 hops for a large network with lots of data. As data is requested the routing tables get optimized so that the node that returned the data is likely to be contacted again for similar data. Also on insertion the data is more likely to go to that node.

Also should you ever request data twice it's one hop. I know I request a heck of a lot of data multiple times...

It's not as efficient as normal internet routing for unpopular data but way more efficient for popular data. And I'll guess that a good %70 of the data I use would be classified as "popular", slashdot front page and stuff linked off of it for instance. (this is ignoring dynamicly created stuff which Freenet can't do anyway)

Number two, it means that, when I click "download," it's got to be actually downloaded and uploaded two or three times for the average case. If one of those links in the middle has a 56k (or reboots in the middle), it's gonna be a REALLY slow process.

As I mentioned network speed will be taken into account in future versions of Freenet so your 56k modem will only be used as a last resort.

As for rebooting nodes just plain don't get rebooted often. Nodes that don't aren't on 24/7 just don't get added to routing tables. Freenet is *very* harsh against nodes that aren't always on. Probably more harsh then it should be really.

You then finish, "Freenet doesn't do a very good job of this because it doesn't take network speeds into acocount...But it will be quite simple to add support for that and will be done after a few more releases." Yes, and, if you do that, it'll become a backbone network, because all clients will "want" to connect to the few fast peers. If I'm the RIAA, I just DMCA notice all those fast peers' ISPs on the same day, and the network falls over.

The fastest peers won't be the same for everyone. It's far faster to connect to a system 3 hops away from you then some system on the other side of the planet. Anyway Freenet is already biased torwards nodes with big datastores. And if you do knock out those nodes not all that much happens. I've tested this on my node and if I block access to even a node with %70 of the references things return to normal in a few hours. And it's unlikely that any node would rival the hundreds of smaller nodes around.

Regardless of specific attacks, my point is that Freenet (or any P2P) is, by definition, going to be more complex (and hence more expensive) than a simple, centralized system, and will therefor only be used when the primary overriding concern is trying to hide who is distributing the content, not for people who want to put up family pictures.

Any *free* system is going to be used for illegal stuff if the users can get away with it. Freenet still provides many advantages over centralized servers when you're trying to serve large data. The automatic caching makes the slashdot effect just not happen. The *more* often a site is requested the faster and more reliable it is to get the site. I use Freenet, this is true and works really well even in spite of all the bugs and other nastyness in the network right now.

Why?

[Sanity]

The thing that irks me, and the thing you have failed to address, is the expectation that all P2P systems should interoperate. Of course, it is good when people standardize on open protocols such as XML and XML-RPC, but using XML doesn't suddenly make your software interoperable, and sometimes it isn't possible. For example, the communication protocol used in Freenet has very specific requirements in terms of crypto, forward deniability, and security, which would mean that we anyone who wanted to talk to Freenet must speak Freenet's language.

Interoperability is a good goal in general terms, but there is nothing about P2P systems which makes them more in-need of interoperability than any other software. In many cases, TCP/IP is all the interoperability they need.

--

-1 Uninformed

[Sanity]

You really don't seem to understand how any of the P2P systems work, particularly Freenet. I suggest you read this paper to get better informed.

--

Re:-1 Uninformed

[the_quark]

I'm quite aware of how ALL the P2P systems work, especially freenet. I'd love to see any particular criticisms you have beyond an outright dismissal of my arguments because I "don't understand." My statements about Freenet are accurate and true; if I request a document from Freenet, unless a node one hop away has it, there are going to be multiple proxy copies made to service it. To quote from the document you link to, "The data is returned from d via e and b back to a, which sends it back to the user." That's essentially a full download to e, a full download to b, and a full download to a. If any one of those guys is on a 56k modem, it's gonna suck.

To reiterate - P2P, by definition, requires more complexity, bandwidth and expense than a simple centralized system. My argument is that, like crypto, the extra hassle and complexity will mean that people will only use P2P when it is more effective than cheaper and simpler alternatives. The only times that will be the case is if you are trying to avoid detection as the distributor of the data.

As an aside, I very strongly doubt that Freenet would survive a concentrated DMCA attack. Each document has a uniquely identifiable string. If I'm the RIAA, I log onto Freenet with a hacked client. I do a search. The server one hop away says "Here ya go." As the RIAA, I don't know if that server proxied it or is storing it. And it doesn't matter. I send out a DMCA notice to the guy who owns the server (I know is IP and the time he was on, so I can do this via is ISP, who will cooperate, because he doesn't care). The guy who owns the server now has "actual knowledge" that copyrighted material is moving through his system. He can either block it (have his Freenet server not respond to requests for that), or simply drop out of Freenet. If he does neither, the RIAA DMCA notices his ISP, which simply pulls the plug on him, because they don't care (and he's prolly violating their terms of service by running a server, anyhow).

I think something like Freenet can only work if most of the documents on it are the kinds of things people will risk going to jail for. I don't see most people risking going to jail for free MP3s.

Pionner in peer-to-peer? Helloooo????

[Pig Hogger]

Napster will likely be remembered not so much for enabling music piracy as for starting a revolution that changed the way the Internet worked.

Napster is the pioneer of a technology known as peer-to-peer networking, or P2P for short. The core idea of P2P is to allow individual computers to communicate directly over the Internet.

Er? Am I missing something here? Isn't "The Internet" already a peer-to-peer network???? Hasn't it always been????

--

Re:Why P2P sucks for legal activities

[evanbd]

There's another important reason: if a node can go down at any time, then it doesn't matter if I wish to run a server on my desktop. Gnutella lets me share files (many of them legal), while only remaining connected for a few hours or days at a time. If I go down, then someone else (hopefully) has the data also, so it can still propogate. P2P in its current invocations doesn't allow nicely published pages, because the clients are intended to download a single file, not a collection thereof. There are cases where I would like to be able to share my (legal) music and videos that other people want (and they do), without having to find a way to provide a server that is up 99%+ and has sufficient bandwidth for all requests. P2P provides an answer, provided other people provide mirrors (something I try to do). Also, the mirrors are so easy to create -- just download the file and *poof* there it is.

Fun links 4 u!

[Bi()hazard]

While we're waiting for the article to come back, take a look at this site about p2p, this article with statistics on gnutella and links, and of course this list of p2p clients. There might be some interesting things in this slashdot article. And yes I am a filthy karma whore, and no I have no shame whatsoever.

How to better Napster...

[Anonymous Coward]

Seeing as absolutely nobody here has read the article (it seems to have experienced some sort of preemptive Slashdotting--must be a new IIS "feature"), I can only guess as to whether I'm on-topic here...

How to make the successor to Napster

Most people don't realize that what made Napster successful was not its network protocol or its technical sophistication. Ease of use made Napster, and ease of use will make or break every P2P system. For a P2P system to work and be useful, it has to be simple, easy to find, easy to install, easy to use, and easy to upgrade. Why upgrade? Because upgrades will be necessary to combat "blocking" technologies instituted by ISPs.

The way to accomplish this is not to write yet another Gnutella or OpenNap or Freenet or Blocks client. There are enough of these today. The solution is not necessarily to create new protocols--too many protocols, and your average Joe won't know what to use or why. Furthermore, you open yourself to liability if you do so, as described in this excellent white paper from the EFF.

The solution is to write a generic P2P interface. Make it extensible via protocol plugins. Users should be able to type in one search query, and have it search OpenNap, Gnutella, and Freenet simultaneously, and display all the results in one window. Plugins should concern themselves only with the P2P protocol, and with comforming to some standard interface.

Do not write both the plugins and the interface yourself. Obviously, a GUI that does nothing by itself is not an infringing device. Better to compartmentalize things and make it harder for people to be hit with absurd lawsuits. Furthermore, the programmers who tend to be good at network protocols are generally not so hot at GUI design, and vice versa.

Try to keep the plugin architecture very cross-platform. Ideally, plugins would be totally portable code requiring only a recompile (even when going from Windows to MacOS, for instance), or--better yet--would just be a bunch of specs in XML format.

Set up a mechanism that allows updates to be distributed via the P2P network. Obviously, some sort of signing/verification will be required to reduce the risk of trojans. Still, the goal should be to make it almost transparent for novice users to add support for new protocols. If it becomes easy to implement, distribute, and adopt a new protocol every month, it will be impossible to stop P2P. Updating the GUI is less important--it's the protocols that have to change.

Conclusion: What this will do is create a highly adaptable, fault-tolerant, lawsuit-resistant network that is easy to use. Why have I placed so much emphasis on ease of use? Without a large userbase, you're nothing. A P2P network is only as good as the libraries of its users. Why is this network so adaptable? Suppose one or two protocols get shut down or blocked or whatever. Someone creates a new protocol, writes a plugin, distributes it by one of the remaining channels, and we're back in business. Finally, the system also encourages modularity and code reuse, which is A Good Thing.

By the way, if you can't figure out why I'm anonymous, you haven't been paying attention.

19021312

Re:How to better Napster...

[quick_dry_3]

Try to keep the plugin architecture very cross-platform

I'd suggest not using XML specs, but including some interpreter for byte compiled code - such as a small Python interpreter. This way all the logic is inside the plugin, and the application using taht plugin can be dumb, it doesn't have to read through a defined spec, build an engine to understand it and then talk that protocol.

It would probably be more of a bear to do this and write the XML plugin spec, than it would be to write a module for an interpreter.

Bytecode compiled modules are platform independant, so the same code piece will run on Mac, Win32, UNIX, QNX, whatever... no need for a recompile.

I suggest python over java since its already ported to more platforms and a minimal interpreter is probably going to be smaller than the Java interpreter - and Guido isn't going to beat you over the head with language spec conformance issues like Sun might.

intent should be general.

[Odinson]

Look it may be simplistic, but it's necessary. It seems to be a noble goal and even if it's as hard as a desktop enviorment to implement it's the only way to insure we are allowed to anonymously file share in the future.

To simplify coding it he/she/it could have suggested...

...breaking the plugins down to client and server modules, but that makes it leagaly vulnerable

...breaking it down by type of service:ptp, central-database-based and single node shares, but that also makes it weak to attack legally.

So the question really is, what do ptp and central-index, and node by node file sharing have in common.

This should be really useful for downloading free software public domain type stuff too. If not it will be a short conversation when intent becomes a topic in the court.

slanted soapbox, preachy rant type thing

One could look at the litigation and destruction of napster as the dumbest thing corporate media ever did. All those users were going to one place, and may-not have-ever-known they could learn a new interface... but they will know now. When the RIAA goes after the next target making the next switch will be much easier for those millions of users. It might be as easier for techie types to adopt an ex-napster user who is wandering the streets like a lost dog, and show them 2 way nature of the internet. After all not ALL musicans are/want-to-be RIAA and for them this kinda sucks.

Ex-napster users have been unAOLified. awoken from their clicky-clicky bad dream. I haven't met anyone who is a born again AOLer, have you?

End rant type thing

Napster - Distributed servers

[powlette]

I'm not one to priase Shawn Fanning, who is obviously a punk kid, but the idea behind the protocol is very good and after studing Gnutella, Freenet, Blocks, and just about every front end out there, I've found they all have a long long way to go.

None of them are as fast, easy to use, or reach 1/10th the number of people that napster does. I believe the solution, at least for the short term is not to create new protocols from scatch, but rather to make the napster protocol itself more distributed.

Rather than having centralized napster servers which can easily be shut down, it would be ideal to have linked servers that not only can share the load, but will propogate the index of the other servers. Then we need some way to make the napster client aware of the servers and find one at random, possibly through a webpage with an index of registered servers, or by contacting other clients on the network similar to gnutellas node catcher.

The client would also need to automatically jump from one server to another if the server goes down because of connectivity or tragic law suits.

The server could even be built into the client so that you can check a box and start operating as a server which would publish your ip to others on the network. Once there are suffcient numbers of napster client/servers out there and changing every hour it will be very very difficult to shut down, and for the average user, they can just download the new client and it will work as it always did.

Napster should abandon their filtering direction and release this type of distributed, fail-over, server as open source along with the matching client. Once its in the wild they can shut down their own servers and just host the napster webpage. They can still enjoy millions of hits to their site, with no libability of running the napster servers.

I suspect even the RIAA won't be able to get a restraining order against a software company to make them stop creating software. Other wise netscape/ms/apache and any other software that allows computers to communicate is next.

So please, before we abandon the well known napster protocol, let's slightly change it to be more resistive to attack. Opennap may be headed in this direction but until i can visit their page and see 10,000 opennap servers all listed and linked together, they're just inviting themselves to another lawsuit.

Also see the O'Reilly P2P conference site

[Twid]

FYI, O'Reilly recently had a big P2P conference in San Francisco, you can find out lots more information on P2P, especially on who all the current playing in the market are and what they are making, here.

$15 includes development costs

[yerricde]

apparently we have been stealing from the companies that charge 14.99 for a cd that costs them about 50 cents to package and market.

It costs more than that to market a CD. It costs to record the music, it costs to license the samples used on the CD, and it costs to produce promotional tools such as a website or music video.

Napster is going down for all the wrong reasons.

The "wrong reasons" you're referring to include the fact that recording artists who publish their work for a free download have a very hard time getting their work on mainstream radio thanks to under-the-table payola systems that the RIAA and NAB maintain.

All your hallucinogen are belong to us.

Infoanarchy.org

[Anonymous Coward]

For discussion of p2p issues and apps please visit infoanarchy.org

And if you are looking for a good p2p don't forget to check the resources page. Thankyou.

WTF?

[DanThe1Man]

Did anyone see that article before it was slashdotted?

_ _ _
I was working on a flat tax proposal and I accidentally proved there's no god.

Sounds like a good article

[Alien54]

but it looks like the site is not answering up or something.

I am have yet to see the article, under 3 different browsers...

between that, and the unusual number of drunken trolls... I think I'll wait wait until morning before trying to making a comment.

New scientist usually does good articles, even if a little light in content (no more that a dozen paragraphs or so).

Should be interesting.