Closed Gnutella System to Prevent Bandwidth Hogs

prostoalex writes: "Salon.com is running a story on Gnutella developers contemplating the creation of a closed or authorization-only system to prevent bandwidth hogging. Turns out, numerous applications, including Xolox and QTraxMax employ quering algorithms that are capable of bringing the network traffic to a halt. While it gets better download speeds for the users of the aforementioned applications, the damage to network traffic as a whole is substantial."

Comments

A few thoughts on P2P

[jukal]

Here's a clip from an email I sent sometime ago to someone, it might or might not have something in it, judge yourself.

- the system must reorganise itself automatically based on current
analysis of the nodes available on the network. - the system must have a dynamic trust model, based on "paranoia".
- the trust model must be utilized in combination of other characteristics of each peer(node) to select best population of the nodes as more important servants. Untrusted/neutral nodes are not to be given any crucial tasks. No-one can do anything crucial alone, confirmation for the action must be confirmed from other trusted ones. - All functionality of the network mut be replicable automaticly. Tasks done by any node must be transferrable transparently.
- Weak nodes will not be given any "community work"
- Every node must pass constant quality criteria to be able to perform any actions on the actual network.

Just to mention a few points. In short, anarchy does not work - even in P2P networks. We need a government, but one which is always on move, but still governs population using strict - but adaptive - rules. :)

Get rid of pop culture vultures!

[Glowing Fish]

The biggest problem with gnutella is not technical. It is that gnutella was invented so that true hardcore underground people such as myself could complete our collection of harcore underground things, such as the entire run of Evangelion. However, gnutella is cluttered with people only interested in Brittney Spears. Here is an idea I first proposed on everything2 for making gnutella less crowded.

Gnutella is one of the best things to come out of Sedona, AZ since the hordes of Alien Invaders who passed through the vortex. At leat for those of us who have DSL or better, Gnutella is the best way to complete our collection of Evangelion episodes, obscure hip hop mp3s and fets.com sets.

The problem with gnutella though, is that it is crowded, and according to my estimates, about 75% of this crowding is due to people looking either for mp3s of that damn song that plays on the radio every half hour and\or nude pictures of celebrities. Often to compound matters, these people are looking for nude pictures of that one celebrity that sings that damn song they play on the radio every half hour.

If we have a tool that allows us to download obscure 90 minute long epic techno ballads from the Slovak Republic, why are we allowing people to use it to listen to music that they can hear by turning on MTV?

The answer is because we don't know how to stop them. But I have a possible solution for our problem. All it requires is for about 100 or so people to put a file in a shared directory called "Brittneyspearsbarebreasts.jpg" or something along those lines. But instead of said picture actually being of Miss Spears beare breats, why not make it something else...such as possibly goatse.cx?

After seeing this picture one too many times (which will probably be the first time), many people will cease to use gnutella as a vehicle for their pop culture stupidities.

Self-policing network

[Anonymous Coward]

How about implementing per-node policing using a credit system like gnunet? (http://http://www.gnu.org/software/GNUnet/)

Nodes individually keep track of the behavior of their neighbors. Bad or expensive behavior like out-of-spec activity or excessive querying lowers the 'credit' of the node. Good behavior like answering queries increases a node's credit. Credit determines the probability that a node's queries will be answered or passed along and the priority with which they will be treated. Abusively written clients will eventually be ignored out of the network.

The solution is to block abusive servents

[Anonymous Coward]

IIRC, the big players on the Gnutella network at this point (Limewire, Bearshare, etc) are able to exchange version information, and to confirm that version information. If this is true, and it's not possible for a rogue application to masquerade as another servent, I believe it's time to lock abusive servents out of the network. If they aren't playing fair, don't let them play at all. Period.

This means you, XoloX. As well as all the other servents which send requeries at ridiculously short intervals, send download requests tens of times per minute trying to force their way into a download slot, support downloading but not sharing, encourage or emphasize web downloading as opposed to participating in the Gnutella network, etc. Freeloaders are as much a problem as they ever were, but (IMO) only because they're being allowed to be such a problem.

The time has come when abusive servents need to be shown the door. I don't mind sharing most of the time. But when the same asshole is hammering me 100 times per minute trying to get a download slot, or sending the same query every 5 seconds trying to find more sources, my desire to share files goes down the toilet. Something needs to be done.

Re:The solution is to block abusive servents

[DNS-and-BIND]

The Gnutella developers see their mission as bringing a new, revolutionary network protocol to the masses. Something on the level of a new HTTP.

The Asia-based Qtraxmax developers see their mission as getting as many software(spyware?) installs as possible, through promising a superior user experience, and they would cheerfully destroy the network to do so.

Obviously, the solution is a new Gnutella option, defaulting to "on", that says "deny resources to abusive clients".

GNL

[TheSHAD0W]

I was a part of the Gnutella development clique a while back, and had made a few proposals on improvements to Gnutella clients.

One such proposal, GNL, was to provide a way to define alternate Gnutella networks from the main system, and include ways to limit their behavior. Another proposal, GNV, was a method for administering these networks, and said administration could be performed anonymously.

Many people liked my ideas, until I made the mistake of mentioning that the end result would probably be differentiation of Gnutella into several networks, each specializing in different types of files; it would be like making Gnutella into IRC, with separate server networks providing different flavors of service. I also mentioned that I thought the original Gnutellanet would wither on the vine. They looked on this with horror and dropped my suggestions.

*shrug* I dunno. Considering that, at the time, the Gnutellanet was scaling itself into bloated nonoperation, I thought splitting the Gnet into different specialty networks was a good idea. Clients could even log onto more than one Gnet at a time.

I'd rather see this as an option

[ShaunC]

I agree with you that some of the more abusive clients are getting out of control. I don't agree with blocking them outright, though. Gnutella is where it is because it's an open network and an open protocol; I think we have to leave it that way if we expect any future genius to appear on the network. Closing things up and locking the doors, these aren't the appropriate solutions IMO.

I think filtering of abusive apps should be done on the client side of the servent equation. The biggest problems I've seen lately don't involve Xolox specifically, but users of varying servents. People who queue up hundreds of different files to download at a time. People using programs which ignore "Not Shared" or "Refused" replies, and continue to pound my box looking for files that don't exist.

I was out of town for a few days last week (all computers turned off, except for my router box). When I came back, I fired up my Gnutella program. Without even connecting to the network, I was immediately serving uploads. That means that someone was trying to download from me for three full days while a) the files were not shared, b) Gnutella wasn't running, and c) the freaking computer wasn't even turned on! Come on, servent authors: pay some attention when you get "Refused" or "Not Shared" responses. Drop such files from the queue after 2 or 3 failed tries, don't leave them sitting there for eternity.

I want a setting that says "drop all packets from hosts who request a no-longer-shared file." I want a setting that says "drop all packets from hosts who attempt to download while the program is running but not connected to the network." I want a setting that says "drop all packets from hosts who send download requests more than $TIMES per minute." My per-user upload limit is set at 1, so someone queueing up 200 files at a time generates an enormous amount of protocol overhead. It might be 5 hours before that user gets all of his 200 files, all the while he's sending a constant barrage of packets which accomplish nothing.

Gnutella is an open network. Yes, we do need to do something about read-only clients, but I think it should be up to the people to decide what gets done. Provide the users with the appropriate filters and let the majority determine what behavior is good vs. bad.

Shaun

Those who do not learn from history...

[Todd Knarr]

It's not like this hasn't happened before.

Sun did it with Ethernet. They set their NICs to use the minimum retry interval instead of minimum + random time like the spec says they must. This got better performance for Sun equipment. Right up to the time where someone put a dozen Suns on a single Ethernet segment and the competition between all of them hammered the network down to 10% of the expected bandwidth.

Various TCP/IP "accelerators" tried this too, by ignoring the exponential-backoff and slow-start parts of the TCP spec. They too improved speeds for the people who used them. Right up to the point where lots of people started to use them, when the competition between them hammered their transfer rates down to a fraction of what's expected.

We've seen it on UDP-based streaming protocols, where lack of flow-control mechanisms causes massive congestion problems and slower transfer rates than when flow-control is applied.

So why didn't anyone expect/predict this when they were designing the Gnutella network and protocols?

Gnutella is the future of the Internet

[Anenga]

Stop the FUD.

People need to realize that Gnutella is now fastly becoming a big player in the function and value of the Internet.

Gnutella, in my view (and many others), is not a mecca for porn, warez, and MP3's - but a pool where anyone can share any type of file.

A bigger trend now showing up is linking to files on the Gnutella network instead of the common http://site.com/file.zip. How does this benfit you? You get faster downloads by utilizing partial file sharing, swarm downloads, etc. It also benfits servers greatly. They now aren't the only source for the download, because once the file gets onto a Gnutella client, it searches for more peers, and shares the load with them. This can save TREMENDOUS bandwidth.

For example, Linux can link to Linux links as such: magnet:?xt=urn:sha1:(InsertSHA1)&dn=Linux&xs=http: //www.linux.org/linux.iso

(not an actual correct MAGNET link, but you get the idea)

When someone clicks that, it opens it up in a Gnutella client. It begins downloading from that source, and searching for the same file on the Gnutella network. Through the entire life of the download, it will continue to add sources. You could then be downloading from over 30 people at once, gaining speeds of up to 10MBPS+.

Oh, the power of Gnutella. Can KazAa (FastTrack) do that?! (Well, it can, kind of :P)

Oh, how do you know if that's the correct file? Hashing. Gnutella servents are implamenting hashing now, where each file has it's own hash. So when searching for files, they can swarm you downloads. You are GUARANTEED that all the sources your downloading from are in fact the same file, because they have the same hash (SHa1). That's whats getting the RIAA so scared :P No longer can they infect files and make them the same file size/file name.

Also new on the scene (well, new as in new popularity) is Bitzi. Bitzi catologs hashs (bitprints). You can search through their database, and find files with hashes. Click the hashes, and you can download a file. Each file on bitzi has a "Bitzi Ticket" where you can rate the file. You can mark it "Invalid/Misleading" which means it is not the file you want. You can mark them if they contain virus's too. I can almost hear the sweat dripping from the RIAA Lawyers foreheads.

Want to see the future of Gnutella? Check out Shareaza (WINE Compatable).

Supports all of what I discussed in this post.

It's called "The Tragedy of the Commons" (1833)

[kriegsman]

This problem was first identified and analized in 1833 by Willian Lloyd. It went something like this:

The tragedy of the commons develops in this way. Picture a pasture open to all. It is to be expected that each herdsman will try to keep as many cattle as possible on the commons. Such an arrangement may work reasonably satisfactorily for centuries because tribal wars, poaching, and disease keep the numbers of both man and beast well below the carrying capacity of the land. Finally, however, comes the day of reckoning, that is, the day when the long-desired goal of social stability becomes a reality. At this point, the inherent logic of the commons remorselessly generates tragedy.

As a rational being, each herdsman seeks to maximize his gain. Explicitly or implicitly, more or less consciously, he asks, "What is the utility to me of adding one more animal to my herd?" This utility has one negative and one positive component.

1. The positive component is a function of the increment of one animal. Since the herdsman receives all the proceeds from the sale of the additional animal, the positive utility is nearly + 1.

2. The negative component is a function of the additional overgrazing created by one more animal. Since, however, the effects of overgrazing are shared by all the herdsmen, the negative utility for any particular decisionmaking herdsman is only a fraction of - 1.

Adding together the component partial utilities, the rational herdsman concludes that the only sensible course for him to pursue is to add another animal to his herd. And another.... But this is the conclusion reached by each and every rational herdsman sharing a commons. Therein is the tragedy. Each man is locked into a system that compels him to increase his herd without limit -- in a world that is limited. Ruin is the destination toward which all men rush, each pursuing his own best interest in a society that believes in the freedom of the commons. Freedom in a commons brings ruin to all.

The problem in general arises when you've set up a situation where if each user acted in both a rational and self-interested way, the system overall would collapse for all the users.

When designing any kind of multi-user system, it's critical to plan for the "what if all the users (or half of them) suddenly got very selfish." What results are things like disk quotas: central-system-enforced limits on individual behavior.

In a system like the gnutella network, where there is no 'central system' to enforce 'community-minded' behavior, the eventual collapse of the system can be predicted as a function of overall population, presuming that there are always a few people who are more selfish than the rest.

Centralized systems like Napster actually had an advantage in that the centralized servers could establish and enforce 'fairness' policies that kept selfish users from triggeringa 'Tragedy of The Commons'.

-Mark