Slashdot Mirror
Closed Gnutella System to Prevent Bandwidth Hogs
prostoalex writes: "Salon.com is running a story on Gnutella developers contemplating the creation of a closed or authorization-only system to prevent bandwidth hogging. Turns out, numerous applications, including Xolox and QTraxMax employ quering algorithms that are capable of bringing the network traffic to a halt. While it gets better download speeds for the users of the aforementioned applications, the damage to network traffic as a whole is substantial."
The solution is not authentication - it's building better network infrastructure.
Now would be a good time to plug the free, recently-opensourced Ozone file sharing program. It interfaces with MUSCLE/BeShare servers to allow people to share files without worries of AdWare and SpyWare and junk like that.
:)
Ozone - Available for Linux, Windows, and OS X.
Beryllium's BeShare Server - use "Beryllium.BeShare.Com" inside of Ozone to check it out!
Enjoy
Is anyone else reminded of the book animal farm after reading this article?
Here's a clip from an email I sent sometime ago to someone, it might or might not have something in it, judge yourself.
:)
- the system must reorganise itself automatically based on current
analysis of the nodes available on the network. - the system must have a dynamic trust model, based on "paranoia".
- the trust model must be utilized in combination of other characteristics of each peer(node) to select best population of the nodes as more important servants. Untrusted/neutral nodes are not to be given any crucial tasks. No-one can do anything crucial alone, confirmation for the action must be confirmed from other trusted ones. - All functionality of the network mut be replicable automaticly. Tasks done by any node must be transferrable transparently.
- Weak nodes will not be given any "community work"
- Every node must pass constant quality criteria to be able to perform any actions on the actual network.
Just to mention a few points. In short, anarchy does not work - even in P2P networks. We need a government, but one which is always on move, but still governs population using strict - but adaptive - rules.
The biggest problem with gnutella is not technical. It is that gnutella was invented so that true hardcore underground people such as myself could complete our collection of harcore underground things, such as the entire run of Evangelion. However, gnutella is cluttered with people only interested in Brittney Spears. Here is an idea I first proposed on everything2 for making gnutella less crowded.
Hopefully I didn't put any [] around my words.
How about implementing per-node policing using a credit system like gnunet? (http://http://www.gnu.org/software/GNUnet/)
Nodes individually keep track of the behavior of their neighbors. Bad or expensive behavior like out-of-spec activity or excessive querying lowers the 'credit' of the node. Good behavior like answering queries increases a node's credit. Credit determines the probability that a node's queries will be answered or passed along and the priority with which they will be treated. Abusively written clients will eventually be ignored out of the network.
How about enforcing UPLOAD/DOWNLOAD ratios to all users?
So users won't be able to download without contributing to other user...
There is a P2P network layer called GNUNet. I've studied the papers on it, and the design looks extremely solid and resilient.
I always find it amusing when someone takes a specific implimentation that happens to be similar to a philosophy that they know about, and take that specific example as proof that the whole theory is worthless.
In actuality, gnutella doesn't paralell any serious anarchist philosophy that I have seen very well at all. Most such systems that I have seen proposed generally call for communities of people that work together for benefit of the community and are run by a direct democracy rather than a representative democracy.
In fact anarchy doesn't advocate a state of chaos or lack of laws as much as a lack of hierarchy. It calls for elimination of the concept of "positions of power" where the laws of the land are decided directly by the people themselves and where no person is forced to live by those rules except as the voluntarily accepted price of living within a given community.
gnutella on the other hand is more of a "free for all". More of an "frontier", which isn't very anarchistic at all, as hierarchy is easily created in the frontier, all it takes is a small gang or some guns. Whoever has the most ability to weild deadly power is the top of the hierarchy.
-Steve
"I opened my eyes, and everything went dark again"
I have gotten the impression that these P2P networks are not good netizens. I access the net via a dial-up connection. Within a few minutes of logging on yesterday morning, I found myself dealing with what appeared to be a DOS attack on port 6346 coming from an adsl connection in Lithuania. I have that port blocked, so I was seeing a large queue of security alerts from my firewall. This has not been the first time this has happened with one of the P2P ports. Shto/WTFO?
IIRC, the big players on the Gnutella network at this point (Limewire, Bearshare, etc) are able to exchange version information, and to confirm that version information. If this is true, and it's not possible for a rogue application to masquerade as another servent, I believe it's time to lock abusive servents out of the network. If they aren't playing fair, don't let them play at all. Period.
This means you, XoloX. As well as all the other servents which send requeries at ridiculously short intervals, send download requests tens of times per minute trying to force their way into a download slot, support downloading but not sharing, encourage or emphasize web downloading as opposed to participating in the Gnutella network, etc. Freeloaders are as much a problem as they ever were, but (IMO) only because they're being allowed to be such a problem.
The time has come when abusive servents need to be shown the door. I don't mind sharing most of the time. But when the same asshole is hammering me 100 times per minute trying to get a download slot, or sending the same query every 5 seconds trying to find more sources, my desire to share files goes down the toilet. Something needs to be done.
The userbase of Xolox and QTraxMax doubled today...
The S&P 500 and the FBI's most wanted lists are going to be merged.
I was a part of the Gnutella development clique a while back, and had made a few proposals on improvements to Gnutella clients.
One such proposal, GNL, was to provide a way to define alternate Gnutella networks from the main system, and include ways to limit their behavior. Another proposal, GNV, was a method for administering these networks, and said administration could be performed anonymously.
Many people liked my ideas, until I made the mistake of mentioning that the end result would probably be differentiation of Gnutella into several networks, each specializing in different types of files; it would be like making Gnutella into IRC, with separate server networks providing different flavors of service. I also mentioned that I thought the original Gnutellanet would wither on the vine. They looked on this with horror and dropped my suggestions.
*shrug* I dunno. Considering that, at the time, the Gnutellanet was scaling itself into bloated nonoperation, I thought splitting the Gnet into different specialty networks was a good idea. Clients could even log onto more than one Gnet at a time.
I agree with you that some of the more abusive clients are getting out of control. I don't agree with blocking them outright, though. Gnutella is where it is because it's an open network and an open protocol; I think we have to leave it that way if we expect any future genius to appear on the network. Closing things up and locking the doors, these aren't the appropriate solutions IMO.
I think filtering of abusive apps should be done on the client side of the servent equation. The biggest problems I've seen lately don't involve Xolox specifically, but users of varying servents. People who queue up hundreds of different files to download at a time. People using programs which ignore "Not Shared" or "Refused" replies, and continue to pound my box looking for files that don't exist.
I was out of town for a few days last week (all computers turned off, except for my router box). When I came back, I fired up my Gnutella program. Without even connecting to the network, I was immediately serving uploads. That means that someone was trying to download from me for three full days while a) the files were not shared, b) Gnutella wasn't running, and c) the freaking computer wasn't even turned on! Come on, servent authors: pay some attention when you get "Refused" or "Not Shared" responses. Drop such files from the queue after 2 or 3 failed tries, don't leave them sitting there for eternity.
I want a setting that says "drop all packets from hosts who request a no-longer-shared file." I want a setting that says "drop all packets from hosts who attempt to download while the program is running but not connected to the network." I want a setting that says "drop all packets from hosts who send download requests more than $TIMES per minute." My per-user upload limit is set at 1, so someone queueing up 200 files at a time generates an enormous amount of protocol overhead. It might be 5 hours before that user gets all of his 200 files, all the while he's sending a constant barrage of packets which accomplish nothing.
Gnutella is an open network. Yes, we do need to do something about read-only clients, but I think it should be up to the people to decide what gets done. Provide the users with the appropriate filters and let the majority determine what behavior is good vs. bad.
Shaun
Thanks to the War on Drugs, it's easier to buy meth than it is to buy cold medicine!
It's not like this hasn't happened before.
Sun did it with Ethernet. They set their NICs to use the minimum retry interval instead of minimum + random time like the spec says they must. This got better performance for Sun equipment. Right up to the time where someone put a dozen Suns on a single Ethernet segment and the competition between all of them hammered the network down to 10% of the expected bandwidth.
Various TCP/IP "accelerators" tried this too, by ignoring the exponential-backoff and slow-start parts of the TCP spec. They too improved speeds for the people who used them. Right up to the point where lots of people started to use them, when the competition between them hammered their transfer rates down to a fraction of what's expected.
We've seen it on UDP-based streaming protocols, where lack of flow-control mechanisms causes massive congestion problems and slower transfer rates than when flow-control is applied.
So why didn't anyone expect/predict this when they were designing the Gnutella network and protocols?
Now, run along and play, or we'll have to airdrop you and Chomsky into downtown Gonaives, and you two can try to explain Bakunin and Kropotkin to the natives, and why an absence of rule is a good thing.
Shutting down free speech with violence isn't fighting fascism. It IS fascism!
Stop the FUD.
: //www.linux.org/linux.iso
:P)
:P No longer can they infect files and make them the same file size/file name.
People need to realize that Gnutella is now fastly becoming a big player in the function and value of the Internet.
Gnutella, in my view (and many others), is not a mecca for porn, warez, and MP3's - but a pool where anyone can share any type of file.
A bigger trend now showing up is linking to files on the Gnutella network instead of the common http://site.com/file.zip. How does this benfit you? You get faster downloads by utilizing partial file sharing, swarm downloads, etc. It also benfits servers greatly. They now aren't the only source for the download, because once the file gets onto a Gnutella client, it searches for more peers, and shares the load with them. This can save TREMENDOUS bandwidth.
For example, Linux can link to Linux links as such: magnet:?xt=urn:sha1:(InsertSHA1)&dn=Linux&xs=http
(not an actual correct MAGNET link, but you get the idea)
When someone clicks that, it opens it up in a Gnutella client. It begins downloading from that source, and searching for the same file on the Gnutella network. Through the entire life of the download, it will continue to add sources. You could then be downloading from over 30 people at once, gaining speeds of up to 10MBPS+.
Oh, the power of Gnutella. Can KazAa (FastTrack) do that?! (Well, it can, kind of
Oh, how do you know if that's the correct file? Hashing. Gnutella servents are implamenting hashing now, where each file has it's own hash. So when searching for files, they can swarm you downloads. You are GUARANTEED that all the sources your downloading from are in fact the same file, because they have the same hash (SHa1). That's whats getting the RIAA so scared
Also new on the scene (well, new as in new popularity) is Bitzi. Bitzi catologs hashs (bitprints). You can search through their database, and find files with hashes. Click the hashes, and you can download a file. Each file on bitzi has a "Bitzi Ticket" where you can rate the file. You can mark it "Invalid/Misleading" which means it is not the file you want. You can mark them if they contain virus's too. I can almost hear the sweat dripping from the RIAA Lawyers foreheads.
Want to see the future of Gnutella? Check out Shareaza (WINE Compatable).
Supports all of what I discussed in this post.
I'm not a coder myself, and am probably not very up to date on the whole p2p scene (other than knowning that Limewire doesn't seem to work real well on my box at work), but one of the real problems on the p2p networks seems to be trust. With the recent news about entertainment industry bodies seeking legislation to DoS the networks, and the common user experience of crap files on the network (incomplete, or incorrectly labled files), I wonder whether someone could make a system based on the same sort of web of trust model than PGP/GnuPG uses.
The Keyserver infrastructure is already there, and the apps (like GnuPG) are readily available cross-platform. So why can't p2p clients allow content to be signed, so that you can establish a web of trust as to whose content can and cannot be trusted. Downloading a signature of a file to check it's validity would certainly help reduce the chance of downloading dodgy content. This should be especially useful as you tend to get groups of people who are all interested in the same sorts of files (anime, divx, certain bands, etc), so you could imagine a good web forming fairly rapidly.
Making a valid OpenPGP key is a computationally intensive task, suggesting that few people would make thousands of them on the possibility they would be blacklisted. They also don't require any form of real identification, making them effectively anonymous. Also gaining a good trust metric would be an incentive to keep the same key, especially if downloading was restricted based on your trustability.
I can't think of any good reason that this couldn't be worked into an existing p2p network. Whether it would work in practice I have no idea. Anyone who knows more about this than me care to comment? Anyone done it already?
So, you need some kind of intelligence gathering agency and millitary force that could detect and prevent a potential outbreak of government. And, of course, you'll need rules for these agencys to follow, so as to protect everyone else from them, and some sort of oversight commity to make shure those rules are followed. And then you'll need a group of people handle the punishment of those who violate the rules, and another set of rules for them to follow to ensure that innocent people are not punished.
You'll also need some meathod of deterring people from lying, stealing, killing, or otherwise abusing eachother. After all, most people aren't very nice. And then you'll need some way of seeing to it that those who do violate the rules of common decency are dealt with, and again, there will need to be a set of rules for how to procede with such matters.
Wow, you were right, anarchy does work. All you have to do is follow these simple guidelines and...
Wait a minute. Oh shit! We've just created a government. Guess we'll have to start over.
If you don't have anything nice to say, shut up you stupid prick.
When someone points a gun at you, it isn't anarchy any more, so none of your examples apply.
What would Lemmy do?
We all complain about the amount of crap (incomplete & low quality files and such) that we receive through the p2p networks. How about someone created a DB where you send the hash, and it returned the actual contents. Maybe you could even send the textual request, and it would return the hashes of files that match - and then you can search for files matching the hash?
Would this be feasible at all, do you think? It would be an additional p2p distributed network (we gotta make sure the DB is accurate and relatively synchronized, so we can't give direct, universal write access). I'm thinking that you open a socket to the server, and just keep sending requests as you search for files, and as you open files. This way, we would also be able to blacklist files we don't want distributed, blocking those from being returned by the initial search.
You think the RIAA guy monitoring this discussion just choked?
Stop the brainwash
[pause]
Now if only I could find out where those elitist bastards are hiding! :-)
--
What short sigs we have -
One hundred and twenty chars!
Too short for haiku.
The problem in general arises when you've set up a situation where if each user acted in both a rational and self-interested way, the system overall would collapse for all the users.
When designing any kind of multi-user system, it's critical to plan for the "what if all the users (or half of them) suddenly got very selfish." What results are things like disk quotas: central-system-enforced limits on individual behavior.
In a system like the gnutella network, where there is no 'central system' to enforce 'community-minded' behavior, the eventual collapse of the system can be predicted as a function of overall population, presuming that there are always a few people who are more selfish than the rest.
Centralized systems like Napster actually had an advantage in that the centralized servers could establish and enforce 'fairness' policies that kept selfish users from triggeringa 'Tragedy of The Commons'.
-Mark
don't allow people with 0 files in their library to download
Then what about one file?
Besides, making the network trade-only leads to a chicken-and-egg problem for new users. How are "honest" users (the ones willing to share) supposed to get into the network in the first place? Where does a new network member get her first audio or video file?
Will I retire or break 10K?
Actually, you mostly don't want to ignore these constraints. The P2P should make use of closer servers (mostly, but not exclusively).
In order to get better performance for themselves, people play "the prisoners dilemma", and rat everyone else out with clients that gang up on requests to ensure disproportionately favorable service.
I don't see that this is necessarily a real issue. After all the server that has the file you want can keep a queue of requestors, and serve it in strict first come, first served order. 'Take a ticket and sit down over there.' It works. Asking more than once doesn't get you anywhere; and may even get you lower down the list.
The only real way to deal with this is to define a new protocol that is not virtual point-to-point linked.
Unclear. Very unclear.
Now take active attacks. "Automatic Karma" can deal with dummy files -- "poisoning"... at least until they start intermixing bad with good.
Yes, but users can usually play files before they've finished and cryptographic hashing of file contents can preclude people spoofing files, even when downloaded from multiple servers simultaneously.
-WolfWithoutAClause
"Gravity is only a theory, not a fact!"I'm pretty tired of all the complaints about freeloading on any system even remotely likely Gnutella. It's the same with Carracho, Hotline, FTP, what have you: you will always have more freeloaders than sharers until equilibrium is achieved; equilibrium, though, will never be achieved.
The ratio of users who have useful, desireable files to share to users who do not will always be low, perhaps 1 to 10 or 1 to 100. This is because the "freeloaders" cannot and do not have files to share until the get them from someone else. They will continue to be non-sharing nodes until such time as the sharers with desireable files open up the portcullis.
The point of the system is filesharing: Why impose restrictions on its primary function? The way to stop "freeloading" is not to restrict downloads, but to *increase* them. The closer to the unachievable equilibrium we come, the less "freeloading" there will be.
Wordnik, a dictionary project which aims to collect
Do you expect the same people who use the network predominantly for breaching copyright to care about the greater good?
Do you actually think they copyrights they're breaching have anything to do with the greater good?
Four companies have collectively monopolized music distribution, using copyright. Is this a good thing?
Get real. Record companies are scum. The artist would get more money if I mailed them a quarter, than if I bought the CD. Meanwhile, I would be giving the RIAA more money to keep it illegal to play legally purchased DVDs on my PC. I hope they all go bankrupt. Then we'll have competition.
I'll participate in a free market, but not the current abusive, short-sighted ologoploy. Tell me where I could legally download my 300 favorite CDs for a reasonable fee? I can't. Thankfully record companies don't have a long term business plan. They just keep trying to stifle new technology and get their business model legislated. They should be trying to provide the services people want. That's what they'd be doing in a free market economy. They're trying to tell me what I want. They can bite me.
Life is too short to proofread.
Why not have the clients block anyone automatically who starts to do instant requeries?
I am not sure exactly how the Gnutella protocol works, but if every valid client had this blocker, then these "super-nodes" would not be able to get any information in or out.
Basically, the idea would be that when one of the malicious nodes starts to send multiple queries to another node with this blocking code. The other node would determine whether or not this is legit. If it is not legit, that node will be blocked. Eventually, a "fence" would be put up around the offending nodes, and the damage they cause would be limited to non-standard clients.
As well, it may be prudent to make the block last for a specific time period. Perhaps even add the ability to pass the offending node addresses to other clients so they block as well.
If the gnutella protocol allows this. It would be the most effective way of preventing malicious clients because as soon as they threaten the infrastructure, they are blocked off.
~ kjrose
I've been reading through some of the news and related sites on this topic and it seems the possibility exists that one or more of these gnutella clients that send massive numbers of request in such short periods could actually be a maliciously intended program. Some of the developers who make these have yet to respond to any of these problems even though there have been repeated attempts to contact them about the situation. The way some (I'm looking at QTRAXMAX right now) word their sales pitch, it sounds eerily similar to some e-mails I've gotten with links to these sites or those mysterious 53k-attachments-to-emails-that-just-say-hi-from-so me-guy-named-boris-in-siberia that are so obviously worms or viruses. The way they currently work looks eerily similar to a DoS attack. Use people's own greed to flood a network with requests. It would actually be a pretty clever strategy - millions of users instantly flock to the program to maximize their gain out of gnutella, only to block each other out when they send 83 gazillion file requests a second. Classic Nash.
Who would be behind such an attack? There are many possibilities. The recording industry is definately one of them. There could be others. Who knows.
The point is you should all be careful what you install on your computer or even download. Millions of people around the world know how to program at varying levels of control over many different kinds of computers with different purposes. It's like the Force - some use it for good, some don't. There's bound to be at least a couple who are going to write a full-fledged application that is really just one big worm.
Just another freak in the freak kingdom.
If they make it so that they can control who is on Gnutella, won't the RIAA be able to sue whomever has this control? Bad idea, folks. The simple solution is bandwidth limiting, and blacklists for IP's that are abusive.
BlackGriffen