Gnutella: Alive, Well, And Changing Fast

Benno writes: "Clip2 has put up a nice story about the latest developments on Gnutella: Gnutella: Alive, Well, and Changing Fast." It's good to see that they're solving the scalability problems.

Comments

Needs an overhaul

[Joe Rumsey]

The progress made by BearShare and others is commendable, but it's all incremental. Nothing that's actually up and running today implements any long-term scalability solutions, and as far as I can see, nothing that attempts to connect to existing gnutella clients ever will. There were a number of good proposals for new protocols from the (apprently dead) next generation group, but none of them have yet amounted to anything. (As of a few weeks ago at least, I have not checked today.)

As a Gnutella developer of sorts, I'd love to see an alternative emerge and add support for it. Of course, I'm not actually taking that initiative myself, so I can't point the finger at anyone here. But if I have some time to spare when and if someone does something truly new, I'll do what I can to add support to Gumshoe. In the meantime, I'm doing what I can to catch up with the improvements made by others.

My Gnutella network audit

[Mongoose]

Since I started work on making a Gnutella client/server; I've noticed lots of bad protocol usage. ( I'm going to focus on using my server as a gnutella 'proxy' to my apche server to serve files. )

Here's a breif listing of little things that are easy to fix:

1. I've see
n TTLs set up to 256! We shouldn't forward these packets at all or at least reset the TTL to like 4.

2. New packet types should be rejected. Here's a few examples my client logged one night:

UNKNOWN FUNCTION 0x69
UNKNOWN FUNCTION 0xdb
UNKNOWN FUNCTION 0xfa

And here is an example with it's data field:

UNKNOWN FUNCTION 0x97
'^@^@^@)^Xt^Q^QH^@^@^Oj@^A^F^Z^@
^@^@R{.!^?-C"?^B22^@^@)^X'

3. Reject unsupported unicode queries until we have a standard! I know I don't serve files in japanese, so I shouldn't forward it to other clients that don't either. These queries are often 256bytes!

We might need to start a gnutella steering committee and just rethink some issues. Just fixing these 3 minor things frees up a lot of bandwidth. Also we need to use Pongs for finding new hosts more than pinging the fuck out of ppl like me that are on dail-up 50% of the time. =)

Re:Gnutella and Mojo Nation

[mbyte]

.. because Mojo Nation is horrible to use. I spend > 3 hours trying to get it work and failed.
(because of weired proxy configurations ...)

BeShare is only a matter of "click install".


Samba Information HQ

Re:Random search strings?

[big.ears]

I think it is a strategy to identify ad-serving clients. Specifically, there are fake clients that return "yoursearch.html" or somesuch. A way to avoid this is to search for random text and block everyone who returns hits to you.

History about Gnutella.it and its Websearcher

[sbenno]

Some clarifications on the websearcher hosted on Gnutella.it

: A few folks accused me of promoting freeloading and giving back nothing to the gnutella community.

First gnutella.it was born for the following reasons :I'm italian and when I discovered that the gnutella.it domain was still available, (In June 2000 I believe), I registered it and put up infos related to the Gnutella technology in italian. (before some jerk would have used it for something different)
A bit later I wrote a nice MySQL + PHP + C++ engine which acts as a gnutella web search engine ad that permits you to download directly from a browser. Just as the zillions out there, but with a few special features (cached+realtime search,sorting,timelimiting etc).

It became quite good and the most read italian newspaper "Il Corriere della Sera" interviewed me , writing an article about it. see here (hehe nice publicity for free :-) )

Eventually downloading from gnutella via a webbrowser became so popular and easy (thanks to gnute.com , mp3board.com and others), that the network began to feel the load of these browser-freeloaders.

I want to point out that the goal of this searcher was not to permit the folks to leech off the gnutella network without giving back anything.

Some have me put in the same league as gnute.com and mp3board.com.

That's just plain wrong !

The LimeWire folks worte such a notice on their pages too, but after a clarification they corrected the stuff.

On my site I've placed notices (even on each search result page) that you should contribute to the Gnutella network using a servent and share files, otherwise there would be no files.

But why worry about webbased-freeloaders ?

BearShare and LimeWire block now browser downloads, rendering all gnutella web search sites basically useless, since as soon as you try to click on a download link, you get redirected to a HTML page which informs you not to freeload and use a standalone client instead.

So basically all these sites, suddenly promote gnutella, which is a good thing.

And in fact this has caused a boost in terms of # of gnutella users , the download success rate (less "Busy" signals) has increased and compared to a few months ago, you find now much more stuff on gnutella.

I'm currently working with the authors of BearShare to find a way to make gnutella web search working again and reliable. This will require a few things like sending out informations (in the query_hit packets) whether or not the user DESIRES to share his files with browsers. And incorporating some protection from mp3 leeching sites (like gnute.com , mp3board.com etc) since this service can be easily abused.

Another way to use a gnutella web search like Gnutella.it is to use a Gnutella client which supports drag'n drop as BearShare does. That way you can use the web searcher to search for files, and most of times it finds much more stuff than a regular standalone client because it caches a huge amount of links to files. (the drawback is that some liks could be a bit outdated, but you can look at the age of the file to estimate its reliability).
As soon as you find a link, you can drag it into BearShare which in turn will try to download it. It will not get blocked since the download does not occur via browser.
And if the download succeeds, then the file will get automatically shared and made available to others.

For modem users it is perhaps more lightweight in terms of traffic to use the web searcher and drag the links into BearShare, since you will save bandwidth comprising search / searchreply and host connection traffic. If you share files, then ONE single host connection will suffice and you will still be reachable by other gnutella users. That way you will be able to devote almost all of your limited modem bandwidth to downloads/uploads. (although the whole process is a bit messier than using a gnutella client alone)

I admit that Gnutella has its drawbacks and may be viewed as crappy and unreliable compared to napster.
But I think the protocol has many possibilities to become a nice and efficient distributed filesharing protocol. (check out the gnutella developer forum: http://groups.yahoo.com/group/the_gdf )

Plus I have not seen a truly distributed filesharing protocol yet, which works much better than gnutella and does not depend from a central entity.
Every centralized filesharing system will sooner or later get attacked (or bought out) by the various RIAA/MPAA etc. So only decentralized system will survive over the long term.

Yes, freenet may become much nicier and efficient, but they have a long way to go until this becomes reality, plus without searching capabilities, freenet will not be that useful. (if you are searching for files, you need the key first, and there is no way to search for keys on freenet). One combination would be to use Gnutella to store keyfiles (with some fulltext extensions) so that you could search on gNet for filename->key mappings and then download the real content off the freenet network.

The important thing is IMHO to become indipendent from centralized filesharing systems (ala Napster,Imesh, etc), at this point users will choose the system that works best AND is easy to use. Unfortunately the holy grail has not been found yet. But I think we are not THAT far away from an easy to use, efficient and distributed system which will attract millions of "Joe Average" users.

We will see ....

cheers,
Benno.
Gnutella.it

Re:The Man(TM) will never take this lying down...

[RedWizzard]

Otherwise, quit yer whining and accept the ISP's rules.

Yeah but they don't tell you the rules. They say "oh, by the way, you can't run a server off this collection", and you think "fine, I wasn't planning on running a Web server or a Quake server or anything". Then you get an email stating that they've decided to shut down the NetBios port due to virus problems. I quote from the email:

Closing Port 139 (also known as the NetBios port) will mean that all customers will no longer be able to use certain functions that this port would normally facilitate. In particular, some network sharing services (file and print sharing) through our network will cease operating. Volume based plan customers only use this feature as hosting servers is a breach of the Acceptable Use Policy

It's not too coherent, I know, but that says to me that they consider a NetBios share to be a server. I'm probably breaching their Acceptable Use Policy right now - I think I forgot to disable the echo service. Am I the only one who finds that overly broad?

Re:The Man(TM) will never take this lying down...

[micromoog]

I think that it will be used as a very unsubtle smackdown for anything that threattens to use all the bandwidth I pay for.

I just have to comment on this remark. Consumer-level connections are not designed to accomodate continuous load at the maximum bandwidth level. They are designed for high peaks with a low average load, because most consumers are interested in pages that load quickly, not sustained performance.

You're not paying for "all the bandwidth" you can consume; you're paying very low prices for bandwidth, based on the consumer usage profile. If you really want sustained bandwidth at a specific level, bite the bullet and cough up some serious cash for a commercial connection. Otherwise, quit yer whining and accept the ISP's rules.

Mojo Nation install difficulties have been fixed.

[Zooko]

You'll be glad to know that we stopped requiring the weird proxy configurations a couple of versions ago. To run the new version is simply: "tar -xzf ./mojonation-*.tar.gz ; ./mojonation/evil/Broker --no-tail ; netscape ./.mojonation/broker/intropage.html".

The windows version is apparently even easier to install -- it has some kind of gui point-and-drool installer that untars and executes these things for you, or something. ;-)

We also recently fixed several really nasty "distributed bugs", which didn't show up at all on any particular broker, but which caused the entire network to degrade. Nowadays Mojo Nation is much faster and more robust than it was last time you tried it.

Regards,

Zooko

Re:The Man(TM) will never take this lying down...

[treke]

>Otherwise, quit yer whining and accept the ISP's rules. Its a great theory to try abiding by the rules, but it doesn't work. My ISP Mediaone/RoadRunner/AT&T/whoever buys em next month) allows servers but has a list of server users aren't allowed to run. I called to get a copy of this list, and guess what? They refused to give me one stating they don't have one available to subscribers. Sure makes it hard to be a good little boy and obey the rules when I'm not even allowed to know what those rules are.
treke

The Man(TM) will never take this lying down...

[Doomsdaisy]

Even if we get Gnutella to a point where even those folks on dial up modems can participate in the grand link-up, I'm a bit worried about what the broadband ISP monopolies (@home anyone?) will do to dissuade their customers from using such a product. Most broadband providers prohibit their users from running 'servers'.
Now you know what a server is, and I know what a server is, but my ISP seems to have a very broad definition of what a server is, and they seem to change it to suit their needs. I think that it will be used as a very unsubtle smackdown for anything that threattens to use all the bandwidth I pay for.
The media industry will whine about a functional Gnutella, and the media providers will be happy to try a ham-handed solution.
just my .02$

These are breasts; this is source code.

Re:Gnutella and Mojo Nation

[Wesley Felter]

This article calls it the download failure problem. Basically, whenever I try to download something from Napster or Gnutella, it either times out or it comes in at less than 1 K/s and then times out halfway through. These problems don't happen nearly as often with Mojo Nation.

Even ignoring all the stuff about freeloaders and economics, Mojo Nation solves other problems, too.

Gnutella and Mojo Nation

[Wesley Felter]

The article mentions that Mojo Nation has already solved the largest problem that plagues Gnutella (and Napster in my experience), so why keep Gnutella on life support? Why not just switch to Mojo Nation?