Gnutella2?

Anenga writes "A Windows (and somewhat WINE compatible) Gnutella client, Shareaza, has released a public preview of its next version which includes a re-designed Gnutella protocol they call "Gnutella2". Gnutella2 (or "G2") dumps the Gnutella broadcast model and uses a new global searching method with UDP connections. It also features compression to limit hub-to-hub (G2 Ultrapeers) bandwidth, Tiger Tree Hashing etc. Shareaza has released a small description of the revised protocol here, but plans to release a full spec to the GDF after the release of v1.7 Final. Gnutella2, which is really a revised Gnutella protocol, will also be free and open for anyone to use in their clients. Shareaza and G2 may give Gnutella - an open and free P2P protocol which has been struggling to keep up with the times against Kazaa, eDonkey and other P2P spin-offs - the stability and power it needs to attract the closed and commercial FastTrack Network users when or if the network folds."

Comments

It's pretty fast...

[Anonymous Coward]

I've tried the beta release and G2 hubs operate faster than the G1 hubs. I was able to get faster and larger searches. If only the other clients included supportf for G2 in the future. Better not be Coke II!

Re:It's pretty fast...

[iofire]

Did anyone else notice that on the beta download page (visit the "next version" link at the top of the page) that there is a button to download it via gnutella? It's nice to see someone make use of this as a way to download software.

Re:It's pretty fast...

[Anonvmous Coward]

"It's nice to see someone make use of this as a way to download software."

Kazaa does that. Their mini-installer logged into the P2P network and pulled the files down from one of the peers. When I started Kazaa again, the first thing that happened was people started downloading it from me.

I thought it was kinda cool. Far less bandwidth use on their part.

Re:It's pretty fast...

[pod]

The installer does that, and they also offer this capability as a service. It's called CloudBurst, of something like that, and lets people pay to upload their content to the Kazaa nodes, where it's stored in a special shared folder, and it's downloadable with a specially written Kazaa client.

Variety of standards

[pc_plod]

Anything that is a move to keep a variety of standards out the in the P2P world is a positive move, stopping record companies finding a way to stop the whole movement by blocking a single protocol, (a la Napster). The more the better.

Re:Variety of standards

[Nerant]

While you have a point, I must point out Napster wasn't strictly a peer to peer network system. Napster had a centralised set of servers, and was only peer to peer in the sense that it utilised the bandwidth of those sharing to upload those files to you and vice versa. As far as I know, these centralised servers are in fact what led to litigation against napster.
True peer to peer networks like Gnutella have no real centralised points: the process of discovery of new nodes does not require a a centralised server or servers, unlike Napster.

Re:Variety of standards

[pc_plod]

Of course, but pressure on client distibution can be equally damaging, as well as the adding of corrupt files to the network and other such underhand tricks. Having a central server to shut down made Napster an easy target but real P2P systems are vunerable as well, especially if one standard emerges

Re:Variety of standards

[adamshamblin]

I just hope the whole "gnutella2" doesn't end up being vaporous. More interesting than yet another client is enhancements to the protocol, but the gnutella2 web site is 'opening soon', and the 'Full specifications will be available soon', but there is already this Shareaza client out there?

I've been a user and supporter of the Gnutella Network since the beginning ( back when it didn't work that well :) ), and I've seen enough clients come and go to know that it takes a well documented protocol/specification to see network growth and improvement.

Until we have the specs, it's just hype.

Re:Variety of standards

[buswolley]

Design the "Trillian" of P2P. One that works on many networks at once.

Re:Variety of standards

[Ari Rahikkala]

Remember one fact about real P2P networks: They have a lot of overhead for searching and, in the case of Gnutella, pinging/ponging. IM networks are centralised enough to not have that overhead, so being logged in to several of them doesn't use that much bandwidth. But unless you have insane amounts of bandwidth or are going to be connected only to very few computers in each network at once, you do NOT want to be in more than one P2P network at the same time. Even if you do have a lot of bandwidth, it's a better service to be a supernode to one network than a normal node to several networks.

Re:Variety of standards

[The J Kid]

You want giFT.

It currently only has the OpenFT protocol as plugin, but you're free (or anybody else for that matter) to write a Gnutella2 plugin or whatnot, and it still connects via the same nice GUI of choice!

Re:Variety of standards

[The J Kid]

You're not seeing the "sharing" aspects. For instance, Gnutella uses HTTP for file transfers, but adds special headers to help assist in the transfer (alternate locations, SHA1 URNs, etc...).

Ah, but OpenFT does that too! HASH search, and HTTP for file transfers, etc. It's just that because of the more stable network (it is based on Fasttrack..) there isn't any need to send all those special headers.

I don't know what a "plugin" can do in giFT, but it would need to do quite a lot to be able to act as a Gnutella server.

The existing "plugin", openFT, can be a Fasttrack-like Index, Search or User node, so that shouldn't be a problem.

Personally, I'm not interested in using a protocol like openFT, because it is a marginal protocol with few users. It does not have the critical mass Gnutella has today.

Ah, but what is the critical mass of Gnutella? Because of the limitations of pureP2P you can only see a small part of the whole network and if you get mixed with good old Morpheus nodes, you ain't gonna see anything in this lifetime!

With openFT (hint, the FT stand for FastTrack, like in KaZaA) the network is far more stable and you can actually see the whole network all the time. And, as of this moment there is a mass of 3.6 TB with 8.2 GB/user ratio. (It has been 5.1 TB, but because of developement it breaks with older versions. (only CVS at the mo)

But to go back to the point, a G2 plugin in giFT would bind both networks together to make Gnutella more stable and give openFT more mass.

It's a win-win situation.

Other OS P2P technologies

[jfrumkin]

I've been working with the JXTA project for a bit now, and they seem to be taking a very nice approach to designing a p2p network that is implementation independant (can be implemented on different platforms, devices, etc.). Besides gnutella (and g2), and JXTA, are there other open P2P networks out there? And if there are, what's the best project?

Re:Other OS P2P technologies

[iofire]

I'm surprised that no one has mentioned it, but giFT is a very nice open protocal modeled after the fasttrack network. (originally it used the actual fasttrack network, but now they use an open protocol called OpenFT)
Check it out at http://gift.sourceforge.net
The ncurses based frontend giFTcurs is very nice, but there also are graphical and even web-based frontends to it.
I use it under linux and have been very happy with it.

Re:Other OS P2P technologies

[Anonymous Coward]

I've had nothing but problems with giFT. Finally started booting back into Windows and using Kazaa.

Re:Other OS P2P technologies

[WWWWolf]

I've had nothing but problems with giFT. Finally started booting back into Windows and using Kazaa.

Curious. I've had very few problems with the CVS version of giFT. In fact, i've found it to be the best-working p2p app on Linux since Napster. Doesn't eat all of the bandwidth for nothing, and downloads are *really* fast compared to *any* Gnutella client. I've actually downloaded things with this thing! =)

The problems have mostly been like "okay, today it didn't work, tomorrow it'll work again"; I think the biggest outage in my case was recently when the new interface protocol was introduced and all clients seemed to work only day or two later.

Re:Other OS P2P technologies

[PureFiction]

are there other open P2P networks out there?

Yes, the ALPINE Network uses a UDP based social discovery mechanism to implement fast, effective searches with minimal bandwidth and dual NAT support.

Some of the features include:

- High concurrent connection support (over 10,000).
- Adaptive configuration for enhanced accuracy and quality of responses.
- True peer to peer network. No hierarchy, no central servers.
- Low communication overhead (small UDP packets, no forwarding).
- Module support to allow extensions to query and transport operations.

You can read an overview of how alpine works here . There is also a frequently asked questions and plenty of developer information .

Enjoy!

Muahahaha.. Slashdot? With a sense of humor???

[Bowie J. Poag]

Anyone here find it just a wee bit ironic that a postabout BMG and their so-called "copy protection" (*chuckle*) is followed immediately by a rather technical article on a new, faster, better, low-density P2P client?

Hell, they haven't even managed to shut the _first_ version down!

Cheers,

It Would Be a Real Shame....

[CatWrangler]

if people tested out this network by trading only BMG files at first. Have to beta test and all though I suppose.

The Gnutella innovation I like

[Anonymous Coward]

Is the Gnutella Web Caching System. It allows clients to find other gnutella peers without any sort of central gnutella server.

Crossing fingers

[ceranta]

Let's hope that this gnew version of gnutella will be better and more scalable than the previous one.

Points from the gnutella2.com site:

Level One: A New Protocol

Gnutella2 introduces a flexible new protocol to support current and future P2P technologies. Packets are compact binary trees of named data items, which allow multi-vendor information nesting and augmentation, selective digital signing and other exciting features. Existing data structures can be modified and improved without disrupting deployed software, and advanced topics such as UNICODE support are handled in a uniform manner.

Level Two: A New Data Transport Architecture

Gnutella2 provides two interdependent data transport mechanisms: reliable compressed TCP streams, and an unreliable and semi-reliable UDP transport provider. The combination of these two systems allow higher level G2 constructs to take maximum advantage of network conditions to deliver data packets quickly and efficiently, with or without assured delivery, within bandwidth requirements and without unnecessary overhead.

Level Three: A New Set of Base Services

Gnutella2 takes full advantage of the first two levels to deliver an exciting new set of distributed peer-to-peer services. Controlled global object searching is implemented using an iterative walker approach, with selective out of band response delivery and translation. Combined with an abstract component interest/response query model, this system goes beyond what is available in any other P2P platform. The Gnutella Addressing System (GAS) provides the ability to reach arbitary nodes based on a known identifier, regardless of their connection method.

Level Four: A New Implementation Standard

One of the problems facing the legacy Gnutella network was the varying level of support for critical network features in different clients. The Gnutella2 Standard requires clients to implement the first two levels completely, as well as the dual transport providers with some form of intelligent bandwidth control, 1-bit universal QHT, simple search response, basic metadata (at minimum), simple query language, link compression, root tigertree as the primary URN, HTTP/1.1, partial transfer and sharing. If able to operate as a hub, the full set of generic routing rules must be supported. Support for G1 is recommended but not required.

CLICK ME!

Re:Crossing fingers

[Adam Fisk]

Almost all of these protocols are existing standards that have been developed by the rest of the Gnutella community. In face, not one of the primary protocols has actually been developed by Shareaza -- they're basically just repackaging the existing collection of newer protocols and calling them "Gnutella 2."

The only new additions proposed here are the binary tree structure for packets and the node addressing system. Otherwise, all of these protocol have been developed by other members of the Gnutella community. "Gnutella 2" is a marketting gimmick aimed at doing things like getting on Slashdot.
That said, it's not all bad. The perception of Gnutella should change, as the network is continuing to develop rapidly, with powerful protocols including the Hash/URN Gnutella Extension (HUGE), the Gnutella UDP Extension for Scalable Searches (GUESS), and the Ultrapeer proposal.

Perhaps the more important issue at hand, however, is whether or not Gnutella will remain an open, interoperable protocol, or whether it will disintegrate into proprietary schemes. As yet, none of the new parts of Gnutella 2 have been posted in public specifications. This is really a first for Gnutella -- the Gnutella community works because standards are published publicly and go through a review process among all Gnutella developers. Gnutella 2 may bode ill for the future of Gnutella as an open network, but I really I hope not. I hope that Mike (Shareaza) will quickly publish any new specifications that he has to alleviate the fears of myself and everyone else in the Gnutella world!

Re:Crossing fingers

[0x0d0a]

Yup. Raphael Manfreti (of gtk-gnutella fame) and the Limewire team (also major GDF developers), get no credit, and these "Sharezilla" wankers get a Slashdot link.

Well, *here* is credit where credit's due:

GTK-gnutella

LimeWire

Gnutella started out as an "interesting project". It is now one of the most heavily developed an analyzed projects -- somewhat less centralized than the Freenet project, but far more skill (and variety of clients) on this than, say, FastTrack and the much-lauded Kazaa.

Kazaa vs. eDonkey

[T-Kir]

Ever since I've been using Broadband (Optimum Online yeah baby!), eDonkey has won me over vs. Kazaa(lite).

Alhough eDonkey needs a little more work than Kazaa to operate, the file hashing/segmented downloads/no leeching is far better than Kazaa, plus the amount of file corruptions I get using Kazaa is way too much (especially with very large files). I've also started trying Overnet, but still have loads of downloads I'm clearing through the Donkey (Yes I have tried using the donkey downloads for Overnet, but only half register in the download tab).

I've tried using Gnutella/Gnucleus on numerous occasions, bit given up due to a lack of being able to do anything with it compared to the other P2P programs... I just hope Gnutella2 will become a viable option for me to use it.

Re:Kazaa vs. eDonkey

[Arker]

If you have a *nix box (even an apple if it's OS X) you can use mldonkey which is a very nice client. You can operate it remotely from another box, it uses both edonkey and overnet protocols simultaneously, it's partially open source (there is a key component kept secret for security reasons, the one flaw in thes protocols is that they require trusted clients unfortunately) and it really gives you the best of edonkey and overnet both, as well as supporting the move to overnet since anything you're downloading from edonkey or sharing out will also be shared to overnet.

Re:Kazaa vs. eDonkey

[Jugalator]

Same progress for me - i.e. Kazaa to eDonkey to Overnet. The biggest advantage with eDonkey over Kazaa IMHO are the "MD5 URL's" or whatever you should call them, where clicking on an URL adds the download to the eDonkey queue, by using the MD5-style checksum. So you're 100% sure it's not a fake file. You can also be 100% sure that's it's not a partial file as well.

I guess the downside with eDonkey is that it requires up to date server lists, although that's a minor hassle really. And that's where the serverless Overnet comes in, which owns Kazaa any time except for the occasional music searches perhaps.

Re:Kazaa vs. eDonkey

[bluehell]

it's partially open source (there is a key component kept secret for security reasons, the one flaw in thes protocols is that they require trusted clients unfortunately)

That's not true anymore. Since emule, anonther edonkey client, released its source code, the mldonkey author decided to open-source the remaining code.

Re:Kazaa vs. eDonkey

[Arker]

Wow, that's a big change. I wonder where that leaves the security issue though, that's of course always been coming, but now I guess it's upon us... the network relies on the fact that even those who try to be leechers can't avoid sharing the parts already downloaded while waiting for the rest... if the complete source is out it will be much easier for someone to put together a full leecher client... and if that becomes very popular the whole network will become untenable. :(

I never thought security through obscurity was a viable philosophy longterm, but it's better than nothing. What now? Have any of the developers addressed this that you know of?

Re:Kazaa vs. eDonkey

[athakur999]

For Kazaa, etc. you can try out sig2dat, which works similarly to the MD5 checksums. You click on a sig2dat link and the program will generate a .dat file with a checksum in it, which Kazaa will use to search for the file.

It's not perfect though, since you have to shut down and restart the client to get it to see the generated .dat files. It's nice when using sites such as Fast Track Movies.

Re:Kazaa vs. eDonkey

[mlinksva]

I don't know, but I'd love to. I'd also like to be able to have mozilla hand off MAGNET links to Lime, gtk-gnutella or the like.

Re:Kazaa vs. eDonkey

[Galvatron]

Okay, on your (and others) advice, I just gave it a try. It doesn't seem to have very many files. "Babylon" (to find Babylon 5 episodes) turned up 0 results. "Terminator" only showed 1 result, and that was supposedly a screener copy of T3: Rise of the Machines.

Kazaa-lite, on the other hand, has enabled me to watch virtually all of Babylon 5 (I've just got the last half season to go). What am I missing, how is eDonkey better?

Re:Kazaa vs. eDonkey

[T-Kir]

OK, first off there is a little knack to searching on eDonkey. You first have to make sure that your firewall will accept connections through ports 4662-4663 (and forwarding to the machine running the donkey)... most of this info is on the eDonkey site.

When it is up and running, you can do a search when you are connected to a server (a good idea is to get an updated serverlist, one of the places I go to is The Donkey Network). If there aren't any of the files there, then click the 'Extend Search' button that pops up to the left of the search button... to do more searches, click the button then press and hold down the enter key for less than a second, do more short bursts to let any server search results get through.

A lot of the files will be dependant on what people are sharing, and the more blue the colour, the more people have the same file. A great place I've recently found that lists certain Sci-Fi files is Varelse's Sharepool, and another site for other links is ShareReactor.

A lot of the server work (updating lists, etc) has been automated in Overnet, but I haven't been using it at all yet. As I said in the first post, it takes a little more work to learn eDonkey, but I've found the quality of files that are being shared far superior to the FastTrack network (esp. for very large files). There are times that I can't find stuff on the Donkey network, so Kazaa still comes in handy.

Just wondered...

[Anonymous Coward]

Does anyone actually use P2P networks for legal uses?!?!?!?!?!? e.g. not mp3/porn..

If so, can you list what you use it for?

Re:Just wondered...

[psavo]

mp3 is perfectly legal. many bands share their music via p2p networks. I also download some drivers & game demos from p2p, as it's often faster way to get them.

Re:Just wondered...

[roryh]

MP3s are not inherently illegal. I download classical music for which there are no copyright issures that I'm aware of.

Re:Just wondered...

[Jugalator]

Does anyone actually use P2P networks for legal uses?!?!?!?!?!? e.g. not mp3/porn..

*thinking hard*

Downloading .nfo's and .cue's?

AFAIK, those aren't illegal. :-)

Re:Just wondered...

[Troed]

If an orchestra records a classic that recording is (of course) copyrighted.

Re:Just wondered...

[Anenga]

Now that Shareaza has global searches (and nativly hashes in SHA/MD4/ND5/TTH) we can post up the hash of linux distro's and begin downloading from the Linxu distro site.

People can download off that person using partial file sharing, people can download off that person using partial file sharing etc. It will save the main site a hell of a lot of bandwidth and you'll be downloading the distro swarming from 10+ people rather than one slow FTP site.

Re:Just wondered...

[falzer]

As far as I know, the mp3s here are free. But many recordings are indeed copyrighted.

Also, in case anyone is interested in sheet music that fell out of copyright, check out this link.

Re:Just wondered...

[Sebbo]

I've downloaded a lot of MP3s of music that I already have on cassette. Since making personal-use copy of works you own is legal, I believe that activity is covered.

I'm sharing out a lot of MP3s I got from mp3.com and Amazon free downloads.

There are also a lot of movie trailers and video samples from pornsites out there

Re:Just wondered...

[reflector]

yes, lots of legit uses. here are some:

-downloading tv shows (futurama, the simpsons, south park, etc.), which is no different than recording the shows on my vcr, except has the advantage of being more easily organized and no commercials.

-downloading copyrighted works that i already own on another media such as cassette tape or vhs that i want in digital format.

-i'm sharing a lot of public domain works that i've downloaded from project gutenberg (plato, aristotle, descartes, thoreau, etc).

-downloading/sharing star office 5.2, which sun no longer offers for free on their web site.

there's probably other uses that don't come to mind at the moment...

Re:Just wondered...

[Abreu]

How about the latest Mandrake to Mexico City?

I'll mail you back the blank CD-Rs to replace the ones you used and some postcards, along with your small fee.

Gnutella2 - The real story!

[smd4985]

I'm a engineer at Lime Wire LLC so I can debunk much of this submission. Shareaza's Gnutella2 isn't so much the second iteration of Gnutella - instead, think of it as a improved Gnutella . In fact, the improvements were actually proposed by Lime Wire LLC (consult the GDF and look for messages about 'GUESS'). The GUESS protocol is a UDP based protocol we developed to allow for Gnutella network crawls/walks. We introduced it for public comment on the GDF *before* releasing it because we understand that Gnutella, as a open protocol, needs support from all Gnutella developers. I'm not sure what exactly Shareaza has implemented (because they HAVE NOT released the specs yet), but it sounds a lot like GUESS.

So this isn't so much Gnutella2 as a improved Gnutella. Perhaps one day it will evolve into Gnutella2 more formally, but at the moment this talk of Gnutella2 is premature.

Re:Gnutella2 - The real story!

[Adam Fisk]

Seconding Susheel's comments, "Gnutella 2" appears to be primarily a marketting gimmick. Gnutella 2 is really just a collection of protocols, most of which have been in use on Gnutella for some time. The one apparently new protocol is a version of the Gnutella UDP Extension for Scalable Searches (GUESS) open standard, that was proposed by LimeWire some time ago, as Susheel mentioned, and that is in experimental stages. That said, perhaps "Gnutella 2" makes some sense as a name, as the computing community seems to be out of touch with how rapidly developments are happening on Gnutella. The collection of protocols used on Gnutella today make it a vastly different network than what people typically think of as Gnutella. If Gnutella 2 changes that perception, then it's great. Just keep in mind that "Gnutella 2" has little to nothing to do with Shareaza -- they primarily contributed the name. The new protocols in use on Gnutella are the result of countless hours of work from many Gnutella developers around the world.

Re:Gnutella2 - The real story!

[YrWrstNtmr]

I'm a engineer at Lime Wire LLC...

So when are you guys going to remove all that crapware & stealware from the LimeWire client?

Re:Gnutella2 - The real story!

[Lukey Boy]

Judging by your comment and your associate's, it seems that you fellows feel as if your toes are being stepped on a little :-) Honestly though, Shareaza came out less than a year ago and is outpacing Limewire and Gnucleus (which was my previous favorite client), and it's written by one guy. Oh, and, NO SPYWARE.

At first I saw that you worked for LimeWire, and felt a small amount of respect - then I remember the bullshit hoops I had to go through to clean my system of the utter crap it installed through my system directories and the registry.

As for calling something like Gnutella2 premature, um, no. The standards of the web were written down by the W3C, just as the Gnutella standards are written by the GDF. But if Shareaza comes out with something radically different and is accepted by the majority of users, it becomes the standard much in the same way that IE (unfortunately) did in the browser war. Now the W3C is playing catchup - and maybe the GDF is as well.

Re:Gnutella2 - The real story!

[Adam Fisk]

I think you're right -- our toes were stepped on a little =) -- ours and everyone else's in the Gnutella world who put in most of the work to come up with "Gnutella 2," many of them open source programmers who donated their time to the effort.

Mike has done a hell of a job on his client and is a very nice guy, but he simply is not the originator of the vast majority of the standards being branded as "Gnutella 2."

The key word in your last paragraph is "unfortunately." Yes, it was unfortunate that IE created it's own standards and bypassed the w3c. Are you truly advocating proprietary standards over open standards? Am I misinterpretting you?

Re:Gnutella2 - The real story!

[Anenga]

Since you work for Limewire, then you'll know that Mike (the developer of Shareaza) was contacted by you with your GUESS proposal as he was working on his own similar proposal (wish is now used in Shareaza). I know those e-mail logs are lurking somewhere at Limewire LLC. Perhaps your accusations that Mike "stole" or "took" the GUESS protocol are a little too "immature"?

Come now. I think were all quite tired of the poor attitude the GDF has shown towards Mike. There is no rule in the GDF against further development of the Gnutella protocol. I could of course have apathy for why your upset (Mike hasen't released the specs yet, or consulted the GDF before hand) but there is no putting off the actual facts: I get better results on G2 than G1. Less bandwidth, more fruitful results, etc etc.

GUESS will be included in Shareaza soon enough as other clients start using it (in Shareaza's G1 capability). If you dislike the G2 design, that's fine. But we also hope that you could put aside your personal matters and actually embrace the network as really a better path to take than currently working with the mess which is G1.

Re:Gnutella2 - The real story!

[Adam Fisk]

I agree with you to some degree -- the key is coming out with the best technology for everyone in the end. It's just that open standards are typically what make this possible. This is as much of a social issue as anything else -- the Gnutella world is a pretty tightly-knit group that relies on trust to a large degree, but maybe it needed a little shake-up.

As far as Gnutella vs. other networks, this is really the crux of the issue. Gnutella has lagged behind in some ways precisely because it is open -- it just takes forever for people to agree. I would argue, though, that it's worth it. Why? Because you come up with better standards in the end. Gnutella is the one network that has a public set of RFC-style specifications precisely outlining how the various protocols should work. This takes time, but it allows interoperability.

eDonkey (and Overnet) are great counter-examples. They work very well, but they are proprietary standards for the most part. This has meant things like the eDonkey web URIs not really being standards compliant. But, then again, they're everywhere, so does it matter? Maybe not.

It's my belief that open standards win in the end because they allow unforeseen innovation and creativity to be built on top of them.

Re:Gnutella2 - The real story!

[Adam Fisk]

On the spyware topic, we originally started bundling software purely out of the need to survive -- we needed to bring in money, or we would have closed down. Since then, we came out with LimeWire Pro and now only bundle TopMoxie with LimeWire, and it's an optional install. TopMoxie really is not spyware in the way people typically use that word. It basically does one thing -- it has affiliate programs with multiple web sites, and when you visit those sites, it pops up a really innocuous window (that times out and disappears) asking you if you would like your purchase to support LimeWire. If you click yes, we get the affiliate revenue. If you ignore it or don't notice it, nothing happens. TopMoxie is primarily used by schools and charities to raise money for them.

So, the short answer is that we would prefer not to bundle anything. As a result, we now only bundle one program that we feel is really innocuous to the user and that they have the option not to install. On all operating systems other than Windows, LimeWire has no bundled software.

Re:Gnutella2 - The real story!

[zapfie]

So when are you guys going to remove all that crapware & stealware from the LimeWire client?

When a good amount of people are willing to pay decent money for a client?

Re:Gnutella2 - The real story!

[YrWrstNtmr]

Get real

"I, LimeWire the user, agree to allow LimeWire, LLC, to redirect authorized funds from a 3rd party during the completion of a transaction."

How is that not stealing? If you tried this at a store, the owner would have you arrested.

If LimeWire, LLC's business model sucks so bad as to not be able to generate enough income to pay their employees, that makes it OK to take money from a 3rd party (who certainly did not agree to this)? Bull.

A crappy business model does not make stealing OK.
If you don't make enough money to support your family, stealing from the supermarket is OK?

Re:Gnutella2 - The real story!

[0x0d0a]

At first I saw you worked for LimeWire, and felt a small amount of respect - then I remember the bullshit hoops I had to go through to clean my system of the utter crap it installed through my system directories and the registry.

I hate to say it, but I'm starting to get a pretty good chuckle every time I see some poor Windows user griping about the amount of pain they go through to get "good downloadz". I hear whining about "pop up" or "pop under" ads. I hear complaining about "spyware". I hear complaining about "mandatory sharing" in P2P apps. I hear people frantic that newer P2P apps can "fake" shares (like on Direct Connect) because of piss-poorly designed architectures involving trusted remote code.

It's all really funny to those of us who have been using open source P2P clients and Mozilla on Linux. *We* haven't seen a single one of these problems, and *we* aren't suffering.

But, you know what? I encourage pop-ups. And intrusive advertising, spyware, and everything else. Why? It doesn't affect me in the least, and it means that *you* are subsidizing the good life for me. Each pop-up you see funds another good, clean pop-up free page for me.

Of course, someday you people are going to catch on. You're going to use Mozilla, use Linux. You're going to use better P2P clients. But until that day, the rest of us are going to enjoy the good life.

Until then, thanks for everything!

Re:Gnutella2 - The real story!

[0x0d0a]

Sharezilla is more than welcome to use their "G2" protocol. They can communicate with the other "Sharezilla" users out there. Woohoo!

Re:Gnutella2 - The real story!

[Adam Fisk]

We also don't mean to imply that Mike "took" GUESS in any way. I fully understand that he was working on a separate protocol when GUESS was being developed -- I understand because I'm the one who had those conversations with him.

The point is that GUESS is a public specification for searching on Gnutella. Hopefully, whatever Mike is doing will soon be public as well. Then again, if it's not GUESS but is very similar to GUESS, then it'll create a standardization nightmare that we've worked very hard to avoid -- the type of incompatibility that wastes everyone's time. We need to have an open network that evolves and innovates rapidly. Perhaps Gnutella 2 will be a positive part of that, but it's not off to a great start.

Re:Gnutella2 - The real story!

[Jeremi]

A crappy business model does not make stealing OK.

That's a little rich, coming from a p2p file sharing advocate.... ;^)

Re:Gnutella2 - The real story!

[YrWrstNtmr]

P2P has nothing to do with it. That technology could be piggybacked on any application.

OBTW, I much prefer emusic.com to any 'file sharing'.

Re:Gnutella2 - The real story!

[karlm]

Interesting... any way to tell if a site is a TopMoxie affiliate? Any way to set the appropriate TopMoxie information for an order without the official TopMoxie client?

UDP and firewalls

[elliotj]

I wonder how this client will perform for people behind firewalls? Many firewalls are setup to deny UDP traffic because most Internet activity is TCP and having UDP open has been unnecessary up to this point.

I wonder if this will halt the spread of Gnutella2? With P2P, it's all about getting as many people online as possible.

Re:UDP and firewalls

[Adam Fisk]

Firewalls block most incoming UDP traffic in the same way that they block incoming TCP traffic -- there's really no difference. Incoming traffic is generally denied except for specific ports.
So, with both UDP and TCP, only outgoing data will not be blocked as a general rule. With TCP, this poses less of a challenge because once you've established a connection, data can be passed both ways. With UDP, you cannot establish a connection in the same way. That said, most firewally will allow incoming UDP from a specific endpoint if you've sent outgoing data to that endpoit "recently." In this way, a quasi-connection can be established.
All that aside, though, the short answer is that non-firewalled hosts, and specifically "Ultrapeers" on Gnutella, act as proxies for firewalled hosts, allowing firewalled hosts to behave on the network almost exactly like hosts without firewalls.

Another alternative...

[fault0]

is OpenFT from the giFT project.. as people may recall, giFT was originally an open implmentation of parts of the FastTrack protocol, used by Kazaa, et al. This was an year ago, and KaZaA was not at all happy about this, so they updated a few times to break giFT (see KaZaA version 1.33).

So, some of giFT's developers decided to abandon fasttrack, and make their own protocol, OpenFT. giFT went from "giFT is not FastTrack" to "giFT: Internet File Transfer". This protocol, primarily written by jasta of gnapster fame, has been development for the last ~8 months. A publically released version of giFT with OpenFT is not available yet, but right now, the CVS version works quite well.. even in some ways better than FastTrack does.

There are also some great advantages to giFT. First of all, it enforces a seperation between the client and the network code. giFT is a daemon that handles most of the interaction with the outside world. There are also a multitude of giFT frontends, which are very easy to write, as no network code has to be created. giFT is also modular.. you can put in bridges or even full support to other protocols and networks.

Re:Another alternative...

[RPoet]

I call bullocks on you. While the giFT project leader and main coder may appear a little antisocial and semi-narcissistic, he and his team are highly skilled (and well aware of it, which may be a little politically uncorrect). giFT is highly usable even now, and it gets better for every cvs update I do.

It's certainly very useful for music downloads, ranging from everything from the latest Pop Bimbo to classical Strauss. While the number of users on giFT range only in the hundreds, these are an idealistic crowd, not free-loaders like on KaZaA. Each user shares in average around 8 gigabytes! And you won't find a higher .ogg ratio on any other network. Ditch those mp3s!

If you haven't, try giFT today! And if you're even more interested in the project, join the discussions at #gift @ freenode (and get a taste of that "attitude" you've been hearing so much about ;).

Re:Another alternative...

[fault0]

> While OpenFT is a good protocol, their support for casual users (even of the linux variety) is rather hostile.

This is mostly because the new giFT is not really supposed to be used by the wide public yet, as there is not a general release yet. You are welcome to test it right now, and thus help improve it, but it's not in a stage where the developers like jasta can give support for it, especially when it's still changing relatively rapidly.

Anyways, giFT is still worth trying. It is really shaping up as quite a project. I've been following it since before the beginning, when a mysterious group of people called the "givers" had reversed the FastTrack protocol, and donated it to a group called kazaa-linux who had been working to make kazaa or a variant work in Linux. It's still unknown who the givers were.

UI

[Mr_Silver]

It's a small thing, but my biggest complaint about these p2p programs is that the user interface just sucks.

Sure, it's useable, but it's horrific. Kazaa's is aweful, eDonkey's just blows and WinMX, urgh, don't get me started.

Admitially I never really investigated Gnutella after trying the original Nullsoft version. The UI was ok, if a little plain, but the time it took to hook up to a bunch of stable nodes, the slow download time and frequency of dropped downloads just put me off.

So really, all i'm asking is that whilst you're concentrating on making an excellent protocol, please don't employ a 7 year old with a crayon to do the UI. Hell, I'd happily help out on an OSS project, however I can't use VC++ to save my life and most people wouldn't like submissions on Visual Basic frm's - i'll probably end up standing on the sidelines shouting but having no-one listen.

There are a few examples of technically inferior applications that do better than others simply because their UI is clean, consistent and works. Lets have that, please!

Re:UI

[Anenga]

Whaa? I actually think Shareaza has the best UI out of all Gnutella clients.

Though, there is that one Searching Status I begged Mike not to put in, which was put in anyways =)

Your suggestions are welcome on the forum.

Re:UI

[0x0d0a]

It's a shame that all the Windows P2P clients use things that look like their UI designer is an ex-web designer.

UNIX P2P clients don't suffer from this problem.

Re:UI

[Pinky]

I get a half dozen complaints about the GUI in my p2p program called Myster every few months. I have asked people to do mock-ups of what they would like to see in a GUI and have not yet received any. Personaly my ideal p2p GUI is in Myster (well, with some more intellegent window behavior). Many simple windows for a small learning curve and ability to do many things at once. or not.. :-)

OSS no spyware Unicode everywhere (lots of japaneese stuff)

www.mysternetworks.com

Re:Hopefully downloads are better with G2...

[bpd1069]

I even got my girlfriend, boss, and brother using Bearshare.

You got a girlfriend, boss and Brother from a P2P applications! Wow what search are you using?!

Re:Hopefully downloads are better with G2...

[fault0]

> I even got my girlfriend, boss, and brother using Bearshare.

Congratulations, you made them install quite a bit of spyware too.

I would recommend something like XoloX, which has absolutely no spyware.

Re:spyware?

[Lukey Boy]

It's actually got a clean record, from day one. Amazingly enough.

This...

[jmu1]

coming from a company who installs spyware? I don't use p2p filesharing apps, but I've had to clean several machines of users who do. Although I fully understand the desire to rob people blind through their own stupidity; I'm in technical support for crying out loud, I just can't see how anything that comes from you guys(or any other company willing to put that sort of trash on a person's machine) could be truthful.

The problems with Gnutella 1

[evilviper]

The (solvable) problems with Gnutella:

Bandwidth Usage (for searches)

Search results. You only get about 4-7 hops. Assuming 4 hops & 4 non-redundant connections per node, that means you are only searching about 256 nodes. Being able to search everyone would make Gnutella for more useful for less-common files.

Fifo queuing. You may have been requesting a file for the past 24 hours, but someone that just requested a file may get lucky, and take what should have been your spot.

Messages. We need messages to tell people that slow nodes downloading from our node gets disconnected, that you are 2nd in the queue, etc.

Upload settings. Each node should be disconnected after a set period of time to prevent slow nodes from causing bottlenecks, or RIAA employees from abusing the limited open slots.

Bandwith Min/Max for Uploads/Downloads. A limit on the min/max speed for each file download/uploaded, and a min/max for the TOTAL of all downloads/uploads.

Dynamic determination of REAL IP (if behind NAT with dynamic globally valid IP).

Solution to the 'PUSH' fiasco. Is there a way that 2 firewalled nodes can connect to a third (non-firewalled) party to open the connection, then tranfer data directly? I don't think so, but worth including here.

Any more?

Re:The problems with Gnutella 1

[evilviper]

Hmm, I've left a few out...

Metadata. So that you can find mp3s/oggs via the value in their id3 tags. e.g. Search for Artist:Megadeth

Searching by (sha1) hash. Some have it, ALL need it. Slackware could put their latest ISO on gnutella, and distribute the (sha1) hash on their website. You then find the file on many nodes, and can download from any or all of them. Instant , easy, free, file mirrioring.

And while I'm here... Long live Napshare!

Re:The problems with Gnutella 1

[glesga_kiss]

Solution to the 'PUSH' fiasco. Is there a way that 2 firewalled nodes can connect to a third (non-firewalled) party to open the connection, then tranfer data directly? I don't think so, but worth including here.

Not possible. The only way it could work is if the third party connection acted as a proxy for all of the data, because neither side can initiate the TCP transfer. The PUSH idea is actually a pretty neat solution to getting around firewalls, but I never liked the way Gnutella used it, you rarely got a successful push.

It won't work if both clients are firewalled, and this is in keeping with the point of the firewall, i.e. preventing incomming connections.

My solution is to run p2p on a sacrifical PC, that only has limited access to my network i.e. read-only access to SAMBA shares. I do this anyway because it's a public PC in my living room, and with lot's of random people around from time to time, it's a good idea to protect my data. My firewall forwards the p2p ports to this host, so I basically can access all of the nodes on the network. Should it ever get "rooted", then my exposure is not quite as bad as it would be for a trusted machine.

Running p2p behind a firewall severely limits the number of people you can access. I see this as a good thing, because it means less people are fighting over the resources that I personally can use. ;-)

Re:The problems with Gnutella 1

[chrohrs]

What may not be clear to many Slashdot readers is that the Gnutella protocol has been steadily improving over the last few months. Let me correct the previous poster on a few points:

Search results. You only get about 4-7 hops. Assuming 4 hops & 4 non-redundant connections per node, that means you are only searching about 256 nodes.

Your math is way off here. Try 7 hops with 6 connections, plus an extra factor of 100 or so from ultrapeers. That said, we are always looking for ways to improve searching. Ultrapeers were one step along that path.

Fifo queuing. You may have been requesting a file for the past 24 hours, but someone that just requested a file may get lucky, and take what should have been your spot.

Many clients (e.g., LimeWire, BearShare, Shareaza, Gtk-Gnutella) have supported this for some time now. They all interoperate too.

Bandwith Min/Max for Uploads/Downloads. A limit on the min/max speed for each file download/uploaded, and a min/max for the TOTAL of all downloads/uploads.

All decent client have features like this. But note that this is an implementation issue, not a protocol issue.

Search by hash

This has been supported for many months, thanks to Gordon Mohr's HUGE proposal.

Metadata

LimeWire has had XML-based metadata for over a year. I believe Shareaza uses the same scheme.

As these examples show, the GDF has been quite successful at driving innovation on the Gnutella network. But caution is sometimes in order; it can be hard to predict the result of thousands of clients running a new protocol. It would be good for Shareaza to submit its new extensions for peer review before rolling out thousands of clients. It is easy to build a client that gets more search results; it is harder to do that without hurting the entire network.

Christopher Rohrs
LimeWire

Re:The problems with Gnutella 1

[XNormal]

Not possible. The only way it could work is if the third party connection acted as a proxy for all of the data, because neither side can initiate the TCP transfer.

I've seen an article somewhere describing a system that exploits the little-known TCP feature of simultaneously opening a connection from both sides to allow two machines behind NATs to talk TCP to each other. It's very tricky and starting the connection requires the assistance of a third party that is capable of sending packets with spoofed IP addresses. After the connection is open, though, the two participants can talk directly without additional help.

It may not be practical, but it is possible.

Re:The problems with Gnutella 1

[evilviper]

we are always looking for ways to improve searching. Ultrapeers [limewire.com] were one step along that path.

Supernodes are anything BUT a solution IMHO. It is simply taking the heavy bandwidth usage from one node, and passing it to another. Far from idea. Besides, you should consider exactly how much bandwidth is used up to provide 6 simultaneous connections with a hop-count of 7. It's more than many broadband users can muster, even if that's the only thing they are dedicating their bandwidth to.

GDF has been quite successful at driving innovation on the Gnutella network

Not really. The only places there have really been improvements is in client-side changes. The bandwidth-heavy 'broadcast' method has been in-use (and despised by everyone) since the very begining. I wouldn't call 'superpeers' anything more than a nasty hack that just shifts the problem around a bit.

Re:spyware?

[Jugalator]

Anybody who knows want to comment on whether Shareaza is loaded with spy/adware?

What's stopping you from clicking the link in the article to find out on the web page? ;-)

This is missing the point

[TerryAtWork]

The whole point of a p2p network is not to share files but to not get caught sharing files.

Last February I got a bigfoot letter from my ISP, Rogers, who had been contacted by the Canadian equivalent to the MIAA, whatever it's called.

I was sharing tons of stuff, 8,000 mp3s, on DalNet and they wanted me to stop. What bothers me is they never contacted ME, they went straight to my ISP and tried to get me kicked off the Internet.

The letter from Rogers said you're in violation etc, stop now etc, or else etc.

So, I stopped.

This close call ruined my career on Dalnet where I had built quite a rep, and trashed my source of free music.

And not popular music either - ancient stuff you cant get anymore, like Robert Crumb and his Cheap Suit Serenaders. Buy THAT at your local CD shop...

Since then the point has been made moot by the fact that my cable modem has been capped at about a FIFTH of it's previous speed. (I am investigating DSL)

However - the crux of the whole matter is this - the record companies hired people to go on the internet and score music for them. Then these people, who, and this is crucial, have the IP of the music source, use that IP to run down the source down and then use legal means to try and get that person kicked off the internet.

IT DOESN'T MATTER HOW FANCY YOUR PROTOCOL IS OR HOW GOOD YOUR CRYPTOGRAPHY IS, IF THEY CAN GET YOUR IP YOUR SCREWED.

I have NEVER seen a p2p system address this issue.

Re:This is missing the point

[Jugalator]

IT DOESN'T MATTER HOW FANCY YOUR PROTOCOL IS OR HOW GOOD YOUR CRYPTOGRAPHY IS, IF THEY CAN GET YOUR IP YOUR SCREWED.

*ears ringing*

I have NEVER seen a p2p system address this issue.

No, it would probably require changing the IP layer to something more exotic. And don't say CPIP-based solutions doesn't work! :-)

Re:This is missing the point

[Lukey Boy]

I have NEVER seen a p2p system address this issue.

I have - it's called Freenet.

Re:This is missing the point

[/dev/trash]

This close call ruined my career on Dalnet where I had built quite a rep, and trashed my source of free music

Your career? Huh? You were distributing files illegally. What career is that????

Re:Hopefully downloads are better with G2...

[Anonymous Coward]

"Skank", "Ogre", and "Homosexual".

Gnucleus & GnucDNA

[DeadBugs]

Gnucleus has been a solid Gnutella client for me.

They are also working on GnucDNA a component for building your own P2P applications.

Re:Gnucleus & GnucDNA

[sshore]

I thought Gnucleus was pretty good too, but on my computer it crashes after a short time if I share any files. It appears to crash while hashing the files and performing queries on those files. If I let it finish hashing the files before connecting to the network, it works just fine - right up until I actually download something and it hashes the download to verify it.

I believe someone on the Gnucleus forum tracked it down, something to do with the shared file lists changing between the start of a hash and the end.

I've been waiting a long time for a Gnucleus 1.8.5 to fix this problem. Nothing's available so far.

MOD PARENT UP!

[yoz]

At last, a straight answer from Limewire people about spyware.

Re:Hopefully downloads are better with G2...

[Anonymous Coward]

I even got my girlfriend, boss, and brother using Bearshare.

I'll bet all three are the same person ;-)

Ignore him, Adam

[0x0d0a]

This guy is a troll. He's pushing IE, closed source, propriatary standards, and the domination of MS over the W3C and standards committees. He's saying that product "foo" is better than your product to try to sting you a bit, and then bashing the GDF, which *developed* the stuff he's lauding.

Don't bite.

Spyware Free Limewire

[asv108]

Well I assume if your posting on /. you are familiar with CVS? Go to this page and grab a copy of the source. All you need to compile is the java sdk and the ant build tool.

A *real* anti-leech/anti attacker system proposal

[0x0d0a]

no leeching is far better than Kazaa

I seriously doubt that. Any current "no leeching" mechanisms I've seen are severely flawed and rely on trusted remote code.

People who whine and bitch that people are bypassing them are ignoring the fact that the design is fundamentally wrong. You cannot trust code on another computer. Period. It *will* be broken.

It is possible to build a trust web (where you have metered trust, instead of just a binary "trusted" or "not trusted" a la PGP). Have each user generate a public/private key pair. Have each person maintain a list of trusted users. These users are identified by their public keys. "Trust values" are assigned to each user in the list-holding user's trust list. The scale is arbitrary -- maybe "100" means trust a lot and "1" means trust a little, and "0" means no trust. Trust is generally positive (more on that later).

When you want to determine "absolute trust" of a user, you run out and download the trust lists of all the users from them in your trust list (this spans only two hops out on the web of trust...you could go further, though I think this is sufficient). Person can grant absolute trust to person B as following: (points of trust A gives B in A's local trust list)/(total points of trust A gives A's local trust list)* (points of trust A has in our local trust list).

Then, attackers like the RIAA will be excluded from the network of trust, having low or no trust values, as they hand out corrupted files.

Trust lists can be redownloaded whenever. Cache 'em for weeks if you want.

Clients could automatically add a point of trust per data unit downloaded succesfully from a remote client...then, if it's a bad download, the local user could strip all trust away.

Trust could be used for ranking priorities to let people download from you, determining which copy of a file is "authentic" and which is bogus, etc.

Other possibilities: the reason we don't allow negative trust or blacklists -- only whitelists -- is because it's usually fairly easy to regenerate a new IP, and this results in bloating attacks against users maintaing blacklists. If a user can present something that "costs" them something to obtain, like a VeriSign cert or other "expensive" (i.e. can't regenerate on your computer easily) proof of identity (doesn't have to be your RL name -- could be a signed cert endorsing a 'nym from Zero Knowledge), then give them automatically a certain number of points of trust (client configurable). Why? Because it's much less likely that they're running out and buying a new Verisign cert for each attack. They're opening themselves up to blacklisting.

You could purge year-old entries from your local trust list to stay up to date...oh, there's tons of possible tweaks.

The trust network simply sits on top of another P2P network. It does not require that users not download from users with zero trust -- it simply provides some extremely useful information which is essential to implementing strong antileech/anti network attack protections, or what have you. It is also very difficult to attack. PGP is much more vulnerable, since you just need one stupid person in your web of trust to okay someone, their binary trust bit flips to 1, and they're in your web. If you don't trust someone much, and they give someone else a little tiny bit of trust...that person is only very slightly trusted.

Drawbacks:
My analysis of this approach has found only two drawbacks. First, there is some disk and memory overhead to store cached trust information locally. Gnutella clients already store IPs for much of the network, so it shouldn't be prohibitive, though -- we don't have to handle the whole network, just *trusted* users.
The second one is that letting people download your trust list -- crucial to the functioning of the system -- can leak some information. It means that you "trust" some user on the network. If that user provides nothing but, say, child porn, anyone on the trust network has circumstantial evidence that you have downloaded child porn. Of course, you could have granted the person trust for any number of other reasons, but it is a small amount of information leakage, and worth mentioning.

I welcome comments.

Re:Very Interesting

[0x0d0a]

WTF would any users donate their disk/bandwidth to a DRM-using legal media distribution system?

That might get IPO dollars, but it's not going to work.

Archie! Live!

[0x0d0a]

I want archie back. Current web-based FTP search engines suck. I like the command line search...pop in the name of a software package like openoffice, and it spits back several sites.

After a lot of work, six months ago I got an archie client compiling. Took some work on the source, but got it up. Then I took another two weeks to find a working archie server. It was the last public one, I'm fairly certain.

Two months after that, it went down. I was probably the only person that used it in ages, and probably the admins were wondering what was going on.

Download Mirror

[nstrom]

Download link http://download.shareaza.com:8825/Shareaza1701.exe seems impossibly slow -- I'm getting 276 bytes per sec on my DSL connection. For anyone who wants to check out the 1.7 prerelease, here's a mirror:

http://nstrom.chaosnet.org/Shareaza1701.exe

Re:Download Mirror

[reflector]

i'm sharing 1701 on gnutella, as are many people. that's the fastest way to download it if you already have a client that understands magnet links:
magnet:?xt=urn:sha1:QQB67YHOQV5BSLCFS7JYV6 2QAPLWCF RB&dn=Shareaza1701.exe

Re:Download Mirror

[reflector]

take out the space between the wcf and the rb&.
for some reason slashdot inserts a space when i post a magnet link.

OK

[chegosaurus]

So does this mean I get more pr0n, or less pr0n?

Re:Very Interesting

[0x0d0a]

That's why the company would want to do it. Where's the incentive for the user?

Has been reviewed

[0x0d0a]

I've run this by one of the gtk-gnutella developers, who's said that this seems like a good idea.

Re:How exactly does TopMoxie work?

[Adam Fisk]

The difference is in the link. In your example, the user links to Amazon from another web site. In this case, TopMoxie does nothing. If the user goes directly to Amazon by typing it in a web browser, though, TopMoxie does prompt the user for whether or not he or she would like the commission to go to LimeWire.

UDP and DDoS

[sshore]

I haven't seen anyone mention the potential of abusing the UDP search extension as a massive DDoS reflector. Simply send a query for something very common, with a faked source address on the packet, to as many Ultrapeers as possible. (I'm assuming that Shareaza implemented the GUESS extension, as many people have suggested.)

The documentation for GUESS is not reassuring:

In the past, a principal objection to using UDP has been that it allows anyone to easily execute a DDoS attack on any target machine. This concern has been based on the assumption that queries would require an extension listing the IP address and UDP port to reply to, however. In this proposal, this extension is not required, as responses are always sent directly back to the node that sent them, rendering such an attack impossible.

This totally ignores the fact that the only way to determine which node sent the packet is to use the source address on the UDP packet! Am I missing something here? Am I misreading the documentation?

Re:UDP and DDoS

[pclminion]

What I don't understand is why spoofing is possible at all. Can't your uplink router just take a look at your spoofed packet and decide, "Well crap, there's no way in HELL a legitimate packet from THAT IP address could have come from THIS interface, so I'll just drop it on the floor."

Ok, so you can still spoof the addresses of other people on your segment (or possibly subnet, if you are using dumb switches), but shouldn't routers be able to figure out when packets are OBVIOUSLY bogus?

Re:UDP and DDoS

[sshore]

Ok, so you can still spoof the addresses of other people on your segment (or possibly subnet, if you are using dumb switches), but shouldn't routers be able to figure out when packets are OBVIOUSLY bogus?

As the saying goes, "That would cost extra". It's one of those things that's only really useful if everybody does it, and it requires more intelligence and processing power in the routing hardware, which equals more dollars.

Many networks already do Reverse Path Filtering, but that doesn't fix the networks that don't do it.

Re:Hopefully downloads are better with G2...

[sgtsanity]

G2 is very, very good. The main improvement is that it has a global search radius. Normally, in G1, you can only see about 20% of the network at any one time. This is due to the design, and to the bad clients (i.e. Morpheus) polluting it. G2 has special techniques (a modified & extended GUESS) to see everything in the network at once.

There are other advancements that improve upon this, but they aren't really the thing that has the biggest impact now, like Tiger Tree Hashing, etc. Shareaza has already improved the network by providing a high-speed, high-efficiency backbone to the rest of the Gnutella network. The Shareaza clients freely connect to the other clients on the network, and so provide a way to see more of the network at once.

So, any way you spin it, it's good for the Gnutella network, especially considering that the specs will be released soon. And everyone who complains about this, even though their complaints may be valid, aren't seeing the tremendous improvement this makes to the workings of Gnutella.

Re:Hopefully downloads are better with G2...

[golrien]

To be fair, Shareaza has no spyware either. What's more, it's almost the *only* P2P client that doesn't eat up half the system resources - Kazaa, Bearshare, XoloX, Grokster (which installs more spyware than I thought was humanely possible anyway) and many others are all huge and silly. Shareaza (and at least one other Windows Gnutella client, though I forget the name) will happily multitask.

FWIW, Bearshare doesn't install insane amounts of spyware. Okay, it's not as good as the 100%-free clients, but they seem to have said "Hey, let's sell an improved version of this to make money" rather than, "Hey, let's piss off all our users to make money"

Re:Very Interesting

[0x0d0a]

No, I mean where's the incentive for the customer in using P2P at *all* if they aren't saving money by not paying for the item (or paying less)?

I would imagine that once the feeling of a "community" working together, where everyone pitches in some bandwidth and everyone saves money, goes away, people are just going to want to buy the product directly from the company (possibly electronically) or from bricks and mortar.

Witness Napster trying to charge for access...

Re:Very Interesting

[0x0d0a]

I'm not *arguing* that it's attractive to the company providing the service. P2P is great for them. The problem I have is why on earth the *consumer* would want to pay *and* pitch in time, effort, and bandwidth.

Re:A *real* anti-leech/anti attacker system propos

[karlm]

Ever signed anyone's PGP key? There are 4 levels of trust. (at least with GPG) Check your facts.

Re:A *real* anti-leech/anti attacker system propos

[0x0d0a]

True (I wasn't aware of this, and I'm not sure that there's actually *support* for this in any actual implementation), but not useful for what I'm talking about. It's still much too rough-grained.

Re:Anonymity: A problem with Gnutella

[evilviper]

I was well aware of this concern, but I don't believe it is really solvable.

The best solution is to have the Gnutella nodes work as routers. Each one passes the data off to the next.

Let us assume that A wants to dowload from B. Well there are a few nodes in between them (4 hops). Well, how is one going to no what the next hop is? Either A or B is going to need to know the complete path the the other.

For that small improvement in anonymity, you are now downloading at 1/100th the speed you would get from a direct connection.

Additionally, the node working as the router directly connected to A or B could tell that they are the end nodes. So, on top of all that, you need encryption.

I've spent some time thinking about it, and I haven't figured out any good scheme so far.

Re:A *real* anti-leech/anti attacker system propos

[0x0d0a]

In a real p2p net you constantly interact with people you have not communicated with before.

But if you have a 200 entry trust list, and all the people on there have a 200 entry trust list, that's a pretty sizeable chunk of data. It isn't just those that you have to interact with.

The traffic required to gather the information needed to keep track of individual users is more of a burden than leeches.

Take the above scheme...200 lists, refreshed on at most a weekly basis...that isn't *that* much. A gnutella client generates more than that with search broadcasts in a short period of time.

When user A uploads to user B only they know that it happened, if user A says he uploaded but user B says he didn't then who do you trust?

Using files was just an idea, one of the many sources of deriving trust automatically. Originally my idea was to manually grant trust -- "Wow, this guy's serving me this great file! I'm giving him some points!". However, I suspect that people are more likely to be willing to do active negative moderation ("These files are fakes -- I'm stripping the guy of all the trust I gave him") and let the client slowly increase trust. It also makes it difficult for someone *cough* MPAA to serve deliberately corrupted files, since their goal is to serve tiny corrupted slices and damage large files -- to gain trust, they have to serve large chunks of data. There is no required, direct link between files downloaded and trust gained. Furthermore, the only person listened to is the person *granting* the trust. He is under no onus to grant trust if he downloads a file -- this isn't "credit" or "money", it's "trust". He could have a client that doesn't trust anyone except those he manually marks.

There is also the privacy/incrimination issue of tracking individual issue of tracking individual users uploads and downloads.

Yes, I mentioned that. It was some of some concern, but the fact that trust can derive from other sources helps obscure it. There's no log of transferred files ever moved around. The best you know is that Person A knows of Person B, and likely engaged in some transaction with them. Hell, they could have chatted with them and liked them and jacked their trust.

I thought about this a lot and tried to create algorithms for working out how much to trust someone

Same here, though my interest lies more in the field of improving resistance of P2P networks to organized outside attack, as the RIAA/MPAA have been hiring firms to do. The robust antileech stuff is a bonus that I suspect some people will be interested in.

The only solution that seems workable to me is for the users to form groups. The group members trust each other. People who want the same sort of files will group together, groups of pr0nsters, electronica fans, ebook people etc. When people in group X have sent you more bytes than you have sent to them, you give people from group X priority in your upload queue.

Actually, rather similar to what I'm doing in essence. My approach is a bit more fuzzy (outward spreading weakening webs of trust), since I'd like to automate as much as possible, which I feel is important for scalability reasons. People tend to interact with those that hae similar interests.

You need some crypto so that group members can prove their membership and people can't falsly claim to be a member of a group that uploads a lot.

[Nods] similar to my conclusions, and my design follows yours here. Note: one drawback of any auth system that I have a small amount of concern over is that suddenly, people are storing a standardized piece of data that has lots of value to crackers on their computer -- their keypair. I could see nasty worms erupting aimed at grabbing keypairs.

The implementation details will need tweaking after some actual s/w is written. My current vision is that a group grows from a single member who holds the group private key. The founder grants membership to people who upload to him. Group members send reccomendations to the founder for new members. When the group grows large it may be necessary to spread the private key to several members.

Again we're similar. A few differences -- I tried to avoid discrete trust level issues -- "I trust you" or "I don't", but no middle ground. You may not trust everyone that gets into the group -- an issue with PGP, for instance. Second, there is no single private key in what I'm proposing -- no person has rights to anything other than their own trust list. Third (this may in particular be interesting, since it could be better either way), with your system trust is transitive. If User A is in Group B, and User C is in Group B, then A and C both trust each other. I allow one way trust -- I can trust Bob the Enormous Fileserver Admin, though he may not trust me.

Your idea sounds almost like a distributed version of Direct Connect. If you can get a userlist, it's like a distributed server...

Thanks for the feedback and posting the ideas. There are precious few people interested in security issues on P2P (despite the fact that P2P needs security more than most areas), and it's really good to talk to someone else interested in the area. Most P2P types like "quick patch" solutions. :-(

Re:A *real* anti-leech/anti attacker system propos

[0x0d0a]

What technical problems are you concerned with?

I have one major advantage -- I suspect that most people doing decentralized trust care about whether someone can be trusted absolutely or not. I'm willing to say "this person is a little trusted" even if there's not a lot of ground to go on (because someone you trust trusts them) or "this person is not trusted" even if they could be (the db doesn't have to be complete at each node). It's okay to have a certain error margin, because this is just used to assist the primary program. That would, I imagine, free me from the nastiest constraint involved...

Sounds like a trust metric to me

[raph]

Hi,

Interesting proposal, 0x0d0a. It's quite similar to the trust metric work I've been pursuing for my PhD thesis, and which is implemented in Advogato.

In general, I find the attack-resistance of your proposal to be sound. However, because your horizon only goes out to a horizon of two, it's an easier problem to solve than the general trust case. If you try to extend your system to more hops, you'll probably find that the trust values fade very fast. This was one of the less expected results I found when I implemented the eigenvector-based diary ratings at Advogato.

In any case, I encourage you to look at my trust metric work, and to use the code I've released as well as the trust graph available at Advogato. This will probably help you test and develop your ideas further.

Here's a url with links to most of the other stuff I'm talking about: http://www.levien.com/free/tmetric-HOWTO.html

Has some trouble

[Sycraft-fu]

I actually remember e-mailing you about this back in the day, how because I was on a super high bandwidth line I ended up having upload traffic an order of magnitude greater than my download traffic.

It's not entirely up to your program either as many connections these days are asymetric. Most cable will pull 1.5mbps or more down when not congested but is capped at 128k up.

Interesting

[0x0d0a]

Wow. Thanks for your time reviewing this.

because your horizon only goes out to a horizon of two

The horizon of two isn't a hard limit -- it might be extended to, say, three, or even be adaptive. However, I strongly suspect that trying to maintain any sort of local cache of an entire P2P network is a losing cause, unless distribution of the thing was a bit smarter (come to think of it, since there are public keys involved, this data can be signed...hmm, interesting, though it still might eat a lot of storage space).

the trust values fade very fast

How about a nonlinear trust degradation?

Here's an url with most of the links

I'll certainly read this (though because of two projects that I'm getting swamped with, it'll probably be a few days until I can properly give the stuff the attention it deserves). Still, thank you very much. This looks like exactly what I like poking around with.

Re:Anonymity: A problem with Gnutella

[evilviper]

With Internet routing, each node only knows what the next hop is, and doesn't have to know the entire path.

That's not exactly true. You see, on the internet, each router knows the final DESTINATION. Obviously, that does not accomplish anonymity. Each node/router needs to know where to send it next, and there's no way to do that anonymously. A router doesn't know where to send it next unless it knows where it is ultimately going.

How did you arrive at that result (1/100th of the original speed)? It is not at all obvious to me what the speed impact will be.

If the routing is organized such that most of the routing nodes are on the backbone, then the speed impact for end nodes on slower connections would not have to be affected so much.

For the speed, I made up a VERY CONSERVATIVE number. It's more likely that Gnutella would actually grind to a halt.

There is no such thing as a backbone on Gnutella. Perhaps you are refering to the fastest nodes (with the least delay)? The problem is that a huge ammount of bandwidth is being used up for the nodes participating. People are furious about the current bandwidth usage, and this would make the broadcasts look completely insignificant. There may be some fast nodes, but even all of them combined couldn't handle a fraction of this traffic, and that's assuming they would be willing to dedicate their bandwidth to this.

Sure, it could work for a handful of people, but it won't take long before it has saturated the network.

But this whole point is moot. There is no way traffic can be routed from source to destination without the nodes being privy to that information.

Re:Anonymity: A problem with Gnutella

[evilviper]

If you were willing to broadcast your file to all the gnutella nodes, then yes, you could establish some decent anonymity.

But that's part of the reason I called my post "solvable" problems with Gnutella.

Re:A *real* anti-leech/anti attacker system propos

[LostCluster]

I see a hole in that. HoneyPot1 absolutely trusts RIAA1, RIAA2, RIAA3, RIAA4, RIAA5, etc. HoneyPot1 earns trust points by giving out the real thing very well, and then transfers that those points to servers that have the dud content by "trusting" them. Whats more, once HoneyPot1 gets up to its desired state of trust, it could flip itself into dud content mode as well. The attackers could repeat this process as many times as they want. A trust system dies the first time a turncoat gets a high-trust status.

Re:200 entry trust list?

[0x0d0a]

You can partially automate this -- on a download, give a point of trust automatically, say. If the user says "this person screwed me", they can block the person from showing up in future results, and in doing so, set their trust to 0.

Re:A *real* anti-leech/anti attacker system propos

[0x0d0a]

The second issue is taken care of -- if HoneyPot isscrews me over, I zero his trust.

The transitive trust is a good point. A is legitimate, but A trusts B, and B is bogus, and A (deliberately or otherwise) does not update his local trust list upon B going bad. Hm.

Okay, here's a possible modification. Allow negative trust values. If I'm C, I trust A, user A trusts user B, and I consider user B a "bad guy" (negative trust), reduce C's trust of user A periodically.

Re:A *real* anti-leech/anti attacker system propos

[LostCluster]

Zeroing out HoneyPot after the fact still didn't prevent you from getting a bogus file. Expanded outward, if ten of your twelve favorite servers all turn bogus for you at the same time, you will have a rather meaningless set of trust data with trustworthy sites with low ratings and bogus sites rated high. What's more, less frequent users will not notice when a HoneyPot flips, so they'll continue to transfer a false trustworthy status to that flipped HoneyPot.