Slashdot Mirror

A friend of mine, Sam Joesph, was working on a project called Neurogrid http://www.neurogrid.net/php/index.php to develop a platform for distributed information sharing. I think he was looking into connecting some of this information using multiple dimensions like time and not just location (space).

PureFiction writes "Peer networks are gaining some attention these days given advances in much more decentralized search architectures and swarming distribution networks. Research has indicated that these decentralized networks are resistant to legal and technological attacks. The continued proliferation of broadband and wireless networking will ensure pervasive deployment of distributed peer networking infrastructure that will drive significant innovations in personal and community digital communications services."

AND FOR THOSE OF YOU IN A HURRY... A- R- T- I- C- L-
E-. -- S-
U- M- M-
A- R-
Y-

One thing Cringley hints at is a coming boom in popularity and capability of truly decentralized peer networks. It is the fully and highly decentralized network architectures that the Microsoft group credits most with resilience against any kind of legal, technological or political attacks.

We are starting to see some of these technologies emerge, awaiting integration into flexible infrastructure that allows fast, easy and efficient distribution of data, content or otherwise, between peers on a local and global scale.

The end result will be a combination of a number of technologies seamlessly interoperating like:

- distributed hash tables

- decentralized search

- swarming distribution

- wireless networks ... and many others.

It is nice to see the word get out: You cannot control the flow of digitial information in decentralized peer networks!

The problem is that gnutella's reliance on broadcast forwarding and indirect communication will always allow rogue peers to exploit bandwidth or queries in the network.

There are a number of alternative discovery mechanisms which do not suffere from these kinds of architectural problems.

For example, NeuroGrid and alpine both use social discovery and peer profiling to prevent bandwidth hogging or query spamming.

There are also hybrid network that use super peers like the Kazaa and Grokster clients.

There is only so much you can do to improve a flooding broadcast architecture. Gnutella will always have some kind of bandwidth and query problems no matter how optimized the clients become.

What I find most interesting are the kinds of projects that have sprung up in Gnutella's wake. Many of these started out as attempts to improve Gnutella, and have since moved on (the Gnutella Next Generation working group never really materialized into anything)

We had napster and one extreme, gnutella at the other, and in the middle a re a number of partially centralized systems with super peers like Fast Track, such as:
Open FT
JXTA Search
GNet
NEShare

and many others...

Then there are the alternative projects that use an entirely different mechanism. For example, social discovery as implemented in:
NeuroGrid
ALPINE

Or distributed keyword hash indexes like:
Chord
Circle
GISP
JXTA Distributed Indexing

And many others as well.

The coming year(s) will see a lot of maturity in these areas, and searching large peer networks will become ever more efficient over time. Gnutella showed us the possibilities of a fully decentralized model, and refinements of its underlying architecture can produce vastly better solutions.

2002 will be an interesting year for peer networking applications...

There is another method for ensuring cooperation and fair behavior in peer networks. And it works the same was as the method described.

It is called social discovery, and it works by having each and every peer create a view of the network that suits their interests and needs. In such an environment, the freeloading peers will not be viewed as valuable peers and will be dropped from your peer group(s); no longer used, and no longer using your resources.

On the flip side, there is a strong incentive to become a better, more reliable peer yourself, as the quality of peers you can associate with is directly related to how they perceive *your* quality to them.

If you want to be able to tap better, higher quality peers, then you should keep your node available longer, more often, and also share more resources (whatever they may be).

The project I am working on that implements this social discovery mechanism is called the ALPINE Network and there is also another social discovery based project called NeuroGrid.

I am biased towards this kind of approach, but I think it provides the best long term solution to resource discovery / searching in large peer networks.

My friend has/is developing a system and tools for creating a p2p search network. This seems like one way to interconnect searches and information as it becomes more interspersed thoughout the know universe. Have a look at Neurogrid