Slashdot Mirror
Hypernets -- Good (G)news for Gnutella
Red Roo writes: "This online article addresses the recent
criticism of Gnutella network scalability by pointing out that it is a Cayley tree. As a viable candidate for massively scalable P2P bandwidth, all trees are dead! But by going to higher dimensional virtual networks (aka "hypernets") e.g., hypercubes or hypertori, near linear scalability can be achieved for P2P populations on the order of several million peers each with only 20 open connections. This concept seems to have been entirely overlooked by critics and developers alike."
Correct me if I'm wrong here, but, the article seems to be saying that packet switching is more effiecient than old style circuit switching (?hierarchal switcing?). It says that bouncing stuff around nodes connected to a bunch of other nodes and letting the stuff find a path to its destination is more effiecient and scalable than any kind of tree structure where stuff goes down to the trunk and back up to a different branch to reach its destination.
Unfortunately with the way things are set up right now, I think our beloved internet is set up like a toroid instead of a cube. You have a backbone as the middle loop and then coming off of it are rings that are local that provide service to local ISPs. Then they sell to thier end users. In the end I picture a fuzzy torroid. And according to the article, those are more scalable than trees, but not as much as the cube. However the article says that they are harder to implement than the cubes, not so, as they seem to have evolved in the marketplace naturally, and setting up a cube like network in the real world is harder.
But they're talking about this applied to software, and virtual networks, not real world hardware. However, seeing as how the real world has moved from a tree based telecom system, to the torroid sceme of the current system, it would be interesting to see what happens when the torroidal system in the real world runs into scalability problems and goes for the cube.
If Mr. Edison had thought smarter he wouldn't sweat as much. --Nikola Tesla
I'm not one for saying "it'll never happen" because sometimes if you just sit there that's exactly what happens.
.ogg files no more.
But I'm not convinvced of this particular threat.
It would require worldwide cooperation and at every level of computing. It would be difficult to draft an international law AND define what a computer was. Does my digital watch need DRM?
To get this law in one country (probably the US) is going to have to implement it unilaterally. Chaos will ensue. I think it's just too much hassle for a government to embark on.
So mark my words, and then punch me with them when you can't play your
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
You don't pay enough to suck down all your bandwidth 24/7. If you were led to believe that was your right when you bought your broadband, then I'm sorry you were misled, but the only people who are sold lines that they are allowed to max out 24/7 are the people that pay for real Internet access, i.e. a T1 from a first tier provider. Otherwise, you have to share.
I've had enough abrasive sigs. Kittens are cute and fuzzy.
This is because it assumes the peers are already arranged in the network in the topology one wants.
If a central addressing authority exists, it is no problem to simply give new peers addresses and the addresses of their neighbors in such a way that the network acquires any topolgy the authority wants. The authority can even cope with peers leaving the network more or less arbitrarily.
However, a real question is -- how do you get peers to "self-assemble" into the desired topology in such a way that a small population of peers that choose not to play by the generally accepted rules cannot dramatically effect the outcome. In other words, how can peers be persuaded to place themselves on the points of a cubic hyperlattice solely by contacting a few already installed peers, some of which may not be telling the truth?
I really cannot understand why we keep harping on about gnutella. Gnutella was the experiment, and not the solution to all your file sharing needs. Ever.
Just get over it. Gnutella came, was good, and went. Now there are better things. Kazaa/FastTrack. Distributed Napster. Etc.
Stop trying to add new components to you VW bug, when there are Ferraris to be had.
Hopfrog.
A hypernet is an interesting idea, although I can think of a number of reasons why current p2p sharing networks would not implement them. Namely, because authoritarian networks like Napster were shut down by trade associations like the MPAA/RIAA, while more anarchic networks like Gnutella are more immune from such actions - we must consider not only the survival of the scaling network due to technical constraints like Dr. Gunther does, but also it's survival due to legal constraints orchestrated by large corporations.
I think your concern here is exactly backwards. Specifically, higher dimensional topology would decrease the need for central "UltraPeer"s (also known as lawyer bait) and thus make the network harder to shut down. If the trend towards depending more on some "peers" than others continues to the natural limit, you wind up right back at Napster (one UberUltraPeer to rule them all, and in the darkness...get eaten by a grue if it's lucky, sued by the RIAA if it's not).
On the other hand, if the topology is made more scalable, the targets won't be as tempting at any given network size, and the whole thing would be harder to take down by force. If all nodes are equal, cutting one will likely create enough publicity to attract seven more to take its place.
-- MarkusQ
Exactly.
Moreover, what are we to make of this?
The dominant constraint for hardware implementations of high-dimensional networks is the cost of the physical wires on the interconnect backplane. Since the hypernets discussed here would be implemented in software, no such constraints would prevent reaching the desired level of scalability.
I really don't see how you can sweep the actual physical infrastrucure under the rug like this. Eventually, virtual hypercubes turn into real packets on a real network. A network that is subject to the very same topological limitations this article discusses. Any wonder that the Tandem Himalaya architecture he mentions was implemented in hardware, rather than as a "virtual" topology implemented on top of a traditional TCP/IP network?
When it comes to complicated mathematics like this, though, intuition has often led me astray...
--Lawrence Lessig for Congress!
I can't believe Slashdot just posts this psuedomathematical nonsense without doing even elementary fact-checking
At first, I thought you were refering to your own post, and was tempted to give you a couteracting +1 Insightful.
-- MarkusQ
You can do all the computer simulations you want of the Gnutella network, but I know that it just works. I've never had any problems finding music that I want, its just that it takes a little more time. Its kind of like stamp collecting. Even after Napster fell and there was an upsurge in nodes, I was still able to use it well. Who cares if you are on a subnet of Gnutella if you still find the music you want?
Until Gnutella starts directing searches intelligently - towards nodes which are more likely to have the data being sought, as Freenet does, it will always be an inefficient way to search for data.
It doesn't. The problem with the tree topology used by Gnutella is that it utilizes the available bandwidth in a fashion that becomes highly inefficient as the number of nodes becomes large. At around 10^6 nodes, it uses 15-20% of the total available capacity, whereas a hypercube topology at 10^6 nodes uses essentially 100% of the available bandwidth.
So obviously, if your application runs up against the physical capacity of your underlying communications systems, you can't send more data. But via correct choices of virtual network topology, you can ensure that the physical capacity is being used productively.
Quantum mechanics: the dreams that stuff is made of.
In layman's terms. If you are a node in one of his example networks, and you're sitting on a DSL connection, does the available bandwidth you contribute to the net change whether you have 20 outbound TCP connections or 2? No. It is constant. The author incorrectly computes the "bandwidth" of these different network topologies like you are stringing a separate DSL to each person you open a TCP connection to.
Available network bandwidth in a peer network like Gnutella is related only to the physical interconnect of the nodes. (i.e. whether they are on an SBC DSL line or sitting a North American OC-3)
The only useful analysis is that which determines the amount of data-transfer required between each node (and all nodes) for common operations when using different topologies. When performing this study, you are looking for the topology which will transfer the smallest amount of data over the smallest number of nodes when performing searches. For a great analysis, see the Gnutella Performance Paper by Ritter, referenced by the above paper.
Careful analysis will tell you the same thing that common sense does -- that the best architecture involves centralized dedicated servers (supernodes), located on machines with the largest physical bandwidth available. (i.e. eactly what Napster did. )
In order to create an efficient peer network which scales, Gnutella 'merely' needs to 1) order the network by physical topology, 2) identify the nodes with the best combination of physical bandwidth, longevity, and CPU/disk resources, and 3) fully utilize those machines as supernodes.
Good luck. :)
To simplify: If the Gnutella network used a different node addressing (and associated routing) scheme, total aggregate bandwidth of the entire network would be increased, even as the network scaled upwards of a million nodes.
The underlying physical wiring is exactly the same for this method as for the existing Gnutella topology. That's the whole point. By simply changing the way Gnutella nodes connect to one another you can increase the aggregate network efficiency. It isn't manufacturing bandwidth out of thin air, it is making more efficient use of what is already there. Read over sections 4 and 5 again perhaps.
Finally, I have to admit to being somewhat boggled by your last paragraph. Are you suggesting that in order to become part of a network you must be part of the network already? Various routing protocols have solutions for how to dynamically insert a new node into a topology. STP, OSPF and EIGRP have differing methods, not particularly analagous to Gnutella, but useful nonetheless. A 'closed cycle' as you refer to it could also be thought of as a routing loop. It is possible to have multiple paths and not have a routing loop, something routing protocols are generally designed to prevent. What you're talking about here is that you have to have connectivity into a network in order to become a part of the network, which is somewhat obvious. I mean, it does generally work better if you plug it in.
Just because you're paranoid doesn't mean they're NOT after you.