Slashdot Mirror

← Back to Stories (view on slashdot.org)

P2P Web searches

Posted by ryuzaki0 on from the one-level-up-please dept.

prostoalex writes "Researchers at UCLA are looking for easier ways to implement Web searches by using peer-to-peer techniques to decrease the workload. 'Queries need to be passed along only a few links rather than flooded throughout the network, which keeps search-related traffic low,' reports Technology Research News."

80 comments

  1. Yes, but... by Anonymous Coward · · Score: -1, Offtopic

    Will it help me find porn faster?

  2. Happy 9/11 From The GNAA by Anonymous Coward · · Score: -1, Offtopic
    Happy 9/11 From The GNAA
    Gnaa, Nigeria

    We at the GNAA (Gay Nigger Association of America) would like to wish you seasons greetings on this most glorious day.

    As we reach the third anniversary of the momentous destruction of two large ugly buildings by the highly trained agents of mossad (Israels notorious anti-gentile secret service), and the war against sandniggers everywhere that ensued, we have entered into what is widely recognised as a golden age of trolling. This could not have been possible without the assistance of YOU!

    Without the glorious genocidal activites of hooknosed Jews, the gulliblity of pinko Slashbots soaking up whatever Michael Moore shits out, the whining of eurotrash clinging to the tattered remnants of Socialism, and of course, without the poorly washed Muslims who, as we speak, continue their holy and righteous Jihad against the evils of the Israeli Bus Service. Without these people, without you people, our jobs would be so much more taxing, and far less rewarding.

    As your family gathers around the Osama Tree to exchange your gifts of toy Boeings and charred Firemen, we at the GNAA hope that you can all get together in celebration. To pass down your memories of the shock, awe, and arousal you felt. To recall your joy as your anti-Bush FUD was modded to +5, insightful. To remember your hilarious puns at the size of John Kerry's chin, and how they earned you +o in #politics. To giggle girlishly at how your crippling zergrush wowed all who viewed your rancidly coloured MIDI infested xanga blog.

    But most importantly of all. The GNAA would also like to issue our fondest desire for world peace. And hold firm in our resolve that the bloodshed can only be resolved through the pacifying and unifying acts of male-on-male sodomy and fellatio, and through a peaceful male-only world. Can the leaders of the world put aside their differences once and for all and love their fellow man? Can we swap our rusty kalashnikovs for a warm meaty dong? Our explosive belts for exploding rectums? We can only hope.

    Happy Holidays!
    From The Gay Nigger Association of America

    About WTC

    The World Trade Center (WTC) was built in the 1960's by well-known drug-addicted architect Guy Niguere. On massive levels of barbiturates at the time, Niguere made many mistakes in the design of the building, but structurally and stylistically. This led to the towers easy destruction by the Israeli.

    About GNAA:
    GNAA (GAY NIGGER ASSOCIATION OF AMERICA) is the first organization which gathers GAY NIGGERS from all over America and abroad for one common goal - being GAY NIGGERS.

    Are you GAY ?
    Are you a NIGGER ?
    Are you a GAY NIGGER ?

    If you answered "Yes" to all of the above questions, then GNAA (GAY NIGGER ASSOCIATION OF AMERICA) might be exactly what you've been looking for!
    Join GNAA (GAY NIGGER ASSOCIATION OF AMERICA) today, and enjoy all the benefits of being a full-time GNAA member.
    GNAA (GAY NIGGER ASSOCIATION OF AMERICA) is the fastest-growing GAY NIGGER community with THOUSANDS of members all over United States of America and the World! You, too, can be a part of GNAA if you join today!

    Why not? It's quick and easy - only 3 simple steps!

    • First, you have to obtain a copy of
  3. fp by Anonymous Coward · · Score: -1, Offtopic

    first post, yeah

    1. Re:fp by Anonymous Coward · · Score: -1, Offtopic

      third post, actually. close, but no cigar.

    2. Re:fp by Anonymous Coward · · Score: -1, Offtopic

      'fp' as in fourth post? Correct.

    3. Re:fp by Anonymous Coward · · Score: -1, Offtopic

      Shit, I was trying for fifth post :(

  4. fp by Anonymous Coward · · Score: -1, Offtopic

    fp

  5. Too many people trying to use p2p by benna · · Score: 1, Interesting

    I'm sick of all this hype about p2p. Its a good technology but its not like we have to use it for everything. The old ways of doing things still work.

    --
    "It is not how things are in the world that is mystical, but that it exists." -Ludwig Wittgenstein
    1. Re:Too many people trying to use p2p by photonagon · · Score: 1

      True, at some point networks and machines will be so congested with various p2p protocals, that everyone will jump back to centralized servers.

    2. Re:Too many people trying to use p2p by jrockway · · Score: -1, Troll

      I'm sick of all the hype about searching. This is very typical in a University setting; hey the Google kids got rich on searching. Let's make a search thing!

      And P2P is irritating. Here is my comment: it's a dumb idea, guys. I think I'll infect the P2P database with my spamvertising. Great idea, indeed.

      --
      My other car is first.
    3. Re:Too many people trying to use p2p by Bryan+Gividen · · Score: 2

      The old ways of doing things still work.

      Which is why I still prefer walking everywhere, using chalk and slate for taking notes, and refuse to use a zipper...

      In other words, wtf?!?!

    4. Re:Too many people trying to use p2p by Anonymous Coward · · Score: 0

      A car is fine for some things, many things, but it is sometimes faster to walk. If I want to go across the street, do I go to my car, start the engine drive across the street and park, or do I just walk?

    5. Re:Too many people trying to use p2p by Anonymous Coward · · Score: 2, Funny

      Which is why I still prefer walking everywhere, using chalk and slate for taking notes, and refuse to use a zipper...

      And why I like using my RPN calculator to change the TV station...

    6. Re:Too many people trying to use p2p by Anonymous Coward · · Score: 2, Interesting

      No, they won't. You need tons of server hardware to cope with the bandwidth of anything even remotely popular. Thus free services tend to be spoiled with ads and whatnot.

      The magic of p2p is that you can build the same out of 'thin air'. There are no expensive server rooms and gigabit lines but just a bunch of nodes that are slightly more complicated than simple clients. You use it, you provide it. Fair game and you get exactly the kind of service you want without strings attached. At least theoretically - the reality still seems to be something different.

    7. Re:Too many people trying to use p2p by kfg · · Score: 1

      . . .and refuse to use a zipper...

      Zippers are obsolescent, you insensitive Luddite.

      KFG

    8. Re:Too many people trying to use p2p by Bert690 · · Score: 3, Insightful
      True, at some point networks and machines will be so congested with various p2p protocals, that everyone will jump back to centralized servers.

      If you'd take some time to actually read the article, you'd see that the story is about research that addresses congestion problems with existing p2p methods.

      Besides, much if not most traffic from p2p networks is from file downloads, not query routing. Moving files to a centralized server isn't going to reduce that traffic at all. In fact, the bottlenecks that result can make congestion even worse.

      Moving files to central servers only seems to help congestion because central servers with anything interesting to download tend to be shut down quickly.

    9. Re:Too many people trying to use p2p by cbiltcliffe · · Score: 5, Insightful
      I'm sick of all this hype about p2p. Its a good technology but its not like we have to use it for everything. The old ways of doing things still work.

      You're right, but consider this:

      The entertainment industry is trying very hard to convince the US government that all P2P can be used for is copyright infringement, so it should be banned completely.
      Any non-infringing use obviously proves them wrong, no matter how out there it is.

      Right now, I think we need as many off-the-wall uses as possible for P2P, even if it's not the most efficient way to accomplish the task.
      Calling mass attention to these uses wouldn't hurt, either.
      --
      "City hall" in German is "Rathaus" Kinda explains a few things......
    10. Re:Too many people trying to use p2p by farble1670 · · Score: 2, Interesting

      ?

      the point is that new technologies are adopted when they improve on an existing method. we already have super-fast, super robust, complete search technologies that are not p2p ... so what problem are they trying to solve? an academic exercise? well, that's okay ... but let's call it what it is.

      google is already so fast as i would not notice it is if were any faster. the best a p2p search technology could achieve would be equivalent speed with the addition of the consumption of my bandwidth.

    11. Re:Too many people trying to use p2p by MoFoQ · · Score: 1

      don't forget about using ur abacus for ur calc classes.

    12. Re:Too many people trying to use p2p by timts · · Score: 1

      u r right, P2P for search will require a lot effort and result isnt reliable

      I think proper cache is enough for web search, as long as you have a farm with tons of memory holding up everything, the search result can be fairly fast.

  6. BOOM! by Anonymous Coward · · Score: 0

    haha, it's not slashdotted yet, but who the hell cares?!!!?!

    Simple search lightens Net load
    September 8/15, 2004
    By Kimberly Patch, Technology Research News

    Researchers working on finding better ways to search the Internet are increasingly turning to methods that require individual nodes, or servers, to know a little bit about nearby servers, but don't require servers to look much beyond their own neighborhoods.

    This type of co-operation, which is also found in many natural networks such as insect communities, uses local rules in such a way that the system as a whole has predictable global properties.

    Researchers from the University of California at Los Angeles have devised a fast search algorithm that uses local rules to find nodes and content in randomly-formed, scale-free networks such as the Internet. Scale-free networks have a few nodes that have many connections to other nodes, and many nodes that have far fewer connections.

    "Without global knowledge of the network, and only doing local operations, we can make the cost of searching an entire network grow less than linearly [with] the size of the network, and still have the query be very fast," said Vwani Roychowdhury, a professor of electrical engineering at the University of California at Los Angeles.

    The search algorithm could be used to increase Internet efficiency by making it easier to find routes between hosts, said Roychowdhury. It could also be used to reduce traffic in peer-to-peer networks running on the Internet that allow people to exchange files, like Kazaa and Gnutella.

    Queries need to be passed along only a few links rather than flooded throughout the network, which keeps search-related traffic low, said Roychowdhury.

    "Many networks are known to be scale free... our search algorithm could be applied to all of them," said Roychowdhury. The researchers' simulations have showed that the algorithm could reduce Gnutella traffic by one or two orders of magnitude, he said.

    In 2001, researchers at Stanford University and Hewlett-Packard Laboratories developed a simple, light-weight random search algorithm for peer networks. That algorithm forwards queries one node at a time whereas the UCLA researchers' algorithm operates in parallel, said Sarshar.

    The researchers' algorithm is based on the bond percolation threshold, or the smallest probability that a message is guaranteed to reach a core sub-network of highly-connected nodes, said Roychowdhury.

    As connections randomly percolate through the network at a low rate, only small, isolated islands form. Once the bond percolation threshold is passed, the core of the network becomes connected. The threshold is an abrupt phase transition like the quick transformation of would in that takes place when water boils or freezes.

    The algorithm involves three basic steps: content caching, query implantation, and bond percolation, said Roychowdhury.

    Content caching happens when a node joins a peer-to-peer network and performs a one-time short random survey, or walk of nearby nodes and adds its content directory to each of these neighboring nodes.

    The query implementation step is similar, but happens at the beginning of every query. When a node has a query, it performs a short random walk and passes the query along to each node it encounters.

    These random walks are long enough that any given node will almost surely encounter at least one highly-connected node, said Roychowdhury. "So after these two steps one of the high-degree nodes has a copy of a node's directory, and a query is implanted at one of the high-degree nodes."

    Once this has been set up, the bond percolation step makes sure that the directory and query connect.

    In this last step, all of the initially queried nodes percolate the query throughout the network so that the query is guaranteed to reach a core sub-network of highly-connected nodes. "Since a copy of the query is in one of the nodes in the core network, a

    1. Re:BOOM! by OneDeeTenTee · · Score: 1

      Darn you!

      You tricked me into RTFAing!!!

      --
      Stop the world; I need to get off.
  7. GNAA EARLY POST - irc.easynews.com #WINDOWS by Anonymous Coward · · Score: -1, Offtopic

    This "early post" is brought to you by the ECFA (Euthanasia for Canus Familirous Association). We are an organization dedicated to the eradication of dogs. ECFA (Euthanasia for Canus Familirous Association) is committed to protect our oxygen, to clean our streets, and to curb noise pollution - through the simple eradication of canine pests.

    We have recently "connected" with the GNAA to form one ECFA. Stay connected. Please note that since we are moving to a larger demographic (the untold scores of people who deal with dog messes, noises, and annoyances daily), most of the current GNAA content is offline. In fact, we're pulling all of it except the "early post", which is now a ECFA-style "early post". The traditional GNAA "early post" will continue to be posted on all SCO stories, as insisted by upper GNAA management and its core team of fans. The illicit images and language will not be a part of the new combined organiztion. We do not condone any sexual lifestyle or race.

    Have you ever stepped in DOG DOO-DOO

    Are you MAD?

    Do you KILL DOGS?

    Are you a MAD DOG KILLER?

    If you answered "YES" to any of the above questions the ECFA (Euthanasia for Canus Familirous Association) is for you! You no longer need change your skin color or sexual lifestyle in order to become a member of an "EVIL TROLLING ORGANIZATION." Instead, you can work toward the noble of goal of INCREASING OUR SUPPLY OF O2! OVERPOPULATION of DOGS is RAPANT in this country. Did you know that DOGS turn BENEFICIAL O2 into CO2 simply to gain their energy to bark, drool, and howl? They ACTUALLY BURN OUR OXYGEN SUPPLY!!! One dog easily waste the Oxygen output of ten mature trees! This country has MANY UNWANTED, ABANDONED DOGS that WE ARE PAYING MONEY TO KEEP ALIVE. We are FEEDING them our food supply while making the homeless STARVE! Are you TIRED of having your TAXES increased? Humane Societies cost our country over $100 million annually. By using a Dog Killing Gadget, a dog can be turned into beneficial food, helping us all. We let children go hungry yet feed our **UNWANTED** dogs like royalty.

    One dog can output over 10 lbs of droppings daily. One dog can aggrivate the allergies of untold numbers of people with its fast growing hair and all too common dandruff. Do you own a dog? Are you tired of its mess? Don't feel like planting ten trees and waiting 10 years for them to reach maturity? Then get it euthanized. Euthanasia is a painless way for a dog to... terminate. However, it can be too expensive to buy these drugs for the LARGE NUMBER of DOGS in the HUMANE SOCIETIES. It is thus proposed that these dogs be turned into food for the homeless. One dog can feed up to five homeless children for one day.

    Many have wondered the best way to exterminate dogs. Euthanasia is by far the most clean method, but it taints the meat and is cost prohibitive. Thus, the most economical method is our K9Zap product featured on TechTV ($29.95), which deals a fatal shock to a dog up to 60lbs. Alternatively, the slightly messier bakers chocolate approach costs only about $0.30 per pound of dog. For more information, reply to this message or contact Gadgets for the Elimination of Dogs (GED). A rifle also works wonders, but may be against local codes, and is generally best to avoid in dog elimination.

    WANT TO SUPPORT THE ECFA? Simply participate in our propaganda campaign to exterminate dogs. You can become a member of our slashdot trolling team, our usenet trolling team, or you can be a member of our local campaigning - by simply handing out brocures or posting signs outside humaine socities. If you have MOD POINTS, alternatively you can moderate this post UP to support our cause.

    ==This post brought to you by the Proud Dog Killers in #windows on EFNET.

  8. If it's P2P... by thebudgie · · Score: 5, Insightful

    The searching load on servers might be reduced i suppose. But from my experiences with P2P searches are long and slow. How would this help exactly?

    1. Re:If it's P2P... by Anonymous Coward · · Score: 0

      Indeed. And the statement "Queries need to be passed along only a few links" is absurd, unless almost everyone has a major chunk of the index already residing on their drive.

    2. Re:If it's P2P... by A1kmm · · Score: 3, Interesting

      From the article...
      > In this last step, all of the initially queried
      > nodes percolate the query throughout the network
      > so that the query is guaranteed to reach a core
      > sub-network of highly-connected nodes. "Since a
      > copy of the query is in one of the nodes in the
      > core network, and since the content list of a
      > node is cached at one of these high-degree
      > nodes, one is guaranteed to find the content as
      > long as at least one node in the network has
      > it," said Roychowdhury.
      So in other words, the "major sharers", i.e. nodes which are "high degree", i.e. have a lot of connections, form the "core network", and collectively host the entire index. However, this is starting to lose the advantages of being a peer-to-peer network. Obviously, you can't have it both ways.

      --
      X-Has-Sig: yes
    3. Re:If it's P2P... by Anonymous Coward · · Score: 0

      Yeah, the more bandwidth/space/proc time the big sharers use for that, the less they'll actually share. And even true p2p like bitorrent works much better when you have a few super-bandwidth people throwing data in all directions.

  9. I foresee.. by Gentlewhisper · · Score: 5, Interesting

    Maybe in future Google will implement a small server in our "Gmail notifier" application, and each time we search for something on google, it will cache some of the results, and should anyone close by ask for it, just forward the old results to them.

    Save the server load on the main google server!

    **Plus maybe some smart guy will figure out how to trade mp3s over the GoOgLe-P2p network! :D

    --
    Online backup with Mozy, sounds like Ozzie, but more!
    1. Re:I foresee.. by LostCluster · · Score: 4, Insightful

      Save the server load on the main google server!

      Error 404: No such main server found.

      Google is such a distributed computing network that when a single computer in a cluster fails, they've discovered that it'd cost them more to go to the broken node and repair it than the vaule of the computing resources they've lost. Google just lets such failed computers sit useless, and waits until there are enough downed computers to justify sending in the repair people.

      Besides, P2P services to respond to your Google query would mean that your query would end up in the hands of a dreaded "untrusted third party", and I don't think anybody here wants all of their searches available to their next door neighbor.

    2. Re:I foresee.. by Gentlewhisper · · Score: 1

      "Google just lets such failed computers sit useless, and waits until there are enough downed computers to justify sending in the repair people."

      Wow, rather than letting it sit there useless and depreciating, I rather they find some cheap and efficient means to just sell that machine (cheaply!) outright, and then they order a new replacement to go back into that empty pigeonhole.

      --
      Online backup with Mozy, sounds like Ozzie, but more!
    3. Re:I foresee.. by Phleg · · Score: 2, Informative

      Google is such a distributed computing network that when a single computer in a cluster fails, they've discovered that it'd cost them more to go to the broken node and repair it than the vaule of the computing resources they've lost.

      This is nothing more than just a myth. They continually have job postings looking for Data Center Technicians, whose entire job is to crawl through their massive cluster and repair downed nodes. I should know, I interviewed for the position just a month or two ago.

      --
      No comment.
    4. Re:I foresee.. by LostCluster · · Score: 1

      They fix the downed nodes eventually... but one down node alone is not worth sending anybody after. They wait until there's a collection of downed nodes to send the tech after them...

    5. Re:I foresee.. by mod_parent_down · · Score: 1
      I don't think anybody here wants all of their searches available to their next door neighbor

      One of the coolest things I've seen was a little ticker that WebCrawler used to run that was just a constant stream of random search queries other people had made. You could click on any of them as they scrolled by and it would bring up the results.

      Totally anonymous, very addictive. Sad to see it gone.

    6. Re:I foresee.. by jackbird · · Score: 2, Informative
      Sad to see it gone.

      me too.

    7. Re:I foresee.. by mrogers · · Score: 3, Interesting

      Actually, I'd rather have my next door neighbour know what I was searching for (and vice versa) than have any single person know what *everyone* was searching for. Power corrupts.

    8. Re:I foresee.. by lewko · · Score: 1
      I don't think anybody here wants all of their searches available to their next door neighbor.

      I'm more worried about my next door neigbbour being able to serve up the search results!

      --
      Do you or your partner snore? - Visit www.snoring.com.au
  10. Last time I checked, by rasafras · · Score: 4, Funny

    Google still works.

    Results 1 - 10 of about 6,290,000 for p2p [definition]. (0.19 seconds)

    --
    webpage
    1. Re:Last time I checked, by LostCluster · · Score: 4, Informative

      Not to mention, Google is often better at searching a given website than the search untility a site tries to provide on its own. TechTV host Leo Laporte used to frequently searching Google with the "Site:techtv.com" marker included to find deeply-hidden articles on the site, because it'd be easier to search that way than using TechTV's own search boxes.

      Google's even encuraging this behavior by linking their free websearch feature with their AdSense service, and giving publishers a share of the AdWords revenue when a search that came from their site results in an ad click.

    2. Re:Last time I checked, by BinLadenMyHero · · Score: 1

      Google leave us with no real need for something better on searching the web.
      But we need a Free search engine, so we don't depend on any big corporation to run our lives, and P2P is the way to overcome the huge cost of running a single system to serve the whole Internet.

    3. Re:Last time I checked, by Deep+Fried+Geekboy · · Score: 1, Informative

      Correctamundo! In fact the google search is a very efficient way of searching sites. Wikipedia uses this to great advantage if your keyword search fails. A big advantage is that frequent googlers have a good sense of how to word the query for maximum valid results.

      I am just about to put a 50,000 message mailing list archive online and the search facility will be Google, which is far better than any of the other solutions I've investigated.

      --

      I'm not wrong. You haven't thought about it hard enough.

    4. Re:Last time I checked, by Technonotice_Dom · · Score: 1

      A big advantage is that frequent googlers have a good sense of how to word the query for maximum valid results.

      I agree - many times I see a search box on a website, with no "advanced search" link and you never know how it'll work. Usually you find that (unlike Google) it'll match any word and not all of them, so you lots of really irrelevant material. You don't know what boolean operators it supports etc etc.

      Another quite simple advantage of using a Google search on your website is that it's a consistent interface - which is a Good Thing.

  11. Newb! Coral Cache It! by Anonymous Coward · · Score: 0

    This is how we prevent Slashdotting in 2004:

    http://www.trnmag.com.nyud.net:8090/Stories/2004/0 90804/Simple_search_lightens_Net_load_090804.html

  12. Question about timely content being served by peculiarmethod · · Score: -1, Troll

    " The algorithm involves three basic steps: content caching, query implantation, and bond percolation, said Roychowdhury. "

    Did anyone else get a bit turned on when they heard this? Sounds like someone on the team also has a few degrees in Human Reproduction / Mating and Reproduction.
    - content caching (otherwise know as the gift giving stage
    - query implantation (umm.. some call this a 'homerun')
    - bond percolation (marriage, otherwise known as pregnancy)

    --
    ** "It's not my job to stand between the people talking to me, and the ones listening to me." -- Pego the Jerk
    1. Re:Question about timely content being served by Anonymous Coward · · Score: 0

      The query lifestyle is a sin!

  13. does that mean.. by datadriven · · Score: 0

    that all the peer would know what i was searching for? No thanks.

    I have 6 gmail invites to give away.

    --
    GETPKG - Package Management for Slackware
  14. UCLA discovers ultrapeers! by Magila · · Score: 5, Interesting

    From a quick read of the article it sounds like what they've done is implemented a slightly more sophistcated/less deterministic version of the ultrapeer/hub system already in use by Gnutella/G2 Basicaly quereies are routed such that they are guarenteed to reach a "highly-connected node" which is the equivalent of an ultrapeer/hub node. The main difference is the folks at UCLA have come up with a novel method of picking ultrapeers, but the end result isn't much different.

    1. Re:UCLA discovers ultrapeers! by shadowmatter · · Score: 4, Interesting

      Not quite... Note: I'm about to karma whore here.

      About a year ago, right before starting my senior year at UCLA, I was offered an opportunity to work on this P2P project. At the time it was called "Gnucla," and was being developed by the UCLA EE department's Complex Networks Group. I turned it down, because I had already committed to working on a p2p system in the CS department. But since in all honesty their research was more novel than ours (and my friend was in their group), I subscribed to their mailing list and kept informed on what they were doing.

      What they've done isn't find a novel way of picking ultrapeers. Let's review what motivated ultrapeers -- in the beginning, there was Gnutella. Gnutella was a power-law based network. What this meant is that there was no real "topology" to it, unlike peer to peer networks that were emerging and based on Distributed Hash Tables (such as Chord, Pastry, Kademlia [on which Coral is based]). It had nice properties: a low diameter, and very resilient to attacks common on p2p networks. (Loads of peers dropping simultaneously could not partition the network, unlike, say, in Pastry -- unless they are high degree nodes.) But the big problem was that to search the network, you had to flood it. And that generated so much traffic that the network eventually tore itself apart under its own load.

      So someone thought that maybe if only a few, select, high-capacity nodes participated in the power-law network, it wouldn't tear itself apart because they could handle the load. These would become the ultrapeers. The nodes that couldn't handle the demands of a flooding, power-law network would connect to ultrapeers and let the ultrapeers take note of their shared files, and handle search requests for them. Thus, when a peer searches, no peer connected to an ultrapeer ever sees the search unless they have the file being searched for, because the searching happens at a level above them. Between low-capacity nodes and ultrapeers, it's much like a client-server model. Between ultrapeers, it's still a power-law network.

      But the ultrapeer network has problems in itself, so this group sought to find a way to search a power-law based network, such as Gnutella, without flooding. They exploited the fact that, in a power-law network, select nodes have very high degree connectivity. If you take a random walk on a power-law based network (meaning, starting from your own PC, randomly jump to a node connected to you, randomly jump to a node connected to that node, etc...) you'll end up at or passing through a node with very high connectivity. Thus, they were a natrual spot rendezvous point for clients wishing to share files, and clients wishing to download files. Perhaps, in this sense, they are an "ultrapeer," but we haven't separated the network into two different architectures like before. The network is still entirely power-law based, and retains all its wonderful properties.

      But that's not the entire story, just the gist of it. There are other neat tricks to it... Trust me, this is really good stuff we're talking about here. They recently won Best Paper Award at the 2004 IEEE International Conference on Peer-to-Peer Computing. (See paper here.)

      "Brunet," as they call it, is designed to be a framework for any peer-to-peer application that could exploit the percolation search outlined above. Google-like searching is just one possible approach (and perhaps a little unrealistic...). Right now I can tell you that they have a chat program in the works, and it is working well. The framework should be released when it's ready.

      Please don't flood me with questions -- remember, I'm not actually in their research group :)

      - sm

  15. The Ask Slashdot section by Man+of+E · · Score: 5, Funny
    P2P searching? The Ask Slashdot section does P2P searching already (in a less fancy-schmancy way), moreso than some would like :-)

    Q: What is $search_term and how does it work?
    A: A simple google search shows that $search_term is $blahblah and you use it like $this (repeated a hundred times)

    Add another hundred replies about how the poster should search before submitting, and how AskSlashdot is degenerating into AskPeopleToGoogleForYou, and there you have it. P2P searching in all its glory.

    --
    Ceci n'est pas une sig
  16. islands of users by bodrell · · Score: 2, Informative

    That wouldn't solve the problem of local areas of users that are disconnected from everyone but themselves. I know this is an issue with other p2p apps. You can only connect to someone who's in your area, and sometimes that just isn't good enough. I know China is in many respects isolated from the rest of the internet.

    --
    Si la vida me da palo, yo la voy a soportar Si la vida me da palo, yo la voy a espabilar
    1. Re:islands of users by TheOtherShoe · · Score: 1

      If they are disconnected from everyone else, how would any kind of search reach them/everybody else?

    2. Re:islands of users by bodrell · · Score: 1
      If they are disconnected from everyone else, how would any kind of search reach them/everybody else?

      Exactly. They wouldn't.

      Do note that I was talking about areas of users not being able to connect to anyone else. P2P is not the same as explicit IP addresses like on the web. For example, it would be a lot harder for me to get to slashdot by only clicking on links than by typing the address into my browser's bar.

      --
      Si la vida me da palo, yo la voy a soportar Si la vida me da palo, yo la voy a espabilar
  17. In other news... by SeaDour · · Score: 1

    A group of researchers from UCLA have been hired by Google Corporation with enticing payrolls and stock options.

  18. Mods on crack? by Anonymous Coward · · Score: -1, Troll

    How the first non-first-post comment redundent?

  19. This was already tried... by shodson · · Score: 4, Interesting

    Infrasearch was working on this, until Sun paid $8M for the company, them had them work on something else, then Gene Kan committed suicide. Be careful what you work on.

    1. Re:This was already tried... by tunabomber · · Score: 1

      I'm glad somebody mentioned Infrasearch, they were pioneering the field of peer-to-peer search way back in 2000. Gene Kan and co. were some of the first to realize that peer-to-peer networks could be used for something other than evading the authorities.
      The brillient aspect of Infrasearch (later JXTASearch) is that unlike most peer-to-peer search implementations, it doesn't just act like a metasearch engine, broadcasting or propagating a query to a bunch of specialized indexing nodes and then aggregating the results. Instead, "provider" nodes (the nodes sharing the content) analyse the metadata of all the content that they are sharing and produce a list of patterns that define which queries the provider node would be likely to have matches for. Then, this data is propagated across a network of search hubs.
      If you want to search for something, you'd send a query to the nearest search hub and it would check if your query is matched by any of the patterns in its index. If it is, the query will be routed to the providers which registered the matching patterns. The providers will then query their own content indexes and forward any matches back to you.
      There are several advantages to this approach:
      1. It relies far less on broadcasting and propagation, which consume large amounts of bandwidth.
      2. Provider nodes won't suffer from high loads because they will only recieve queries that are pertinent.
      3. Since the search hubs index generalized query patterns and not the actual content metadata, the provider nodes can provide highly dynamic content without having to worry about updating the indexes in the search hubs all the time.

      The main downside is that the queries you send would have to be more qualified, i.e. if you are searching for the song by a certain band, you wouldn't be able to find anything unless you included the band name in your query.

      --

      pi = 3.141592653589793helpimtrappedinauniversefactory71 ...
  20. Huh? by Ars-Fartsica · · Score: 2, Insightful
    Aren't searches sent to, and derived by, single search engine domains?

    Google, Yahoo etc of course crawl the web at large, but even if you want to throw a peer network at crawling, aren't you mitigating freshness?

    What I can see is a DNS-like system for propogating metadata in to the interior of the network, and maybe a caching mechanism as a result...not sure if this is what they mean.

  21. There already is distributed crawling by Anonymous Coward · · Score: 3, Interesting

    It's called grub.

  22. An alternative idea for complete indexing.... by i_want_you_to_throw_ · · Score: 4, Interesting

    Feel free to shoot full of holes as needed....

    Every website has DNS servers so what if that same company that ran the DNS servers indexed the pages of the sites that it hosted? Daily?

    Wouldn't that then provide a complete index of the web?

    Start a search and somehow get the results back through that distributed method. Haven't figure that out yet...... but if you can...
    PROFIT!!!!!

    1. Re:An alternative idea for complete indexing.... by timealterer · · Score: 2, Interesting

      So, under this theory... everybody indexes their own content? Implying, everybody would provide legitimate "indexes" and not simply provide whatever is most likely to bring in search engine visitors? "Look, here's my index! My site has a MILLION pages of free porn warez!!" Indexing needs to be done by a third party, that's just the way it is.

      --
      - Allen Pike
      Altering time, one time at a time.
    2. Re:An alternative idea for complete indexing.... by otisg · · Score: 2, Informative

      This, or something akeen to this has already been tried years ago with Harvest and its SOIF records (I think that was the name). The idea was to index locally, while being a part of a larger index network. Obviously, it never worked.

      There is a mailing list for people involved with writing and running web crawlers (aka spider or robots), and several years ago there was a lot of talk about making crawling and indexing more efficient by enchancing the 'robot exclusion protocol' (i.e. robots.txt) by creating a system in which sites would advertize what pages have changed, so that web crawlers could do less crawling and poking around (read wasting time, bandwidth, CPU power, electricity, etc.) and just pick up the pages that needed to be (re)indexed. That was never realized either.

      Over time corporations that run search engines have become more and more closed and protective of their propriatory technology and practices, which gives them the edge in our highly competitive world. Who can blame them. As the result, it is more and more difficult to enhance things like 'robots exclusion protocol', as it requires open talk, agreements, etc.

      Ah.... this makes me said, I'll stop reminiscing.

      --
      Simpy
  23. On a related note, the official song of the GNAA by Anonymous Coward · · Score: -1, Troll

    I know what boys like I know what guys want
    I know what boys like I've got what boys like
    I know what boys like I know what guys want
    I see them looking I make them want me I like to tease them
    They want to touch me I never let them I know what boys like
    I know what guys want I know what boys like, boys like, boys like me
    But you, you're special I might let you You're so much different
    I might let you Mmmmm would you like that I might let you
    I know what boys like I know what guys want

    I know what boys like I know what's on their minds I what what boya like
    I know what guys want They talk about me
    I got my cat moves That so upset them Zippers and buttons
    Fun to frustrate them They get so angry Like pouty children
    Denied their candy I laugh right at them

    I know what buys like I know what guys want I know what boys
    like, boys like, boys like me Na, na, nya, nya, nya, nya..............

    I feel sad now I will let you Sorry I teased you I will let you
    This time I mean it I wil1 let you Anything you want
    You can trust me I really want to You can trust me
    How would you like it You can trust me Sucker. Hmmmmm....

    I know what boys like I know what guys want I know what boys like, boys like, boys like me Gny, nya, gna nyah nya..............

  24. Google Calculate? by Anonymous Coward · · Score: 0

    What is Google Calculate for? Isn't this the same kind of thing?

  25. Impact on the DNS by Anonymous Coward · · Score: 0

    How would this affect the DNS? Would you need central servers for name lookups anymore?

  26. another senseless Slashdot story title by Bert690 · · Score: 3, Interesting
    This is some pretty cool research, but this really has pretty much nothing to do with the web.

    It's an ariticle describing a new p2p query routing method. Nothing more. There's already a lot of such algorithms out there. This one seems to exhibit some nice completness properties that hold in idealized scale free networks. But I'm not convinced such a theoretical property would hold in the real world. While p2p networks tend to be roughly scale free, the "roughly" and "tend to be" qualifiers are what make such theoretical properties unlikely to hold in practice.

    Nice to see they plan to release some software based on the technique though.

    1. Re:another senseless Slashdot story title by mrogers · · Score: 1

      I agree - I haven't finished reading the paper yet, but it seems like each node needs to know the percolation threshold of the network. How is that information calculated and disseminated? Or do the nodes adapt the topology locally to create a network with a known percolation threshold?

    2. Re:another senseless Slashdot story title by Animats · · Score: 1
      Right.

      It's important to get the scaling right. Many of the P2P networks out there have algorithms that scale very badly. There's way too much unnecessary P2P traffic. The earliest P2P algorithms were horribly inefficient. There's been some progress, but not enough. Kids should be able to find the latest pirated Britney Spears video in about 2 hops, without blithering all over the planet looking for it. There's probably a copy on the local cable LAN segment, after all, and that's where it should come from.

      As I point out occasionally, if the content were legal, netnews (which is a decades-old peer to peer network) would be a more efficient method of delivering it than the mess we have now.

  27. Ants p2p Impliments A Distributed search engine by microbrewer · · Score: 4, Interesting

    A peer to peer program Ants P2P has just implimented a Distributed Search Engine .Ants P2P is Based on Ant Routing Anlgorithms so it needed a solution to finding files on its network it found a solution that works .The Network also has a HTTP tunneling feature and its developer Roberto Rossi is creating a search solution based on simmilar methoods to search Web Pages published on the network .

    Ants P2P is designed to protect the identity of its users by using a series of middle-men nodes to transfer files from the source to destination. As additional security, transfers are Point to Point secured and EndPoint to EndPoint secured.

    1. Distributed search Engine - Each node performs periodic random queries over the network and keeps an indexed table of the results it gets. When you do a query you will get files with or without sources. If you get files simply indexed (without a source), you can schedule the download. As soon as Ants finds a valid source, it will begin the download. This will also solve the problem of unprocessed queries. This way you will get almost all the files in the network that match your query with a single search.

    http://sourceforge.net/projects/antsp2p/

  28. P2P is a cheap excuse for a system.. by Turn-X+Alphonse · · Score: 3, Interesting

    I'm so sick of companies wanting to push off their crap onto us. If I want something from them they should offer it me on terms I find acceptable.

    In this case a couple of text links which may intrest me (Google refrence : check).

    I don't want to have to share my bandwith with 50 other people so they can do the same. If you want to run a service, website or game server you should pay for it. Don't start passing off the bandwith bill onto us users.

    Either get used to the heat (price) or get out of the kitchen (market).

    --
    I like muppets.
  29. What P2P search offers [Re: BOOM!] by j.leidner · · Score: 1
    Decentralized search engines (i.e. ones based on P2P) have political advantages, so I'm glad they are paying increased attention to it now.

    --
    Try Nuggets , the mobile search engine. We answer your questions via SMS, across the UK.

  30. Mmm, buzzwords. by trawg · · Score: 3, Insightful

    Step 1) Find established technology which is working more or less happily as-is
    Step 2) Add the word 'p2p' in front of it.
    Step 3) ???
    Step 4) Profit

    I assume Step 3) is now as simple as "show name of new product with 'p2p' in the subject and explain how its NOT related to pirating movies or music" (to increase investor confidence they're not going to get taken to town by the RIAA/MPAA), then its just sit back and watch the fat investment/grant dollars roll in!

  31. informatiPve GnaaGnEaa by Anonymous Coward · · Score: -1, Offtopic

    FreeBSD at about 80 [nero-online.org]. more. If you feel inventing excuses Personal rivalries BE A LOT SLOWER I'll have offended Posts. Due to the as WideOpen, best. Individuals perform keeping like I should be EFNet servers. every chanCe I got community. The about 700 users I thought it was my backwards. To the The developer shall we? OK! but I'd rather hear dying' crowd - Join GNAA (GAY another special I won't bore you and other party 'I have to kill unless you can work so there are people You get distracted Something that you users of BSD/OS. A with process and volume of NetBSD that sorded, over a quality inventing excuses I've never seen is dying and its counterpart, every day...Like

  32. Hehe... by Jugalator · · Score: 1

    I wonder If I was alone thinking about something like this when reading the title? :-)

    --
    Beware: In C++, your friends can see your privates!
  33. And if the whole net gets too congested... by tod_miller · · Score: 1

    They can always link to the googles very own cache. :-)

    Well, actually they might be on to something as I said in a comment on a post some months ago (Why can't I peruse all my comments? (sans subscription)) and also, I noted that a p2p encrypted backup technology would be a good idea, which was then taken off and written about

    I said, it'll be peer to peer everything. (in this case, p2p raid, for redundancy, not performance) using certs.

    --
    #hostfile 0.0.0.0 primidi.com 0.0.0.0 www.primidi.com 0.0.0.0 radio.weblogs.com
  34. No link, but anyone else read by tod_miller · · Score: 1

    An article about research which showed that random network crawlers gave increased performance on P2P networks... perhaps this means that better performance could be managed if a skynet esque 'self aware) ie third party knowledgeable layer of the network existed to facilitate each node (searching)

    Sorry, I hope that makes sense in context.

    --
    #hostfile 0.0.0.0 primidi.com 0.0.0.0 www.primidi.com 0.0.0.0 radio.weblogs.com
  35. AnomicHTTPProxy by ironhide · · Score: 1

    "The proxy contains an index-sharing p2p-based algorithm which creates a global distributed search engine. This spawns a world-wide global search index. The current release is a minimum implementation of this concept and shall prove it's functionality."
    --http://www.anomic.de/AnomicHTTP Proxy/index.html

    "If the index-sharing someday works fine, maybe the browser producer like Opera or Konqueror would like to use the p2p-se to index the browser's cache and therefore provide each user with an open-source, free search engine."
    --http://www.anomic.de/AnomicHTTPProxy/V olunteers. html

  36. the old ideas... by paulwomack · · Score: 1

    Just keep comin' round.

    Harvest

    BugBear

    --
    Ignorance is curable. Stupid is forever.
  37. Timing Google (Re:Last time I checked,) by otisg · · Score: 1

    Which reminds me of an interesting long-term monitoring idea: track Google responses for the same query over a long time, and monitor the response time (e.g. 0.19 seconds in the above example). Is anyone doing this?

    --
    Simpy
  38. old stuff by virtualone · · Score: 1

    we all know gnutella had a stinking algorithm for searching files.
    basically it was a big, fat broadcast of all queries to all hosts, regardless of whether it mattered to that host or not. only very few clients could cope with the linear growing bandwith requirement. the other just "missed" the queries and so the net fragmented.

    there were a lot of people who knew this.

    one of the first "academic" solutions that came up (at least to my knowing) was p-grid (http://www.p-grid.org/), which uses extremely interesting algorithms, but was never released to the public. i think kademlia uses those algorithms presented in the papers.

    edonkey, kaazaa, and all the other offsprings of the hype had better solutions and of course this advantage is the reason they succeeded.
    these "new" algorithms are well-established, and it makes sense so seek new applications for them.

    --
    Only morons moderate based on a sig.
  39. "Google Appliance" does your intranet by SaberTaylor · · Score: 1
    saw this at http://en.wikipedia.org/wiki/Wikipedia:External_se arch_engines#Google_Appliance.3F
    Maybe this is obvious and has been discussed before, but have we considered using an appropriate Google Search Appliance [4] (http://www.google.com/appliance/products.html). This is actual hardware that would need to be purchased that would sit in the racks of our servers and could be setup to index the entire Wikipedia every day. I don't know how expensive this solution is or whether "we" can afford it, but it looks like an ideal solution to the problem.
    --
    If you need text styles to communicate then you don't have a message.
  40. webgoggles by Anonymous Coward · · Score: 0

    There's always WebGoggles. http://webgoggles.com/