Slashdot Mirror


Mr Anti-Google

MrNovember writes "Salon is running a story on some guy named Daniel Brandt who they call "Mr. Anti-Google." Mr. Brandt runs a sort of anti-establishment database of citations called NameBase as well as Google Watch. He claims that Google's PageRank system is undemocratic primarily because it doesn't rank his NameBase information very highly. He also points out that Google maintains a log of all you've ever searched for associated with a long-term cookie. Google's system seems to work the best if you ask me but, on the other hand, link popularity may not provide the most intelligent top rankings."

21 of 494 comments (clear)

  1. Link popularity works in the long term by Anonymous Coward · · Score: 2, Interesting

    The older your site is (and the better it is), the more likely that it will be linked to, and linked to well. If your site is new, small, or bad, very few people will link to you.

    Compared to the other search engines, Google is great, and that's what matters. Is it possible that someone could make a better search engine? Maybe. Please, try. Competition is good for everyone.

  2. Re:Cookie? What cookie? by Goldberg's+Pants · · Score: 2, Interesting

    I don't either. It's not like they FORCE you too use it. Google is the only decent search engine anymore since Altavista went to shit.

    I think the guys claims are probably bullshit, or at least gross exagerations. Google do seem to be one of the good guys, so it's inevitable that someone would come along and try and dig up stuff, real or imagined.

    Makes you wonder who might be bankrolling the guy...

  3. PageRank discriminates? by The+Magic+Yak · · Score: 2, Interesting

    I don't mind someone having their point of view, in fact I applaud Mr. Brandt for furthering what he beleives. However, search engine popularity is so flighty, if I think another engine is better than Google, I'll use it. Honestly, I have no ties to any search engine and feel I never will. However, Google has been able to stay at the top of the list (at least my list) for quite some time and has also managed to put the least amount of advertising (or harrassment) in my face. I used to used yahoo, until the pop-ups and ads overwhelmed me. I think much of Google's success came from the fact they never went public. This and the text based ads are incredible decisions when every other search engine was greedily grabbing web based advertising revenue. I like Google, I'll continue to use it, but I'm not going to fight for it either. Just my two cents.

    --
    Bill, can you factor this prime number for me?
  4. Namebase: pretty handy by Anonymous Coward · · Score: 2, Interesting

    Last year we were able to use Namebase to identify a rogue investor as having trained at the knee of Robert Vesco. Remember Vesco? The most successful international swindler of all time, and friend of the Whitehouse plumbers? Same guy. Ordinary due diligence did not turn up this information. Brandt may be offkey on Google, but he gets my vote of thanks.

  5. Moderation of hits? by paladin_tom · · Score: 3, Interesting

    If letting Google rank the pages is undemocratic, what about a system in which, when you go to a page from a Google search, Google adds a frame at the top of your page that let's you vote on how useful this page was on a scale of 1-10?

    Then, the most popular hits for a given set of search words would have their Google ranking rise. Now that's democracy.

    --
    #define sig "Every social system runs on the people's belief in it."
    1. Re:Moderation of hits? by actiondan · · Score: 2, Interesting

      If letting Google rank the pages is undemocratic, what about a system in which, when you go to a page from a Google search, Google adds a frame at the top of your page that let's you vote on how useful this page was on a scale of 1-10?


      Are you serious?

      Do you think someone might think to abuse this system? Automated form filling, anyone? Even if that were prevented, it wouldn't be too hard or expensive to hire hundreds of low paid data entry people to vote a site up.

      The google alogorithm can be manipulated to some extent but it has stood up pretty well so far. A voting system could be manipulated much more easily.
  6. The problem with this is.. by happystink · · Score: 3, Interesting

    People can theorize about Pagerank all they want and come up with 100 theories of why it's not correct and won't give you good results.. but guess what, that's all in theory, and in reality, Google gives amazing results. Pagerank will probably fall by the wayside in the years to come as more sophisticated algorhythms come along, but for now, it is ludicrous to suggest that it doesn't work, when you just have to search for anything on Google to see it's usefulness.

    Also, this guy claims that Google keeps a record of what everyone searches for.. what proof does he have of this? That Google sends a cookie? That cookie is more likely than anything just used for tracking how often most people use the site, so they can create aggregate numbers of unique users, etc. Sure they could be tracking every search term, but why would they, think how much storage space that'd waste for no return. If the FBI ever wants to find out what this guy searches for, they'll just contact his ISP and have him monitored that way.

    --

    sig:
    See the "..for smart people" banners Wired runs here? Look elsewhere guys.

  7. His real problem... by Farley+Mullet · · Score: 2, Interesting
    When you type "NameBase" into Google, Brandt's site comes up first, but Brandt is not satisfied with that. "My problem has been to get Google to go deep enough into my site," he says. In other words, Brandt wants Google to index the 100,000 names he has in his database, so that a Google search for "Donald Rumsfeld" will bring up NameBase's page for the secretary of defense. -- From the Salon article

    So it seems that this guy's real problem isn't with how Google ranks his site, but rather that Google isn't pushing his product to every searcher who hits their site. So he talks about the "undemocracy" of Google, but when it comes down to it, his main issue is that Google isn't helping his business, or rather, that Google's ranking algorithm isn't compatible with his business plan.

    Too often, when people say something is undemocratic, it's just because they aren't getting there own way.

  8. link to article, a quote, and my response by Bogatyr · · Score: 5, Interesting

    First, a link to the article:
    http://www.salon.com/tech/feature/2002/0 8/29/googl e_watch/index.html

    (might be a space inserted in the URL by the browser submission, apologies)

    Second, a quote from the article:
    "Brandt sees this as Google's major flaw. "I'm not saying there aren't some sites that are more important that others, bu t in Google the sites that do well are the spammy sites, sites which have Google psyched out, and a lot of big sites, corporate headquarters' sites -- they show up before sites that criticize those companies.

    In other words, Brandt recognizes that ther e has to be some order to Google's results, and that some sites might deserve to come up before others. He just disagrees with the way Google does it. In Brandt's ideal world, if you searched for "United Airlines," you would see untied.com -- a site crit i cal of United -- before you see United's page. And if you searched for Rumsfeld, you'd see NameBase's dossier on him before the Defense Department's site on the "The Honorabl e Donald Rumsfeld."

    I must disagree with the ideal expressed here as Mr. Bran dt's. If I was searching for material on the Web about Donald Rumsfeld, I would rarely search for information critical of him *first*. If I was ego surfing on myself, I'd want to see my own material about me returned by Google, ahead of negative reviews and sites. I don't think that's an unfair way for Google to operate. While some of the issues Mr. Brandt raises might be valid, I do not feel that Google is required to promote or support Mr. Brandt's agenda over the agenda of the people and organizations Mr. Brandt chooses to focus on. M

  9. k00k? by TheTick · · Score: 2, Interesting

    Is it just me, or does this guy sound like yet another internet kook? Get "untied.com" ranked first when searching for "united airlines"? That makes no sense.

    Google is a system -- a system that works a certain way. His complaints about PageRank are like complaining about an automobile for the way its wheels go 'round and 'round.

    I'm surprised salon dedicated any article space to this.

    --

    --
    bachiatari na torisetsu o yome!

  10. Google can be hijacked by Anonymous Coward · · Score: 2, Interesting

    This is especially popular with protesters, using the pagerank algorithm to rank higher than the company / organisation that they are protesting about. Its called google bombing. If he hates google so much then google bomb his site.

  11. Re:Boo Hoo by wfrp01 · · Score: 3, Interesting

    However Google isn't used by most folks as a directory - it's a search engine. It simply pulls up entries according to a formula (see pigeonrank [slashdot.org] for the inside scoop) and gives those back.

    I agree, and I wish Mr. Brandt would suggest a workable alternative, rather than whining. He clearly has a monkey on his back.

    However, I do wonder about the efficacy of google's formula. My concern is that google's popularity turn it's page rankings into self-fulfilling prophesies. It's a positive feedback loop: a site w/ a high google rank gets more views and more links, which increases its google rank, ad infinitum.

    Like you say, I'd rather not have search engines be driven by agendas or money. But I believe anything can be improved upon. Personally, I believe perhaps a bit of randomness might help. Instead of recieving an absolute page rank, pages should recieve a probability of being listed higher or lower. Just a thought.

    --

    --Lawrence Lessig for Congress!
  12. Re:Google Cookies by Anonymous Coward · · Score: 1, Interesting

    Of course cookies aren't a big deal since you can just refuse them from a site. I'm not all that concerned with using them or not (often I'll enable them at google so it'll memorize my search settings). After all, I don't care if they have my static IP and my search phrases or my static IP, search phrases and a session-id. Either way they can detect me and locate me (through my ISP) just as well (as can any other website on the face of the earth).

    The only way they'll tie in searches to ME is if they contact my ISP and get my personal information from them. And they can do that with or without a cookie.

  13. Re:Boo Hoo by wfrp01 · · Score: 3, Interesting

    While I'm at it...another idea. Google should publically state that if you put some particular meta tag in your document, that they will publish the contents of that tag (or tags?) in their page rank summary. This would encourage people to write good summary overviews of their pages, and would help users find things easier. With their clout, they could easily create a de-facto open meta-data standard. Use it or lose.

    As opposed to summaries that typically look like: ... Fri Dec 29 2000 Claudio Matsuoka : 5.49-39cl; put $CHKROOT
    inside ... fixes suport to "linux confirm"; make utmp group 22. Sat Aug 21 1999 ...

    --

    --Lawrence Lessig for Congress!
  14. I Disagree by FreeUser · · Score: 4, Interesting

    That if you don't like Google... then you shouldn't use Google. Duh. Why the holy crusade? If you think Altavista or hell, Netscape Search meets your needs, then use it. Why do people find it necessary to attack everything instead of being constructive.

    I think, to be quite blunt, that this is a crock of shit.

    One of the most important things in a civil society are the checks and balances critcism offers on any service, any government, any individual, indeed, any endeavor undertaken. These checks and balances, and the importance of public criticism, because of vastly greater importance when the perceptions and lives of many people are impacted.

    This is true whether one is criticizing GNU, Linux, Richard Stallman, our corporate masters in the form of George Bush, Enron, WorldComm, Microsoft, Apple, Sun Microsystems, Red Hat, or whomever else happens to be in the hotseat at any given time.

    If Google really were stacking their search results, criticism and a 'holy crusade' as you so snidely put it, would be a very important counterbalance in offsetting the corruption and distortion inherent in such a thing, particularly given how trusted Google is.

    I disagree with the guys criticism, for what it is worth, and am an ardent user of Google. But I agree whole heartedly with the need for such criticism to keep the likes of Google honest, and to call them on the carpet when they do something shady or wrong (like they did when the caved to the Cult of Scientology's pressure to censor the search results revealing critics of that particular organization).

    This "if you don't have something nice to say, don't say anything at all" is a fine creed for slaves or submissive corporate drones, but it has no place at all in the marketplace of intellectual thought or debate.

    Now, on the other hand, if you'd like to argue for civil discourse instead of flame fests and random insults, I will be the first to add my voice to yours, but lest we forget, civil discourse can and must include criticism, sometimes vehement criticism. Indeed, such can often be the most important civil discourse being conducted.

    --
    The Future of Human Evolution: Autonomy
  15. Re:Google Cookies by Compulawyer · · Score: 3, Interesting
    True, but search quality is measured by the closeness of the match between the search perfomed and the result of that search. If a high-ranked result is an excellent or even good match to the search performed (meaning that it is what the searcher wanted in the first place) then the fact that the high ranking was sold rather than generated by an algorithm that does not account for financial/business relationship factors is completely meaningless. The searcher got what s/he wanted - a quality result. The high-listed site got what it wanted - high placement in Google's result listing. Google got what it wanted too - payment for steering traffic to the site.

    As in this example search for snowboard retailers, Google even tags the top results as "Sponsored Links" so even the searchers know that those sites are ranked first because they paid Google to be ranked first. If it is what the searcher wanted, it doesn't matter.

    IMO, this is no different from a company purchasing a large ad in the yellow pages of the phone directory. Does/should anyone think that the ads are bigger for certain companies because those companies are better? People know that companies buy those ads. Searchers should also know that "sponsored" = paid. I don't see anything inappropriate. In fact, I credit Google for being above some of the slimy companies on the web and staying above the board with its business practices. Google's ability to charge for ranking will be nil if its search results reduce to the point of being roughly equivalent to random advertising.

    --

    Laws affecting technology will always be bad until enough techies become lawyers.

  16. He deserves it! by Fujisawa+Sensei · · Score: 2, Interesting

    After reading about this Jackass and his ranting. I'm really happy his crap isn't at the top of google's page ranking. I would say, "If only Google would let me filter out his website", (I won't post the URL because that might acutally help his ranking.) But he seems to have done a good job of this on his own.

    For example if I'm searching for United Airlines, I want UAL.com, I'm not interested in untied.com. If I were interested in "How UAL treats its Own" I would type that into my search engine.

    If I search for "Mickey Mouse" I want a site about the rat, not one about how Disney is abusing trademark, copywrite laws, or the DCMA.

    I would say, "If only Google would let me filter out his website", (I won't post the URL because that might acutally help his ranking.) But he seems to have done a good job of this on his own.

    BTW I tried searching Google for Donald Rumsfield at www.namebase.org and I got the following result:
    Your search - Donald Rumsfeld site:www.namebase.org - did not match any documents.

    I think Google's page rank for his site is dead on.

    --
    If someone is passing you on the right, you are an asshole for driving in the wrong lane.
  17. Here's my essay by Everyman · · Score: 4, Interesting

    Hi all. I'm the evil Daniel Brandt who has the gall to criticize your beloved Google. Sorry the site is down. We're being synflooded, apparently by one or more slashdotters, since it started with the slashdot post. It's probably one of those who posted here, saying that if we can't keep our site going, then we don't belong in Google. We have our own router, so we hope to be able to clear things up shortly.

    A few points missed in the Salon piece:

    I specifically pointed out to the author of the piece when he interviewed me, that I felt my site did okay in Google, and that I was speaking for the public interest. The so-called "royal we" that Mr. Manjoo, the interviewer and author, refers to sarcastically, is used because I'm speaking for a tax-exempt, nonprofit public charity, Public Information Research, Inc. We do not sell widgets. Some of the comments in Slashdot have me mixed up with another person who is selling ads based on PageRank. But then, who expects Slashdotters to actually read the article?

    My main site in Google is www.pir.org and it has a PageRank of 7. The www.namebase.org, with a PR of 6, is a streamlined CGI version of the main site, without all the essays and cartoons. NameBase began in the early 1980s and has been on the Internet since early 1995.

    The other problem I have with the author's spin is that a good half of the interview was about Google's cookie. Most of the work I put into www.google-watch.org has to do with the cookie. In the article, the cookie is briefly mentioned, and most of the article is about how selfish and silly I am to think that Google should rank me higher.

    My complaint about Google is not that PIR got the short end of the stick from Google, but that Google's stick should be longer.

    My essay about PageRank is below.

    _____________________

    PageRank: Google's Original Sin

    by Daniel Brandt

    By 1998, the dot-com gold rush was in full swing. Web search engines had been around since 1995, and had been immediately touted by high-tech pundits (and Forbes magazine) as one more element in the magical mix that would make us all rich. Such innovations meant nothing less than the end of the business cycle.

    But the truth of the matter, as these same pundits conceded after the crash, was that the false promise of easy riches put bottom-line pressures on companies that should have known better. One of the most successful of the earliest search engines was AltaVista, then owned by Digital Equipment Corporation. By 1998 it began to lose its way. All the pundits were talking "portals," so AltaVista tried to become a portal, and forgot to work on improving their search ranking algorithms.

    Even by 1998, it was clear that too many results were being returned by the average search engine for the one or two keywords that were entered by the searcher. AltaVista offered numerous ways to zero in on specific combinations of keywords, but paid much less attention to the "ranking" problem. Ranking, or the ordering of returned results according to some criteria, was where the action should have been. Users don't want to figure out Boolean logic, and they will not be looking at more than the first twenty matches out of the thousands that might be produced by a search engine. What really matters is how useful the first page of results appears on search engine A, as opposed to the results produced by the same terms entered into engine B. AltaVista was too busy trying to be a portal to notice that this was important.

    Enter Google

    By early 1998, Stanford University grad students Larry Page and Sergey Brin had been playing around with a particular ranking algorithm. They presented a paper titled "The Anatomy of a Large-Scale Hypertextual Web Search Engine" at a World Wide Web conference. With Stanford as the assignee and Larry Page as the inventor, a patent was filed on January 9, 1998. By the time it was finally granted on September 4, 2001 (Patent No. 6,285,999), the algorithm was known as "PageRank," and Google was handling 150 million search queries per day. AltaVista continued to fade; even two changes of ownership didn't make a difference.

    Google hyped PageRank, because it was a convenient buzzword that satisfied those who wondered why Google's engine did, in fact, provide better results. Even today, Google is proud of their advantage. The hype approaches the point where bloggers sometimes have to specify what they mean by "PR" -- do they mean PageRank, the algorithm, or do they mean the Public Relations that Google does so well:

    PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important."

    Google goes on to admit that other variables are also used, in addition to PageRank, in determining the relevance of a page. While the broad outlines of these additional variables are easily discerned by webmasters who study how to improve the ranking of their websites, the actual details of all algorithms are considered trade secrets by Google, Inc. It's in Google's interest to make it as difficult as possible for webmasters to cheat on their rankings.

    It's all in the ranking

    Beyond any doubt, search engines have become increasingly important on the web. E-commerce is very attuned to the ranking issue, because higher ranking translates directly into more sales. Various methods have been designed by various engines to monetize the ranking situation, such as paid placement, pay per click, and pay for inclusion. On June 27, 2002, the U.S. Federal Trade Commission issued guidelines that recommended that any ranking results influenced by payment, rather than by impartial and objective relevance criteria, ought to be clearly labeled as such in the interests of consumer protection. It appears, then, that any algorithm such as PageRank, that can reasonably pretend to be objective, will remain an important aspect of web searching for the foreseeable future.

    Not only have engines improved their ranking methods, but the web has grown so huge that most surfers use search engines several times a day. All portals have built-in search functions, and most of them have to rely on one of a handful of established search engines to provide results. That's because only a few engines have the capacity to "crawl" or "spider" more than two billion web pages frequently enough to keep their database current. Google is perhaps the only engine that is known for consistent, predictable crawling, and that's only been true for less than two years. It takes almost a week to cover the available web, and another week to calculate PageRank for every page. Google's main update cycle is about 28 days, which is a bit too slow for news-hungry surfers. In August, 2001 they also began a second "mini-crawl" for news sites, which are now checked every day. Results from each crawl are mingled together, giving the searcher an impression of freshness.

    For the average webmaster, the mechanics of running a successful site have changed dramatically from 1996 to 2002. This is due almost entirely to the increased importance of search engines. Even though much of the dot-com hype collapsed in 2000 and 2001 (a welcome relief to noncommercial webmasters who remembered the pre-hype days), the fact remains that by now, search engines are the fundamental consideration for almost every aspect of web design and linking. It's close to a wag-the-dog situation. That's why the algorithms that search engines consider to be consistent with the FTC's idea of impartial and objective ranking criteria deserve closer scrutiny.

    What objective criteria are available?

    Ranking criteria fall into three broad categories. The first is link popularity, which is used by a number of search engines to some extent. Google's PageRank is the original form of "link pop," and remains its purest expression. The next category is on-page characteristics. These include font size, title, headings, anchor text, word frequency, word proximity, file name, directory name, and domain name. The last is content analysis. This generally takes the form of on-the-fly clustering of produced results into two or more categories, which allows the searcher to "drill down" into the data in a more specific manner. Each method has its place. Search engines use some combination of the first two, or they use on-page characteristics alone, or perhaps even all three methods.

    Content analysis is very difficult, but also very enticing. When it works, it allows for the sort of graphical visualization of results that can give a search engine an overnight reputation for innovation and excellence. But many times it doesn't work well, because computers are not very good at natural language processing. They cannot understand the nuances within a large stack of prose from disparate sources. Also, most top engines work with dozens of languages, which makes content analysis more difficult, since each language has its own nuances. There are several search engines that have made interesting advances in content analysis and even visualization, but Google is not one of them. The most promising aspect of content analysis is that it can be used in conjunction with link pop, to rank sites within their own areas of specialization. This provides an extra dimension that addresses some of the problems of pure link popularity.

    Link popularity, which is "PageRank" to Google, is by far the most significant portion of Google's ranking cocktail. While in some cases the on-page characteristics of one page can trump the superior PageRank of a competing page, it's much more common for a low PageRank to completely bury a page that has perfect on-page relevance by every conceivable measure. To put it another way, it's frequently the case that a page with both search terms in the title, and in a heading, and in numerous internal anchors, will get buried in the rankings because the sponsoring site isn't sufficiently popular, and is unable to pass sufficient PageRank to this otherwise perfectly relevant page. In December 2000, Google came out with a downloadable toolbar attachment that made it possible to see the relative PageRank of any page on the web. Even the dumbed-down resolution of this toolbar, in conjunction with studying the ranking of a page against its competition, allows for considerable insight into the role of PageRank.

    Moreover, PageRank drives Google's monthly crawl, such that sites with higher PageRank get crawled earlier, faster, and deeper than sites with low PageRank. For a large site with an average-to-low PageRank, this is a major obstacle. If your pages don't get crawled, they won't get indexed. If they don't get indexed in Google, people won't know about them. If people don't know about them, then there's no point in maintaining a website. Google starts over again on every site for every 28-day cycle, so the missing pages stand an excellent chance of getting missed on the next cycle also. In short, PageRank is the soul and essence of Google, on both the all-important crawl and the all-important rankings. By 2002 Google was universally recognized as the world's most popular search engine.

    How does PageRank measure up?

    In the first place, Google's claim that "PageRank relies on the uniquely democratic nature of the web" must be seen for what it is, which is pure hype. In a democracy, every person has one vote. In PageRank, rich people get more votes than poor people, or, in web terms, pages with higher PageRank have their votes weighted more than the votes from lower pages. As Google explains, "Votes cast by pages that are themselves 'important' weigh more heavily and help to make other pages 'important.'" In other words, the rich get richer, and the poor hardly count at all. This is not "uniquely democratic," but rather it's uniquely tyrannical. It's corporate America's dream machine, a search engine where big business can crush the little guy. This alone makes PageRank more closely related to the "pay for placement" schemes frowned on by the Federal Trade Commission, than it is related to those "impartial and objective ranking criteria" that the FTC exempts from labeling.

    Secondly, only big guys can have big databases. If your site has an average PageRank, don't even bother making your database available to Google's crawlers, because they most likely won't crawl all of it. This is important for any site that has more than a few thousand pages, and a home page of about five or less on the toolbar's crude scale.

    Thirdly, in order for Google to access the links to crawl a deep site of thousands of pages, a hierarchical system of doorway pages is needed so that crawler can start at the top and work its way down. A single site with thousands of pages typically has all external links coming into the home page, and few or none coming into deep pages. The home page PageRank therefore gets distributed to the deep pages by virtue of the hierarchical internal linking structure. But by the time the crawler gets to the real "meat" at the bottom of the tree, these pages frequently end up with a PageRank of zero. This zero is devastating for the ranking of that page, even assuming that Google's crawler gets to it, and it ends up in the index, and it has excellent on-page characteristics. The bottom line is that only big, popular sites can put their databases on the web and expect Google to cover their data adequately. And that's true even for websites that had their data on the web long before Google started up in 1999.

    What about non-database sites?

    There are other areas where PageRank has a negative effect, even for sites without a lot of data. The nature of PageRank is so discriminatory, that it's rather like the exact opposite of affirmative action. While many see affirmative action as reverse discrimination, no one would claim (apart from economists who advocate more tax cuts for the rich) that the opposite, which would be deliberate discrimination in favor of the already-privileged, is a solution for anything. Yet this is essentially what Google claims.

    Those who launch new websites in 2002 have a much more difficult time getting traffic to their sites than they did before Google became dominant. The first step for a new site is to get listed in the Open Directory Project. This is used by Google to seed the crawl every month. But even after a year of trying to coax links to your new site from other established sites, the new webmaster can expect fewer than 30 visitors per day. Sites with a respectable PageRank, on the other hand, get tens of thousands of visitors per day. That's the scale of things on the web -- a scale that is best expressed by the fact that Google's zero-to-ten toolbar is a logarithmic scale, perhaps with a base of six. To go from an old PageRank of four to a new rank of five requires several times more incoming links. This is not easy to achieve. The cure for cancer might already be on the web somewhere, but if it's on a new site, you won't find it.

    PageRank also encourages webmasters to change their linking patterns. On search engine optimization forums, webmasters even discuss charging for little ads with links, according to the PageRank they've achieved for their site. This would benefit those sites with a lower PageRank that pay for such ads. Sometimes these PageRank achievements are the result of link farms or other shady practices, which Google tries to detect and then penalizes with a PageRank of zero. At other times professional optimizers get away with spammy techniques. Mirror sites and duplicate pages on other domains are now forbidden by Google and swiftly punished, even when there are good reasons for maintaining such sites. Overall, linking patterns have changed significantly because of Google. Many webmasters are stingy about giving out links (which can dilute your transference of PageRank to a given site), at the same time that they're desperate for more links from others.

    What should Google do?

    We feel that PageRank has run its course. Google doesn't have to abandon it entirely, but they should de-emphasize it. The first step is to stop reporting PageRank on the toolbar. This would mute the awareness of PageRank among optimizers and webmasters, and remove some of the bizarre effects that such awareness has engendered. The next step would be to replace all mention of PageRank in their own public relations documentation, in favor of general phrases about how link popularity is one factor among many in their ranking algorithms. And Google should adjust the balance between their various algorithms so that excellent on-page characteristics are not completely cancelled by low link popularity.

    PageRank must be streamlined so that the "tyranny of the rich" characteristics are scaled down in favor of a more egalitarian approach to link popularity. This would greatly simplify the complex and recursive calculations that are now required to rank two billion web pages, which must be very expensive for Google. The crawl must not be PageRank driven. There should be a way for Google to arrange the crawl so that if a site cannot be fully covered in one cycle, Google's crawlers can pick up where they left off on the next cycle.

    Google is so important to the web these days, that it probably ought to be a public utility. Regulatory interest from agencies such as the FTC is entirely appropriate, but we feel that the FTC addressed only the most blatant abuses among search engines. Google, which only recently began using sponsored links and ad boxes, was not even an object of concern to the Ralph Nader group, Commercial Alert, that complained to the FTC.

    This was a mistake, because Commercial Alert failed to look closely enough at PageRank. Some aspects of PageRank, as presently implemented by Google, are nearly as pernicious as pay for placement. There is no question that the FTC should regulate advertising agencies that parade as search engines, in the interests of protecting consumers. Google is still a search engine, but not by much. They can remain a search engine only by fixing PageRank's worst features.

    *

    [Daniel Brandt is founder and president of Public Information Research, Inc., a tax-exempt public charity that sponsors NameBase. He began compiling NameBase in 1982, from material that he started collecting in 1974, and is now the programmer and webmaster for PIR's several sites. He participates in various forums where webmasters share observations about the often-secretive algorithms, bugs, and behavior of various search engines. Brandt has been watching Google's interaction with NameBase ever since Google, in October, 2000, became the first search engine to go "deep" on PIR's main site by crawling thousands of dynamic pages.]

  18. Re:Google Cookies by afidel · · Score: 4, Interesting

    Google probably does have the space, they have about a half dozen (probably more now) copies of their database at each of their colocation facilites, they do this for load balancing and redundancy purposes, by increasing their server farms by say 10-20% they could pretty easily store all of the searches they run every day in some compressed format.

    --
    There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
  19. Re:Google Cookies by NanoGator · · Score: 3, Interesting
    I'm not the one who made that claim. It's in the article (guess I shoulda quoted the guy.)

    Here's the quote because people obviously read my post before reading the article:

    More than that, says Brandt, Google is a careless custodian of private information. When you search for something at Google, it saves your search terms and associates them with a cookie that is set to live on your machine for 36 years. Brandt fears that law enforcement officials could muscle Google into divulging all the terms you've ever searched for. Those terms could be "a window into your state of mind," and are therefore a clear violation of your privacy, he says."


    Mr Anti-Google said that, not me.

    Buried on the next page of the Salon article, Google responded with basically what you said:

    In an e-mail, Nathan Tyler, a Google representative, told Salon that "Google uses cookies to enhance the user search experience. With cookies, Google can store a user's preferences such as their search language, SafeSearch settings, the number of results per page, etc. Our users tell us time and again that they want their preferences saved even if they don't return to Google for a long time. In addition, a longer cookie life means less data is transmitted at every visit (to refresh a cookie, check if it's current, etc.) and therefore speeds download times.


    Mr. Anti-Google probably mistook the history bar for a cookie heh.

    The point I was making was that even if Google did store search terms in the cookie, so what? There's a big difference between storing that locally on your machine and storing it on their own servers. Even if this guy was absolutely 100% right about how it works, his point carries no weight.

    --
    "Derp de derp."
  20. An exercise in character assassinations by ergo98 · · Score: 2, Interesting

    I've skimmed plenty of the below comments and they all seem to agree that this anti-google guy is a goofball.

    This whole bizarre Salon article and the followup Slashdot postings seems like a horrible, reprehensible character assassination because someone said something that someone else didn't like (is it too late, and Google has gotten too powerful :-}). If you read the Salon article with a critical eye, you'll see an article slamming someone who actually made a fairly logical and reasonably thought-out complaint about the PageRank system, carefully interspacing comments about his counterculture past with his simple belief that the "democractic" nature of PageRank isn't democratic at all. With wink of the eye comments like "(using the royal "we")" make it very clear what the bias of the author is: Disparage this guy no matter what. They went so far as to make claims on behalf of him (which I can't see in his article), such as "In Brandt's ideal world, if you searched for "United Airlines," you would see untied.com -- a site critical of United -- before you see United's page. And if you searched for Rumsfeld, you'd see NameBase's dossier on him before the Defense Department's site on the "The Honorable Donald Rumsfeld."" : Funny, but I don't see that in his paper, but instead that appears to be Salon making some rhetorical exaggerations to push his opinion to extremes.

    The bizarrest thing is how quickly everyone hopped on the bandwagon to slam this "kook", all based upon the carefully manipulative wording of a Salon article. It is especially disconcerting given that this is the type of guy (questioning "the establishment") that the Slashdot crowd usually hoists on their shoulders and casts as their hero. This Salon article is DISPICABLE, and the methods that the author uses to villainize this guy is a study in evasive techniques (Google's cookie and search tracking doesn't matter, you see, because there are sites that are worse).