Slashdot Mirror
Mr Anti-Google
MrNovember writes "Salon is
running a story on some guy named Daniel Brandt who they call "Mr. Anti-Google." Mr. Brandt runs a sort of anti-establishment database of citations called NameBase as well as Google Watch. He claims that Google's PageRank system is undemocratic primarily because it doesn't rank his NameBase information very highly. He also points out that Google maintains a log of all you've ever searched for associated with a long-term cookie. Google's system seems to work the best if you ask me but, on the other hand, link popularity may not provide the most intelligent top rankings."
And I think the Us Monetary system is unfair because I dont have enough of it!!!
No I didnt spell check this post...
a few links will fix namebase's spot on the list
...he'd increase his page ranking on google if he removed the little tin foil hats from his servers.
As for the point made that this guy thinks that Google is "undemocratic," give me a break! Google is not a government - it is a search site! They exist to make a profit. They will make money by providing a quality search result, thereby attracting users. They are not in the business of being the arbiter of democratic principles on the web.
Laws affecting technology will always be bad until enough techies become lawyers.
-brian
To his agenda perhaps.
However Google isn't used by most folks as a directory - it's a search engine. It simply pulls up entries according to a formula (see pigeonrank for the inside scoop) and gives those back. No bias beyond what smart webmasters can impart, no artificial clustering, etc.
If Google were to start doing as Brandt wants it would quickly run into endless battles, loose it's searching edge, become just another pay(or agenda)-for-play roadkill.
No thanks.
I don't read ACs: If a post isn't worth so much as a nom de plume to its author then I wont bother either.
Upon reading the article, you find that Mr. Brandt's main complaint about Google is that he believes that when you type in, say, "Richard M. Nixon" into Google, the material he has compiled on Nixon should be ranked #1.
Okay, so I did a search on Nixon on Brandt's site. Here are the first couple of results:
(1) How the Vatican conspired to hide Nazi war criminals.
(2) How various activists were persecuted by the CIA and FBI.
Nowhere did I even SEE Nixon's name in these abstracts. The only relevance is that Nixon was alive at the time, or maybe president when some of them took place, but hardly the man personally responsible for all of them.
When I type "Nixon" into Google, I expect to see biographical material, both good and bad, not totally unrelated rantings. Google is doing its job, in my opinion. It is giving low rankings to Brandt's irrelevant materials. His complaints are pure self-centered sour grapes.
This guy's just whining because Google doesn't rank pages according to his crackheaded counterculture views? And this is news?
Google must be doing pretty well if this is the worst criticism they can find about them.
Maybe I'm missing something here, but how is this a violation of your privacy? I mean, the whole thing is that you are using their service for free and willfully sending them the data that you choose. Everyone gets to choose what they search for in a search engine. This isn't private information in any real way. Google is providing you the free service of looking up words that you have intentionally provided. You don't like them being associated with a cookie? Refuse the damn cookie! Really paranoid? Go wander the web on your own without a search engine!
At what point were you guaranteed the free and anonymous use of a search engine? You're not being forced to use it. The world doesn't discriminate against people who do not choose not to efficiently search the web.
People like this are blurring the privacy issue and focusing attention away from legitimate privacy issues.
Nonperiodic Central Trajectory
"I am some poor guy who runs a second-grade website and since I can't get google to list me high, I will elicit some news media to get my site slashdotted"
hope you like your servers toasty, bud.
My life in the land of the rising sun.
What this guy wants, by abolishing PageRank, is a return to the free-for-all of early search engines, where the loudest voice rules. If one page has more keywords, it's ranked higher -- whether or not those keywords appear in the context of relevant content.
Here's his real problem: he thinks that linking to "Donald Rumsfeld" should bring his site's page to the top, despite the fact that he has no actual content -- just a list of links to other pages with content.
He calls this a failing of PageRank. I call it whining. If he wants more links from Google, he should get the word out about his site (preferably without manipulating Salon.com into doing it for him) and add some actual information about the people he's archiving by hand, instead of just building a big hotlist about them.
Basically, he wants to be the tyrant he imagines Google to be. Well, let him want all he likes. Google's popular because it's good and it's relevant; the fact that a tiny tiny minority think it's not isn't a good reason to overthrow the whole system.
People who live in glass houses shouldn't throw stones. He should start by making changes on his own site, not insisting Google make changes on there.
Google responds by stating that now all of their pigeons will go through an "intruduction to democracy" short course, and all "bird seed" websites are now ranked by humans instead of the patented "pigeon rank" system.
My life in the land of the rising sun.
In the meantime, anyone who would like to cover their tracks can use my cookie:
.google.com TRUE / FALSE 2147368045 PREF ID=111439b95052c72a:TM=1030056425:LM=1030056425:S= v7T9QSFKEkI
;)
Of course, if it turns out that Google is planning to give a prize to the most active user, or they have some kind of search engine green stamps, you're screwed.
Proud member of the Weirdo-American community.
First, a link to the article:0 8/29/googl e_watch/index.html
http://www.salon.com/tech/feature/2002/
(might be a space inserted in the URL by the browser submission, apologies)
Second, a quote from the article:
"Brandt sees this as Google's major flaw. "I'm not saying there aren't some sites that are more important that others, bu t in Google the sites that do well are the spammy sites, sites which have Google psyched out, and a lot of big sites, corporate headquarters' sites -- they show up before sites that criticize those companies.
In other words, Brandt recognizes that ther e has to be some order to Google's results, and that some sites might deserve to come up before others. He just disagrees with the way Google does it. In Brandt's ideal world, if you searched for "United Airlines," you would see untied.com -- a site crit i cal of United -- before you see United's page. And if you searched for Rumsfeld, you'd see NameBase's dossier on him before the Defense Department's site on the "The Honorabl e Donald Rumsfeld."
I must disagree with the ideal expressed here as Mr. Bran dt's. If I was searching for material on the Web about Donald Rumsfeld, I would rarely search for information critical of him *first*. If I was ego surfing on myself, I'd want to see my own material about me returned by Google, ahead of negative reviews and sites. I don't think that's an unfair way for Google to operate. While some of the issues Mr. Brandt raises might be valid, I do not feel that Google is required to promote or support Mr. Brandt's agenda over the agenda of the people and organizations Mr. Brandt chooses to focus on. M
Well, you could have submitted a link to the freakin' article instead of just a URL for it.
That if you don't like Google... then you shouldn't use Google. Duh. Why the holy crusade? If you think Altavista or hell, Netscape Search meets your needs, then use it. Why do people find it necessary to attack everything instead of being constructive.
I think, to be quite blunt, that this is a crock of shit.
One of the most important things in a civil society are the checks and balances critcism offers on any service, any government, any individual, indeed, any endeavor undertaken. These checks and balances, and the importance of public criticism, because of vastly greater importance when the perceptions and lives of many people are impacted.
This is true whether one is criticizing GNU, Linux, Richard Stallman, our corporate masters in the form of George Bush, Enron, WorldComm, Microsoft, Apple, Sun Microsystems, Red Hat, or whomever else happens to be in the hotseat at any given time.
If Google really were stacking their search results, criticism and a 'holy crusade' as you so snidely put it, would be a very important counterbalance in offsetting the corruption and distortion inherent in such a thing, particularly given how trusted Google is.
I disagree with the guys criticism, for what it is worth, and am an ardent user of Google. But I agree whole heartedly with the need for such criticism to keep the likes of Google honest, and to call them on the carpet when they do something shady or wrong (like they did when the caved to the Cult of Scientology's pressure to censor the search results revealing critics of that particular organization).
This "if you don't have something nice to say, don't say anything at all" is a fine creed for slaves or submissive corporate drones, but it has no place at all in the marketplace of intellectual thought or debate.
Now, on the other hand, if you'd like to argue for civil discourse instead of flame fests and random insults, I will be the first to add my voice to yours, but lest we forget, civil discourse can and must include criticism, sometimes vehement criticism. Indeed, such can often be the most important civil discourse being conducted.
The Future of Human Evolution: Autonomy
in Google the sites that do well are the spammy sites, sites which have Google psyched out, and a lot of big sites, corporate headquarters' sites -- they show up before sites that criticize those companies."
In other words, Brandt recognizes that there has to be some order to Google's results, and that some sites might deserve to come up before others. He just disagrees with the way Google does it. In Brandt's ideal world, if you searched for "United Airlines," you would see untied.com -- a site critical of United -- before you see United's page. And if you searched for Rumsfeld, you'd see NameBase's dossier on him before the Defense Department's site on the "The Honorable Donald Rumsfeld."
He wants google to be a political action site that favors his views. He's a whiny little baby.
Sites that critisize corporations should appear before the corporations main site? Why? Did you search for the company or for criticism? If the company/group in question was something he agreed with, perhaps some environmental organization or the democratic national commitee, would he want criticism of them to come up first too?
A quick stop at google shows that if you search for "United Airlines" you get their site first, and the site he thinks should be first shortly thereafter. If you search for "United Airlines criticism" you get the site he reccommends first. Looks like google is doing it's job correctly to me.
Why is salon publishing the crap?
Soon to be announced: Google for Wackos! With a clean-cut, cookie-less interface free of CIA influence, Google for Wackos will return search results based not on the listed sites' popularity, but on the wackiness of the conspiracy theories they present. Most popular search terms include Zapruder, tin foil, UFOs, and of course sex (but only the dirty illegal kind that politicians have.)
Genocide Man -- Life is funny. Death is funnier. Mass murder can be hilarious.
I never understand why people make such a big deal about cookies. If you don't want to be tracked (like me, like most of us here at slashdot) there are countless ways of protecting yourself via browser settings, CookieCop, Proximotron, etc. Anyone who really cares about privacy probably already knows how to disable cookies. And anyone who doesn't know probably doesn't care about privacy (my grandmother, etc.). It seems like people just enjoy complaining about a standard web technique even though it is easily circumventable.
Second, why the hell is slashdot even posting this article? I've skimmed plenty of the below comments and they all seem to agree that this anti-google guy is a goofball. Just because Salon ran an article on him doesn't mean that this fruitcake's complaints have any merit. Considering how many stories get rejected from slashdot on a daily basis, why was this chosen? Is it just me, or did anyone enjoy/learn from that article?
GMD
watch this
Short answer: It's a bug/quirk/feature of IE that, somehow, the page came across screwed up and got cached that way and, despite anything and everything you may have told IE about "check for a new page every time I visit...", it still checks this screwed up cache version first. The solution is to delete your temporary Internet files (Tools->Internet Options->"Delete Files" in "Temporary Internet Files")
Long answer:I had this problem with a site I frequent quite a bit. Since I know the author personally I told her about it. When I would actually save and view the page as prompted I would see all the HTML like I was supposed to but tons and tons of gibberish right before it. I told her to republish her blog but that didn't do it. No one else on her forum was having these problems and I figured since I told IE to check for a new version of the page every time that that couldn't be it. However, after clearing my cache out that did it.
Slightly More Elegant Solution: Instead of setting your homepage to Google, get the Google Toolbar. This way you can set your homepage to whatever and use the Toolbar to do whatever Google searching you want. With all the options its got it's easily the most useful thing I've ever used. Be sure to check the experimental options as well.
Schnapple
Hi all. I'm the evil Daniel Brandt who has the gall to criticize your beloved Google. Sorry the site is down. We're being synflooded, apparently by one or more slashdotters, since it started with the slashdot post. It's probably one of those who posted here, saying that if we can't keep our site going, then we don't belong in Google. We have our own router, so we hope to be able to clear things up shortly.
A few points missed in the Salon piece:
I specifically pointed out to the author of the piece when he interviewed me, that I felt my site did okay in Google, and that I was speaking for the public interest. The so-called "royal we" that Mr. Manjoo, the interviewer and author, refers to sarcastically, is used because I'm speaking for a tax-exempt, nonprofit public charity, Public Information Research, Inc. We do not sell widgets. Some of the comments in Slashdot have me mixed up with another person who is selling ads based on PageRank. But then, who expects Slashdotters to actually read the article?
My main site in Google is www.pir.org and it has a PageRank of 7. The www.namebase.org, with a PR of 6, is a streamlined CGI version of the main site, without all the essays and cartoons. NameBase began in the early 1980s and has been on the Internet since early 1995.
The other problem I have with the author's spin is that a good half of the interview was about Google's cookie. Most of the work I put into www.google-watch.org has to do with the cookie. In the article, the cookie is briefly mentioned, and most of the article is about how selfish and silly I am to think that Google should rank me higher.
My complaint about Google is not that PIR got the short end of the stick from Google, but that Google's stick should be longer.
My essay about PageRank is below.
_____________________
PageRank: Google's Original Sin
by Daniel Brandt
By 1998, the dot-com gold rush was in full swing. Web search engines had been around since 1995, and had been immediately touted by high-tech pundits (and Forbes magazine) as one more element in the magical mix that would make us all rich. Such innovations meant nothing less than the end of the business cycle.
But the truth of the matter, as these same pundits conceded after the crash, was that the false promise of easy riches put bottom-line pressures on companies that should have known better. One of the most successful of the earliest search engines was AltaVista, then owned by Digital Equipment Corporation. By 1998 it began to lose its way. All the pundits were talking "portals," so AltaVista tried to become a portal, and forgot to work on improving their search ranking algorithms.
Even by 1998, it was clear that too many results were being returned by the average search engine for the one or two keywords that were entered by the searcher. AltaVista offered numerous ways to zero in on specific combinations of keywords, but paid much less attention to the "ranking" problem. Ranking, or the ordering of returned results according to some criteria, was where the action should have been. Users don't want to figure out Boolean logic, and they will not be looking at more than the first twenty matches out of the thousands that might be produced by a search engine. What really matters is how useful the first page of results appears on search engine A, as opposed to the results produced by the same terms entered into engine B. AltaVista was too busy trying to be a portal to notice that this was important.
Enter Google
By early 1998, Stanford University grad students Larry Page and Sergey Brin had been playing around with a particular ranking algorithm. They presented a paper titled "The Anatomy of a Large-Scale Hypertextual Web Search Engine" at a World Wide Web conference. With Stanford as the assignee and Larry Page as the inventor, a patent was filed on January 9, 1998. By the time it was finally granted on September 4, 2001 (Patent No. 6,285,999), the algorithm was known as "PageRank," and Google was handling 150 million search queries per day. AltaVista continued to fade; even two changes of ownership didn't make a difference.
Google hyped PageRank, because it was a convenient buzzword that satisfied those who wondered why Google's engine did, in fact, provide better results. Even today, Google is proud of their advantage. The hype approaches the point where bloggers sometimes have to specify what they mean by "PR" -- do they mean PageRank, the algorithm, or do they mean the Public Relations that Google does so well:
PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important."
Google goes on to admit that other variables are also used, in addition to PageRank, in determining the relevance of a page. While the broad outlines of these additional variables are easily discerned by webmasters who study how to improve the ranking of their websites, the actual details of all algorithms are considered trade secrets by Google, Inc. It's in Google's interest to make it as difficult as possible for webmasters to cheat on their rankings.
It's all in the ranking
Beyond any doubt, search engines have become increasingly important on the web. E-commerce is very attuned to the ranking issue, because higher ranking translates directly into more sales. Various methods have been designed by various engines to monetize the ranking situation, such as paid placement, pay per click, and pay for inclusion. On June 27, 2002, the U.S. Federal Trade Commission issued guidelines that recommended that any ranking results influenced by payment, rather than by impartial and objective relevance criteria, ought to be clearly labeled as such in the interests of consumer protection. It appears, then, that any algorithm such as PageRank, that can reasonably pretend to be objective, will remain an important aspect of web searching for the foreseeable future.
Not only have engines improved their ranking methods, but the web has grown so huge that most surfers use search engines several times a day. All portals have built-in search functions, and most of them have to rely on one of a handful of established search engines to provide results. That's because only a few engines have the capacity to "crawl" or "spider" more than two billion web pages frequently enough to keep their database current. Google is perhaps the only engine that is known for consistent, predictable crawling, and that's only been true for less than two years. It takes almost a week to cover the available web, and another week to calculate PageRank for every page. Google's main update cycle is about 28 days, which is a bit too slow for news-hungry surfers. In August, 2001 they also began a second "mini-crawl" for news sites, which are now checked every day. Results from each crawl are mingled together, giving the searcher an impression of freshness.
For the average webmaster, the mechanics of running a successful site have changed dramatically from 1996 to 2002. This is due almost entirely to the increased importance of search engines. Even though much of the dot-com hype collapsed in 2000 and 2001 (a welcome relief to noncommercial webmasters who remembered the pre-hype days), the fact remains that by now, search engines are the fundamental consideration for almost every aspect of web design and linking. It's close to a wag-the-dog situation. That's why the algorithms that search engines consider to be consistent with the FTC's idea of impartial and objective ranking criteria deserve closer scrutiny.
What objective criteria are available?
Ranking criteria fall into three broad categories. The first is link popularity, which is used by a number of search engines to some extent. Google's PageRank is the original form of "link pop," and remains its purest expression. The next category is on-page characteristics. These include font size, title, headings, anchor text, word frequency, word proximity, file name, directory name, and domain name. The last is content analysis. This generally takes the form of on-the-fly clustering of produced results into two or more categories, which allows the searcher to "drill down" into the data in a more specific manner. Each method has its place. Search engines use some combination of the first two, or they use on-page characteristics alone, or perhaps even all three methods.
Content analysis is very difficult, but also very enticing. When it works, it allows for the sort of graphical visualization of results that can give a search engine an overnight reputation for innovation and excellence. But many times it doesn't work well, because computers are not very good at natural language processing. They cannot understand the nuances within a large stack of prose from disparate sources. Also, most top engines work with dozens of languages, which makes content analysis more difficult, since each language has its own nuances. There are several search engines that have made interesting advances in content analysis and even visualization, but Google is not one of them. The most promising aspect of content analysis is that it can be used in conjunction with link pop, to rank sites within their own areas of specialization. This provides an extra dimension that addresses some of the problems of pure link popularity.
Link popularity, which is "PageRank" to Google, is by far the most significant portion of Google's ranking cocktail. While in some cases the on-page characteristics of one page can trump the superior PageRank of a competing page, it's much more common for a low PageRank to completely bury a page that has perfect on-page relevance by every conceivable measure. To put it another way, it's frequently the case that a page with both search terms in the title, and in a heading, and in numerous internal anchors, will get buried in the rankings because the sponsoring site isn't sufficiently popular, and is unable to pass sufficient PageRank to this otherwise perfectly relevant page. In December 2000, Google came out with a downloadable toolbar attachment that made it possible to see the relative PageRank of any page on the web. Even the dumbed-down resolution of this toolbar, in conjunction with studying the ranking of a page against its competition, allows for considerable insight into the role of PageRank.
Moreover, PageRank drives Google's monthly crawl, such that sites with higher PageRank get crawled earlier, faster, and deeper than sites with low PageRank. For a large site with an average-to-low PageRank, this is a major obstacle. If your pages don't get crawled, they won't get indexed. If they don't get indexed in Google, people won't know about them. If people don't know about them, then there's no point in maintaining a website. Google starts over again on every site for every 28-day cycle, so the missing pages stand an excellent chance of getting missed on the next cycle also. In short, PageRank is the soul and essence of Google, on both the all-important crawl and the all-important rankings. By 2002 Google was universally recognized as the world's most popular search engine.
How does PageRank measure up?
In the first place, Google's claim that "PageRank relies on the uniquely democratic nature of the web" must be seen for what it is, which is pure hype. In a democracy, every person has one vote. In PageRank, rich people get more votes than poor people, or, in web terms, pages with higher PageRank have their votes weighted more than the votes from lower pages. As Google explains, "Votes cast by pages that are themselves 'important' weigh more heavily and help to make other pages 'important.'" In other words, the rich get richer, and the poor hardly count at all. This is not "uniquely democratic," but rather it's uniquely tyrannical. It's corporate America's dream machine, a search engine where big business can crush the little guy. This alone makes PageRank more closely related to the "pay for placement" schemes frowned on by the Federal Trade Commission, than it is related to those "impartial and objective ranking criteria" that the FTC exempts from labeling.
Secondly, only big guys can have big databases. If your site has an average PageRank, don't even bother making your database available to Google's crawlers, because they most likely won't crawl all of it. This is important for any site that has more than a few thousand pages, and a home page of about five or less on the toolbar's crude scale.
Thirdly, in order for Google to access the links to crawl a deep site of thousands of pages, a hierarchical system of doorway pages is needed so that crawler can start at the top and work its way down. A single site with thousands of pages typically has all external links coming into the home page, and few or none coming into deep pages. The home page PageRank therefore gets distributed to the deep pages by virtue of the hierarchical internal linking structure. But by the time the crawler gets to the real "meat" at the bottom of the tree, these pages frequently end up with a PageRank of zero. This zero is devastating for the ranking of that page, even assuming that Google's crawler gets to it, and it ends up in the index, and it has excellent on-page characteristics. The bottom line is that only big, popular sites can put their databases on the web and expect Google to cover their data adequately. And that's true even for websites that had their data on the web long before Google started up in 1999.
What about non-database sites?
There are other areas where PageRank has a negative effect, even for sites without a lot of data. The nature of PageRank is so discriminatory, that it's rather like the exact opposite of affirmative action. While many see affirmative action as reverse discrimination, no one would claim (apart from economists who advocate more tax cuts for the rich) that the opposite, which would be deliberate discrimination in favor of the already-privileged, is a solution for anything. Yet this is essentially what Google claims.
Those who launch new websites in 2002 have a much more difficult time getting traffic to their sites than they did before Google became dominant. The first step for a new site is to get listed in the Open Directory Project. This is used by Google to seed the crawl every month. But even after a year of trying to coax links to your new site from other established sites, the new webmaster can expect fewer than 30 visitors per day. Sites with a respectable PageRank, on the other hand, get tens of thousands of visitors per day. That's the scale of things on the web -- a scale that is best expressed by the fact that Google's zero-to-ten toolbar is a logarithmic scale, perhaps with a base of six. To go from an old PageRank of four to a new rank of five requires several times more incoming links. This is not easy to achieve. The cure for cancer might already be on the web somewhere, but if it's on a new site, you won't find it.
PageRank also encourages webmasters to change their linking patterns. On search engine optimization forums, webmasters even discuss charging for little ads with links, according to the PageRank they've achieved for their site. This would benefit those sites with a lower PageRank that pay for such ads. Sometimes these PageRank achievements are the result of link farms or other shady practices, which Google tries to detect and then penalizes with a PageRank of zero. At other times professional optimizers get away with spammy techniques. Mirror sites and duplicate pages on other domains are now forbidden by Google and swiftly punished, even when there are good reasons for maintaining such sites. Overall, linking patterns have changed significantly because of Google. Many webmasters are stingy about giving out links (which can dilute your transference of PageRank to a given site), at the same time that they're desperate for more links from others.
What should Google do?
We feel that PageRank has run its course. Google doesn't have to abandon it entirely, but they should de-emphasize it. The first step is to stop reporting PageRank on the toolbar. This would mute the awareness of PageRank among optimizers and webmasters, and remove some of the bizarre effects that such awareness has engendered. The next step would be to replace all mention of PageRank in their own public relations documentation, in favor of general phrases about how link popularity is one factor among many in their ranking algorithms. And Google should adjust the balance between their various algorithms so that excellent on-page characteristics are not completely cancelled by low link popularity.
PageRank must be streamlined so that the "tyranny of the rich" characteristics are scaled down in favor of a more egalitarian approach to link popularity. This would greatly simplify the complex and recursive calculations that are now required to rank two billion web pages, which must be very expensive for Google. The crawl must not be PageRank driven. There should be a way for Google to arrange the crawl so that if a site cannot be fully covered in one cycle, Google's crawlers can pick up where they left off on the next cycle.
Google is so important to the web these days, that it probably ought to be a public utility. Regulatory interest from agencies such as the FTC is entirely appropriate, but we feel that the FTC addressed only the most blatant abuses among search engines. Google, which only recently began using sponsored links and ad boxes, was not even an object of concern to the Ralph Nader group, Commercial Alert, that complained to the FTC.
This was a mistake, because Commercial Alert failed to look closely enough at PageRank. Some aspects of PageRank, as presently implemented by Google, are nearly as pernicious as pay for placement. There is no question that the FTC should regulate advertising agencies that parade as search engines, in the interests of protecting consumers. Google is still a search engine, but not by much. They can remain a search engine only by fixing PageRank's worst features.
*
[Daniel Brandt is founder and president of Public Information Research, Inc., a tax-exempt public charity that sponsors NameBase. He began compiling NameBase in 1982, from material that he started collecting in 1974, and is now the programmer and webmaster for PIR's several sites. He participates in various forums where webmasters share observations about the often-secretive algorithms, bugs, and behavior of various search engines. Brandt has been watching Google's interaction with NameBase ever since Google, in October, 2000, became the first search engine to go "deep" on PIR's main site by crawling thousands of dynamic pages.]
-
It's a search engine. You find info by typing names into a form. There are no obvious links to the content. How's that supposed to get spidered?
-
His search engine is overloaded right now and just returns error messages. Maybe that's what Google sees.
-
The good data is by subscription only:
"And ask your library or student government to subscribe to NameBase ($200 for two years of unrestricted access from any campus computer) so that we can continue to add names, and you can continue to find them."
-
<meta NAME="GOOGLEBOT" CONTENT="NOARCHIVE"> can't be helping.
-
This guy is very picky about who gets to spider him. Here's his "robots.txt" file:
-
He uses one-pixel GIFs to trap spiders. He also uses cookies and web bugs, providing a long-winded explanation of why what he does is OK, but what Google does is evil.
In conclusion, this guy created his own problem.I run three web sites. Each is at the top of the Google rankings for its obvious keywords, and I've done nothing whatsoever to make that happen. I just have useful content that people like.
I don't get it - you go on at great lengths about what google should do; about how bad Pagerank is, and how it should be fixed. But you don't say why you're not doing it yourself.
/.'ers) think that Pagerank works, and works very well. If you believe otherwise, why don't you simply go ahead and prove it?
Google became what it is because it saw an unfilled niche, and filled it. They "built a better mousetrap", and the world did indeed beat a path to their door. There is nothing stopping you from doing the same. If you're half as smart as you seem to think you are, you should have no problem implementing a search engine, and becoming as successful as Google is now.
Google is NOT a public utility, nor is it any form of monopoly. It needs to be regulated just as much as YOUR site does.
Unlike so many other companies, Google got where it is today solely on the merits of it's technology. It didn't succeed by pumping millions of dollars into marketing, it didn't succeed by using underhanded business tactics to squash its' competitors. All it did was make the best product.
Contrary to your essay, I (and I think many