Slashdot Mirror
Search Engines-Does Obscurity Prevent Exploitation?
GeekLife.com asks: "Search engines refuse to release (and often change) the exact criteria that determines their ranked results, presumably both to prevent competitors from stealing their techniques and to stop (or at least make less successful) attempts at "cheating" - optimizing a site to exploit these criteria, resulting in a higher ranking than it deserves to be. Is this an example where keeping the specifics a secret actually improves the tool? Or would releasing all the rules result in enough feedback ('given enough eyeballs...'), honing the criteria towards unexploitable results?" Interesting though. Can current systems be improved to give better results or have we reached an 'accuracy limit' as far as keyword-based searching is concerned?
Since there's a strong incentive to get your site listed in the search engines, the search criteria will always be exploited.
A friend of mine left the company I work for and started making porn pages for an australia based porn company.
He is supposed to make 400 pages per month, all somewhat different. He gets a bonus based on how many hits are generated, and a commission based on signups from his banner ads.
He's doing pretty well financially
********* sig: If you don't like the law, get filthy stinking rich, and buy a better one.
The only way to achieve true search engine accuracy is to have an actual person search for pages on request. Why no company has thought of this, I'm not sure, as this could certainly be an explosive business opportunity here. The difficulty of finding trustworthy information on the Internet is legendary, and I'm a sure plenty of clueless newbies would pay a monthly fee to get better search results.
I think the many-eyeball argument doesn't apply here, because the it's not about finding bugs in the rules, but preventing cheating. When a site decides to "cheat", it doesn't exploit a bug. The scoring system is not like a kernel, where you know exactly what should happen, it a (generally) complex AI system. These systems are designed so that they work well enough in 95% (or 80%, this is not the point) of the time. There're going to miss a couple percent, but what else can you do. Now if you have access to the rule, you can make sure your site uses the 5% errors to go on top of the list. Unless someone thinks he can have 100% accuracy (how do you measure accuracy anyway!), the scoring rules shouldn't be released.
Opus: the Swiss army knife of audio codec
I think that we may have reached an "accuracy limit" with search engines until such time that people don't mind search engines leaving cookies on their hard drives, so they can examine a user's past queries and use those to try to present more relevant results for that users current query. I really think that will be the only way for them to grow, because most search terms I've seen (basically, referrer logs for my site and few other sites i've worked on) only consist of 3 or less words. It's a rarity that someone enters more than that, so that doesn't give a search engine much to work with...
However, if say google knew that I'd done searches for "albini" and "shellac" in the past, it could probably surmise that when i did a search for "big black", i'm actually looking for Steve Albini's first band, and not BIG BLACK BOOBS, et al...
I can't figure how else something like that could be accomplished without a sacrafice of our hope for privacy...
I like the concept of ranking pages by the number or pages referring to it. That seems to be the best method of keeping it somewhat difficult to cheat. The only problem is that newer material will take longer to reach the top, regardless of the relevance. It would be nice to have a search engine that could record poeples input regarding the results that were found. How many times have you searched for something only to find the best result buried five pages down. A method to go back and vote the ranking results yourself would seem appealing to me.
The google "algorithm" is explained on the Why Use page on Google. Although it doesn't give the *exact* code used, it explains (in english) the whole process pretty well.
--Xandu
With enough experimenting someone can find out how the system works. Either through keywords,page text,bribing .. etc whatever. People will find out how it works. Just a matter of time
Well, while there are user-submitted lists of sites (yahoo,whatever) I think it's just about time for a moderated search engine.
The users could submit links in different subjects or categories with different keywords adding them to the harvested ones and, most important, registered users would be able to get x moderator points a week and vote down spam links or links that don't make much sense with the search one conducted.
Add a healthy dose of meta-moderation (maybe three levels) and some obvious anti-cheat prevention techniques and it should work much better than a normal search engine.
God knows many times even on google obviously poisonous sites come up in the search, it would be so nice to have a button to click to moderate down the page or the domain itself...
-- the cake is a lie
Let's presuppose that their ranking criteria are reasonably accurate. If you search for "girl's soccor" (sans the quotes, since most people don't use them, or know the option is available to them), or something pertaining to female gender in a non sexual sense, why are most of the top results pornography related material? Is it just because there are so many adult oriented sites, or is it because they have some sort of technique that allows search spiders to place them in all sorts of unrelated catagories, say using meta information?
-- From my Best Friend (Written to me over ICQ): "i was gonna go to a party...but i had to reinstall windows"
I thought someone already did this, posting enough links on /. that goatse.cx became the
number one response for searches on "natalie
portman grits"
--
The scalloped tatters of the King in Yellow must cover
Yhtill forever. (R. W. Chambers, the King in Yellow)
The scalloped tatters of the King in Yellow must cover
Yhtill forever. (R. W. Chambers, the King in Yellow
Here's an idea: rotate the criteria between several open-source methods of sorting. Design each to both low-score attempts at exploiting the other methods, while still maintaining an adequate order for people doing searches.
So, at best, a company could be really popular on occasion, but suck the rest of the time.
---
Sure, you could pay people to link to your site -- it's done all the time. But only pr0n sites and the Big Portals are gonna have enough coin to make that a factor in a generalized site. If your site is at all focussed or specialized in nature, you're not likely to have the funds or time to pay people to stay linked to your site.
And that brings me to another point: with sophisticated enough keyword ranking algorithms, it'll become more of a pain in the ass to come up with spam that makes it through the filters than to simply put up a good site in the first place. 600 repetions of "sex" are easy to pick out. And if HotGrammar 2.0 can pick through my dangling participles in a reasonable amount of time, then it can't be too much more difficult to point it at a website and say "Does the content match the keywords?"
At that point I think you really would have some reasonably close to a non-hackable search engines: The Google Algorithm to pick out the sites that people have blessed with links, and The Grammar Nazi Algorithm to make sure there's content to match.
Carousel is a lie!
If your algorithm is good, it can't be exploited. Google's algorithm, which is quite well-known, seems to work quite well. For that matter, dmoz.org isn't doing badly.
What I'd suggest, though, is a compromise. The basic popularity rating is based off of number of links, like with Google. However, people would be able "rate" the effectiveness of a given site when it comes up in the list (nothing fancy, just "relevant," "neutral," or "irrelevant"; maybe a five-point scale rather than three-point). No individual vote could do very much to the system, but as votes add up trends start to show, and relevance ratings can be modified based on this. This would require a user registration system to keep track of moderations (though definitely not searches), but so long as registration isn't required to actually use the search engine itself I fail to see the problem in it.
----------
The biggest exploit I could think of for this, would be for a big company like Amazon to pay a lot of people to link to their site (oh wait, they already do). Thus they can use money to skew the results, without having to corrupt Google directly. Hmm.
"Destroy science and religion. Science would re-emerge exactly the same; but not religion." - Penn Jillette, paraphrased
Well, also the fact that a huge chunk of the web isn't even indexed at all.
Other than that, though, the interfaces that most search engines use are pretty bad. There is usually no way to filter through a set of results to eliminate things that are obviously not what the searcher wants. Just being able to eliminate a set of domains from the initial results would make a huge difference for me.
Also, most people have no clue how to effectively use search engines - and they're not all that interested in doing so. I've been working in the web industry for quite a long time, and most of my colleagues seem to have no idea that changing the settings can yield better results. The setting 'phrase' for instance, makes a HUGE difference much of the time - yet I've never seen a colleague change any default settings when doing a web search. If you're not willing to do so much as even toggle an individual setting, you deserve the crappy results you get.
Oh, another thing - many of the links I get back are of dubious quality - even on the setting 'phrase', many results don't come back that match what I specified. If you play the the rules and the results STILL don't match, I have little faith in ANY results, even if the web site operators are trying to override accuracy. This is aside the very common result of '404 not found' pages.
Right now, the best search engine I know of is a meta search engine called 'ProFusion' - I've had much better luck with it than with Google. Not enough control over Google...I also like that the results with Profusion ( http://www.profusion.com ) come back with an option next to each result to open in a new browser window - now THAT's a nice idea!
How about a 'vote' icon? Not for each and every click, as no one will do that, but rather a 'demote' button next to a link. You can keep on organizing results by the number of people who click on the link, but if someone clicks through and finds something other than what they're looking for they'll be heading back to the search anyway, so they can give a 'demote' button a quick click as they continue down the list. If you made such a system immediately detract 100 clicks from that site, misleading links would soon be phased to the bottom.
After all, more people like to complain about a bad link than to promote a good one. Let human nature work in our favor, for once.
---
Two points:
Since from when I can first remember seeing banner adds, I can also remember seeing "please click here to support this page" right below them. People often end up clicking adds in order to 'support' a site and generate money, rather than being interested in the content of the adds. If people can exploit something for money, they will.
Secondly, banner adds are what give people incentive to cheat on search engines. If they can get more hits per month, that directly translates to more clicks (or impressions) per month, and more $$$. In today's society with the commercialization of the internet and 'dot-commies', I would bet money that the resulting information would definately be used to make somebody a quick buck.
That being said, I would wager that the final result would end up being more reliable search engines, but the potential problems may or may not be worth it.
Be the Ultimate Ninja! Play Billy Vs. SNAKEMAN today!
Some really good points by previous posters that I want to recap:
If you open up the criteria such that *everyone* exploits the criteria, then there is no discrimination. When the criteria is closed, only those who have found the exploits can get increased exposure, making it inherently unfair.
Another issue is that what a search engine wants you to see is different than what you want the search engine to give you, in some cases.
We want the union of two criteria; the results that give the search engine the most use/reuse(usefulness of the search) and the results that give the search engine the most financial recompense(so that the search engine can grow, get better, get faster, etc)
They may not be correlated, but they are both very important. The most useful pages may not give them the most money, and the pages that pay them the most may not generate enough repeat use for them either.
Perhaps the best search algorithm is two step:
Rank according to links (the more links to a page, the more useful the page)
Count repeat use (the more times a search has to be refined, the less useful the pages returned)
Rank according to links already occurs at Altavista and Google.
I don't know that anyone does the second.
Say you do a search on Google; if you hit the next button, then the pages that were generated get knocked a few points. If you hit Google again a few minutes later with a variant search, then knock a few points to *all* the pages that got listed in the previous search. If a user goes back, and hits 'related' pages, increase the points to that page, and all the related pages. Repeat the above algorithm for every hit to Google.
The nick is a joke! Really!
GPL Deconstructed
Having many-eyeballs thing could improve ranking algorithms. However, the more people know about the algorithm, the more complex the algorithm is going to have to be to defeat cheaters. It's a catch-22. Eventually the algorithms would be so complex that they'd have to render the page to determine the relevance of different elements and parse out sentences to determine if they're gibberish or not.
So what we need is to work on developing an open, complex, and nigh-uncheatable algorithm while search engines continue to use their own closed methods.
-- dR.fuZZo
Perfect example, wouldn't you say? IIRC, Google rates their sites based a good deal on how many other sites link to them. That is going to be non-trivial to hack
Nah, easy. You ever look into reigstering a domain name. Notice how there are all those bulk registration specials, like "only $9.95 each if you register 250 or more?" That's because there are a good many web companies out there with dozens or hundreds of domains. In the case of porn, maybe thousands. It's pretty easy to cross link everything so you look more popular than you are.
. . . "have we reached an 'accuracy limit' as far as keyword-based searching is concerned?" and he didn't really have a conclusive answer, but was able to suggest a places where I could buy foundation and how to apply it.
dunno i can think of at least 18 different ways
:)
ummm oh i see, as a keyword
Why not...:
Increase relevence points if a link is followed
Decrease relevence points if a link is ignored(next is hit, instead)
Decrease relevence points if a new search is defined(none of the prior search were sufficient)
Increase relevence points if 'similar pages' is followed
It should behave something like what you propose, without additional cookies, work, voting, or otherwise. Other than normal behavior!
The nick is a joke! Really!
GPL Deconstructed
If your trying to make a search engine that isn't easily exploitable how about identifying how current engines are exploited and design around that....
One way is to include the keyword many many times in a comment tag. A possible solution is to grade the keywords via their entropy with the words surrounding it. Such as testing for repeating patterns in a comment tag. Hell use the HTML tags to help you out by not grading anything in HTML comments. Another way is to do some syntactic analysis on the content of tags like and if they are not unique then they are only counted once. Certainly people with bad grammer skills and languages other than the language intended will suffer but you can add verification critera for each language you want to index. Before the trolls hunt me down and say this method is censorship (americans love to say anything is censorship) then you should just stick to the simple useless search engines we have now. Plus one can always implement this as a searching option to an existing broken wheel.
This isn't really that hard it just requires a bit cleverness and lots of prepratory work.
Or maybe I'm just on glue. :P
"Survival of the fittest Max, and we've got the fucking gun!" - Pi
I was reading these posts and started rolling around ideas in my head for scripting a little meta-search engine that learns from your searches and is customizable, when I realized that such programs already exist (sort of). Search Rocket is one of them (I know there are more). It searches all the big search engines at once, lets you filter and sort results and save your work in a nice xml file for later. I use this tool when I am doing in depth research and need to come back to the links I found later (but don't want to shuffle bookmarks for an hour). Still, it would be even cooler if it did some of the smart auto-filter type stuff mentioned above.
Oh, and just to be completely on topic, if the big guys change their search results, Search Rocket needs a quick patch. So in a way, they could prevent tools like this from working by changing the results a lot, but I would think they want to keep it standard due to cross licensing etc etc
Ok, enough rambling. Time to go home.
The biggest problems with Search Engines, is relevancy. The problem being that when I do a search for a word like "magic" the search engine will return results based upon its algorithm, but trying to produce relevancy from a single search word is just about impossible as a task. With a term like "magic" I could be looking for:
Or any of a large number of subjects that I could have in mind at the time of my search. The results from a search engine such as Google, will rank pages which contain the word magic in the page title, multiple times in the body of the page, in the META tags, in or near HREF links, or which are linked to by many other sites higher than those which do not meat these criteria. It differs from search engine to search engine, depending on criteria.
None of these criteria for ranking take into account the nature of my query - what I had in mind when I did the search. In other words they do not directly address the relevancy of the results. If a search engine offered me the opportunity to pick from results it returned and gradually refine the search to produce better results it would be addressing this situation. Some do with a "search again in this result set" or "more like this" type option on their results pages, but its still kinda mechanical, and not all that reliable.
I think it will take some sort of AI analysis of search requests based on user-feedback of some sort and with a learning capability to surpass the current crop of search engines. Until such time as we have some smart systems working behind the scenes on searching any improvements will no doubt be incremental rather than radical.
Now, as for keeping the specifics of how a page is ranked secret I think its absolutely necessary. There is a constant, quiet, war going on between the search engines and the folks who want to get their websites listed at the top of the page when a result set is produced. The people who regularly submit their sites to the various search engines, with each search engine receiving a specially made page generated just for its benefit to ensure that the website gets the best ranking possible etc, are not interested in how accurate the search engine is, they simply want to come up first. The folks at the search engine generally want the most relevant pages to be returned. There is an essential difference of purpose between the two camps.
On the side of the search engines, they have control over their ranking system, and change it peridically to prevent abuse of the system. The folks who are seriously trying to get to the top of the heap in the search engine results are constantly trying new methods to get ahead.
For instance, at one point some webmasters were creating their webpages with a lot of text at the bottom of the page that was the same font color as the background, so that the search engines would spider the contents of the page but users would never see those contents. This let them list all sorts of words that scored higher in the search engines returns, but had little or no relevancy to the page contents. The search engines got wise to this trick and now most will penalize you for using it.
Opening up the search engines ranking rules would only make the system easier to abuse more precisely. No matter how many eyeballs pour over the code, it will still not change the nature of the guy who will use any method at his disposal to get his porn page returned as Link #1 when you do a search for MP3 because its the hottest term currently being searched for.
Google has altered this battle somewhat by ranking pages higher in their results based on how many other webpages contain links to that page (and also based upon the nature of the linking page. They use a distinction between pages which contain a lot of links - like a web directory such as my own Omphalos - and those which are linked to by a lot of other pages. Both get points for different reasons and in different instances. I don't remember the details), but even this is open to abuse, although with a bit more effort required. I know of a website which has over 200 different URLs registered and operational, all of which contain pages which point back to the main URL they are promoting. When a search engine such as Google goes to anaylize this website, it will rank it higher because it is linked to by so many separate domains and so many separate pages on those domains. Its harder to abuse, but it can be done.
Of course, this is all basically irrelevant, since each of the search engine companies keeps their methodology and their source code highly protected. It is worth millions of dollars in revenue, and I cannot honestly see any of them deciding to release their software in this way.
If you have not noticed, practically every graduate student who devises a new and effective method of indexing and ranking search results ends up creating their own company once they have delivered their thesis and entered the real world. That is certainly how Google started, and I believe is also how Ask Jeeves got going. I am sure that most of the other main search engines have gotten going in the same or similiar manners.
All that said, If you want to play with a true search engine that is GPLed and works quite well, although not on the scale of a Google or an Altavista, try UDMSearch. It runs just fine under Linux or FreeBSD (I have installed it on both in the past) and I am using it on my site under Solaris. It is still in an intense development cycle and new versions are released regularly, but its worth exploring if you are interested in how a search engine works, and want to get your hands dirty.
For more information on the big boys, check out Search Engine Watch, and finally, if you are simply interested in Space, Space Exploration or Space Science, check out SpaceRef.
"The first time I got drunk, I got married. The second time I bought a chimpanzee, after that I stayed sober" Arian Seid
usually -fuck or -cum works for me. I mean, I may be reading something about "the male sex" or whatever, but I'm not usually reading something about the word fuck.
Also, -"credit card" and -"member"
usually filter out a lot of shit.
willis/
there is no thing
what else could you want?
Yahoo released a method for exploting the rankings on Google. It is known as the "strategic alliance exploit"
It's important to note that most search engines are just like every other internet buisness, they thrive on the number of hits they get, since most of them depend heavily on advertising dollars. How does a search engine get hits? By improving the results that queries produce. Almost all search engines are so bad anyway that when I find a better one, I'm almost instantly converted. (Hence I use google all the time)
The sorting mechanism used by the search engines is their way of creating a great search engine! It's relatively easy (given enough resources) to catalog web pages. The big problem is figuring out which one of those web pages somebody is looking for.
Asking a search engine to release it's sorting mechanism is like asking Big Bob to release his secret barbeque sauce recipe in the name of improving barbeque sauce around the world.
Mad Software: Rantings on Developing So
How does this Windows (I know, it is Windows!) software relate to this story question? Any opinions? :)
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
If the users could 'rank' sites as well, the the criteria of 'getting the best results' would be fulfilled pretty quickly, regardless of search-sorting.
If all the users get are Coca-cola sites, no matter how they search, and these are not relevent sites, then they will get devalued to the point of non-existence.
So any gross abuse gets moderated by the user population.
Thus any useless site, no matter how hard they try, will not be able to maintain a high rank.
The nick is a joke! Really!
GPL Deconstructed
Come visit the new MegaSearch©®(tm) Engine, the only engine to find exactly what you want, every time, GUARANTEED!
Features Include:
Never coming from a startup
IBM had PL/1, with syntax worse than JOSS,
And everywhere the language went, it was a total loss...
Pages I'm looking for are often not designed to exploit search engines anyway. If I'm looking for some technical documentation, it is often just stuck together by someone in a rush, or on a mailing list archive. These pages often don't have optimised META tags, etc, so some engines don't index them well.
Many people are suggesting that making the rules open is good because then everyone will exploit them equally. Perhaps, but that will just make it more difficult to find pages that don't try and get a highly rated search result.
On the other hand, I find Google works pretty well, and how it works is pretty well known.
This isn't security as much as it is in the same argument base...
The arguments against "Security by obscurity" apply here.. so just insert those arguments [here] and I'll move on...
It works not by prevention so much as "reduced body count" and I guess thats the best a search engen can hope for.
When someone thwarts security thats it.. your dead...
When someone tricks a search to give them top results it's just a few websites.. it CAN be overlooked.
So say... 1 person hacks AltaVista.. it's down... blah.. 100 persons hack AltaVista.. it's still down... 1 cracker vs 1,000 crackers... makes very little diffrence... it only takes one defect and one joker to ruin your day...
But with searches... a defect becomes known and you don't fix it in time... 1,000 jokers and your screwed...
1 joker however isn't a problem.... your still online and USUALLY you still give good results... just one bad result...
You get bad results by random chance and user mistakes... so big deal...
But your expecting the joker.. once he's discovered this little trick... won't make it public....
Right now this dosn't happen...
But it's a lot to risk...
Recomendation.... sence obscurity is effective... but not perfict... give away the OLD system...
Provide a liccens that basicly says "Any changes may be used by us at any time with out notice... but only we may do this... all else is open source"
I don't actually exist.
A lot of the "preferred" listings on search engines are paid for - that is the company who is at the end of a link is paying a certain amount per click through. GoTo, and I think Yahoo and About provide preferred search results to many sites such as Netscape search and AOL. So essentially, to get yourself to the top pay more $$$. Now this is not a necessarily a horrific practice if you compare it to, say, how the yellow pages works. If you are looking for a plumber, chances are you are going to be drawn to the biggest ad on the page. And how does the plumber afford the biggest ad? - He must have a lot of business - which may infer that he has lots of happy customers (ok I know, I prefer the mom and pop businesses, but we are talking about the masses here). So I suppose relevance is determined by who can afford the most for advertising, and who can afford the most advertising is doing the most business, and they are doing the most business because they are obviously the best at what they do - ahhh, capitalism ladies and gentlemen...
We have not reached the accuracy limit. A search engine should be able to read my mind and infer the best sites to go to.
No, Thursday's out. How about never - is never good for you?
Aren't search engines supposed to remove the bad link when a visitor click on the broken link? Let's say I was using Google, and it gave me a link that I want to click on. I click on it and it gives me a 404 error page/missing page message. I think search engines should remove that particular link when I found this.
:)
Would that help a lot with these broken links? What do you think? Is this possible?
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
In algorithm design, we often think about the input to our algorithm as being designed by an "adversary" -- a really smart entity which knows our algorithm and aims to defeat it. This is good, because it gives us robust algorithms and data structures which perform well in the worst case.
The solution often involves randomness. If search engines built randomness into their weighting criteria, an adversary with the algorithm would still not be able to influence the random (or "pseudo-random") aspect of the weighting.
Other things beyond the control of the adversary are usable too (some people have mentioned excellent ideas such as external links, voting, etc.).
Even "good" search methods have embedded social values. For example, Google's backlinking methodology tends to reinforce traditional power structures since heavily commercial sites tend to link to each other a lot.
Search engines are in the business of controlling what you become aware of. There are lots of things that become interesting just because lots of other people are also aware of it (e.g., Survivor, Big Brother, etc.).
Search Engines don't really try to maximize relevancy; they try to be relevant enough so that you don't leave. That's why Yahoo uses google search results as a placeholder, but that's just to create more space to promote its own stuff.
Proprietary SE techniques are a bad thing(tm) from the perspective they obscure the embedded social values in their design.
Yes, this is lots of random thoughts but I think this is an important topic.
Hey democracy lovers, add Quorum as a c
You can't view Profusion with the Linux version of Netscape (V4.74). Well, you can VIEW it but you can't do anything after that because it messes up the browser (Linux Netscape users are used to this on sites with amateurish HTML). I'll be intrigued when they take pride in their work and hire authors that can write browser-nonspecific HTML. For now I'll just yawn and use the ever more annoying google.
I like using Oingo. Honestly the hits it turns up are not always great and it is tricky to use but it is not a "stupid" search engine. It attempts some level of ontologizing (ontologization?)... and its about time someone tried that. There are many times when I can't get a decent hit via one of the old fashioned search engines like google or AV but Oingo will find plenty, mostly for searches that require more than just "keywords"... where I'm searching for a related concept or idea and not just a "token".
We know that telling people what the rules are will get them to change their behavior. Game theory tells us that the set of possible equilibria may change. In short, it is not necessarily better to tell everyone what the rules are, and try to iterate to some new steady state, in which we all try to exploit, and "it all balances out".
This really is not related to the security-by-obsecurity issue, I think. It involves security in the sense of "keep your passwords secret", not in the sense of "build a system with no bugs", which seems to be where security-by-obscurity fails. The issue here is that rules which people could know about and not be able to exploit might well be significantly inferior to rules which are exploitable.
I suppose that someone who wanted to find out an answer to this could try to get a grant to set up a search engine with a public rating system, and see what came of it. If we can come up with a reasonable metric for the signal-to-noise ratio which resulted from searches, we could find out what really works. By the way, I suspect that one problem with such a proposal would be that no one will bother gaming the system unless it is REALLY popular, and delivers loads of hits. That is, I don't think that this could be done on a small scale: go Google-size, or you won't get any data that applies to the big engines. But you could find out how alternative rules work, with a site that was below the radar screens of the home shopping network and the porn queens.
I have a question for you CS grad students: is there any academic literature on this issue, looking at how people react to this sort of structure, and how the structure must be designed to get them to react in a way which doesn't screw things up? This seems to relate to the information theory field of economics.
See what I've been reading.
GoTo and some other sites already do this.
.02 anyway
Check out the "cost to advertiser" link under the top 'N' results on GoTo.com...
usually it is just a couple of cents, but I've seen some site pay more than a US$ dollar for your click-through.
Makes me wonder if they knew what they were doing when they submitted their bid...
hey--what's the difference between 2 and
---
Interested in the Colorado Lottery?
Interested in the Colorado Lottery or Powerball games?
check out http://colotto.com
The primary reason why it sucks is exactly that it doesn't have moderation... :-) They have a problem that in some categories lots of spammers sign up only for self promotion, and their response is to reject 90% or so of those signing up. Instead, what they should do is to make sure that no individual has too much power, including the meta editors (who are not always awfully clued).
The reason why it never gets corrected is that writing a comment like I do now is considered "illoyal" to the directory.
I have a bunch of ideas for a better web directory. In the meantime, I'm thinking about a classification for skeptical resources.
Employee of Inrupt, Project Release Manager and Community Manager for Solid
Actually, I know a bunch of research librarians who have been snapped up by a new company out on Rt128 which sells their services for doing web hunts....
----------------------------------------------
-*- Any technology indistinguishable from magic is insufficiently advanced -*-
If your sites only link to each other, and there are few or no links to them from the outside, then they form an "island". This shouldn't be that hard to detect. Further, if none of the authoritative sites link into the island the credibility of the cross-links isn't high. The Google method isn't quite as easy to spoof as it might sound; I've never been directed to a pr0n site from a Google search.
--
Build a man a fire, and he's warm for a day.
Time is Nature's way of keeping everything from happening at once... the bitch.
Guess you didn't read what I wrote, eh?
;)
Try searching for 'babelfish translator' with PHRASE mode, and it comes up as number one.