Slashdot Mirror
Search Engine Results Relatively Fair
perkr writes "The Economist and PhysicsWeb report on a study from Indiana
University claiming that search engines have an egalitarian effect
that gives new pages a greater chance to be discovered, compared to
what would be the case in the absence of search engines. Based on an
analysis of Web traffic and topology, this result contradicts the
widely held 'Googlearchy' hypothesis according to which search engines
amplify the rich-get-richer dynamics of the Web."
First of all any time you want to analyze Google, you have to realize that they've had ten PhDs crunching the problem already for years. Google is designed to give the best results for whatever its users are searching, thus any apparent bent towards egalitarianism, monopolism, antidisestablishmentarianism, or what-have-you, is purely incidental.
If you're searching for something obscure, Google will instantly tell you the one startup company building it. On the other hand, if you want something mainstream, they'll give you a prioritized list of the best sources. There's no alterior motive it seems - they just give you what you searched for... imagine that! I've seen a business through from obscure geek hack to the mainstream consumer, and Google has been there at every step of the way, working exactly as users expect. To accuse them of favoring any particular stratum of that chain is awfully unfouned IMHO unless there are some specific examples. Indeed, answering users' needs instead of pandering to the status quo seems to be he most valuable bit of what google does.
I've had worldwidewingtour.com live for about 3 days and I have a good google ranking. Even a search like "hooters wing tour" places me at number 7 on google.
Those who know, do not speak. Those who speak, do not know. ~Lao Tzu
It's all in the rhythm of the algorithm.
I've established a number of websites primarily for small groups of users, and every one of them has been ranked, even one set up for a friend strictly to put up family pics for his brother to see. If it's out there it's googlable. And no, I don't care if it's not a word :)
Check out the google results for "rebate lawsuit".
As of this writing, they go to "some guy's blog", namely mine. No links to it that I know of, either, which is sorta weird.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
google "ranks" on a 0-10 scale. you have a 0. anything under 4 is not a "good ranking". whatever searches you're getting good search placement on (which is different thank pagerank.. PR is only one of a few factor in a search) must not be highly contested searchs heh.
It makes no attempt to filter spam, which like email will soon account for about 80% of content.
Try this search for Tartfuel http://www.google.co.uk/search?q=tartfuel once a local band. When Google claims to have 28,600 results, in fact there are only 36. Now that's a con. When I give search advice look through all the results and they look me to say "but there's millions". Never, if you're doing a specific search, Google won't even display a tenth page (which is the max).
So, of the 36 results, how many are real? There are 10 relating to the band, one about alcopops, and the other 20+ are spam. Their old website and every subdomain is of the generic spam-generated marketing page this is about 10, and theothers are spammed guestbooks linking to them.
http://classic-motor-bikes.tartfuel.co.uk/
I'm not interested in Google until it can search through time. Damn, I wish the net archive had done a better job. Most of the content is of popular, mainstream commercial sites, which are so unoriginal, it's off little interest. And how many images did it save? Not enough.
Mod (-5) Google bashing. No. This applies to all the search engines.
Comment removed based on user account deletion
...is how to get a number one spot in the natural listings. Sigh.
I used to have a better sig but it broke.
I've made sites with fairly mainstream content before, which were totally ignored by google. But then, I put an article on my blog about the history of a certain group of elite English schools in Taiwan. Previously, this information had not been on the internet anywhere. Now, if you type the name of the original school of that group (Modawei) into google, my article comes up #1.
I'm a gnu world man.
Google only updates their publicly visible Pagerank data every couple weeks or so, to make it more difficult to game the system. New pages will show up as PR 0 until the next public update, but (of course) Google updates their private Pagerank database more often. That being said, WTF?! World Wide Wing Tour?!
Then, when I search for "digital voice recorder", the first 2,000,000 results will be lame ass "coming soon" pages, or pages that suck so much nobody has ever felt like linking to them. Usefull sites, like this one would be near the bottom so as to ensure that "the rich are robbed".
Hell, while we're at it, why not make roads that way too! Let's rob the "population rich" metropolitan areas and focus our road building on the isolated rocky passes passes which have been deprived of people and infrastructure for far too long.
Maybe we could do elections that way too... oh wait. That actually could be an improvement.
I'm a gnu world man.
Most web newbies would form their impressions of the web from their ISP's portal site. That would give a lot of power to corps like AOL, who for a long time tried to persuade their subscribers that there was no web outside of AOL hosted content.
There might still be blogs and social networking sites, but the take up would be slowed since fewer people wold have heard of them, and both might have failed to ignite into the movement we see today.
Which would probably mean that if you wanted something outside of the main ISP channels, you'd be reduced to digging through the spam on USENET to find it.
Google as an egalitarian influence on the web? I think it's a bit of a no-brainer, personally.
Don't let THEM immanentize the Eschaton!
There is another paper out of UCLA that is similar to this one except with somewhat opposing results. In which, the authors show analytically that the rich-get-richer phenomenon does exist. http://oak.cs.ucla.edu/~cho/papers/cho-bias.pdf
It seems tough to reconcile these two sets of findings, and this new paper even makes mention of this:
"The connection between the popularity of a page and its acquisition of new links has led to the well-known rich-get-richer growth paradigm that explains many of the observed topological features of the Web. The present findings, however, show that several non-linear mechanisms involving search engine algorithms and user behavior regulate the popularity of pages. This calls for a new theoretical framework that considers more of the various behavioral and semantic issues that shape the evolution of the Web. How such a framework may yield coherent models that still agree with the Web's observed topological properties is a difficult and important theoretical
challenge."
Big companies can pay more for advertising, so if you bury them in the listings they pay to appear at the top of the adverts. It's good solid business sense.
It's also not Egalitarian because Egalitarianism assumes all people are equal, so company of 100,000 employees is 10,000 more important than a company of 10 employees.
It's more like positive discrimination, you discriminate against big companies for your own benefit and pretend its for some greater moral purpose.
In the same way positive discrimination for blacks is really just negative discrimination against whites, spun to sound like a good thing.
As is the case with many things the truth is somewhere in between these two. While I will quite commonly end up visiting an obscure site through google because it has high relevance, the larger sites will almost certainly be listed alongside these results.
For instance slashdot is highly ranked and grows because it has high relevance to a wide selection of technical topics and is also linked from a large number of sites because it is well known.
I have never let my schooling interfere with my education. - Mark Twain
I'm quite certain that new domains get a boost by Google for a little while and then a push down. Or the other way around. Google does something special with new websites (or possibly only domains) anyway. I wish I could be more verbose, but I'm afraid I've forgotten all I read about it, and I'm too lazy to search for decent SEO websites.
Well duh! Otherwise you'd have to browse. and browse. and browse some more, hoping to find a site with the info you wanted. And you'd probably only know about sites that had a big budget to advertise. Search engines are inevitable - some bright spark is always going to realise that there must be a better way to automate the process by having a computer browse for you, so you can ask it later if it found anything on your topic.
Drag n' Drop DVD Recommendations
I didn't even have to read through all the details of the study to see how it was bunk. I was quite suspicious as to how someone could conduct a small study to determine this, especially considering the extremely large amount of sampling and data analysis require to do such a study. And then, it would rely on a bunch of assumptions about your relatively uknown and non-established testing procedures being accurate. Of course, reading the study it appears they have used the notoriously unreliable Alexa rankings and google's also equally unreliable "link:" feature to gauge popularity (the backlink feature returns a tiny subset of pages that link to it).
I'm sorry, but this study doesn't even attempt to index a large set of data itself, it relies on existing crappy search engine data to validate that those search engines are OMG SO GREAT AND EGALITARIAN. Hello? You can't use search engine data, especially ones known to be unreliable, to confirm shit about them.
Even google admits that phenomenon such as Googlebombing and similar phenomenon is a serious problem and yet, these idiots, with their shitty meausures, think they know better than the people who run the damn search engine.
Search Engines give new pages a greater chance to be discovered
This just in - Yellow Pages give new businesses a greater chance to be
discovered.
Some kind of cypher communication, using /. as a way to exchange data pretty anonymously? It's certainly interesting to observe that each block is repeated once, the ones between the "HELLO WORLD" maybe identifying a key, though I don't believe in a complex scheme (I'd bet you can en- & decode this thing without using a sheet a paper even), probably just subtract them. Any crypto-gurus around, I'm not interested in wasting my time on this one? Could be just trolling :-P
Here's how: the wealthy get to decide who receives their spending, and those people in turn decide how strongly to weight their suppliers' votes in the allocation of resources. This perpetuates through in a cycle that reaches a very rough, shifting equilibrium that very much resembles Google's "pagerank", IMO.
Compared with outright hierarchy, this kind of inequality is still going to appear relatively fair, but it doesn't measure up to equally weighted votes. That is, it isn't democratically fair. However, this, or at least some inequality appears to be essential to making useful discrimination, if you're going to use the "intelligence" of the web itself to do it. Ideally, the results would be based upon the quality of the content itself, no matter how obscure, but the artificial intelligence required to do that would be mind-boggling.
Besides, people often want to find something that they were surfing the other day (ie. relatively more likely to be strongly linked), or else read up on what others are talking about, so that they need the same points of reference... An objectively better site might actually be inferior for socialising with one's peers, or engaging in political tribal virtual warfare: a third point of reference in such cases leaves you out of the discussion!
Wikileaks, no DNS
Blogs are an example of Google's support for new content. It's excellent indexing of blogs supported the popularity of the concept. And as a consequence of the importance give to new content, many keywords (esp technical) list blogs right on top.
Supporting new content is essential for the growth of the web. A web NOT weaved around high profile websites, built by media monsters (CNN, BBC...). The new web is about independent content, free thought and free speech. Yours and mine.
Life is just a conviction.
The bit this ignores is that it's up to the search engines to decide who gets chosen. So for example, the google sandbox penalises most new sites for up to 18 months. If the best answer to your question is on a new site, sorry, but google probably won't find it. Whatever rules they make up determines what happens.
Read reviews of shopping cart software
It looks more like something which is trying to look like a cypher. The repeated blocks are redundant, nothing more.
Perhaps it is a social experiment, designed to draw out paranoid theories.
(Accepts pat on tinfoil hatted head)
http://michaelsmith.id.au
...egalitarian communes breed dissidents!
If you RTFA you'll notice some of the arguments against it.
But beyond that, common sense alone tells you winner takes all, and it continues to be that way, with google or with anyone else.
The entire pageranking algorythm is there to point you to the most likely result you're looking for. They base that on popularity, number of links coming in, and the importance of the referring sites linking you. The net effect is, the more popular you are, the more relevent you become and the higher ranked you are.
Also, when you type in say "windows" Google automatically assumes you're talking about the Windows OS. What if you were looking for real windows? The search engines are always assumming based on popular demand. This steers people's thoughts and pushes them in a non-neutral direction. As a word's context changes to favor a certain direction, search engines rank that as more relevent, which leads to it being more favorable, etc. Cycle repeats.
eTrade SUCKS
google cannot be completly fair since people are aware of its methods and can design ways to negate them. i can't see how this test was carried out scientifically. the test should not be whether the results are mainstream or not, it should be whether the results are relevent. it appears that economically poor websites have as much ability to fix search results as rich ones do. great. the information content relevence is scored, not by an independent expert, but by the number of other websites that link to it. what we have here is a problem of the popular becoming more popular. not necessarily because of relevence, but because if a webdesigner wants to find an address to link to in their web page, more often than not they're gonna search for it using something like google. __________________________________________________ _______
don't lick the end of a live USB cable even if it looks yum
Of course. Doesn't anyone remember what the web was like before search engines became popular, when the main way to find pages was by following links there from other pages? If you could get someone to link to your page who in turn was listed prominently on the Humor, Jokes, and Fun page on akebono, then you were all set, but otherwise, it would take *months* for anyone to find out about your page, if they ever did.
Don't even bother replying to this unless you know the significance of akebono.
Cut that out, or I will ship you to Norilsk in a box.
Search for Ikioi on Google, MSN, Yahoo, take your pick. (Not yet an entry in H2G2.) I'm obscure and #1! Hmm, not sure if that was a fair trade off...
I think there is a fairly straight forward relationship between rating and specialization, and it has everything to do with competition. And, obscurity is the best way to avoid competition. For instance, the top results are still funny for something so utterly obscure as "French Military Victories".
I8-D
The end goal is to be able to return the single 1 page that a user wants that contains ALLLLL the information possible for him/her. Too bad for us these pages dont exist. So the logical thing to do is return all the pages that would satisfy this need, and just those pages - no more, no less. But what google attempts to do is, return ALL pages that it determines as relevent and rank them. This is why we get queries of 100,000+ for somewhat broad terms. It sucks and is a crappy way to do searching. Then on top of that their page rank algorithm is pretty much popularity based and you get even worse results. Eventually there will be a search engine that can determine a fair ranking of pages without the need to check its popularity on the net.
After surfing the web,... I can now safely say:
The name of a Hawaiian sumo wrestler after which the server that first hosted Yahoo! was named after.
as long as I get what I'm looking for.
If anybody has used Freenet, then perhaps they've already encountered some interesting bookmarked pages. Then again, if I'm not mistaken, most of the content there is in Freenet, is most likely some sord of eye-twitching pr0n.
Then again, this quote from their FAQ is interesting:
Is Freenet searchable?
No search mechanism has yet been implemented. One of the design goals was to make it impossible to locate the exact place where any piece of information is stored. Even a server operator cannot determine what is stored at his own node. This naturally makes searching very difficult. Information is currently retrieved by "keys" which should be guessable, or communicated by some other means.
How about give sites with correct XHTML that follows webstandards a higher rank.
And penalize websites with popups, JavaScript, flash, shockwave, ActiveX, etc and get them a lower rank?
Scrapers are all those pages, that simply crawl SE results with expensive keywords, then put them onto a page with ads (adsense many times).
They would even include a search, and poll via yahoo/google api, html strip YOUR pages and present it without backlinks to YOUR site.
With the recent Bourbon Dance (recent reindex/algo change is called a Dance in SEO world) it seems that some of these are gone.
Sad is that your site can be penalized for dupe content, and it happened to me multiple times. That means from yahoo number one to the dustbin on sometimes ALL engines. That can effect a small business in a devastating way.
Sorry for the SEO talk, but I am sure some guys are into it too here, so for those that is no news, while for others it might explain what these crap collections are.
Want to find some? Enter your domain name as a search phrase. Alternatively put a very rare phrase on a page that has widely searched keywords, and try this phrase in those sites' search box, you will PUKE I am almost sure.
Than the flame begins, about "relying on search engines as an income". For your information, lots of small businesses cannot afford to bid anymore, at least not in the competitive areas. Weight loss, debt consolidation clicks can cost around $10 per click, and if you think that's all scam business, try PDA accessories or laptop computers, I am almost sure you will end up paying around $8 for a US visitor.
Now then you say "following guidelines" content is KING so you spend money or time (do yourself or pay someone) to find yourself on the first page, and then scrapers find you in those results, and you get penalized for dupe content.
Devilish cycle, and at the end ad services make a double buck:
1. ads on scraper pages
2. inflated PPC (pay-per-click) costs
It seems more and more that the internet turns into television: small businesses cannot affors crap, unless they sell a niche service/product. But how much can you really make on obscure things no one searches for or looks for?
Now combine that with price wars (read an affiliate/SEO forum) and you see prices going down, costs up, and again at the end only BIG_COMPANY makes a real buck.
Now you wonder why your mail box is getting more and more full with spam -> yes some of those are advertisers who turn to this (crappy method) as they gave up on SE results or PPC. I am not saying it is legit (I hate spam) but I know for a fact that some started spamming because they gave anything else up.
Just for fun, sometimes I'd like to see what the 3 millionth ranked page on The Beatles looks like or other very low ranked pages. Anyone know how to do this?
I use it all the time and I'm constantly frustrated by clicking on an image or link and then getting a page that tells me I don't have permission to access it. Can they could fix the search engine so such images are excluded? That's one area in which all search engines seem to be lacking, so far as I know.
I dream of a better world... one in which chickens can cross roads without their motives being questioned.
I am glad to see so many people here who dont' think Google is the wonderful business that the main stream media make it out to be.
Unless we start doing something along the lines of the pressures that have been put on M$ for the last 10 years, and KEPT-UP until things change, we will have the same lousy state of affairs when it comes to this important issue of searching the InterNet.
A product is NOT FREE when you have to spend countless hours and sometimes days to figure out the most basic things about a subject you NEEEED to research.
Google has failed MISERABALY in this-- even with some of the most basic questions being asked of it.
And now that they are worth $400 per share and the Monstrous-Military-Industrial-Media-Complex has given its blessing to this sad excuse of a business, where is the motivation for it to improve to the point where it will be a truely viable search tool?
Obviously a OSS/GNU type business format will need to take this task on, as it has/is in other important areas for mankind.
Waiting for something 'to happen' is not going to improve this dangerous situation from spiraling down further.
Massive lawsuits and the taking back of any patents and copywrited material owned by companies like Google & Microsoft, by the use of flagrent and vicious 'so-called' violations by the masses (us) are the ONLY sure ways of removing these hand full of criminals who have hijacked our world.
--SlashDots Moderation systems is NOT broke. It is 'Fixed'.
I will gladly loose all of life's battles.. in order to win the war..
Word of mouth gives new businesses the best chance to be discovered.
Yellow Pages only give existing businesses a greater chance to be discovered; new businesses cannot afford large ads and only get one line, no-charge listings. Also, yellow page ad salespeople are the most offensive telemarketers.