Slashdot Mirror
Search Engines Leech Value from Web Sites
bigenchilada writes "Jakob Nielsen, former Sun Distinguished Engineer and now usability guru, proposes "that search engines are sucking out too much of the Web's value, acting as leeches on companies that create the very source materials the search engines index."
He says that the value provided by search engines may be tilting too much in favor of the search engines. The web sites that create content are now simply fodder for the search engines' revenue stream."
This "tilt" intrigues. It's interesting in that it describes an unfolding and evolving business model to which companies must react.
I like that he doesn't just whine about the problem but offers solutions too, to provide the "stickiness" required to keep customers coming and interacting with companies' sites. This is the way the evollution should work.
Oh, that the RIAA and the music industry would have to abide by the same principle now that their business model has changed, rather than buying legislation to cripple advances in technology (which, btw, will NOT work).
Maybe, maybe, the music industry could learn something from this.
This is likely why Google and Yahoo are offering monetization options for content publishers (and creators). Plus, if you don't like search engines "leeching" from you, just set up robots.txt and say no to everything -- they'll go away.
I find that search engines account for nearly 70% of my visitors overall, and account for nearly 60% of my return visitors. I don't believe I can rely on my websites to generate income for me (even if I start selling more products on some sites). As I don't copyright any of my text (I am anti-copyright and put all my creations into the public domain immediately), I use my writings to try to increase my income in my regular life -- speaking engagements, one-on-one consulting, and professional advice to companies and individuals in the markets that I'm valuable in.
Nielsen is nuts if he thinks that the web should scoff at search engines. Search engines are (to me) the biggest reason for the web's overall explosion. Bookmarks help, links from other sites are great, but Google, MSN and Yahoo are the big reasons people can find what we want when we want. If they can't index our sites, how can they send us traffic? Sure, he acknowledges this in his article, but he says that web sites are going from information stores to answer engines. This is completely true, and we all fall victim to our own stupidity when it comes to creating content in an "answer" fashion. I've been working over the past few months to try to create extended interest in my most popular pages (found via search engines) by offering crosslinks to other articles. The longer I can keep the people interested, the more likely I am to see them come back again and again. If you make old "answer" pages, offer links out of those pages that give people MORE information, or give them more questions to find answers to.
Content is worthless without distribution. I prefer face-to-face distribution for profit by using more generic information to "catch" the customer who will hire me. Yet without the search engines, how will I get the word out? Hire a publicist?
Slightly off-topic here:
I think its crazy to put quality profitable information on a website (or even in a book, on a CD or in a movie) that you don't want used by others. Copyright may "protect" you from someone knocking it off in high quantity, but that isn't always where information is the most valuable. Using information in an expert situation is how you can turn quality profitable information into that quality profit -- by selling your advice on a person-to-person level (I call it a performance).
... then learn to use robots.txt. Simple really.
__Laugh daily funny adult videos
I'm sorry. Search engines are to the web site's benefit as well... at least to commercial sites.... well research sites too. Let's face the reality. There cannot be a way to "balance" the benefit. You either do or don't benefit -- it's an on or off situation. If you don't benefit, there's "robots.txt" right? Whiners!
The question, then, is how much will the growth of Wikipedia negativly effect Google? I know I've started doing Wikipedia searches for things I would have Googled for before.
I read this article when it went high on the del.icio.us/popular list. Long story short: this guy is complaining about *advertising* links in a search engine. Then he goes and compares a bunch of apples to oranges and concludes the sky is falling (yes, I meant to mix metaphors, as this is what this guy does in his complaint).
If you look at his analysis, he is coming from this from a perspective that most of the Internet can't really related to: a business to business commerce site that uses no advertising revenue and pays a high "click-through" cost for each visitor from a search engine.
After all of those constraints are in place, he further comes up with the idea that by making $4 per visitor (after COGS and conversion rates) "the site can pay $3.99 per click". Well, I guess if you really are hellbent on giving your profits away you could...
He tries to justify this by saying that "if you don't pay this, other sites can outbid you". He justifies this by saying that others will use his sites methods to improve conversion rates and therefore they will outbid you with the increased revenue. Well, maybe, or maybe they will keep some of the profit.
This commentary is not applicable to those with advertising supported models, nor those who are willing to differentiate themselves by more than hyper-competition in search engine optimization. Which means pretty much most web sites are *not* going to see the results that are predicted here. The ones that *will* see this are those that don't have a differentiator and live and die by the converted sale. I think I will cry now... [sniff]... poor toner refill sites.
His solution: #1, spam the user. #2, notification spam. #4, multi level marketing.
Sig under construction since 1998.
Tell it to the florist i know who registered 18 different domain names and put up six different websites for his 1 business and stuffed them silly with keywords.
It's total and utter bullflop, and it works. And we hate him for it . . .
This is just like television, only you can see much further.
The problem is that copying stuff from my website is too easy. We need stronger copyright laws.
No, wait -- we need *weaker* copyright laws because then I can use anything I find on my site.
Er, no, wait -- we need *stronger* copyright laws because big-money search engines are destroying the value of the little guys.
No, no, no, wait -- we need *weaker* copyright laws because then I can download movies legally.
Ah. I've got it. We need *weak* or *zero* copyright laws for me, and *strong* copyright laws for everybody besides me.
...Free. I don't pay them to index my sites, and they send me potential customers. Somewhere around 80% of my traffic is related to search engines. Sure, they're getting money from advertising other sites that may compete with you, and they don't produce content of their own. But at least I don't have to pay 400 dollars for a page in the yellow book and reach a whole city of customers when I can have my site indexed and reach a whole world. (unless they don't speak english, then they can't get much value from my site)
The author's point is, in a nutshell, that web business are reducible to the cost of their traffic and the revenue it can generate.
His example is something like this: 100 users with a 1% conversion rate for a $4 net profit means you pay $3.99 to the Search Engine for traffic to make 1 cent. Since the search engines are effectively a traffic auction, you always pay exactly as much as your competitors are willing to pay plus a small amount...
I find fault with this argument, because search engines are not a traffic auction, exactly. Google sells adwords but it primarily gives users what they ask for, not what others pay for. Still this is the reality and the mindset of many online businesses, if there are 10,000 other companies like yours, you can only be seen by buying traffic.
His concern is that the search engines' position is too strong - they're the bottleneck, and they price like it. They've created a market where they take most of the profits from any online enterprise. If web businesses find a way to increase margins then it instantly translates into increased search engine fees rather than increased profits, and google earns it by sitting back and "doing nothing."
Of course, they do something, but just like Sony and Tower records, their indispensability may have been converted into a disproportionate amount of the profits of global enterprise.
From 20,000 feet, thinking in a way we seldom do anymore, we could consider alternate regulatory regimes that might tinker with the market. For instance, if you accept that this state of affairs may not be optimal (a few megagiants and millions of small businesses beholden to them), you could flatten it by reinterpreting things like copyright, so that the search engine is not entitled to list anything without splitting a cut of the profits of that enterprise with the content creators.
I'm not actually suggesting this, just trying to seed discussion. One thing that this vaguely reminds me of is the Neal Stephenson concept of the free-market encyclopedia, where anyone can write anything and upload it into the system, and then you get paid, more or less directly, for traffic... presumably by redistribution of fees paid into the system to view content. It's appealing in the way it incentivizes creation of content, especially in such an egalitarian way.
We've got an all-you-can eat model where you pay for access and others pay to publish and writers can pay the rent with advertising or subscriptions... and of course, we have a free market for search services... I like it well enough, but I do sympathize with content creators, who still seem to struggle to realize the value of their intellectual works.
Tired of Political Trolls? Opt Out!
The search engine benefits from the ad revenue; the sites benefit from the increase in visitors. Both sides win.
"He says that the value provided by search engines may be tilting too much in favor of the search engines. The web sites that create content are now simply fodder for the search engines' revenue stream."
Yes, and this is exactly why everyone I know in the e-commerce business spends an exorbitant amount of time trying to figure out how to prevent their site from becoming "fodder" for Google's revenue stream. Because, of course, Google brings absolutely no value what-so-ever and and does nothing to drive traffic.
Riiiiiight....
No wonder Sun has so many problems. They used to be the "dot in dot com"
until IBM took over that in 2000 or so*.
The value of the web is priceless (and free!), how could a search engine
to find stuff on it decrease that value?
This is the one of the most silly things I have read since Taco said the
iPod was lame.
TFA says, "We've known since AltaVista's launch in 1995 that search is
one of the Web's most important services."
Then, "There's no doubt that search engines provide a valuable
service to users. The issue here is what search engines do to
the companies they feed on -- the companies that fund the creation
of original information. Search engines mainly build their business on
other websites' content. The traditional analysis has been that search
engines amply return the favor by directing traffic to these sites."
I use Google for everything. I never type in a blind url, because I make
mistakes from time to time and get some typosquatter or other troll.
What I do is go to the Google box next to the url box and type something
like "barns an noble". Notice I mispelled the name, but even with my
error, the first linke was "Barnes & Noble", which is what I was
looking for. (DNS is already dead because of this, Google is the dot in
dot eveything).
I would have no idea that the url would have been barnesandnoble.com.
Many users don't know that a & is not a valid url character, and
they would mistakently put it in the url if they were to blindly type
it. Most web browsers would give a worthless error message like "The
specified server could not be found." Thanks. I used to run a web proxy,
I've seen everything in the world typed into the URL spot.
What Google and other search engines have done is flood the market with
worthless, fly by night companies. Stable ones have no issues. Search
for Apple computers, or Oracle database, you get useful links. Search
for a commodity item you can get anywhere for the same price, and you
get every sleezeball in the world trying to get your $25, when you could
also just have walked to the store and got it for $30, and played with
it that day.
Niche items are different. I can find them via Google. I have a nice
whip cream dispenser that had the rubber grommet freeze because I was
making so much whipped cream (right). I paid something like $40 for it.
I wanted a new grommet for it, and I used Google to find a store in
Pennsylvania that sold me the grommet for something like $2. I ordered 2
boxes of whipped cream to offset the shipping, and in less than a week I
was back to making whipped cream!
I could have never, ever have found a $2 part without Google.
Google is a monopoly for a reason. Why would you need more than one
place to find all of the questions you have?
* Sun boxes used to run the root DNS servers, and that is were they got
the idea that they were the "dot in dot com". IBM took over that role in
2000 or so, I'm not sure if they still have it or not.
I find that google blows away most commercial sites for finding content on those sites. I use google when looking for products on web store sites. In that respect at least, google is doing them a favor. I'm finding products I want and buying them because Google is there; many of the sites have crappy or nonexistant searching capabilities. Heck, I've tested some; I can be looking at a product I found on their site, and I can type in the words that are in the name of the product into the site's search, and it will tell me there's no such product. Ridiculous.
After I RTFA, I basically got one thing out of it. The writer was complaining that unless you keep working at a website you made, you are going to not earn as much money.
I've got news for him - you can't expect to earn money out of nothing. I know some people manage to do it, and good for them. But its not something you can expect to do. If you don't work on improving your site, and others do, its not the search engine companies' fault that people will be more interested in their work and so they can afford more on advertising.
I do see his point - the search engines will get paid more because your work improves, and other's work improves. But this is not something that is unique to search engines. It is part of advertising in general. The larger a company gets, the more it can put into advertising, which means the competing companies need to keep up with them and put more into advertising themselves. It works in a bit of a different way, but its the same concept.
It doesn't matter what advertising it is - TV, Radio, Newspaper, Search Engine - the way the companies make their money is the same. Google, microsoft, yahoo, etc. are not doing anything new here, they are just brining proven concepts to a new medium. Why do we critisize them for it? If anything we should be critisizing the people who have drummed it into many a programmer that once you've written something, you shuoldn't need to maintain it.
Next he claims that just when you are making money after you hyped your article with a massive advertising campaign your competitor will do the same because he doesn't want to loose forcing you to run an ad campaign again.
Wow. How odd. Lucky nothing like that happens in the real world.
Coca Cola has just the one ad campaign in the late 1900's (or is that 1800's) and has been coasting ever since.
OF COURSE NOT Gee whiz. News flash, if you sell your product through paid advertising then you got to keep paying to advertise. More and more and more and every time there will be some new upstart who runs an ad campaign for a similar product forcing you to do it again.
What the fuck do search engines got to do with it? This is just plain old advertising.
No this fucktard has just learned that pay-per-click advertising has better statistics (you can actually tie the ad to the sale) and then used some magic math to prove that ad costs can sky rocket. Someone tell Intel. How much are they spending on that new logo again?
But you can tell this guy is a nutjob. He seems to think that because software/service X is available for free this will stop competition. Gee, look at my tagline for two free editors. Now google for other text editors. How many do you find? Rounded to the nearest hundred.
The bubble is over, there is no new economy, all the same old rules still apply. Oh and it says a lot about Jakob Nielsen that quality of your product doesn't seem to enter into equation. The only determining factor in how many people come to your site is how much you pay for ads and the only factor in how much you sell is your site.
Eheh. Explain this to google please. Exactly where did google advertise? Thank you.
Of course even an idiot gets some things right. Who here uses slashdot own search or uses google to search slashdot for old stories (oh and the third option for editors "Search old stories? What for?"). It is far easier to google with a question the find an answer site. Gamefaqs.com is about the only site I search directly.
If you don't want people to search you via google then A disable google from indexing you or B improve your goddamn site so the fucking search works properly.
Oh and if you don't like paying several dollar per google ad click, then don't. Word of mouth can work wonders if you are selling quality. There are plenty of companies that never advertise. They survive because they are the best and everyone knows it or they are so common people don't even think about it anymore. Anyone else. Welcome to the world of the ad agency sucking every last dollar out of you that they can. It is their way of making a living.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
In the long run, every time companies increase the value of their online businesses, they end up handing over all that added value to the search engines. Any gain is temporary; once competing sites improve their profit-per-visitor enough to increase their search bids, they'll drive up everybody's cost of traffic.
This is a simple fact of economics: There is no profit in a competitive market. (That is the economics definition of competitive - not the pedestrian definition.) The point is that you have to differentiate yourself from the competition in order to (successfully) charge a premium for your product - either through website improvement, or having a different product that you're selling.
The fact that the proliferation of auction models has made many markets more competitive is a fabulous thing. If you draw the conclusion that the author should have drawn, it becomes ainfully clear: search engines make it harder to be a retail "squatter" and make money. (That is, to run a site that doesn't have any innovation in either site design or product.)
There is one valid economic objection that the author could have made (but didn't). That is that the web advertising market is asymmetric. Google has a near monopoly (AFAIK), which allows it to extract (close to) the full consumer surplus for the ads it sells (the site makes $4 per visitor. Given these assumptions, the site can pay up to $3.99 for each click on its search engine ads...) This wouldn't be the case if there were substantial competition for Google (and I might honestly be wrong about the lack of competition - I just haven't heard of any).
My point is that there might have been a substantial argument to be made about the search market, but if there was, the author failed to make it.
argumentum ad fallacium: Fallacy of defining a fallacy which allows one to dismiss the argument in question.
I don't know about anyone else, but I only use search engines to find the site that has what I am looking for. I don't go around typing random URLs in to try and find a site. I type what I am looking for into the search engine, and it responds with a list of sites that match my criteria and the context (usually) of the site. If the context I'm looking for isn't on the first page, I refine my search.
You wouldn't do research in books looking only at the index (or even a library's card catalog system) would you?
I will probably never find your site... so go ahead and block search crawlers if you feel you are getting screwed.
The plain fact of the matter is there is SO MUCH data on the itnernet as to make it nearly impossible for me to find your site by chance without search engines indexing and suggesting the content on a related search. I don't have time to go and independantly discover your site when looking for a topic or product, if it does not come up with a few google searches it effectivly does not exist and you get 0 revenue from me AT ALL.
"What about all the filthy, filthy porn?" -- RvB
Michael J. Ryan - tracker1.info
I agree with this post. I liken search engines to super markets who make a ton of money from charging PepsiCo for prime "shelf space" Essentially, this is what google and yahoo are doing. They are setting up prime "shelf space", ie that top of the list on the right hand side. They are setting up payment mechanisms (a super market check out stand), which of course they will take a small portion of. When I need to find stuff, I use google, sometimes those adds on the right help me. Just like when I want my favorite tortilla chips, I go to safeway. However, one promise of the 'Net was that creators of goods wouldn't have to go through a middle guy to sell goods. They could sell directly to the customer, and not have profit skimmed off the top. Imagine a world in which you couldn't find the product you were looking for unless you paid google for "compute time" . What you say, pay google for search? You pretty much are going to HAVE to go through a search engine to find stuff, and suppliers are going to HAVE to "register" in order for you to find them. It could happen, look at people who pay for CableTV and still have to watch commercials..... In short, we have to be aware at the power search companies hold, even the ones that "Don't be Evil"
Where you turn a profit. You mention Thousands of dollars over years. $2,000 over 5 years isn't a profession, it isn't even worth reporting on your taxes.
From your post it sounds like you are saying content is worthless, but the author has value. But to me as a consumer, I couldn't give to shakes of a monkey's ass over who wrote my SQL Bible, but it's content has saved me hours of head banging frustration.
Why would I pay $20 to meet the author of the book when I already have the book? Why would I pay $20 for the book when I can get it online for free? Why bother with the ebook when I can use Google to get the same information faster?
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
very thing we record in the studio will instantly be public domain. We'll be watching for others to take the content and redo it, and then we'll be able to use that content as well for our own gain.
What if they copyright their work?
This idea sounds great in a world where everyone who hears or uses copyright free music contributes back to the cause, but that's not the case.
By "public domain" do you mean some sort of creative commons-like license, or are you really going to produce it and then let it be used for whatever?