ebay vs Search Engines
runlevel6 sent us worthwhile article about eBay vs Search Engines. The gist is that a small time auction site was returning search results into eBay. eBay was less than fond of this claiming that people ought to use their search tools. Little company claims this is counterintuitive to the whole point of having a search engine. Check it out.
If eBay is so concerned about search engine's not archiving their site, why don't they just set up a robots.txt? It's a simple, technical solution and no one gets hurt..
--
Reject
--
Reject
reject@metaphorcity dot com
See, change the situation a bit though.
You own a grocery store, one of several in town, and the more people you can get to go to your store, the more money you get through ads. Now you would have one of two reactions:
1) You could be happy that there will be more people using your store (these won't count as customers for ads, because they don't see it)
2) Or you could view it as someone using the infrastructure you already built and profiting on it himself, because he gets money for ads as well. and you don't.
I agree with you, but their reasoning is 100% fiscal, they don't care about how often someone bids, or how often a sale gets made, they just care if they get their cash through marketing.
Just another perspective...
--[Patryn]
Not that I completely agree with eBay, but their complaint (at least, their *public* complaint :-) is that these spiders cache obsolete information and slow down the auction site.
I can believe it to some extent. General purpose search engines always cache stuff and a lot of the results are so out of date that the pages have gone, servers have gone, etc.
But these are auction-specific search engines, so they *know* that caching results more than a few hours old is worse than useless; in fact, it would give the auction search engines a bad rep.
Which brings up eBay's second complaint. To keep from showing obsolete results, the search engines trawl often enough that it ties up the auction site itself.
Now it seems to me that one solution would be for auction sites to have a compact data file they update once an hour or three times a day or sufficiently often, in a format meant to be machine readable, expressly for these auction search engines. That way, the auction site would get the benefit of the search engine with minimum fuss.
--
Infuriate left and right
You moderators call this insightful?!?
Problems with your metaphor:
1) Lets say the guy comes to your store once an hour, instead of weekly.
2) Since bandwidth is limited, let's make the comparison that the guy brings 100 people with him, getting in everyones way, so that he can quickly look at every item in your store.
3) In your example the guy making the glossy insert isn't profiting in any way. Lets change it to the guy places an add with your prices saying he will provide the lowest prices if you join his low price club for $FOO. W
DrLunch.com The site that tells you what's for lunch!
That would be an excellent reason to deny all packets from the search engine's network.
The "cue the foo posts in 3, 2, 1..." posts will commence with no subsequent foo posts in 3, 2, 1...
Because ebay isn't in the service of providing the best deal or lowest price on anything. That is completely up to the people who use their site.
Actually most mata searches just use the cgis and such on the target webstite.
Nice rant about advertising. Too bad eBay doesn't have ads.
--
Win dain a lotica, en vai tu ri silota
That would be a nice theory were it not for the fact that eBay doesn't sell advertising on their site. They make all their income from a listing fee per auction, calculated based on the starting price.
---
"'Is not a quine' is not a quine" is a quine.
"'Is not a quine' is not a quine" is a quine.
Quine "quine?
I don't blame them though. At the time, they only had one ad visible on the whole site, and it was in a frame separate from the main page. Deep links got around the frame-setting page, and so their ads never showed up. This site design was also real annoying for those of us who don't like frames for a variety of reasons (lynx, or using Netscape on SGI which still doesn't get frame-focus right for the keyboard) and so I greatly appreciate them going to a flat format instead; this also makes it less of an issue. However, what's also an issue is the fact it's an issue - it's a collection of stories, setup like a newspaper. Having a channel interface into their site ruins that.
---
"'Is not a quine' is not a quine" is a quine.
"'Is not a quine' is not a quine" is a quine.
Quine "quine?
Thank you for that bit of sanity. Most of the other posts in this article seem to be along these lines:
"eBay should just deal with it, the Web is supposed to be free" or something like that. Yes, but bandwidth isn't free.
"eBay should stop advertising." It's kinda hard to stop when they never did it in the first place.
"First Post."
"It's just like..." (flawed analogy follows)
--
Win dain a lotica, en vai tu ri silota
There are a few tiny ads on the front page. I've never actually been any deeper than that, and the front page was just for a little bit of research for my post. I thought for sure they'd advertise. Oh well.
I'm not saying eBay should XML their data up to make it usable by outside search engines. I'm saying they need to pull their own search engine into the 19th century and allow a little leeway on criteria.
---
Linux MAPI Server!
http://www.openone.com/software/MailOne/
(Exchange Migration HOWTO coming soon)
Of course the correct thing for Ebay to do is put a robots.txt file that limits the area that search engines can traverse. This has been a standard for a long time, and it is understandable to not want search engines to index highly dynamic pages, as the content of those pages would be out of date quickly.
Some things are linkable, some things aren't. Welcome to the dynamic nature of the web.
It's not the hyperlinks that eBay has a problem with. Meta-search engines continually query the eBay database. Imagine you sat on ebay's search page and sent in 1000 searches every minute, every hour of every day. Would you be surprised if eBay got pissed at you? I sure wouldn't. I'm sure eBay would be ecstatic if every one of these meta-auction searches put links to eBay's home page, but they have every right to get upset if these engines make such intensive use of their servers and databases.
None of those sites link to specific auctions on Ebay, they don't update often enough for that sort of thing.
DrLunch.com The site that tells you what's for lunch!
It seems to me that eBay has to do this. They are in the biz of publishing content on the web, and as most dead trees publishers they get a large part of their revenue from adverts on their site.
They also get some revenue by subscribers - people who buy and sell on their site.
What NO conventional publisher would tolerate would be publisher B taking their content ($$$ to develop) and publishing it with their own advertising. Imagine how the NY Times would react to The Daily News publishing their articles to attract readers? Surely their would by lawyers all over the place.
In short, the reasons that this AC gives are probably not really eBay's motivations. As others have pointed out, web-crawlers constantly going over your site would be a great pain and would slow things down. But if eBay licensed the data -- found some other way to distribute it -- they wouldn't have this problem and would have a new source of moolah.
So why aren't they doing this?
I'd block the domain from my subnet, and I wouldnt listen to anything anyone had to say about it. Is this a technical incompetence issue rather than a legal one?
Indeed - did ebay actually attempt to limit access by the 'bots using the Robots Exclusion Standard? (ie install a robots.txt file at the root of their document tree)
Given that www.ebay.com/robots.txt returns a bog-standard 404 I doubt it. If they had tried this, maybe they would have more of a case?
S.
Perhaps E-Bay is trying to eliminate future corruption of their customer profile?
If you read the marketing-speil coming from all the current e-commerce sites, be very afraid. They want to know how who, what why, where, when and what you spend. Now if a robot that emulates a human search (or aggregate metasearches) then GIGO (Garbage In, Garbage Out) rules apply. For people who work in the database field, the technology is the least of the worries as cleaning up the data is probably 80% of the work.
similar analogies would be
- creating artifical users on Hotmail
(their name space is NOT infinite)
- inserting mega dynamic indexes to Yahoo
(overwhelm the human classifiers)
- creating recursive reviews to Amazon
(drowning the human comments with noise)
In short, anything automated can be screwed up to some degree. If the script kiddies ever get a clue, I suspect there will be some sort of arms race between robot automation and exclusion. Fun and games.
LL
Not really. When Sherlock first came to be, a lot of sites started calling Apple telling them they didn't really like this. Apple cuts them in on the deal and even serves ads from the searched site to sherlock. It was all done above the table so to speak. This only counts for the plug-ins that actually ship with MacOS. There are plenty of home rolled plugins that don't cut the searched site in on the goods (so to speak).
-- Are you an EFF member yet?
presumably ebay wants links to its content, but it also wants to avoid spiders hitting its database frequently. a technical solution would be to provide a file somewhere on the site that lists all updates to the database every minute. then the spider just needs to get that file every minute. the file can be written in a standard format, say XML, that can easily be parsed.
Alexium - open source software and articles for web publishers
Sherlock is really just a limited function scriptable web client. I don't think eBay is really going to care about it.
Ebay is the market leader by far. Anybody seriously in search of something will go to Ebay. If you have a meta-site that searches Ebay and other places, then that gives a leg up to the competition. Ebay has much more to lose by allowing the competition to compete on equal ground than by not allowing itself to be listed in meta search engines.
--SolidGold
Everything you know is wrong. Or more accurately, inaccurate.
>eBay doesn't have ads
Ebay charges more for better placement of your auction. Better placement is useless if your auction is listed in a place where ebay's placement is ignored.
--SolidGold
Everything you know is wrong. Or more accurately, inaccurate.
wrong you are. ebay sells screen estate. A "bold" of the title cost 2 bucks, a "gift icon" cost a couple of bucks, and the "feature" that put you on the top of the search result costs more.
It doesn't use graphical banner doesn't there is no ad.
cy
ebay doesn't lose money from simple search results. Think of this. If I use another search engine and find a bid on ebay? Does that mean I don't buy it from ebay? of course not! I bid at ebay! that is how ebay makes it money, ebay makes it money when people sell stuff. The more people that sell, the better for them! The more people can find out what is being sold on ebay by these search engines, the more people are likely to buy from ebay, thus the more $$$. What is their problem?
------ Curiosity killed the cat. {satisfaction brought it back | it didn't die ignorant | lack of it is killing mankind
I'd hardly count that as advertising. Those are just additional extras on the listing fees. Though I could see how that would count as something which a third-party search engine would destroy.
---
"'Is not a quine' is not a quine" is a quine.
"'Is not a quine' is not a quine" is a quine.
Quine "quine?
Okay, so a direct link'll avoid a bunch of ads, but ultimately eBay are going to lose business on this one. I've never bought stuff in an auction, but if I did I'd be more likely to go to a sort of meta-auction (pardon) like BiddersEdge than to restrict myself to one site. If BiddersEdge can't list ebay items, then eBay customers are going to lose out because I couldn't bid.
The other side of the argument, I suppose, is that the only reason these companies are worth the billions that they are is because of eyeball counts. If that really is the case, we can expect to see lots more suits of this type.
The jist is that the aggregate auction sites continuously spider the 2.8 million items on ebay's site multiple times each day. Given their current scaling problems I can see why ebay would like to go it alone.
Obviously, if they can't solve the scaling problems and the public chooses auction portals over the single ebay site then ebay may have problems. For now they certainly have name recognition to get 'em through the short term.
Andy
Let's say I have a grocery store, one of several in a town. It's good, regular customers, I'm doing alright. One day, I open my newspaper to find a new, weekly glossy insert from a firm who goes from grocery store to store, finding the best items for the best price. I guess I could have one of two reactions.
1)I could lavish the guy with praise, because with faith in my service, I know he'll be bringing more customers my way, due in part to name recognition, and in part to the fact that I've got better prices on some items than my competitors.
2)Or, I could have a complete lack of faith in my services, thus basically admitting that others offer a better deal, and try to keep this guy from publishing his rag.
If they don't want this guy searching eBay for deals, how can they even pretend to stand behind their service?
Didn't we see a story on deep linking a few weeks ago? What's wrong with all these companies, thinking that they can take the linking out of HTML?
Once again, as The Onion put it, 79% of Americans 'Just Don't Get It'. The whole point of the web is to make the information "Site Transparent". You don't care where you are and you can drift from one server to another as you wish.
Is there anything we can do to stop this stupidity?
-- IANAEG - I am not an elder god.
First, if eBay doesn't want to make things public, they shouldn't be on the Internet. There are plenty of tech fixes to this "issue".
Second, if they want people to use their search engine, why not make it usable? Simple text matches is not enough. I want to be able to say: "Give me all Nintendo 64 or N64 games under $25. Return back the seller, the price, the closing time and shipping terms".
---
Linux MAPI Server!
http://www.openone.com/software/MailOne/
(Exchange Migration HOWTO coming soon)
>Pursglove said that, although it's not yet occurred, search queries from outside engines could potentially slow down eBay's database.
hm...they seems to be having enough problems as it is. seriously, ebay should be doing their own meta-auction service. this would be the smart way to do it...
-- your knees hurt, don't they?
Did the internet ever have a href-blockade before?
Maybe alta vista, yahoo, hotbot, excite, etc. shoud remove ebay from their search results...
This is a dumb move on e-Bay's part. If they are successful in stopping these auction search engines from searching their sites, then these auction search engines will only show items for bid at e-Bay's competitor's sites. I fail to see how that helps e-Bay.
Too many companies are full of control freaks. Just having any old person linking to their page drives them nuts and causes them to act against their own self-interest.
The cake is a pie
Ok, I know information on the web should be free to access and all that. My feeling is that if you don't want everyone to see something, don't put it on a website.
But the whining about "They're a multibillion dollar company and we're just a little $9 million start-up" made me gag. Please don't try that David vs. Goliath nonsense on me. You're just another loser with an unoriginal idea trying to ca$h in on "this big web auction craze." You're either too lazy or too wasteful with your capital to build an eBay-sized database and site, so you figure you will just link to their content. All of the traffic with none of the work!
Nice try. No cigar. Next!
Well, what is Yahoo and other search engines doing, then? They're providing an access method to different websites. If you want to search a website by going through another search engine, sure, it's a pain, but it's feasible.
I'd hate to see eBay win on this. As far as I'm concerned, the method of accessing a website hierarchy is irrevelant, and you are free to propose any method you wish. eBay is a huge milk cow at the moment, and it sounds as if this is just another example of someone trying to leech off some of their revenues.
However, I have a sinking suspicion that eBay will win its point. It's time Linux takes over the world... We wouldn't have that silliness under Linux. Okay, so if LinBay has an (OS) search engine and someone else picks it up and adapts the code, LinBay wouldn't sue or something as silly; they'd take the new code, and better it and so on.
I know, I'm dragging this whole thing into another "Linux rulz!!!" issue... But I've got to say it: this news piece makes me say this: Open Source is to the Internet what the Middle-Class was to 1950's economy: perhaps the only hope against the self-destruction of capitalism.
Make of that what you will. :)
"There is no surer way to ruin a good discussion than to contaminate it with the facts."
EBay is doing this for number of reasons IMHO.
1. Brand recognition - They don't want to be one of a number of auction sites. They feel they are the biggest, original, and baddest, so why put yourself in the first of a long list of auction sites.
2. Full site searches are slow on their 8088 - They have problems enough handling their users. Someone searching their full site 24 times a day or so is going to cause system slowdowns.
3. Not their pretty little ads - They want users to see their ads. Not some schmuck who's getting a "free ride"
Of course, this isn't in the best interest of users, but it's actually probably in the best interest of EBay.
He also outlined company concerns that outside search results might not display the most up-to-date, accurate information.
This is full of crap. eBay's 'current' bid prices from their own search engine are typically many hours old, I have even seen day-old bid prices displayed on bids.
And for some odd reason, the search seems to miss some items quite often, and I haven't figured out why yet. A search checking for a certain word in the title or description overlooks items that have that word in it. I am sure it isn't a spelling problem, and I believe it isn't a caps problem.
'Deep Linking' is one thing, but constantly spidering a dynamic database is stupid, and eBay has a right to limit it, but I believe that they should keep it open until it really shows signs of causing trouble.
I'm sorry ebay, but what you're doing is hard to make sense of and almost counter-intuitive to anyone with even a smattering of common sense.
These auction search sites not only get you a wider audience for your auctions, the audience they give you is more likely to buy than someone meandering aimlessly through your behemoth of a site. Oh, yes; it is a behemoth. Slow, painful, approaching the experience normally associated with a cheese-grater to the forehead.
I can understand you're concerned about these engines slowing down the experience of those directly viewing your site, through their incessant search crawl of your database. Unfortunatly, you're seeking a short term solution; what you really need is much more power.
Until then, throw a few more Ultras into the mix and enjoy the extra business the search engines deliver to your door. Don't risk larger amounts of dubious revenue when one of these companies takes you up the lawsuit trail.
.sig: Now legally binding!
Does Sherlock spend all day combing and re-combing and traversing and re-traversing the site it is searching?
No. It just makes a call to eBay's own search engine. Unlike these other sites that eBay is complaining about, Sherlock does not tax their site anymore than any other user does. Less, in fact, since you don't load up the front page. Sherlock users certainly aren't downloading all of eBay once an hour.
Since eBay doesn't have advertising, their complaint must rest in the fact that spiders slow them down -- and that's the last thing they need. They have enough problems as is, eh?
So why would eBay have a problem with Sherlock II -- especially since it doesn't have a problem with the original Sherlock?
don't blame e-bay. with their servers they can't even take the hits they get now.
For example:
last week I was checking adult educational materials on e-bay and saw bill gates in a bikini. I was hit hard, and trying to hit stop on my netscape i hit reload couple of times. ( I know stop and reload are far, but my eyes were hurtin'). Two days ago I got e-mail from e-bay saying that my reload pressing caused their servers to crash and that I own them zillion $.
Anyone knows of a good GPL lawyer software, cause I don't got zillon to spare.
self-rating (-1, offtopic).
btw,
I think e-bay makes more money from sales then from ads.
eBay seems to be getting all uptight about information that they put on the web and made publicly available being used by another company.
are they upset because the other company is making money? or because they are losing consumer traffic, and therefore advertising dollars? if their business model was "we sell the service of putting your lot in front of many many people and you pay us for it" or "we provide you with many many lots which you can bid on, and you pay us for it", then they wouldnt care how many search engines distilled their content, because they'd still be making money.
unfortunately, their business model seems to be something like "we make money because lots of people come to our site and see the ads we run", and this is why they are throwing a hissy fit.
advertising on web sites isn't a good idea. there are too many ways to break the advertising-on-webpage model and there aren't any ways to fix it except by suing people who break the model. why should a company like the auction search engine company be penalized because eBay can't figure out a good way to make money?
the Web has quickly become a networked system of banner ads and popup windows, and it sucks. if businesses tried to provide services to consumers in return for payment, things would work a lot better.
Any search engine which ignores the robot exclusion standard deserves to be spanked in the harshest way possible (DENY ALL FROM *.luser.com comes to mind).
Any web admin who threatens legal action without even implementing this simple procedure (http://www.ebay.com/robots.txt does not exist) deserves to be pointed at, laughed at, and then forced to RTFM until they understand why what they did was wrong.
Sure, I don't think that this would stop someone who was determined to send a spider through their site, but you can't very well blame someone for walking on the lawn if you can't be bothered to put up a 'Keep off the grass' sign.
-D
It's a pity that stupidity isn't more painful.
This makes me somewhat nervous as I have a Sherlock plugin for eBay that I wrote mostly for fun. Of course it uses eBay's search engine and can be thought of as a non-standard browser, but it sort of begs the question of what is an acceptable use of their content. Sherlock certainly doesn't display all their cruft but following any link that is returned takes you to an eBay page.
You will not drink with us, but you would taste our steel? - Walter Matthau, The Pirates
Because the auction is not the money maker.
/. screen I'm typing on.
/. bills.
The advertising is. Just like the advert for thebazaar above the
E-Bay sells ads to pay their bills. Andover sells ads to pay the
If you can search the auctions without seeing the ads, then E-bay has lost some possible revenue, depending on their advertising model.
Airneil
Isn't this a copyright question?
They own the right to their content and they decide, what is allowed to do with it.
If they decide to give the information to individuals directly, but not to any seach engines, this is their right, isn't it?
I dont see how this is a court issue at all. If I noticed spiders slamming a website twice a day I'd block the domain from my subnet, and I wouldnt listen to anything anyone had to say about it. Is this a technical incompetence issue rather than a legal one?
If you have pages on your site that you only want to be linked from other pages on your site, then you use CGI/mod_perl/apache modules/whatever to check the referer for a valid link source, and you return an error if you didn't get a good referrer.
If people make bots fake their referrer fields to get in anyway, then you've got a case for wire fraud. If not, then you can shut up and be happy.
Listening to companies whine that "Those people are accessing the data we made publically accessible! Make them stop!" is getting annoying.
Because when MacOS 9 ships in a few weeks, it'll have Sherlock 2.
Sherlock 2 has internet search categories, like people, and news, and auctions.
Tell it you want to search for auctions of rare mono The Who vinyl, (to send to your good buddy Rob) and it'll find them for you, on eBay, on Yahoo, on Amazon, ect. It'll tell you the current bids and how long they have to run.
I think they're just upset because someone else is grabbing impressions with their content. I probably would be to.
Don Negro
Don Negro
Perl 6 will give you the big knob. -- Larry Wall
I've been playing with the betas of MacOS 9 for a bit and one of the nicest new features is a direct search of online auction sites like eBay using Sherlock II. Last I heard Microsoft is also planning to add similar functionality in InternetExplorer 5.0 for the MacOS (and presumably 5.x for Windows as well).
It'll be interesting to see how eBay handles this one, seeing as how a Sherlock plugin just taps into a remote search engine using their own code--it just calls their perl, PHP, etc. from your local machine. I don't know if eBay could really get irked about this, seeing as you'd technically still be using their search engine.
----
Am I the only one who thinks Microsoft is a misnomer? Perhaps Macrosoft would be a better fit?
If I didn't want other people accessing my information, the last thing I would do is make it available to the whole world via the Internet.
I want to stand out in the middle of the street all day, but I don't want any cars to hit me, what do I do?
eBay appears to be dead set on destroying their customer goodwill as fast and hard as they can. The 35mm movie print fiasco (where they put up a policy page that labelled film collectors as criminals) has reduced my goodwill and respect for eBay down into the single digit range.
The problem with competing with eBay is that it's nearly impossible to get enough eyeballs to visit your site to build volume and name recognition.
I think that non-eBay auction sites banding together to form common search databases is a good thing, and I also think that it is a good thing that eBay will NOT be part of this, as it will hasten the backlash against eBay. There are plenty of people who really wish that there was a viable alternative.
-eBay charges sellers fees for listings and more for completed sales.
-Meta searches expose those auctions to a wider audience, thereby increasing eBay revenue derived from completed sales.
-eBay derives revenue from banner ads.
If eBay is not interested in exposing its sellers' auctions to the widest possible audience, which source of revenue do you think is more important to them? Which source of revenue do you think will (and currently is) get greater support from eBay?
How will removing load on eBay's servers from remote search engines and placing that very same activity back on the eBay servers (NT boxes that are always crashing and require a weekly four hour reboot) alieviate server load on eBay's notoriously unreliable servers?
-Jake
Well, what is the purpose of all this technology anyway? To make folks lives easier, or to make businesses money?
From the seller's point of view, meta auctions are good. They get more eyeballs on their products - on the other hand, this could have the effect of devaluing their product, if forced to compete against lower-priced items on other sites.
From the buyer's point of view, meta auctions are good, they get the customer a wider selection, and more power to find the lowest-priced item.
From the auctioneer's point of view, they're bad, because he's now gotta compete with other auctioneers with his service. His service is ultimately devalued, and his marketshare is likewise devalued.
From the meta auctioneer's point of view, of course they're good, they're his free-ticket to hijack someone else's service and offer it.
Would it be nice if the internet was site-independent? Yes, that's what HTML was designed for - but on the other hand, computers were designed to do what they're programmed to do, if that's limiting where a user can go from point A, that's what it was programmed to do. So, just because the feature exists in HTML - doesn't mean the site designer has to use it.
So, what is the purpose of an auction site? to make the auctioneer money. If that goes away, then so do the auction sites. (unless metering content to collect revenue from the meta auction sites somehow becomes feasible).
"The number of suckers born each minute doubles every 18 months."
These are my friends, See how they glisten. See this one shine, how he smiles in the light.
Imagine not just one insert but several from competing companies. Now imagine that they publish the prices from a week ago when your prices change daily. Now say that the people who print these inserts send their employees into your store with no intention of buying anything themselves. Instead they push your paying customers out of the way and tie up your sales/stock people with price checks on every single item in the store.
This is a more complete picture of the situation.
Now does it really sound like this is to your advantage as a store owner?
Ebay sues other people for linking to them, so you post a link to it on slashdot? I hope their lawyers don't come after slashdot/taco.
The shareholder is always right.
Speaking of The Onion, once upon a time there was a "The Onion" channel on My Netscape linking back to The Onion site for individual stories. It was taken down at the request of The Onion.
This channel would have provided lots of traffic directed back to their site, more advertising dollars, etc. However, they didn't want their content linked in from anywhere else.
They just don't get it.
I've often wondered how meta search engines get away with this type of practice. I have noticed that Metacrawler doesn't advertize much, attributes links to the engines that provided them, and seems to sometimes have ads from the other engines that it searches.
Site that try to find the best price for items must pose an even more troublesome problem for online retailers. There's a site that search 20+ online booksellers and returns their prices. A site like this will bring some business to your site but will force you to lower your margins to compete.
This seems fine to me - ebay has a right to their information and if they don't want anyone using it that's their business. It's like going to McDonalds and them having a Taco Bell menu in the lobby - you would think that it would be good for taco bell but it depends on the pricing. I think if taco bell wants to make them remove the menu then McDonalds should have to comply. Yo quiero ebay!
just my 20 lira (approx. $0.02 US)