Slashdot Mirror
Judge Bars eBay Crawler
matty writes: "A judge has said that Bidder's Edge could no longer use its crawler to gather information from eBay. 'Even if its searches use only a small amount of eBay's computer system capacity, Bidder's Edge has nonetheless deprived eBay of the ability to use that portion of its personal property for its own purposes.' So what about Yahoo! and all the other search engines? Don't they use similar technology? Read the article and see for yourself." Or maybe it's not such a bad precedent; it'd be interesting if such a ruling helped discourage hard-drive searching by software which searches for "undesirable" content without your consent or knowledge.
This ruling, if it stands, could have rather far-reaching consequences for the industry. Here are a few scenarios, both good and bad:
... amount of [system X's] computer system capacity," but automated search systems aren't? They both use part of system X's capacity, but the automation makes it evil? Is this because the automated searches potentially use up more bandwidth? If so, will we see future rulings where judges determine that linking or searching is okay as long as it doesn't exhaust more than BLANK percent of a system's capacity?
.02 cents worth.
1. [Good?] Programs which sift the Internet for e-mail addresses (for later spamming) will be banned because such sifting "use[s] a portion of that property," wherein "that property" is defined as any webpage or newsgroup - ANY on-line resource - that can legally be defined as someone else's property.
This creates a puzzle: webpages can be defined as property, but a domain name can't? So, the address which signifies and, in the mind of the consumer, effectively "houses" said property isn't property itself? So, you can steal my property deed, which _isn't_ property so no theft has occurred, and sell it to someone else who then has the right to profit from the webspace or other virtual property which the URL represents?
2. [Bad?] Deep linking has been okayed by the courts. So, deep linking is okay, which inarguably "use[s] only a[n]
3. [Good? Bad?] Search engines which use web spiders are now feasibly verboten. I know that robots.txt has long existed, but many millions of sites don't use it. Will they now be legally required to if they don't want to be searched? Can I sue, then, if you index my site and I don't want you to, because indexing my site uses up some portion of my bandwidth that I didn't want to give?
Just my
Neopets - the best free game on the Int
Tom Swiss | the infamous tms | my blog
You cannot wash away blood with blood
I think that the robots.txt could be seen as a "No trespassing" signs. For those who don't know robots.txt is a file that advertise limits for robots and spiders.
I have just one little doubt, how can one define what is a robot? If I have a cgi-bin that gets information from a site and presents this info to user on request (pehaps in a simplified way or otherwise processed way) woudn't this be a user-agent? If you say no, then maybe you should be aware that you just made proxy-servers a robot. But then you may say that a proxy dosen't change the information, so it appears as the author intended (not aways true, but...), then you just said no to lynx, since almost every site want you to see the graphical adds.
This question is rather hard to answer. I myself think that a cgi-bin could harverst information on the web. I think that if ebay is so worry about this matter, they should simply make security matter to forbidden such access, or at least make them harder.
That's my R$ 0,04 (this is more or less 2 cents of a dolar in Real the Brazil's currency)
--
"take the red pill and you stay in wonderland and I'll show you how deep the rabitt hole goes"
[]'s Victor Bogado da Silva Lins
^[:wq
Let me preface this post with this
::
:: I don't know what the fuck I'm talking about
Having said that, are they truly stealing ad revenue from eBay? The crawlers count as hits yes? Is it not these hits that are used for the marketing of ad space? I don't know how this works, but I'm guessing it's something like this.
Customer : I'd like to purchase ad space on eBay... How much does that cost?
eBay : Well, for your ad to be viewed by 100 people, it'll cost $100. For 500 people, it'll cost $200. For $1000 people, it'll cost $250.
Yes, I understand that customers going through Biddersedge are customers that perhaps AREN'T going through ebay, and they are therefore losing that revenue. Still though, you would think that eBay would have tried to come up with a way to get money from that, rather than shut down a potential source of revenue. Something like:
eBay : Hey, biddersedge... how many people do you have viewing our auctions?
Biddersedge : Well, we usually get about 100 a month.
eBay : Hmm... you're stealing our ad revenue. You should pay us the money that we would make off of that ad revenue. Lets see, we charge $100 for 100 viewers, you owe us $100 this month.
This would enable eBay to reap the benefits of having someone else advertise for them. Now, if Biddersedge couldn't, or wouldn't reimburse, and refused to stop crawling their servers, which is unlikely, then you would move to a robots.txt file, or perhaps DENY host settings. Combat a high-tech problem with high-tech solutions, not legal ones.
Anyway, this is just my take on things. Feel free to correct my mistakes.
Or maybe it's not such a bad precedent; it'd be interesting if such a ruling helped discourage hard-drive searching by software which searches for "undesirable" content without your consent or knowledge.
... "Be a beacon?"
It's pleasant to think that this ruling has potential to only be used for Good Things®.
However, the stark reality is that if it can be used to prevent breaches of privacy à la Aureate Media aka Radiate, it can also be wielded as a weapon in cease-and-desist letters as well as in the courtroom when a site doesn't like another (a competitor, perhaps?) crawling them on a regular basis like the recent AuctionWatch brouhaha.
Gibson Research Corporation has a fantastic links page regarding privacy. Many links on this page really ring true with this ruling.
On another note, Slashdot is really on the ball -- eBay's Press Releases page has absolutely nothing to say about this (as of when I wrote this, of course).
--
"Give him head?"
"One World, one Web, one Program" - Microsoft Ad
(usual disclaimer about my total lack of legal training, and the face that this is only my opinion)
This decision appears to be legally shaky and could get reversed on appeal. The judge has ruled that BE cannot crawl eBay because the crawler is using some small percentage of eBay's server resources, and eBay has the right to limit who uses their "personal property" (in this case, their server resources). However, there are some limitations to the application of law regarding protection of personal property.
- The property should be private, i.e. not a public resource. While it can be argued that eBay servers are the property of eBay, these servers serve web pages to the public Internet. eBay constantly advertises asking people to come to their site. It could be argued that eBay's website is an open accommodation. This is a somewhat weak legal argument due to the disclaimers that are sure to be somewhere on www.ebay.com, but legal disclaimers will not apply if the appeals court rules that their website is an open accommodation.
- To classify an open accommodation as private, the owner of the item should have taken reasonable precautions to inform the infringing party of their trespass. This is why land has "No trespassing" signs on it. I don't see any such signs on eBay, and in any case it can be argued that accessing auction information on eBay is retrieving the information that eBay is designed to serve, and cannot be termed trespass.
- If the property is ruled an open accommodation, the owner must show that he has suffered reasonable loss by the trespass. This is why, if I leave my house door open by mistake and you come in and steal my stuff, I still have legal recourse to recover it (assuming I find you). In the case of software which can be copied freely, this law is applied by the copyright holder under the (perhaps dubious) justification that it causes them economic loss. It appears eBay attorneys have claimed protection under this law, saying that their servers are under load from BE's spider. However, if it is ruled that the primary purpose of the accommodation is the only purpose to which the infringing party puts the accommodation to use, this should not be a valid argument. The situation is akin to a bulletin board you put up outside your house. The space in front of the bulletin board is a limited resource (since if you are not standing in front of it, you cannot read the notices). If somebody walks by once every hour, stands in front of the board for five minutes and reads the notices, this isn't trespass until you can show that this person is causing you reasonable loss. If he stood in front of the board all the time, thereby preventing anybody else from reading it, that would be a valid argument.
Just my $0.02-- Before you moderate: Do you really believe somebody called 31337 d00d has anything useful to say?
But how do you know if you're allowed on a web page until you actually bring it up? It's like posting a billboard and saying only certain people are allowed to look at it. Plus, this case sounds like one aimed at killing off spiders completely. It starts off with just Ebay, then other companies decide they want to control who can visit their sites, and so on. Pretty soon, the people writing the spiders don't know which sites they can legally visit. That's what robots.txt is for, BTW.
Generally, if you put up a web page, it meant for everyone to see. If you want to limit who can see it, add logins and passwords. Think about sites like NY Times -- they require a free login to get access. This is one way to keep the spiders out, though I suspect it is also used to keep statistics on how many people view their stories.
--
For every post, there is an equal and opposite re-post.
The problem may be easily solved if Bidder's Edge made use of some sort of distributed HTTP cache which would already have the relevant HTTP objects stored and would avoid requests of eBay's servers. These caches are populated by other individuals making use of them through other local caches (you ARE making use of a local HTTP cache, aren't you?)
This is not a precident!
For it to be a precident, it has to be ruled on by an appeals court. Unless, it is something that an appeals court does not usually rule on (ie. a motion to remand).
Fight Spammers!
The "using CPU cycles" is not a new legal argument. Back when there were no specific laws against computer cracking, I believe one of the earliest convictions came from proving that a cracker "stole electricity" by using cycles on the machine they'd broken into.
You'll first note, from following the provided link, that this was simply an injunction. There was no mention of damages. There is nothing in this that sets any precedent against webcrawlers, or deep linking. All the case says is that yes, the owners of a webserver may choose how that server is used.
Bidders' Edge were doing something to Ebay's site that Ebay didn't want them to do. Ebay asked them to stop. Bidders' Edge didn't want to stop. So it went to court. The court told Bidders' Edge that what they were doing constituted a use of the Ebay system (their CPU cycles) against the wishes of its owner, and ordered them to stop.
Now what, exactly, is wrong with that? Do we really want to set the alternative precedent, which is that as soon as you put a public resource on the net, you have no say in how that resource is used? I can imagine all the IRC script kiddies drooling at the thought of that one. "Hey! It's a public server, my clonebots were only a few hundred out of the thousands of users online, so I can't have been using a significant amount of the net..."
Charles Miller
--
The more I learn about the Internet, the more amazed I am that it works at all.
The worst effect of this decision will be on meta-search engines and shopping bots. Those sites rely on the ability to act as a proxy on behalf of the user to request pages from other sites in real time and add value to this data by presenting it in one screen (and removing duplicate listings, etc.). Bidder's edge does not use a "spider" and is thus not concerned with robots.txt. Instead, it is simply acting as an intelligent proxy server--requesting information from multiple auction sites on behalf of its users.
Ebay's objection to this practice had nothing to do with ad-revenue. They were upset that they were being compared equally with other auction sites. It is as if Amazon wrote to the companies operating the shopping bots with a statement that they were "Amazon" and should not be compared with mere booksellers. Ebay's brand is their major asset and they are willing to protect it at all costs.
As a result of the injunction that Ebay obtained, consumer choice and fair competition can be undermined by any e-commerce site with a superiority complex.
ByteMyCode.com: A Web 2.0 code sharing community.
This could create a really scarry legal precedent. It is unfortunate that public policy is, more and more, being decided in through litigation and not legislative bodies with input from the communities effected. I hope that the judge eventually removes this injunction.
No, I think this is more like if /. started mirroring content on their own servers, then it would be the same thing.
/. currently does so, increases their(the news services') page views, not vice versa.
Linking to articles on C|Net or Wired, the way
Y'know with a directive or something? If I were the judge, I would have required eBay to use reasonable technical defenses. If those failed, then, perhaps a legal solution could be pursued.
--
Honestly, I can't see how they can determine how much wasted resources is too much..Like matty said, what about these search engines like yahoo..don't they do the same thing (except perhaps on a smaller scale) ?
In other words, where do you draw the line between acceptable and not acceptable..or can you?
~Steve
--
~Steve
--
"<r-xr-xr-x> Just try to edit me" -- www.ircnews.com
If EBay wants to stop a particular client from using it's services, they hardly need to go to court to do it! They can weed out requests based on browser IP, based on whether the client is a spider (robots.txt), based on click-thru agreements...
Basically, imagine that every day, I ask you for a few dollars and you give it to me. Should you then take me to court to force me to stop asking? No! You should just stop handing me your money!
If EBay has such a system in place, and BiddersEdge is ignoring robots.txt, lying about it's client type, or otherwise circumventing it, then fine, moderate this to (-1, Idiot)... but it's late, I'm too lazy to read the article before going to bed, and I've used EBay once or twice without them asking to make sure I wasn't a competitor first.
(Now, if the search engine provider doesn't respect the ROBOTS.TXT exclusions, I say go after them with legal means.)
Wimp. When did the age of Technology for Technology end and the age of Technology for Profit begin? Sysadmins shouldn't go crying to the DOJ every time somebody they don't like pings their servers. A true Sysadmin would have just added a DENY tage to the httpd config file, rather than trying to complain to the courts.
Search engines spider your site and you get more traffic (and maybe more money).
Bidder's Edge spiders your site and you get less traffic (and maybe less money). See the difference?
So will everyone please stop suggesting that this applies to search engines!
Drag n' Drop DVD Recommendations
- No public/anonymous browsing - have to log on to see eBay content.
- Make it against usage agreement to use bots
- Make the site bot-friendly either through optimised pages or a separate connection (ODBC-esq)
- Enter into specific licencing agreements with 3rd parties to allow sorting, sifting and filtering of eBay contect. If they breach the licence, pull their access.
Easy. Then the courts don't have to rule that the public aren't allow to use publicly accessible information if the provider doesn't like the way you look at them.Ah! I see ... I always use the search function, and never go directly into the catagories, so I never see the "featured" section and forgot about it.
Because Ebay makes their money from ad revenues. To use an analogy: Slashdot is cool and harbors extrememly libertarian views about the web, but everytime someone views their headlines on www.redhat.com or any one of a million other places that posts them, they are deprived of a hit to their homepage. Which means that advertisers pay them for one less impression. Over time, these can add up - they probably have cost /. a nice chunk of change in the long run.
This is the same exact thing that happened to ebay. By mirroring only the content of their site (the ad listings), less people are going to www.ebay.com to see that information. Hence, less are viewing the ads on www.ebay.com, which means they get less money in the long run.
The fourth clause for fair usage under the copyright act states "(4) the effect of the use upon the potential market for or value of the copyrighted work." Obviously, fair use has to end here - what this site was doing obviously drove down the "market value" of Ebay advertising, because it lowered their impression count. The fact that it is publicly accessable has nothing to do with it; there are plenty of publicly accessable things out there that are still copyrighted. At the bottom of every page you still find "Copyright © 1995-2000 eBay Inc. All Rights Reserved."
There has been a widescale trend by the courts in narrowing the scope of commercial fair use recently, and this case fights right in with that trend.
--
I think there is a world market for maybe five personal web logs.
I am a lawyer, but this is not legal advice. If you need legal advice, contact an attorney licensed in your own jurisdiction.
:)
> While IANAL, I have a friend that is and I think that giving out legal
> advice (which can consistitue answering legal questions as such)
> without charging for it can be considered a violation of the Bar
No; that is an ancient joke--though if you go back far enough, you find Barristers robes with a pocket in the back so that they could conveniently be given a payment--as it was improper/unethical/ungentle to ask--Esquire is a derivitave of Squire, a class that wasn't supposed to work for a living.
Anyway, free advice is legal, and so is free work (pro bono publico--for the public good).
That said, anywone dispensing free advice in a public forum like this is nuts. There is *no* way to adequately gather enough information from a short posting to give competant advice (save in the most trivial situations). Additionally, proving what you did or didn't say is difficult. WHether you charge or not, you are still liable for the advice.
Extreme example: voice on phone asks question, gets answer, does something, comes back and sues. You have no idea who the person was, or if this is the person you talked to. But you're on the hook, and you insurance carrier is less than happy. This is why I have a hard, no exceptions, policy that I only give advice to clients.
On the other hand, I do frequently comment on legal issues around here--but never in a way that could be legal advice.
Lastly, note that the legal questions that get asked are generally ones for which a proper answer would take several hours of research. I'm not going to do that every time a complete stranger asks. If someone wants to pay my hourly, sure--but I'm not going to do several hundred or a couple of thousand dollars of free work on a regular basis
[Note: I work for a different online auctioneer].
We've been discussing whether or not Bidder's Edge is a Good Thing (from our point of view) or not. My company does B2C auctions (e.g. we actually warehouse the products unlike eBay's C2C model). There's two schools of thought:
1. Good: It potentially drives traffic to the site. The whole point of an auction is to allow as many people as possible to see it to find the one who is willing to pay the highest price.
2. Bad: Bidder's Edge (AFAIK) aggregates auctions (e.g. it consolidates auctions from eBay, Yahoo, Amazon, etc.) and so if there are 2 of the same product, people will obviously bid on the one they can get cheaper. This essentially drives traffic away.
Our current opinion is that 2 is the dominant factor. Plus, these sites train users to go to Bidder's Edge before going to a specific site.
The real problem is that there is a significant difference between a person viewing a page and a bot "viewing" a page. It is reasonable to assume that a person might (however remotely) bid on the listing. Bots can't bid.
I agree with your assessment of the ethics of the situation.
Cheers,
Slak
This is entirely short sighted, unrealistic, and fails to take into account any real world scenerios.
The internet is a public place, no different from the local shopping mall, grocery store, public library, movie theater, whatever...
When you go to the mall they expect that you will and won't do a number of things, commonly accepted criteria for activity in a public place (read: no shirt, no shoes, no service).
On the internet nobody gives a damn if your in your shorts, but suddenly your not allowed to do any number of things that you could realisticly do physically.
For example, lets say I'm doing market research and I send 50 people to the local mall to run around and look at what kind of (product) is being sold, and how many of them, and how much they cost. This is perfectly legitimate, and legal since that information is publicly available. I probably can't go into the stock room and see how many of (product) are not on the shelves, but I can certianly just have a look around the store (just like anyone else).
If we apply the latest internet precidents to this senerio I would NOT be allowed to do this without breaking the law. Suddenly I would be using that stores resources and denying them use of that resource for whatever reason they deemed more important. So I'm not allowed to go the store if I'm just window shopping now?
Publicly accessable resources are held up to a very high standard. Anyone can find out how much a store charges for , this is good for everyone, the store, the customers, the manufacturers.... The same applies (or should apply) to the internet.
As far as purely internet related impact is concerned, can anyone who hosts a site look through their server logs and sue anyone who connects to their site to much? Or all those search engines that come through on a regular basis? Or anyone that pre-caches the site automatically (gee, this is even a feature in Internet Explore... more MS trials?).
My opinion: The internet is a public place, and fair use of a public place is already governed by a certian set of rules and regulations (at least here in the US). Let these rules and regulations do their job and stop creating "special regulation" for a situation that isn't radically different from anything else we humans do on this planet. Just because you do something on the internet doesn't mean it requires special regulation. -Gentry
Isn't this basically what Slashdot does? Slashdot steals bandwidth and CPU cycles by making us all /. a webserver. Isn't it the same thing?
As far as ebay...I kind of side with them and kind of don't...if you are posting content, expect it to be viewed. I don't know if copyright can be involved in this or not? Repackaging information from places has been around for awhile.
It's been around, yes. Several people here have pointed out search engines, but there's a big difference: AltaVista/Google/etc. honor robots.txt. According to some posters here BiddersEdge appears not to have honored it. That changes everything.
In law there's a concept of intent. By not honoring robots.txt, BE demonstrated at best deplorable ignorance of the generally accepted responsibilities of spider-users, and at worst an intent to circumvent measures taken against services such as theirs.
The article doesn't go into detail but from BE's stated position it's not hard to imagine that even if eBay blocked their address block, that BE would shell out the relatively nominal money to buy, say, a bunch of dial-up accounts.
What eBay has done is analogous to me running, say, a playground and kicking off somebody who appears to be using it for purposes I don't agree with. I'm not required to do any more than ask that they leave (robots.txt) before I'm entitled to call the police to assist.
A lot of people have called eBay a "public" service. That's a term that should be used with extreme caution. Not everything that's public is a "common carrier". "Public" services that aren't common carriers still have most of the property rights of a private entity. There are exceptions, such as anti-discrimination laws, but outside of those "protected" conditions, a business owner can kick you out over anything he wants. Or he can just kick you out for no reason at all. Abridging that right in the name of "freedom" will do more to hurt the Internet than any site denying access ever could.
Government-franchised monopoly utilities/services (cable, phone, power, etc.) usually are classed as common carriers. Check out your state/locality's regulations regarding common carriers and then ask yourself if you'd want to operate a large website under those restrictions. I'm sure it's different in other places, but here in Virginia, state-franchised monopolies have to get approval from the state before they can raise their rates, not something most site owners want to (or should) be subject to.
-- Old Man Kensey
"Even if its searches use only a small amount of eBay's computer system capacity, Bidder's Edge has nonetheless deprived eBay of the ability to use that portion of its personal property for its own purposes. The law recognizes no such right to use another's personal property."
I really don't see how this is different than any other web-crawling search engine. By the judges argument, he is denying all use of ebay since every transaction takes of a small amount of resources.
From the technical side, Bidder's Edge may actually be reducing the load on the ebay servers by offloading some of the search load to their own servers. Without knowing how much they are caching and how many users they have it is hard to tell for sure.
Now the interesting bit: would the ruling prevent me from writing software to do the same thing and then selling it? Yeah, you'd need a big/fast network pipe to use it, but that's not really the point here...
-p.
Imagine a sign in the future ... "only humans allowed in park. no robots, cyborgs, automated machinery. or semi-autonomous devices. Strict infobit-control zone. Violators will be sued and their datacores purged."
:-) ) limits on who can access their information (much like a restaurant can say no thongs) but I suspect it is a little disingeneous to call it "stealing cycles" when their ulterior motives is to probably save them the extra hassle of stripping out electronic tracks and to discourage potential superior services. An analogy can be made as to who "owns" the final stock closing price which is used to create index and long-run analyses.
Ebay wants to analyse the human activities (your click stream) and denying robots and other automated search sequences (unless you're willing to pay a hefty access fee) is probably a deliberate strategy. Now whether this is right or wrong is a moot point. Obviously as providers of a service, they can put arbitrary (even downright discriminatory as no League for artificial robot fair access rights exists
The question down the line then arises whether this is a "fair" business tactic or anti-competitive behaviour. Much like certain countries have obstensibly low tarrifs but erect substantial hidden barriers to entry, it can be difficult to judge. If you look at the latest theories on infowar (IT applied to battlefield) you can see the importance of denying the enemy access to information. Much like you can forbid a competitor from wandering into your shop to browse the prices and observe your layouts/fitouts, e-commerce sites are putting in place technical and legal challenges to make it difficult for their peers to emulate their design and scan any price-sensitive movements. Of course in a technology arms race it's probably not too difficult to come up with a distributed random sampling system from multiple access points but it does make life a little bit more challenging for new entrants, especially those wishing to arbitrage between multiple auction sites.
Too bad we don't have educational futures as in the coming techwars the only sure winners will be the overpriced salaries of consultants and MBAs.
LL
I think the answer lies in compilation copyright. This is a special type of copyright granted for an aggregate of information, even if the individual bits of information have others' (or no-one's) copyright. This is what protects, for instance, an anthology or index.
eBay could claim that even though their pages are freely viewable, they own compilation copyright on the whole of the work, i.e. the entire bidding collection and its indexing. There is merit to that claim, as they undoubtedly have to invest a lot of energy in collecting, maintaining and indexing all biddings.
In this case, by essentially copying the entire collection and offering their own search facilities, Bidder's Edge have copied and violated eBay's compilation copyright on the work. Note that the violation lies not in downloading the entire archive (they are entitled to it as much as anyone is entitled to remember what she reads), but in offering that archive to other people.
Other search engines do not violate compilation copyright, because they are essentially having their own compilation copyright: they index a much broaded range of works, and do not offer direct access to the individual compilation of eBay. They have the same rights as the 'keyword lookup' index books you occasionally find in libraries, which are an immense help when doing research.
...Unlike Auctionwatch and other tracking sites, Bidder's Edge was tracking in realtime, making the act of "sniping" so much easier for bidders who would otherwise have to log onto eBay directly.
In such a setup, a bid is cast (usually at an abnormally high rate) within the final hour of the auction, in many cases, within the last 2 minutes to 3 seconds. Anyone else who bid on the item early on in the auction (like a day or four) gets screwed. This also rasise suspicions of "shill" bids, bids cast by the seller & their associates to jack up the price (& hopefully demand) of the item.
While eBay goes to some lengths in saying they disapprove of shilling, they absolutely love sniping. It's where much of their revenue comes from, as hot items like Pokemon, Voodoo X cards, etc. can attract a Dutch tulip-like frenzy.
Meaning more customers, more items in demand for eBay's prersonalized (read: proprietary) site
trackers to keep you informed of, more opportunities for folks to bring their crap to eBay, etc.
Bidder's Edge, therefore, was doing two thirds of all their work for them. No wonder eBay was pissed.
But eBay allows this because this "client-side gathering" places advertisements on retinas.
Will I retire or break 10K?
First of all, he declined to address the copyright and trademark claims made by eBay, thus avoiding the whole troublesome can of worms that would open.
Second of all, he bases his ruling on property rights. Read the ruling, Bidder's Edge is using eBay's servers and bandwidth to generate their content. That is server capacity and bandwidth that eBay is unable to use. As others have pointed out, a ruling in favor of BE could, if broadly interpreted, given a green light to DOS attacks.
Finally, Bidder's Edge is violating the eBay User Agreement. From Section 7: Access and Interference, "You agree that you will not take any action that imposes an unreasonable or disproportionately large load on our infrastructure." While eBay is a public site, there is a User Agreement which user (in theory) must abide by, if they don't, they can expect consequences such as what BE is getting. Not only is BE "stealing" cycles and bandwidth, they're doing it when they agreed to not via the TOS.
This is as perfect of a judicial ruling as I've seen covered on Slashdot. The judge didn't give ownership of the content to eBay (taking it away from the sellers), he didn't forbid linking to eBay (chilling deep linking), he didn't give eBay a patent, he didn't pour hot grits down their shorts.
This ruling doesn't prevent search engines from spidering. Why? Because most sites are completely open for spidering. They welcome the bandwidth usage. Bidder's Edge, however, was using a robot in violation of the TOS. In effect, they ignored a big huge "No Trespassing" sign (a very narrowly tailored "No Tresassing" sign.) As long as you follow the site's TOS, and respect the robots.txt, you're fine to spider it.
While some of us probably consider eBay with the same disdain as we hold toward AOL, we should cheer this ruling.
-sk
Bidders edge isn't some ebay leech, its actually a meta-auction that categorizes and searches many different auction sites, not just ebay. I don't see why the consumer shouldn't be allowed to make cross-auction searches for products. Obviously, the reasons the judge gave would shut down every search engine.
Instead of embracing a system that helps consumers and gives them choice, Ebay has decided to use its finacial muscle to eliminate easy to access competitor information. Essentially ebay is telling its customers, "Don't try for better deals just stick with the brand name you know, us."
This is a ruling that is ignorant, anti-competitive, and needs to be overturned.
---
This is just plain stupid; if you have a page on your website which is viewable by the public then it is available for the public to download.
---
Hrm. How far do you want to extend this, though?
Couldn't it be said that someone launching a Denial Of Service attack by simply requesting documents at an extremely high rate of speed is 'viewing the documents' as expected?
I do think that the judge's excuse was a little suspect - if it were server resources alone, you'd think that this second company would be saving them load by diverting their customers (and advertising dollars) elsewhere.
I do agree that they don't deserve to win, though, by virtue of how they're stealing someone else's content. It's not always certain what kind of precedent this will set though...
- Jeff A. Campbell
- VelociNews (http://www.velocinews.com)
- Jeff
I am rooting for them to lose because they are in effect *competing* with eBay for advertising dollars by *using* eBay's content.
You mean the way Slashdot -- a technology news site / community / portal -- deep-links to articles on other news sources all over web for its own content? Admittedly /. links to the sites, but if the content didn't exist neither would the bulk of /.
Wired and C|net in particular would have /. for breakfast if this set the kind of precedent that seems to be happening.
Why do you all use Yahoo for an example for a spider/search Engine? ;-) "Wie immer sind alle Angaben ohne Gewähr"
This is really interesting as Yahoo is not one of these. (And AFAIK never has been). It's just a directory.
What you probably mean is Inktomi, which provides this service for Yahoo.
What's the English word for Klugscheisser?
I use Junkbuster, which means ebay doesn't get any ad revenue from me in any event. So am I "stealing" cycles from ebay too? Is my conduct indefensible and really lousy?
"The legitimate powers of government extend only to such acts as are injurious to others." Thomas Jefferson.
This could have interesting consequences. Most sales-oriented sites use scripts which your average spider will avoid (due the the *.cgi URLs). On the other hand, to reduce the system load, some sites re-generate their pages at regular intervals (say, every minute) instead of every time a user loads the page. Looking at ebay's site, it appears their URLs end in *.html. Perhaps what is needed here is something in the robots.txt file (does ebay have one of these?). But then again, not all spiders respect this file, so I don't know. If the web site designers don't take the necessary precautions to protect the private information on their site from spiders, how can the spiders know not to catalog them?
--
For every post, there is an equal and opposite re-post.
Bah, a legal "solution" to a technical problem. What is ROBOTS.TXT for?!
(Now, if the search engine provider doesn't respect the ROBOTS.TXT exclusions, I say go after them with legal means.)
Well, seeing as hiw there are only "first posts" up so far (five of them, no less), I guess I'll take a shot at an on-topic post.
The problem here is where do you draw the line at fair use? ebay is providing a publically-accessible database; why should a search of that database (by robot or not) be considered an abuse of their servers? Of course, if the search puts so much strain on their servers that no-one else is able to access them (effectively a DOS), then an injunction would be reasonable, but this search doesn't seem to have caused undue server load.
The point that timothy brings up is not really relevant here, I think - your HD is not a database offered for public access, and should thus be protected from undesired searching, whether or not this injunction is upheld.
...server side? Like, adding a deny ... to their web server config file?
Aside from that, this is really a problem. I mean, not the individual story, but the additude that a company can simply deny somebody like this the right to use a service that they provide. If would be like if you owned a software shop, but you kicked the reviewers out everytime they wanted to write up a story about what you have there. It's just not right. I mean, most everybody on Slashdot should realize that the amount of server time these requests are taking up is less than the amount of time their legal department has spent looking into this. Less than the amount of time needed to post this, even. I mean, eBay isn't running on a 486 anymore. They can handle thousands of people a day, so what's wrong with a web-bot or two? Actually, web-bots, in addition to bringing more people to the site than a normal person connecting to the server would, are also much faster in completing their communication with the server, giving eBay more customers faster and quite possibly easing the load on their servers, by (if I read the story right) mirroring product data on another site. This isn't like somebody is launching a DDOS against eBay. This is a web-bot, for goodness sake! Why do they even care?
At first blush, it seems like this is a stupid ruling, mainly for the reasons the judge gave for making it. He claims that they are essentially stealing cycles from eBay's servers and this could slow down ebay's service and have a negative impact on their customers' experience.
This is just plain stupid; if you have a page on your website which is viewable by the public then it is available for the public to download. That's the point of having a public website. Hey, I'm a customer of eBay's, am I guilty of using server cycles and slowing down the eBay website for other customers? You bet. eBay should secure the entire site and require authentication if they really want to pick and choose who can view.
On the other hand, I think what Bidder's Edge does is really indefensible from an ethical standpoint and I am rooting for them to lose because they are in effect *competing* with eBay for advertising dollars by *using* eBay's content. If you view content from ebay through Bidder's Edge, that's advertising revenue eBay doosn't get which BE does. Seems really lousy.
So it seems like the right ruling, but for totally the wrong reasons. The way the judge worded things it sounds like you could make a case for suing Yahoo, AltaVista, Google etc., if they dare to spider your site.
Whot crap!
-- "Vote Democrat. Because the current crop of conservatives are just bugnut crazy."
I really don't see much similarity with search engines. Yahoo, for example, will scan your site (within parameters you can specify via robots.txt) once in a blue moon. That's VERY different than a robot that scans the same site repeatedly 224 hours a day. I'd get pretty miffed, too, if Yahoo, Altavista, Google, and all the rest of 'em hammered away on my site constantly just to see if I'd changed anything.
On a lower-bandwidth site than eBay, you'd call it a DoS attack.
---
Consult, v. t. To seek another's approval of a course already decided on.
This is absolutely awful. Republicans in the US always like to use the term "legislating from the bench" to describe rulings by liberal judges which overstep the bounds of the case being argued. This is actually worse -- the judge is essentially making up technological terms as he goes along. Using someone's resources that they can never get back? Give me a break, the crawler that Bidder's Edge uses uses an infinitesimal amount of EBay's and other auctions' server capacity compared to the legions of "legitimate" EBay users. This judge is speaking from pure ignorance, and his ruling endangers everything the Web is based upon.
Where do you draw the line? Are we only going to allow "manually" retrieving information from a Web site? What does that mean? Do I have to write code for each page I want to see? Are offline browser caches now going to be illegal since they automatically "drill down" into sites and grab several pages at a time for later viewing?
When you create a Web site, you do it under the implicit assumption that people are going to connect to it and retrieve informaton. End of story. There is no "right" to only have your pages viewed by means you approve of. Every time _anyone_ connects to your site they use some of your resources, and doing it by automated means is no more onerous than by doing it "manually".
Trying 216.32.120.97...
Connected to ebay.com.
Escape character is '^]'.
GET
HTTP/1.1 404 Not Found
Date: Fri, 26 May 2000 09:21:38 GMT
Server: Apache/1.3.6 (Unix)
Connection: close
Content-Type: text/html
404 Not Found
Not Found
The requested URL
Apache/1.3.6 Server at ebay.com Port 80
Connection closed by foreign host.
While IANAL, I have a friend that is and I think that giving out legal advice (which can consistitue answering legal questions as such) without charging for it can be considered a violation of the Bar (not originally used to make money but basically to prevent the legal profession from being blamed for giving out bad advice when casually asked for it.)
"Pinky, you've left the lens cap of your mind on again." - P&TB
"I can see my house from here!" - ST:
If I understand this correctly, the reason the judge forbade Bidders Edge to crawl eBay is that eBay told Bidders Edge not to do it but they did anyway. So it seems fair to me: if I ask you not to keep following me around and you keep doing it, I take you to court to get a restraint order slapped on you.
Whether it's a smart move by eBay to do this is an entirely different matter, however. Apparently all the experts agree that in the near future all transactions on the internet will be mediated by robots, which search out the best deal on the product you want to buy. So as far as the consumer is concerned, it makes no difference if a book comes from Amazon or Barnes and Noble, he'll just get it where his robot tells him it's cheapest. Most probably, he won't even know it's from a certain retailer until he looks at the box in which it was sent to him. Probably he won't care very much even then. At this moment, there's already a number of services that operate on this principle: Bidders Edge does it for auctions, and in NL there's a site called ElCheapo that lets you find the best deal on products ranging from airplane tickets to records.
By banning a robot-service from their site, eBay is, in effect, shutting itself off from the way business will be transacted in the near future. They will not merely lose some ad revenue, as is the case now, but they will lose all their customers.
If this really is what they want to do, I'd say let them.
News and bla for computer musicians: http://lomechanik.net/
it'd be interesting if such a ruling helped discourage hard-drive searching by software which searches for "undesirable" content without your consent or knowledge.
Yeah right.
eBay is a big company with money. Even more, they have a little 'e' at the begining of their name.
Are we (as individules) big conpanies? No.
So why on earth would you think the law would protect us?
Finkployd
I do not object to individuals making use of my site in an interactive manner. I don't mind if people make use of the information on my site. What I do object to is people using a disproportional amount of resources that I am paying for. Many spidering and site sucking programs do exactly that.
If all programs out there that act like "spiders" (not just the search engines) actually used and respected robots.txt and other mechanisms in place to tell spidering programs how to behave, we wouldn't have to use the courts to enforce our right to tell people if, when, and how they can use our resources.
-- PhoneBoy
The views expressed herein are not necessarily those of anyone, including the poster.
here is http://search.ebay.com/robots.txt:
It isn't like eBay is disallowing access to everything, crawlers are allowed to index anything on www.ebay.com (no robots.txt) and whatever is not excluded search.ebay.com. IMO whether the judge knows it he is upholding a standard and that is a good thing.Citrix
Leknor
http://Leknor.com
"So many idiots, so few comets"
No scan, no ban?
eBay has decided that it would be much more profitable to simply sue all of its customers for infringing upon its personal property.
.001%, and response time was instantaneous.
After a freak DNS outage Monday morning, executives internal to the network noticed that their servers performed thousands of times better than when all "those damned customers" were using them. CPU load dropped to
It is this degradation of performance that enables eBay to lay claim to the pocketbooks of all its consumers, and auctioneers. eBay is currently debating lawsuits on volunteers helping to run eBay.
When asked "Isn't this going to jeopardize the good PR that has made eBay so successful?", eBay's response was "Ah, fuck it.. Microsoft has been doing this to customers for years, and they appear to love it... Besides (excitedly) WE'RE FINALLY GOING TO GET OUT OF DEBT!!.
Seriously though, this is a bad idea. If we can't expect our high-tech businesses to make more informed decisions of how to deal with a problem, then who can we count on?
The problem itself could easily have been resolved with a robots.txt, and failing that, DENY host settings. Aside from negative PR, why would a company want to take this to their lawyers before taking it to the same techs that made the company to begin with?!?
The next biggest question, how do we protect ourselves from this?
As someone who has written search engines, both as spiders and as meta search engines, I found the existence of the robots.txt standard to be a real blessing. It was a way for me to easily follow rules that kept my site from simply being BLOCKED at a firewall. By further doing things like not going to a specific site twice in a row and other wierd tricks, I could know that I was not abusing other peoples site with my code running automatically.
In the case of a system that is essentially placing their own interface ontop of another system, what I don't understand is why EBay didn't simply refuse the packets. It should not have been THAT difficult. No user has a right to see a site. I have blocked parts of my site from certain users during times when they perhaps inadvertantly caused trouble. I have blocked use of my email system for the purposes of redirection. All of this I do with the technology available to me. The fact is that IF the improper usage of the site is impossible to detect by the owners of the site it isn't a technical problem. On the other hand, if the issue is that they are successfully using the site in a non intrusive manner to profit, it is a business problem, and the problem is NOT with the offendor but with the offended.
The problem is the notion that a site can control how it is viewed.
If you tell me that it is illegal to look at a site without seeing its ads then we are in for a long hard battle. Browsers are being built with the ability to recognize advertising banners and exclude them from being viewed. Text based browsers have always avoided banner ads. The use internet for advertising sales will someday be impossible unless the advertising is completely the same as content, which it IS to a large extent. Content can be filtered, and should be. I look at slashdot with a WAP enabled phone. No ads are sent. Sooner or later someone will fix that somehow but in truth the ad will be just a link to another site, which is fine with me.
In the future it is considered likely that everyone will have a semiautomated search engine customized for them personally. Near future. In the long run, automated agents will be the primary conduit between a user and data, the web browser will be a thing of the past, and the idea that people won't filter out advertising when it bothers them/wastes their bandwidth is nonsensical.
The Robot.txt file says, "Don't check these URL's for content" which basicly means "Don't read me". In the future sites that restrict use of their pages won't be read. If the only revenue a site gets is through advertising, then they will have to find another way to deliver it.
You know, news like this is really depressing. I got on the internet in college before the web. Back then, (listen up you young whipersnappers!) if you tried to sell something on the internet you were spammed into oblivion. These days with $$$$,Lawyers, and Uncle Sam's long arm, the internet is fast becoming an environment owned by corporations that control how and what you can do. Pretty soon it will be easier to start a radio station than launch a web site.