Slashdot Mirror
Texas Court Blocks Screen-Scraper
An anonymous reader writes "A Texas court has granted American Airlines
an injunction against Farechaser to stop them from using a screen-scraper to copy airfare information from their website in violation of the terms and conditions. In a stunning display of hypocrisy, Farechase.com's own terms and conditions prohibit users from doing to them exactly what they are doing to AA.com. The EFF is involved, but it's unclear whether they're supporting the enforceability of a website's terms and conditions or Farechase's right to violate them."
Legislations, counter-legislations.
Did the web die suddenly or did it just wither away without anyone noticing?
Yes, I am a Muslim. No, I am not a Terrorist.
If you are making information available to the public, what right do you have to be angry when someone, *gasp* uses that information?
The article also fails to mention that Farechaser was sentenced to death by lethal injection in the same decision...
Roving Web-Teleoperated Robot
Fortunately, American isn't basing their case on the idea that their prices are their "intellectual property," but instead are claiming that Farechase was trespassing on their chattel. I wonder if the decision on this could effect cases involving DoS attacks?
My employer has "requested" that I write a screen scraper to grab information from a competitor's site. The data will then be put into our databases, and sold as our own. This is in violation of the competitor's terms of agreement, and I have thus far not done this.
I am unable to find anything on Google relating to the legality of this, but I believe that it is probably not legal. However, I was told to "do it and let the lawyers deal with it".
Whether it is legal or not, I do not feel that it is ethical, and may leave the company if I am pushed to do this.
Does anybody know where I can read any case law on screen scraping? Aside from the current article, that is.
I can't say that I don't give a fuck. I've just run out of fuck to give.
As much as I love them, this is is one of those cases where they can't tell if they're coming or going. Maybe they'll write an Amicus Curae for both sides, and confuse the shit out of the judge.
:P
And for the record, I sent in my annual EFF cheque last month
Cue The Sun...
There is a fundamental distinction between programmatically scraping someone else's site and posting it as your own and an individual drawing down the website via a browser: fair use.
Provided fair use conditions are posted, I don't see where the scraper has a leg to stand on. If you are a competitor, you have different rules, as your intention and the actions that follow your intentions separate you from a normal consumer. To illustrate, it is fair use for me to go to the library and photocopy an article out of a journal and use it as source material for a paper. It is NOT fair use for me to photocopy the article and put it in my own magazine, publishing it as if it was mine, copyright and all.
That being said, I would be very interested in an informed reply from a lawyer that specializes in these matters.
Regardless, Farechaser should not tell its customers one thing then turn right around and do it themselves. Especially when they stand to make money doing so. Sure, it's legal, but that doesn't make it right. Personally I think they ought to nationalize most of the airlines given the current economic situation but that's not likely given the stockholders interests and the current administration.
As long as there is a Second Amendment, there will always be a First Amendment.
In Bidder's Edge, the federal court granted eBay an injunction preventing Bidder's Edge from harvesting information from eBay's website for the purpose of using it on the Bidder's Edge site.
IANAL (but I will be in a couple years), so don't expect legal advice from me.
I think the Bidder's Edge case was decided on a weird basis (eBay has a right to protect its servers from harm), rather than the reasons you'd expect (eBay's data shouldn't be jacked by competitors and used to hurt eBay's business). Nonetheless, I expect other courts would rule the same way on a similar case.
Note: in this case, eBay has specifically told Bidder's Edge not to take the data.
Er, dude, that wasn't precendent. That was what Walmart claimed. They then proceeded to back down when their victim pointed out that they'd claimed that under penalty of perjury.
At least try to pay attention.
Jon Udell created a bookmarklet called LibraryLookup to take the URL of the page you are on and pull off the ISBN and check your local library for the book. It can be used at places like Amazon.com. It is a very creative idea of using different websites to create the useful application. Now one would hate to see applications like this be outlawed. Maybe the difference is that the use is personal (fair use) and not "commerical".
If American Airlines wins this case, what happens to froogle.google.com?
Part of the Terms stated You agree that you will not take any action that imposes an unreasonable or disproportionately large load on our infrastructure.
Does posting the link on Slashdot count?
here and here. And that's just in a couple moments spent searching on the ACLU's own site.
--Dg
This goes along with the article on pricing. In industries where pricing is heavily competitive (e.g. airlines, rental cars, computer equipment) pricing information is constantly shopped. The companies can't ask each other for rates because it is banned by by anti price fixing laws. So they shop competitors' prices on information services, some even have shopbots to do the work. Sabre, the largest travel booking network in North America, which is a closed network, has blocked some paying customers due to excessive shopping. They block shopping because automated shopbots elevate CPU and network usage. These shopbots can't tell when rates have changed, so they continuously hit Sabre. Wonder how hard Farechase was hitting AA.com?
At my company, a major major market site, we publish all kinds of advice, hot socks, bond rates, mutual fund performance calculations etc., etc., etc... All this is publicly avalible on the site.
I noticed that a bunch of smaller companies were scraping our content, stuff we owned, and redisplayed it on their sites..
So, what we did was block the servers that were scraping us, and it largely has stopped the problem. Trying to stop the remaining theifs from manually copying our content isn't worth our legal department's time, and since they are all teny tiny sites doing this, we don't care that much anymore.
But the larger theifs who were profiting from out content --and scraping in large amounts-- don't do so anymore as a result from denying their scraping servers.
There is a connection between screen scraping and privacy arguments.
Many here are posting "What's wrong with screen scraping? The information is there, I can get it, what's the problem?"
By the same token, many decry things like Amazon.com selling their address just because "they have it".
Information is information. If you have the "right" to scrape whatever you want and distribute it however you want, then companies have the right to distribute your personal information to whomever they want, under whatever circumstances they want.
I prefer to live in the world where control over information on the societal level is still allowed; no concept of privacy can exist without it. Not letting people screen scrape isn't even something I'd consider a "price".
To be a bit more theoretical, there is value in information transfer.
This is a summary of a much longer and more complete argument, but it should get a lot of the point across. I won't be defending this on a point-by-point basis in replies as a result.
My advice here is make sure you have a new job lined up. I "took a stand" about 14 months ago, ended up being pushed out, and just got a new job a few days ago. Moral of the story? Have a fallback position, or a big bank account.
See, they can't fire you for refusing to break the law, but "you're not a team player" situations like this can lead to people getting fired for forgetting to refill the printer paper, or failing to turn off the lights on Friday night or something stupid like that...
And definitely don't tell somebody you're interviewing with the reason you want to leave--You will almost automagically be eliminated from consideration for saying something "negative" about your former employer...Even if it is the truth.
You have to come up with a creative quasi-lie to explain your desire to take the risk of changing jobs in a shitty economy.
Who did what now?
TERMS AND CONDITIONS OF THIS COMMENT: YOU MUST AGREE TO THE FOLLOWING TERMS AND CONDITIONS BEFORE READING THIS SLASHDOT COMMENT. Copying this comment is prohibited. Thinking about this comment is prohibited. Once you have begun reading this comment, you must leave at least one window of your browser open at this comment for the rest of Time. Prohibited circumvention measures include, but are not limited to, scrolling up or down, closing the browser window, or restarting your computer. All readers whose browser does not stay open to this comment for the rest of Time will be prosecuted to the full extent of the law. You attest that you will be fully responsible for this obligation regardless of any outside forces that may inhibit your ability to carry out this obligation, including but not limited to unstable Microsoft software, power outages, plagues of locusts, or the Apocalypse. Reading this comment while not under the influence of alcohol is prohibited. You may not tell your friends about this comment. Telling anyone that this comment was funny is prohibited. Moderating it "Funny" is prohibited, and offenders will be prosecuted to the full extent of the law. Reading this comment while sitting in front of a computer is prohibited. All violators of this comment are required by law to pay the specified fine of $10,000 to xenon@@microsoftsucks.org using PayPal. This EULA is considered a part of this comment and is the intellectual property of the owner. ALL RIGHTS RESERVED.
NO WARRANTY: THIS COMMENT IS PROVIDED "AS IS" AND IF YOU DON'T LIEK THE SPEELING YOU HAVE TO PAY ANOTHER $10,000 FINE TO ME.
--
They can't force customers not to tell their friends about the prices, so how the hell do they have a right to stop anyone from relaying that information automatically? This kind of nonsense reminds me of software EULAS: You can't see the license before you buy it, but you must agree to the license to use the software and you can't return the software if you don't like the license because there aren't any stores in the world that accept returns on opened software. They're esssentially extorting rights from you without giving you any choice other than to throw away the software you just paid $$$ for. Posting something on a website is equivalent to posting a comment on slashdot, which is equivalent to yelling it out in public. You can't have a website listing prices or goods and have a website EULA prohibiting other sites from parroting it (xxxxxxxx company is selling xxxxxx for $xxx) any more than you can yell something out on the street and then force others to not tell anyone what you just said (or else get sued) by then dictating a verbal EULA for whatever you just yelled out.
Repeal the DMCA!
I'll never forget their position on Microsoft "Smart Tags." That's when I realized I didn't understand EFF anymore. The very idea that a web browser displaying a page "incorrectly" could possibly be copyright infringement... *sigh*
This was no simple case of "incorrectly" displaying a web page. This was a case of intentionally modifying the text of a potentially copyrighted work, providing links to some external site.
In my opinion, and obviously many others (since it seems to have gone away on its own anyway), what they did was sneaky and wrong, not unlike many similar spyware programs.
As for this case, hm, I'm not sure exactly where I stand on this. I kinda want to say that, copyright aside, just the fact that they took the time to compile the list, and are making it available *to their visitors*, makes it wrong to just copy the data and post it on your own site. But, then again, I'm not sure. It's a tough one...
NGWave - Fast Sound Editor for Windows
To clarify EFF's position, we're not involved in the case at this point. We do strongly support Farechase's right to access a publicly available website and repost uncopyrightable facts gathered from it.
We may get involved with the case as it proceeds, but for the moment, we've just posted a copy of the injunction for reference.
-- Openlaw: Fighting for fair use and the public domain
This logic is BACKWARDS.
If people scrape you and mirror your data, the load on your server should go DOWN. If this is not actually the case, then you can't really claim any sort of harm.
This would be better described as "involuntary mirroring" than "screen-scraping".
A Pirate and a Puritan look the same on a balance sheet.
Wanna see something stupid up in Canuckistan? Our post office is suing a company that makes StrongPost, claiming that the post office owns the word "Post". So much for Post cereals, Post-It Notes, etc. Now THAT's stupid
I've developed a free program called MileTracker that consolidates users' frequent flier accounts. One of the accounts that I pull information from is of course at AA.com. I haven't received any complaints yet, but there are several reasons why I can think of that:
- I do the scraping from the client. There is no central server involved at any point.
- There are only a few hundred users at this point, each generating at most 2-3 requests a day. More likely they're only generating 1 request every few days.
- I've been very careful to simulate an actual human user using a Web browser. There are some steps that I skip for performance reasons (such as downloading images which could be cached anyway), but I make sure to handle all cookies, redirects, Javascript, etc. I also throw in some delays and randomization.
Someone looking through logs would probably have a lot of trouble telling the difference between a real user and my "ghost" user.The application really isn't harmful to AA. In fact, there's a good chance that I actually increase real traffic to AA.com amongst their loyal customers. When there is a change in a user's frequent flier account balance, I provide an easy way for them to actually login to AA.com so that they can view the details. If AA complains, I'd probably obey since I have more to lose than I do to gain. It's AA's customers that use MileTracker that would suffer.
Our company gave up its internal call center ops. Another company bought it lock, stock, and server.
They also made an offer to all the current phone people for continued employment. Not all accepted.
Now...we (I) had written a personnel database to manage the phone peoples schedules, hiring, firing. This included name, address, phone, SSAN, paygrade, etc, etc. The new company also bought this.
I get a phone call from the IT VP "Send so and so a copy (incl design, etc) of the personnel program. Including all the data. They want to get up and running as soona s possible."
"WHAT??? We can't give out all that personal information! What about all the people who did NOT join the new company?"
He pushed, I pushed back.
"OK...I'll do it, on one condition. A signed letter (not an email) from:
The head of HR
The CEO
The COO
The head of our law firm
and you (the IT VP)
Otherwise, I'll send them a sanitized copy. No personal data whatsoever."
After much discussion, they backed down.
I went over and installed the sanitized program.
Bottom line...if you feel it is illegal/immoral, stand up and be counted. If you're wrong, you'll find out soon enough. They you do whatever they want.
If you're right, the powers that be will not stick their neck out that far.
Protect yourself. Otherwise, when the shit hits the fan, you will be one of the scapegoats.
Some things to remember about pilot salaries:
1) They can only work until age 60. FAA rule.
2) They typically spend 8-10 years before making $75,000/year. My buddy is 32 years old, has flown for a 'regional' since 1997, and is still only making $50,000/year. And with the downturn since 9/11, it could easily be 7-10 years before he moves up the ladder (and he'll have to take a pay cut when moving to a 'major' airline the first few years).
3) Any number of health conditions can end your career without warning, or just leave you unpaid. Have a heart attack at 55? Forget it, you're done.
4) These guys have to be tested in a full-motion simulator every 6 months. Blow more something here, you could find yourself out of a job.
5) No ability to "laterally" change jobs. If you fly for an airline (such as Braniff) and have 15 years seniority and are a 747 captain, and Braniff goes out of business, you're new job will have you sitting in the co-pilots seat of a DC-9 or 737, making 36 grand a year again.
6) Oh yeah, you're in charge of the lives of hundreds of people each day. Your mistakes don't cause downtime, they cause DEATH.
Now tell me again why pilots are overpaid?
Steve
American uses Sabre internally (in fact used to own the WHOLE thing) and owns 25% of Worldspan (got it when they bought TWA): both customers of Farechase.
Ultimatly this is about controlling the distribution channel for tickets: first stop paying commission to travel agents (that's right! they make 0% commission now) and now try to put a clamp around screen scrapers.
Then there's the the fact that Orbitz is owned by the majors and is the very definition of collusion.
The owners of Worldspan (which powers Orbitz and is Delta, American and Northwest) plan on bypassing SABRE and putting inventory on Worldspan for sale through Orbitz. This has already been announced. Why? Because if you buy a ticket through a SABRE channel (like travelocity), SABRE charges AA (and every other airline) a $4 segment fee. Not so when you buy it through your own CRS/GDS system.
Again, it's about controlling the distribution channel.
I wouldn't know this except I used to be employed by American as a SABRE programmer.
American is nothing more than a bunch of slimy bastards (like all airlines with the exceptions of Southwest and JetBlue) and will deserve their bankruptcy filing when they do it.
And they WILL given the failed airline economic model that the "majors" subscribe to. In a very funny ironic turn of events two new 747s are valued at more than the whole of AMR (AA's parent).
Don't buy their stock anytime soon except to short it.
it's not "stealing". It wouldn't even be "stealing" if this information was worthy of copyright. HOWEVER, the fact remains that this information is NOT worthy of copright protection.
But, its not just about the information, they are accessing their hardware using their bandwidth ignoring the agreement. Its the same thing spamers do that make everyone here yell. Funny how people can change their opinion when the afected is a for-profit company.
I can open a restaurant and give menus away so my clients can order by phone, but if a fellow comes every day and takes a menu of mine just to copy the precies to his own menu I would never give one to him again, is my damn right after all since are my menus I printed in the paper i bought with the ink i bought, same with bandwidth.
Sigs are for morons... Wait a minute...
Oh yeah, from now on, they get a nickel every time you buy Canadian bacon, too.
I see even classic Slashdot is now pretty much unusable on dial up anymore.