Slashdot Mirror
Hacking Web Services
siduri writes "Udi Manber, chief scientist at Yahoo!, gave a great talk on the kinds of hacks that Yahoo sees at the IEEE's Symposium on Security and Privacy. I wrote an overview of his talk for Dr. Dobb's Journal. While some of the message is well-known stuff (like that people will spend a lot of time hacking the most trivial things), the details of what Yahoo has to deal with are really pretty interesting."
Interestingly enough, by copy-and-pasting the whole text of this story from Dr. Dobbs to Slashdot, you have unwittingly done one of the more common "hacks" that Udi Manber describes as being dangerous. Information stealing is easy to do, and sometimes doesn't even feel like it's a crime.
Congratulations for illustrating his points so directly.
--Mid
If anonymity disappeared from the web, "a lot of the problems would go away," he said.
That's especially true if you equate users with problems ;-)
But he dismissed legal solutions altogether, saying that measures like anti-spam legislation are completely ineffective. "This has to be solved technically, not legally," he warned. "If we can't solve these problems, we'll see less and less services."
That's a point that is occasionally debated in anti-spam circles. The problem there is that the Internet mail delivery system was designed for the kinds of users we had 25 years ago. Heck, it wasn't until somewhat over 5 years ago that all the MTAs [that mattered] would ship with relaying turned off by default. Looked at from that perspective, it seems like a technical problem... change the delivery system and you make the abuse irrelevant. The problem is, how do you implement such a change? It's not so much a question of designing a new system... I've seen a number of proposals that looked fine. The problem is, how do you get all the mail servers on the net to switch over?
At that point in the debate is where the division usually comes in. Some folks will propose various systems for gradual adoption of new systems (essentially having two delivery systems in place until the new one is widely adopted enough to drop the old), while others pull back at that point. They'll say that spam is a social problem and, as a result, it can't be solved technically. Usually those folks will go on to pursue legislative attempts at a solution. The problem is, the track record of using legislation to solve social problems is nothing to write home about.
If he can come up with a technical solution for Yahoo!, of course, then he is all set. The problem, as he said, was that you only have so much identification information available to you at the server end. That makes it nontrivial to reliably separate the valid users from the rest. The thing is, just how much personal identification information are you comfortable giving to Yahoo! to get a mailbox...?
Perhaps, but I actually know Udi. He teaches an advanced computer science class at the University of Arizona (or at least he did in the mid 90's). In terms of problem solving and cleverness, this guy was high on the list.
For what it's worth, however, I totally failed his class. Way over my head.
In 1998, we had started a company with the sole purpose of proving who and who is not a robot on line. We developed a range of techniques for detecting bots and stopping spammers -- images, rate limits, statistical techniques, etc.
The two most important techniques were what we called the "Visual Turing Test" and a reapplication of a cypherpunk scheme called HashCash.
The Visual Turing Test is widely used today, it's the image generated with a code that you have to type in. Our technique started with that, but went much further to defeat OCRs by including AI-level questions, such as displaying an image with a dog, a cat, and a horse, with instructions in the image that say "click on the one that is not a house hold pet."
Back then, we ran a free webmail service for people, without adds, using these techniques to stop email spam.
We were a very poor start up, working over a year with no pay. We went to Yahoo and had a meeting with their engineers and biz-dev people, under a *nondisclosure agreement*, we demoed all this anti-spam, anti-fraud technology. We were looking to sell them the scalable image generation server software we wrote, statistical analysis software, and our services, and potentially our patent on these techniques.
Yahoo basically said "not interested" after several meetings, and one yahoo engineer basically said "We could implement this all myself, why do we need you?" We never heard from yahoo again, didn't get any more meetings. But magically, about a year later, we noticed yahoo using our techniques.
Our company was eventually bought by one of those "pay to watch ads" companies, because they had massive fraud of people installing fake clients, and signing up for hundreds of accounts. Unlike Yahoo's fraud problem, these companies were paying out tens of millions of dollars in cash to people who were signing up bogus accounts.
But it still doesn't take away from the fact that Yahoo is a dishonest shark. If it wasn't for the fact that I am morally opposed to using software patents against people (only had one to make our biz plan look good for investors), I would have sued them.
Word to the wise. Don't present your ideas to yahoo as a small startup and expect they will abide by an NDA.
I've met, and worked with, several Phd holders who could best be described as "morons", and whose ability to solve problems was limited to applying their hammer in a manner that presumed that everything is a nail. Again: I have no doubt that there are some brilliant Phd holders (often in exclusive fields however), just as I know that there are some brilliant non-Phd holders, however blanket claiming that one title indicates a superior being is ridiculous, and I'd love to see an intelligence and "cleverness" ranking between Phd holders and general comp. sci. grads.
Have the humans do something that machines can't do very well, say image recognition and/or categorization.
A simple "Tell me about this picture" and an associated image and a text box would do. If the text submitted does not match a previously stored description well enough, no deal.
Every one in five or so, put out a new, previously un-cataloged, image and log the description...That would also be an easy way to beef up their image search engine.
Good judgement comes from experience, and experience comes from bad judgement.
- W. Wriston, former Citibank CEO
The solution exists, it's just that the transition to the solution will be painful, so we're desperately trying to avoid it.
The solution is whitelists and "postage".
Put all your friends in a whitelist. Main from them is delivered instantly.
Anyone else who emails you gets an autoreponse, "I don't know you. To ensure that you're a real human being, you'll to need to run the postage program to get the result for the code ABAASDFFEFEF". The program needs to be open source and easily verifyable for security reasons. The program solves some problems that is hard to compute (say 60 seconds), but easy to verify. One example would be a brute for cypher break on a simple cypher. The senders email client can handle this autoreponse automatically, shielding the sender from needing to deal with it (Gee, my computer gets slow for a bit when I email someone new). Spammers, on the other hand, would need to either limit their spamming so they have time to generate valid responses, or would need to invest in expensive hardware to generate the responses fast enough. End result: It's no longer cheap and easy spam.
There are a few other details to make mailing lists feasible, but it's doable.
However, this effort would require everyone to upgrade their mail clients or to use external programs to manage this. Given that extremely slow adaptation of other email security features, I'm not optimistic.[B
I went to this talk (and this conference). He basically said that a lot of attacks are just sequences of actions, any of which individually are not a problem, but when combined are a problem. I'd call that a "security" issue. The result is that he can't offer certain services. There's a social good issue there, and an interstate commerce issue, so Congress could easily claim jurisdiction. Not that I'm suggesting that that's the right solution...
/. is also theft. Selling premium services is a valid business model; some people subscribe once, scrape the screen, and have their own premium service. If that isn't "theft," you've effectively said that "information has no owner," in which case you have no recourse for your ISP selling all your packets.
As for "theft," whether you like it or not, taking my data and selling it without permission is theft. Yes, spyware is theft; reposting NYT articles on
Couldn't ebay break apart the login username from the bidding username? This wouldn't eliminate the problem, but it could certainly help since the attacker wouldn't immediately know which account to attempt to block out through bad access violations.
Dodger_
Sometimes, it's much easier to use information if it's not tied down to a browser page-- perl programmers have been parsing web pages for years. Various versions of Excel can do this as well, importing data from Yahoo! Finance 's stock ticker directly into a spreadsheet. Sherlock (for MacOS) parses search engine results. BioPerl parses NCBI webpages (among others) into sequence data...
Obfuscated code makes this type of activity less useful. The trouble is that most of the services are tied to an archaic, and annoying advertising based model. Sherlock gets around this problem by actually parsing the ads and displaying them to a mac user. But most clients are built not to avoid ads so much as increasing the usability of the data. For some things, web browser interfaces leave a lot to be desired.
I was a bit hand-wavy. (Ooh, look at me, I'm a futurist!)
The key is to just add a very small cost. The advantage using CPU time as the cost is that it's easy to automate. However you have a good point.
If we don't change anything else, yes, mail from slower machines will take longer to be delivered. A problem that takes my computer a minute might take a lesser machine ten minutes. However, it's not that terrible, you should be adding friends, coworkers and other people you want to get email from to your whitelist, so they'll be paying the penalty only once. In fact, this can be automated as well: anyone who answers the question one can either be added to your whitelist (and if you later decide you change your mind, moved to a blacklist). Or your mail reader could return a ticket to avoid the answer after answering the question once. Again, you could revoke a ticket if you determined someone was harassing you.
The other solution is to skip computers and force human interaction. Each user would generate a simple puzzle that is hard for computers to parse. The sender will get the puzzle back and his email won't go though until he answers it. You would only need one puzzle, the key is that it needs to be hard to parse with a computer. For example "What is 6 times seven? Add one to the result. Subtract three. Repeat the second step with a tenfold larger number."
Now that UPS is attempting to charge us for "excessive use" of their web site, we track or competitors shipments too
you lying sack of shit how can you track your competitors shipments. you need tracking numbers.
Mod down people who tell people how to mod in their sigs
And how long would it take until we see distributed cypher breaking and adress sharing amongst spammers?