IE Should Use Google's Malware List
But I had a different question: Since Google allows anybody to use the Safe Browsing API, why doesn't Internet Explorer use it as well, in conjunction with their own blacklist, so that a site will be blocked by IE if it's present on either list? This would almost certainly increase the block rate for IE (unless the set of sites blocked by Safe Browsing was entirely a subset of the sites blocked by SmartScreen, which is extremely unlikely). Google's Terms of Use for the Safe Browsing API do require parties to obtain written permission for any usage that will result in more than 10,000 users sending "regular requests" to the API, which would obviously include Internet Explorer. But Google already serves requests for all Firefox users who have the SafeBrowsing API turned on, so for them to process requests for all Internet Explorer users might require four or five times as much computing power, not orders of magnitude more. It's impossible to guess what kind of deal Microsoft and Google would make for the right to have IE do lookups on the Safe Browsing API, but if Microsoft placed a dollar value on increasing the protection for their users, and that dollar value exceeded the cost to Google of running the servers to process the additional queries, then in theory they should be able to agree on a price between those two amounts. Google might well offer to service the queries for free, just for the prestige of being able to say that the Safe Browsing database provided protection for almost all major browsers on the market.
(Microsoft's SmartScreen team declined to comment on the record about their reasons for not using the Safe Browsing list in addition to their own database. I couldn't get an official response from Google about what position they would have on Internet Explorer using the Safe Browsing list, although unofficially an employee said the team would probably be "delighted" if IE were to use it.)
It's worth underlining what a strong statement Microsoft is making by not using the Safe Browsing list. They're not just saying that their own list is better. They're saying that the Safe Browsing list is of such low quality that adding it to their own product would actually make the product worse.
This is different from, for example, what McAfee and Symantec might say about each other's anti-virus lists. Consider the set of all viruses that McAfee blocks and the set of all viruses that Symantec blocks. Let List X be the overlap — the huge swath of viruses that are blocked by both McAfee and Symantec. Then let List Y be the set of all viruses that are blocked by McAfee but not blocked by Symantec, and let list Z be the set of all viruses that are blocked by Symantec but not by McAfee. (So McAfee blocks viruses in the set X+Y, and Symantec blocks viruses in the set X+Z.) Now, representatives from McAfee and Symantec will each say that their list is the better one, which they may or may not believe. But even McAfee is not claiming that List Z — that portion of the list that is blocked by Symantec but not by McAfee — is so worthless that McAfee wouldn't incorporate it into their own product if they could get it for free. If Symantec allowed any anti-virus maker to download Symantec's anti-virus signature database, then presumably McAfee would scratch their heads a bit about why Symantec would do this, but if they cared about giving their users maximum protection, they would incorporate it into their product as well (so that McAfee would then be blocking all viruses in the set X+Y+Z, instead of just the set X+Y as they were before). But Symantec doesn't make it available for free, so McAfee doesn't have the option of using it and the issue doesn't come up. Other than each company claiming their product is the better one (which is par for the course for competitors), the two companies' positions are not contradicting each other.
But consider the analogous situation for anti-malware lists, where X is the set of all sites blocked by both IE's SmartScreen and by the Google Safe Browsing API, Y is the set of all sites blocked by SmartScreen but not by the Safe Browsing API, and Z is the set of all sites blocked by the Safe Browsing API but not by SmartScreen. When Microsoft says that they don't want to use the Safe Browsing list in addition to their own — that they would rather block just X+Y than block X+Y+Z — they're saying that they're estimating that the list Z is of such poor quality (too much risk of containing too many false positives) that it would be better not to block it at all.
In this case, Microsoft's position really is contradicting that of Google, Firefox, Safari, and others who use the Google Safe Browsing API. To achieve the best tradeoff between user safety and convenience, should the sites on List Z — the set of sites on the Safe Browsing API blacklist but not on the SmartScreen blacklist — be blocked, or not? If the answer is Yes, then IE should use the Safe Browsing API in addition to their own SmartScreen list. If the answer is No, then Google should take the URLs in the Safe Browsing API list, run them through IE using some automated script, and then remove all the URLs that weren't blocked by IE — in other words, remove all the URLs on List Z from the Safe Browsing blacklist. But I can think of no consistent set of assumptions that would lead one to recommend that both companies continue doing what they're doing now — that IE should continue not to use the Safe Browsing API, and that Google should continue publishing the Safe Browsing API without trimming URLs that aren't also blocked by IE. Microsoft is saying that the URLs on List Z should not be blocked; Google is saying that they should be.
(Note that this argument is independent of the relative weights that you assign to the benefit of blocking a genuinely malicious site, versus the cost of accidentally blocking a site which is not malicious. Different users might assign different values to these costs and benefits, and depending on what values they assign, those users would want different thresholds to be used in deciding whether to block a site or not. And Microsoft and Google have picked default thresholds that they estimate will meet the needs of the average user. But no matter what values you assign to the benefit of blocking a malicious site and the penalty for blocking a false positive, it's still the case that blocking the sites on List Z either does increases the total cost/benefit score — in which case IE should block sites on the Safe Browsing list in addition to its own — or it doesn't — in which case Google should remove sites from the Safe Browsing list that aren't blocked by SmartScreen.)
I suspect, of course, that the answer is the former — that the set of sites on List Z, those which are blocked by the Safe Browsing API but not blocked by SmartScreen, are probably approximately as likely to be malware as the rest of the sites on the list, and that it would make Internet Explorer safer if Microsoft augmented SmartScreen to use the Safe Browsing API as well. So why don't they?
The answer is probably what people have been shouting out from the back of the classroom since the first paragraph: That for political reasons, Microsoft doesn't want to be seen incorporating anything from Google into their own flagship application. It's not news that a company would prefer to promote its products over its rivals'. But this goes beyond, for example, Microsoft bundling Internet Explorer with Windows instead of Google's Chrome browser. Chrome and Internet Explorer do virtually the same thing, so it would look positively odd for Microsoft to promote IE over Chrome. But IE's SmartScreen list and Google's Safe Browsing list can be used simultaneously, providing more protection than either one by itself.
Still, Microsoft has already calculated that it would be an unwise move politically to use Google's Safe Browsing list. So I'm not trying to second-guess the calculation that they made, based on data that was available to them at the time. Rather, I think that if some publicity can increase the political benefit that they could get from using Google's Safe Browsing list in conjunction with SmartScreen (and increase the political cost of not using it), that might lead them to recalculate and make a different decision. To that end, let me raise up a banner that people can gather under if they want to:
Microsoft, we will not think any less of you if you use the Google Safe Browsing API in Internet Explorer in conjunction with the SmartScreen filter! We'll give you credit for setting aside petty rivalries and using the technology of a competitor in order to make users safer.
The IE team's blog post about the initial success of the SmartScreen filter, from March 2009, cited statistics showing 10 million malware blocks in the previous six months, and asked readers to think about those numbers in terms of their impact on real humans and the grief it saved them: "These are BIG numbers — each malicious download blocked helps prevent compromise of that user's computer." Since then, Microsoft has released new statistics showing that SmartScreen has delivered about 70 million blocks since IE8 was officially released. Of course, not every one of those blocks made the difference between infecting a machine with spyware and keeping it clean (many users wouldn't have downloaded or installed the software that the website was trying to send them), but the IE team is right to be proud anyway. However that also means that if adding Safe Browsing support to IE resulted in only a small percent increase in the filter's effectiveness, it would mean several million additional malware blocks over the same period, and cumulatively tens of millions of more in the years ahead. Isn't that worth Microsoft forming an alliance with Google, especially if doing that would make them look good?
meh, we supposed to read all of that ^^^
You see, IE would go up to the malware list, find itself, and then consider itself to be malware and implode. Multiply times a billion and you know what we'll get? That's right, 911 times a billion!
is why shouldn't Firefox, Opera, et al. use IE's list as well, if it's so much better?
"Frequent contributor Bennett Haselton writes with an idea that he thinks could help keep broswing on Microsoft's browser more secure for users -- and benefit Microsoft as a result."
I have an idea that I think could help keep Slashdot from embarrassing itself even more than failing to ask Blizzard about bnetd - use a spellchecker.
From Bit Tech It should be noted that the NSS Labs testing was sponsored by Microsoft. In comments posted online, NSS Labs president Rick Moy suggested that Microsoft's security engineering team had originally commissioned the study, whose results were then picked up by Redmond's marketing department for use. However, a number of sources online, including Ars Technica and The Tech Herald, feel that Microsoft's sponsorship could have introduced a biased element into the study. The testing also found that Internet Explorer 8 needed an average of 4.96 hours to add a requested phishing URL to its block list, while Firefox 3 took 5.24 hours and Opera 10 Beta needed 6.19 hours. The mean time for a browser to block a site was 16.43 hours, a number exceeded in testing only by Safari 4, which needed an average of 54.67 hours to put a site on its block list.
is at least two orders of magnitude more in base 2.
You mean they should use that obnoxious Google feature that tries to stop one visiting crack sites? At least they could provide a link to continue, after the user is informed of the risks - to not include one is simply irritating.
I've recently heard about a concept called single point of failure, maybe you should look into it. If anything goes wrong and Google goes down with its malware list or they simply choose to block IE, we'll be completely defenseless.
You forgot one thing: Steve Ballmer.
Ah, didn't see Jason1729's similar post...
this is only becuase thier crippled web standards make it hard to execute malware.
All browsers using Google's blacklist effectively gives Google the power to censor the Internet.
Where I come from, "baked" means stoned. With that in mind, I can't help thinking that "more than half-baked" is a very good description for this plan. The blindingly obvious flaw is that if Microsoft did use Google malware list, people would immediately start asking why Microsoft wasn't sharing it's list. While it might be easy for many Slashdotters to say that Microsoft should indeed do that, it does not make any more sense for Microsoft to do that than it does for it to open-source any of the rest of it's products. It's a commercial entity looking for competitive advantage over other players in the marketplace, and that does not go hand-in-hand with these airy-fairy sharing ideas.
Oh no... it's the future.
11/16 baked?
Basically Firefox and Google can be much more conservative when estimating a site's malware potential. Since the browser is more secuire, it can let it more attacks and trust Firefox to protect itself to a large extent. IE has a long history of being used in intranets of corporations, and making the browser secure will break tons of installations and companies will not accept it. Their only choice is to find all the malware hosting sites and block them.
Children who grew up in farms with contact with animals dont develop asthma. The nose and lungs are insensitive to some of the irritants. Kids who grow up in ultra clean, sanitized environment develop asthma. Japan is a basket case in this example. They need a bubble around them.
IE grew up in friendly benign corporate environment. It needs very good filters and blocks. Unix cut its teeth in multiuser, college enviornment. So its derivatives Linux and its cultural progeny Firefox and other OSS have immunity built into them deep down.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
such a BS bash microsoft story. i could spend an hour trying to outline how contrived and flawed the whole article is but i dont have that hour to waste.
I didn't notice that. Granted, I only skimmed the huge summary, but I only saw two things:
1) IE has a better anti-malware feature than anyone else.
2) IE could do even better by combining their own anti-malware set with Google's.
"I zero-index my hamsters" - Willtor (147206)
To paraphrase: "blah blah blah bllah bllah blah everyone should use Google blah blah blah."
Look, monoculture wasn't a good thing the last time and it isn't a good thing this time either. Multiple, competing sources of data please. I don't want a mistake in Google's data to mean it will automatically get propagated to MS' products, nor do I want a mistake in MS's list to automatically propogate to Google.
As for Microsoft having calculated this politically, I'll bet it never gave the matter a moment's thought. MS have to be answerable for their own product - sticking a reliance on a competitor and changeable competitor APIs in there just doesn't make any sense at all.
Surely a slightly better solution would be for the Smartscreen server to import Google's data rather than everyone's version of IE? That way they could insert the results directly into their own database and so there would only be one hit to Google's API (rather than several million), they could vet and filter the data prior to importing (assuming they were mad enough to want to do it) and - best of all - it wouldn't need an update to IE.
Whilst it's a nice idea, I don't think Microsoft will do it. If it was someone else apart from Google (like Yahoo) then there would be a chance - but with Google, I seriously doubt it.
Avantslash - View Slashdot cleanly on your mobile phone.
This advocates MS also using the Google list. How about Firefox, etc, also access the Microsoft API?
I think you are assuming Microsoft cares about customer security.
If that were really the case then this would have already been implemented or in the works to be.
Better yet, why should Microsoft care?
Most people don't fix computer and just go out and buy a new one ever few years
Sounds like another Microsoft fee for a new computer to me.
Maybe I am just to cynical?
... because you can never send your complete browser history to enough 3rd parties :D
The story really is:
1) Microsofts anti-malware is better than Google's.
2) ZOMG Micro$$$oft is EVIL for not being even BETTER
2a) Google, however, is fine.
tl;dr:
Author thinks IE should use SafeBrowsing in addition to Microsoft's technology to catch more bad sites.
And because the authors opinion is a fact, "IE Should Use Google's Malware List".
Google has shown their political bias in the past (the "miserable failure" search manipulation, censoring/banning content from Google News, etc.). Why would Microsoft blindly trust something from a competitor that can, regardless of intent on either side, be (mis)interpreted by the public and/or their shareholders as a political endorsement? It's not worth the risk, especially if their existing product surpasses their competitors.
The focus of the story is colored by the blogger's own bias. Rather than focusing on why MS isn't doing better than 81%, the focus should be on why Google's product performs so abysmally in comparison to Microsoft's. Sure, MS could in theory make marginal improvements, but Google is the one that really ought to be taken to task for their poor results.
I know the conventional wisdom is MS == bad, and Google == good, but trying to find an MS-bashing angle to every bit of news is counterproductive and tiresome.
Shouldn't IE itself and microsoft.com be on any decent malware list?
I read this as Troll.
It contributes absolutely nothing useful to the discussion - but instead simply feeds on the modder's visceral hatred of everything Microsoft.
I agree. It got +4 Insightful from sheer hatred.
Can someone tell me if IE8 provides any ability to clear out the index.dat files IE creates?
Why, for the love of god, can't you buy Opera, slip in IE6,7&8 support and call it IE9?
Here be signatures
"It's worth underlining what a strong statement Microsoft is making by not using the Safe Browsing list. They're not just saying that their own list is better. They're saying that the Safe Browsing list is of such low quality that adding it to their own product would actually make the product worse."
or
"It's worth underlining what a strong statement the airline industry is making by not using teleportation. They're not just saying their planes are better. They're saying that teleportation is of such low quality that adding it to their own product line would actually make their service worse."
Seriously. Features don't grow on trees, SOMEONE has to spec them, write them and test them.
Gotta love those posts. I thought I'd add some on my own:
1. Apple should switch to Windows (in fact I remember Dvorak writing that one).
2. People should type with the mouse and move the pointer with the keyboard.
3. Pink should be the official colors of men.
4. USA should switch to Islam.
5. Slashdot should report news that matter.
I know, I know: they're all hilarious, since it's absurd any of these would happen!
If IE gets anywhere close to even a list of malware, it'll get infected by it.
/* No Comment */
Really, mods? I can understand Funny or Troll, on either end of the spectrum, but Insightful? Seriously?
1. If MS were to adopt Google's API, there would be pressure within MS to eliminate (or at least downsize) the Smartscreen team. Perhaps someone with friends in high places is defending their turf by mandating its exclusive use in IE. This is just like the mandatory silliness we have in the corporate world, where corporate IT creates policies primarily for control and job security, often with diminished productivity for everyone and no real effect on data security.
2. Google is a competitor in the browser market. As mentioned above, if MS adopts the Google API, pretty soon MS is out of the smartscreen business. Nothing prevents Google from slacking off in their detection of sites that are toxic only to IE + Windows, while all other browsers are well-defended. MS has a legitimate concern, primarily because MS would do this to Google within milliseconds if the roles were reversed. MS probably feels "married" to Smartscreen because it's a last line of defense for IE (whose bugs are hard to remove thanks to corporate intranets and MS products that practically depend on them).
In a previous job, there were many opportunities to make things better, but sometimes we had to pass. Thinking a few steps ahead, we realized that the CFO would misinterpret our idea, jump to conclusions, overestimate the sustainability and long-term benefits, and ultimately blame the consequences on us. No thanks.
Do a few things to really address the security problems inherent in the current designof IE, like eliminating ActiveX and fixing the helper function quoting issue, before worrying about blacklists.
"It's worth underlining what a strong statement Microsoft is making by not using the Safe Browsing list. They're not just saying that their own list is better. They're saying that the Safe Browsing list is of such low quality that adding it to their own product would actually make the product worse" Talk about a major leap. Completely ignores all the business, competitive, financial and other reasons that Microsoft may not choose to use it, then states the only interpretation is that Microsoft thinks the list is of low quality. Given the authors ability (or lack thereof) to reason, this article sounds like nothing more than self-adulating hype.
Utter rubbish. The best way to secure IE is to uninstall it. There are no alternatives. Get a life.
Microsoft probably doesn't include these lists, I speculate, because Microsoft and Google are in direct competition; i.e. Chrome vs. IE, Windows vs. (Googles new desktop), Google vs. Bing (live search), Android vs. Windows Mobile, etc. Microsoft doesn't want to include the competitor in their software. Remember when Windows 98 came out, and the anti-trust issue came about? Microsoft did not want to include Netscape Navigator in their software because they (Netscape) were a direct competitor with Microsoft. The evil empire does not share power. One software mogul to rule them all. lol
In Soviet Russia, road forks you!
tl;dr
However, I imagines it boils down to:
"Hey Microsoft, you should use your competitors blacklist in addition to your own!"
Key word there being competitors.
Hey, you know what, they should make the IE search box default to searching both Bing and Google! They should make Live Messenger support both MSN and Google Talk! They should make Microsoft Office include both Word and Google Docs!
Oh, even better, Mozilla should make Firefox render pages in both Gecko and Trident (at the same time, mind you)! Thunderbird should support its own mailboxes and Outlook (if it's installed).
Do I need to continue beating this dead horse?
GLaDOS for President 2016! "Well here we are again. It's always such a pleasure." -- GLaDOS, 2011
I have been looking at this recently in a completly different way. I was looking around for implementing a n2h2 or websense server. However i have not been able to locate any technical information about the 2 protocols. The reasonf or doing it with these specific protocols are because that is what is currently implemented on cisco and is reasonably common against other routers. The reasons or doing this is of course obvious. To make sure all lists on a larger office network are enfored all the time and to prevent people being able to ignore the current lists from within firefox. Can anyone paste some links to info about these 2 protocols.
If you're running Windows XP/Vista, many users install a full Internet security suite program. Don't these programs override the browser security settings and use the malware list from the security program itself, a list that is usually automatically updated on a regular basis anytime you're connected to the Internet?
I'm running Norton Internet Security 2009 on my Windows Vista machine at home and this program imposes its own malware monitoring list into Internet Explorer 7.0/8.0 and Firefox 3.0 and 3.5.
"Arrogance" would have been my guess. I mean who knows better about how you should use your computer than Microsoft?
CUR ALLOC 20195.....5804M