Slashdot Mirror
Consumer Reports Creates Viruses to Test Software
Maximum Prophet writes to mention an MSNBC article about a Consumer Reports plan to test anti-virus software by creating viruses. Security companies are objecting, on the grounds that it's a generally accepted practice not to create viruses for any reason. From the article: "Consumer Reports didn't create thousands of new viruses from scratch. Rather, it took a handful of existing viruses and created hundreds of slight variants, changing the malicious programs just enough to evade detection by an antivirus program with a list of known threats. That's a common trick in the virus writing world; it's standard for a successful virus to inspire dozens of variants. "
Clearly this is all just a cover. The Templars are using Consumer Reports as a cover to train a stable of elite Black Hat hackers, with which to take over the world. They're in a race against Communist China, the Russian Mob, and the NSA.
Any sufficiently well-organized community is indistinguishable from Government.
You know you're in trouble when Consumer Reports is pointing out that your software is worthless. As just about every /.er knows, pattern / signature based detection is all too easily circumvented. Unfortunately it's pretty much all we have. It has been my experience that enabling Heuristic based detection (in Symantec Corporate AV) at any level other than the default just leads to too many false positives.
Consumer Reports destructively tests many things. Why should it matter what they do to their own computers? As long as they don't release these viruses into the wild, there is no problem.
Security companies are objecting, on the grounds that they do not want the gaping holes in their software revealed to the public by Consumer Reports.
Track and chart data from your bike computer.
for one of their viruses getting out then by all means I think Consumer Reports should be allowed to continue.
Catching them after they are out is easy. The consumer really has so very little to go on from a "trusted source" in regards to virus scanning that the obscurity benefits the AVG companies. With a little more light on the subject we all benefit, all except the AVG companies. Guarantee that whomever CR picks is going to parade that around regardless of their stance before testing occurs.
Again, if CR is willing to accept liability for one of their tests getting out into the wild then I say go for it! Perhaps they should register their "new toys" with someone for backup? Of course that makes for another hole too.
* Winners compare their achievements to their goals, losers compare theirs to that of others.
(See my Journal entry for the gory details) ... I would sincerely recommend they don't play with fire. There are too many ways that self-replicating programs can go wrong... or too-right, as in my case :-(
If they can guarantee containment, of course, a virus is completely harmless to the rest of the world. The problem comes when containment is breached because of something you didn't think of - and the problem with things you didn't think of, is that you didn't think of them [grin].
Simon (now a thoroughly-reformed character, honest guv)
Physicists get Hadrons!
Be sure to read our other Consumer Reports articles, where we:
- and -
Thanks, Consumer Reports. Thanks bunches.
____
~ |rip/\/\aster /\/\onkey
coming from them... I was under the distinct impression the vast majority of viruses only existed in their labs...
Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
1) Virus writers will write exactly the same code, unless the boys at Consumer Reports are dedicated enough to come up with truly innovative virus variations. So there's no fear that someone out there will "get ideas."
2) Why not vet your software against somebody else's test suite? If CR wants to function as an extension of Symantec's R&D, let 'em. It's a win-win.
Human being (n.): A genetically human, genetically distinct, functioning organism.
No wonder the AV companies are up in arms - its a standard industry requirement to make sure that there is a PR rep assigned to each engineer to "interpret" results, whenever doing tests that shows how well the software actually works!
Security companies are objecting, on the grounds that it's a generally accepted practice not to create viruses for any reason.
You mean they aren't already doing this internally? If not... what the hell are they doing all day? If they're just being reactive without testing their software against possible variants then their software isn't really useful. Though frankly I find antivirus software to be a cure worse than the disease. A 1/100 chance I'll get a virus that does bad things to my computer, or a 100% chance that my computer will run like crap due to NAV.
Solution? Backup all my documents (mostly pics) to a dvd monthly and trust my Linux box firewall/router/proxy to keep the bad bits out.
rooooar
You can use these files to test if your AV program is working
http://www.eicar.org/anti_virus_test_file.htm
That is exactly what virusscanner sellers do. They create new virusses, mutate them and test them out. Of course they don't do that in a internet or network-connected environment. In all cases this should be in a lab environment completely closed off from the exterior world.
What's the big deal here? A bunch of Windows computer with antivirus software running in a closed off network as to benchmark some programs. Happens with games, office software etc... nothing to see here, please move along.
Of course this way you also get stories (hoax, urban legends) like the one about Symantec releasing virusses to sell their software...
Custom electronics and digital signage for your business: www.evcircuits.com
This is a very good idea, IMO. I mean, for years the major security companies have been using fear tactics to push their software. For an almost equal amount of time, security-concious geeks have been critical of this software. Having a trusted, disinterested third-party like Consumer Reports put it to the test sounds like the perfect solution to this situation.
Its been a long time since someone outside of Norton has talked about how good a Norton product is, but they've been in the game for such a long time that they are trusted by the general public to do their job. I wonder how many would uninstall if Consumer Reports said that their product was utter crap? Or rather, how many would try to uninstall only to find that the uninstaller is broken too?
There is no mod option "-1: Disagree" for a reason. "Overrated" is not an acceptable substitute. Post something instead.
As a CR subscriber, I am utterly amazed that they even had the IDEA to construct a test like that, much less actually find capable programmers and do it. Perhaps that security company cold-called them and suggested it?
CR's technology reviews are often wrong in ways that would be laughable if they weren't so influential. Off the top of my head:
Has anyone here heard of this "Independent Security Evaluators" biz? I wonder how many of the viruses were still functional (not just infectious) after twiddling.
Folks,
This one is REALLY obvious. Consumer reports is going to prove that these security products can't detect things they haven't seen before and the Virus detection companies don't want you to know their dirty little secret, i.e. this stuff only works after the cow is out of the barn, i.e. a virus has already been seen in the wild, measured, and characterized.
Have you compiled your kernel today??
If this helps wake people up to the fact that anti-virus programs simply don't work, all the better. For example, at one time or another, nearly every antivirus package has declared applications with NSIS installers as malware. I remember having a McAfee trial on my computer, that would regularly make up infections. Yet, when a slightly updated version of a worm comes out, you're unprotected.
I'm scared of numbers that can't be written as a fraction. It's an irrational fear.
This is what real engineering is all about. It takes real software engineers, not code monkeys, to expost the vulnerability of a product, and report it to the consumers.
It's the duty of every engineer (those that can rightfully call themseleves engineers) to protect the public.
Clearly, classical antivirus software is not protecting us. Kudos to these folks for pointing out what should be the painfully obvious.
"Creating new viruses for the purpose of testing and education is generally not considered a good idea," wrote Igor Muttik of McAfee's antivirus lab on a public company blog this week. "Viruses can leak and cause real trouble."
All these years I've assumed that AV Companies created hundreds of virus varients in a closed lab somewhere so that they could proactively test their product against against new probable varients? How does McAfee anticipate new threats? Do they wait for a new virus to be released into the wild, and then release a corresponding patch? I thought they tried to be more proactive...
I'm not some AV expert, but it seems like most new virus outbreaks are due to varients of existing viruses. Some of varients are caught when the AV software finds a match to an existing virus signature, but sometimes the signatures do not match.
"Can of worms? The can is open... the worms are everywhere."
We don't need to see the results of any anti-virus software tests. The outcome is known before the tests are even performed: failure.
If you're using a platform that requires the use of anti-virus software for even the most basic level of protection, whether or not that anti-virus software actually works is the least of your problems. Your operating system itself is inherently flawed. Any software that runs above it is likely just as flawed. So while running anti-virus software may afford you some small degree of protection, it's likely that there are far more obvious and serious security holes affecting your computers and your networks.
When there are so many comprehensive and freely-available alternatives out there (Linux, FreeBSD, NetBSD, OpenBSD, Dragonfly BSD, and OpenSolaris, to name a few), one should not be using a system that essentially requires the use of anti-virus software.
Crashing cars to test automobile safety?
Chopping vegetables to test kitchen appliances?
The mind shudders.
I would have thought there are enough viruses (malware, trojans etc) in circulation to sufficiently test software to a high degree of confidence. Existing viruses are extremely diverse with many having had significant resources used in their development.
With this team are trying identify an exploit that a future virus might use, they are essentially trying to predict the future. What if the Black Hat uses a slightly different approach that ends up undetected. Does this not make their time worthless?
This is like creating a range of new biological diseases on the off chance one develops independently and starts to harm people.
I believe the money would be better spent training developers to (try to) 'think like black-hats' where every step of their development they focus on 'how could this be exploited' and 'what can I do to prevent it'.
Soon they'll propose testing car safety by doing test crashes! Or testing fire retardants by trying to set them on fire. Damn those Consumer Reports fools!
It is by the juice of the coffee bean that thoughts acquire speed, the teeth acquire stains. The stains become a warning
FTA: "'Those viruses exist right now only on a CD in a sealed container in a locked cabinet in our computer lab,' Beckford said."
Seriously, it's not like these will ever exist outside of a lab, right? And if they do, the AV companies won't have any problem finding the source code, will they?
Isn't that kind of like telling the insurence institue that they can't change their car crash tests because car makers designed their cars only for specific crash tests? Gee, better not create anything that a car might run into, it's bad ethics!
----- Connection reset by beer
Here are the scores:
BitDefender Standard - 87
Zone Labs ZoneAlarm Antivirus - 85
Kaspersky Labs Anti-Virus Personal - 82
Norton Antivirus - 80
Norton Antivirus for Macintosh - 80
McAfee ViruScan - 77
Trend Micro PC-cillin Internet Security - 75
Alwil Avast! Antivirus - 68
F-Secure Anti-Virus - 66
Panda Software Titanium AV - 64
CA/eTrust EZ Antivirus - 57
PC Tools AntiVirus - 41
However, I don't have a lot of faith in CR's ability to rank high tech items.
The eicar test-virus file is a great way to see how your computer/av-suite will react to a virus. However, it's not an effective test to see how the heuristics systems and such react. It's non-destructive, and every AV vendor makes sure that they can "catch" it. That's nice for making sure that your AV is running, or that your AV on some workstation reports back to the management computer that it caught a virus, but not for testing the ability of AV software to find new viruses that don't necessarily have definitions written for them yet.
----- Connection reset by beer
Windows 95 Windows 98 Windows Me
AV software WILL protect you from new viruses... Just not McAfee and Symantec's crap. Well I suppose I should rephrase: Their software can protect you, but not very well, not as well as others. Bitdefender appears to do the best job at finding viruses that it doesn't have in it's DB. AVG also seems to do a pretty good job.
That's what they are afraid of. Not that it will be revealed their software does nothing, it does work, just that there is cheaper software that works better.
I casually perused CR here and there, but I'd never really known much about them until a relative gifted me with a subscription. Here are a few things I like about them:
/.ers care about, like RFID and general privacy protection; taking strong pro-consumer stances that you don't see in other national publications.
1. They pay their own way. They purchase *all* of the products that they test and destroy, since cozying up to get sample products would tarnish their credibility.
2. They don't accept any advertising dollars within their magazine, since that might bias their reporting and tarnish their credibility.
3. They take a strong stand on protecting consumers beyond just good product recommendations. They do editorials and special reports on subjects that
When my gift subscription runs out, I plan on purchasing my own. Not only because I find the product articles useful and interesting; but because the Consumer's Union does other good things with my money.
Why are you letting these clowns ruin our country?
As long as new software is being written, there will be some black hat trying to find the buffer overrun of the week. Everyone knows this. It is simply life.
Consumer Reports knows this also. This article is probably going to be geared to cause controversy. They can splash on their cover "What you don't know about your AV software!" and scare the crap out of every AOL Mom with a $499 Dell Desktop. This is just a type of marketing and it sells magazines. If I was a major AV producer I would be calling foul too.
They will not lie. The article will be true. Everyone who takes the time to learn Internet Security 101 and a little programming will know how easy it is to make a variant of just about any virus and defeat AV security.
What Consumer Reports isn't going to mention or they will, as a short blurb at the end of the article is the true value of AV software.
Upon discovery of a new virus in the cloud, an AV Company should be judged on:
How quickly they issue an effective patch for the virus from Day 0.
How easily the patch is for consumers to deploy and protect their home computers or corporate infrastructure.
How transparent the whole process is for the end consumer. (Ex. Does the AV software suck up a lot of Ram?)
IMHO opinion AV software isn't going to be fool proof, but as soon as a virus or variant is discovered; it should be judged on how quickly and easy it updates.
Now if Consumer Reports really wanted an article that has the potential to shake up the AV industry and reveal how crappy the current AV offerings are. They should just compare the performance of all the major AV vendors against the above criteria over a period of time.
>
The /. summary says that "plan to test anti-virus software by creating viruses."
TFA says "Consumer Reports recently conducted one of the most thorough tests ever of antivirus programs. But to really put these security programs through the paces, the magazine hired a firm to create 5,500 new viruses, using them to test the antivirus software products for their ability to detect unexpected threats."
By the way: "In the results, McAfee scored in the middle of the pack. BitDefender and Zone Labs scored at the top, in part for the two program's abilities to detect new viruses."
HIV is so difficult to cure, in part, because it mutates rapidly.
CR's model which provides its independence also means it doesn't tend to have the chummy, early access relationship many other outlets have with manufacturers. Them actually doing really substantial tests also means that they tend to take longer than some other outlets. OTOH, I've rarely been led astray by a CR review on anything, computer related or not, so I'm pretty happy with them despite their limitations.
When I was working in the Network Security field doing malicious traffic generation, we had a threat database of over 700 various attacks. Of course the PR Whitepaper for the Major IDS/IDP Maker was carefully worded such that of the "Selected Threats" they managed to stop 100% and their competitor only stopped 35%. For that subset it was true, of course the subset was selected as all the threats their product stopped, while overall they stopped less of the total database than their competitor did. The Joys of Spin.
Yes, this should be allowed and encouraged. See comments about...
:O he he he..
/Leo/
Plus CR should do the automobile equivalent crash test for various OSs and make it public. I just want to see that
Best,
FTA --Testing security only emboldens the terrorists!
Where, in Soviet Russia!?
I have subscribed to "Consumer Reports" for over twenty years and have never seen a serious discussion of Linux.
You would think that the advantages of Linux and BSD would make it a natural choice for an organization that tries to help the consumer to get the best deal available. All I have seen are discussions about whether a PC or a Mac is best. It is as if the Consumers Union is in the bizarro universe.
Even in the latest issue (September 2006), they persist in assessing the rate of Mac OS X spyware and virus infections by conducting a survey, an annual gaffe on their part. Rather than checking around and discovering that no such malware exists in the wild, they assume that computer users are able to judge for themselves the cause of computer difficulties.
This would be like studying the mechanisms of natural selection by way of a survey. Hey, whaddyaknow, turns out there's no such thing as evolution, a survey of Americans would have to conclude.
Consumers Union knows better. I don't know why they keep repeating this mistake.
-Waldo Jaquith
I want to read the report now. If they really didn't want this report publicised, the correct response is "whatever".
There are no trails. There are no trees out here.
"I help out unwed mothers... hey, I just help them get their start!"
All this crap only applies to Windows XP. While is is true that MS-DOS, 3.1 and 98 can be infected by explicitly running an infected program (rare), XP is the only thing you can install, hook up to the net and expect it to be infected withing hours if not minutes.
For the one machine I have at home that has to use winbloze I use 98 and have since, well, 98. Although it has in typical MS fashion shit itself a few times it has NEVER become infected. Not once.
Other than an ill fated XP experiment here briefly the last virus I saw was when my idiot boss in 1989 said "here you need this new assembler" and it was infected with the stoned virus.
Need Mercedes parts ?
"What is Internet Explorer 7? Are you saying we can't access the normal internet?" - I love tech support. Really.
No anti-virus software will ever produce an alternate ending to "Terminator 3".
Move all sig!
Consumer Reports Rocks!
FTA: "'Those viruses exist right now only on a CD in a sealed container in a locked cabinet in our computer lab,' Beckford said."
Seriously, it's not like these will ever exist outside of a lab, right?
And as I recall that's what Morris claimed about the internet mail worm, too: That he was experimenting with it on a set of local computers and it got out accidentally due to a connection he wasn't aware of or hadn't properly shut off.
(The timing of when it got out (when most of the relevant people for fighting it were in transit to a conference, giving it max time to spread) argues aginst his claim. But the fact that the worm was clearly not debugged yet, containing partially implemented code for features (including some intended to shut it down if it were in the wild) argues for it.)
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
From the article: "I understand .. if you want to test a car's performance, you test the car put on road with lots of bumps on it," Marcus said. "But when you are talking about malicious code, there's a threat to public. There are professionals who know how to handle viruses. It should be left to them." (emphasis added)
Well, that's why Consumer Reports hired computer security professionals to work with on this. Maybe they're just mad that CR didn't ask them to be the security consultants... oh wait, that might be a conflict of interest for the product review. Tough.
$nice = $webHosting + $domainNames + $sslCerts
Consumers Reports is the most trusted amoung consumers. They put products through their paces and ensure they work well. With that said, yes Consumer Reports create viruses. They already have done so for testing lastest virus programs. Consumer Reports September 2006 issue has said this. They have rated Bit Defender as the best. The issue specifically said they created new viruses to test how well they did against new viruses not already in the signature lists.
People like Igor Muttik are just scared their crappy anti-virus software sucks. Mcafee ranked #6 in the Sept 2006 issue. And even if a CR virus got loose, CR can release the viruses details to venders immediately. The virus wouldn't last more than couple days.
\
Bitdefender doesn't catch all new viruses, updates are still important, it's just very good at finding new variants. That's what CR is testing here. Say a virus comes out that your software knows about but a variant comes along that it doesn't yet: Can it catch that? For some (like Sophos) the answer is no never, they check against a database and if it's not there you are SOL. For some like Bitdefender the answer is usually. They have a heuristic checking that works pretty well.
There's no magic bullet, there's no "buy this once and be secure forever" kind of solution, but there are better and worse ones out there. Bitdefender and AVG (probably others those are just the two I know) are reasonably good at stopping new, unknown variants. Synametc, well not so good.
If I had a dollar for every time a large corporation disliked how Consumers Reports tests merchandise, I'd have, uh, some number of dollars.
Time and again it seems like testing based on "real world" scenarios is what upsets the corps the most. The corps are right, too; as if anyone lives in the "real world".
RTFM; please, I beg you.
Based on results it is obvious that the Anti-Virus vendors are failing to give consumers a product that averts issues. At best you can hope that your AV vendor releases updates that get installed in time on your machine prior to an infection vector like email or web being triggered on your system. Should that fail you can hope that the virus is not malicious and that your AV software can clean your system up.
Obviously this isn't working. Viruses continue to run rampant. We've been using the same techniques largely for over a decade, perhaps if we try them for another decade they will suddenly start to work.
So why aren't we using different methods that might actually work, especially the proactive ones?
Doh, now I remember, it's because I can continue to sell you an AV subscription at 10-60$ a year forever.
This is much like the consultants dilemna, do you fix the problem and have to find a new job, or do you band-aid the problem and maximize your billable hours.
I don't think AV companies are going to provide any long lasting relief anytime soon.
Perhaps it is time to start asking your vendors why this problem has continued for over a decade, and will continue with no end in sight.
prevents a pregnancy. It doesn't work if you are already pregnant.
They should follow suit with testing bodies such as http://www.av-comparatives.org/ They have a Retrospective test which is kind of like what Consumer Reports is trying to accomplish, but is makes more sense. Basically, what they do is regress the detection database six months, and test the detection of new malware collected during the last six months. All detections will be from heuristics or generic signatures. Seems much better to do a real-world test than a sloppy create-your-own-virus test which will most likely make the antivirus companies look better. Remember, malware writers release malware that they know will specifically not be caught, whereas CR is just randomly modifying.
Hey, let's double... triple dare them to create a Linux virus and release it to the wild.
Speaking of Macs ... and viruses, malware and exploits, the OS X Server list has some folks that make some wild claims on immunity (because of architecture, etc.). Don't even waste your time installing AV and the such. .img file permission problem on 10.2?
s &ul=macos-x-server&s=DRP
0 6/Aug/msg00664.html
I'm a Mac guy, but this is just asking for it. - When some "genius" finds a hole, not to mention passing the files downstream to your Win folks.
Remember the
Yeah, AV SW sucks. (and they need to start doing their jobs)
Whistling through the graveyard.
Priceless.
http://search.lists.apple.com/?cmd=Search!&q=viru
http://lists.apple.com/archives/macos-x-server/20
~hylas
Everyone knows that the anti-virus companies create some of the viruses themselves. It's all about sustaining business.
You moved your mouse. Please restart Windows for changes to take effect.
There's nothing like a good gunfight to uplift the spirit--Calvin
It's not Consumer Reports job to educate the anti-virus company's developers how to write better code; it's their job to tell consumers how effective the AV company's products are.
And consumer reports isn't trying to predict the future; they're doing the simplest of tests (take existing virus, make slight change test) and see what happens... and that is.... nothing. That is, the AV companies fail the simplest of tests. They're useless. They're taking $35/year from us to do nothing.
That's the story. Everything else is an excuse. Stop it and make better products instead of whining about how consumer reports should spend money to train Symantec and CA to write better code.
Consumer reports Hacked my Thalamus!
The problem is that AV software at the moment scans for signatures of known malware. Essentially they are reactive.
What they should be doing more heuristic scanning, identifying malware by characteristics rather than looking for particular malware signatures.
This is a fundimental weakness in most existing AV software. Certainly this is harder to because legitimate software can do similar things to malware. That doesn't change the fact that AV companies should be concentrating more on this. This is particularly true as most "successful" worms get modified and re-released. As a result it should be possible for the AV companies to detect the altered worms.
Consumer reports is doing us all a service here by exposing this weakness. Provided they ensure the worms don't get out I'm all for it. This is a perfectly valid way of testing the malware. In addition FTA they are doing what most malware writers do anyway: altering the worm just enough so that it is likely to get past the signature based scanning software.
Shame on you McAfee.
meh
Auto manufacturers are dismayed with Consumer Reports crash testing of automobiles. "It's a generally accepted principal in driving not to slam into a wall at 65 mph," commented a spokesman for the industry.
they'll never have to pay for job recruiting ads again.
Tech Public Policy stuff
Were you doing this with Acorn RISC OS systems?
If virus creation were rewarded in order to further explore options for improving software then perhaps improved development would increase. Draw up a set of rules and reward monetarily those who can create the most effective exploits of other "secure" software. Generally acceptable to develop virus software if you're competing in a contest. Sounds so simple its probably been done already and I haven't heard about it.
For some reason I refuse to use either spell check or the spacebar properly.