'Leak' Test of 21 Personal Firewalls

mork writes "Matousec.com, as part of a larger analysis of personal firewalls on Windows, has conducted a thorough leak test of 21 pieces of firewall software. Leak tests imitate common methods used by trojans or spyware to send your information from your computer. Windows Firewall XP SP2 fails every test, so the fears that the days of third party firewall software was over seem groundless. Surprisingly the two top programs are both freeware." From the article: "Some firewalls totally failed tests made against their default settings but their results on the highest security settings were much better. Kaspersky Internet Security 6.0.0.303 is the product with the biggest difference between the default settings score and the highest security settings score. Another such product is Safety.Net. Some products like BitDefender, F-Secure, McAfee, Panda, etc. include antivirus engines. The sad and funny thing in once is that lots of them mark leak-testing software as viruses or malware."

Comments

From the Steve Gibson school of thought

[Beryllium Sphere(tm)]

What is "sad and funny" about catching a program that uses the same techniques as malware, techniques which are outside the range of normal software, and flagging it as potential malware?

It's also annoying to see a firewall listed as a failure because it's a firewall and not a host-based IDS.

I'd also argue that the host-based IDS programs are being sold for a purpose that is not their best use. Once a system has malicious software on it, expecting a process on the same machine to protect you and itself is, um, optimistic. Sure they try to defend themselves but that puts them on the wrong side of an arms race.

What they're best for is monitoring and control of "legitimate" software. I have Zone Alarm set to prompt me every time a program tries to run IE6, and to block media players from phoning home to whisper about what I'm watching.

XP SP2 firewall is inbound only

[MagicM]

The "personal firewall" in Windows XP SP2 was never advertised to block outgoing connections. In fact, this PDF states: "Windows Firewall blocks unsolicited incoming traffic. However, you cannot configure Windows Firewall to block outgoing traffic."

So of course it failed every test.

Typical Security Guys

[jandrese]

I notice that there was no column in there about how aggravating the installed firewall rendered your system. How many of those firewalls are going to try to pop up a dialog box on a game that just went full screen and freeze the game (so you can't even alt-tab out) until you click on a box you can't even see? I mean I could have designed a firewall that would easily pass their tests with 100% reliability, it's called "unplug the network firewall", and it's very simple to install, just reach behind your computer, find the ethernet cable, and pull it out. Viola! Perfect Score!

One thing that struck me about Windows Firewalls as compared to Unix firewalls is that Unix firewalls are focused on keeping malicious traffic out of your machine. Windows firewalls are designed to keep malicious traffic from getting out to the internet. In the end, it's no surprise that the results are a mixed bag, once your system is compromised you really can't expect these firewalls to save you. It's a lot like the antivirus market, where you have a constant arms race between the virus writers (do people write honest to goodness viruses anymore?) and the antivirus companies.

My final complaint is that programs like ZoneAlarm Pro are exceedingly resource hungry for what they do. ZoneAlarm takes over a minute to start on my fairly modern laptop, whereas everything else in the system takes about 30 seconds or so total. Why does a firewall need 24 MB of resident memory?