On August 8th @ 6pm at LinuxWorld, we’ll be holding the AntiVirus Fightclub! What is the AntiVirus Fightclub? Its an all-out public test of different anti-virus vendors to see how they really compare. Why would we do this? Because you will be surprised by the results.
The background: Two years ago we decided to add antivirus to our network gateway platform. As usual when adding an application, we did a ‘bake-off’ of the varying solutions available plus a few well known ones for good measure. My theory was that anti-virus was fairly commodity and that all vendors would be roughly equal. I couldn’t have been more wrong!
Some vendors were embarrassingly terrible (clearly selling dead donkies), and yet the little ole’ open source project, ClamAV, was among the best of them. We decided to go with two solutions so the user could choose, one of which was the open source solution ClamAV. We couldn’t be happier with the performance of ClamAV.
A year later, I was talking to one of the testing labs about testing our appliance (for the little sticker certifications). They were eager to work with us for firewall and VPN testing, yet refused to test any antivirus functionality because we were using “the open source solution.” They would not explain why they refused to test ClamAV, although they did offer that they had tested it and it had done poorly relative to the commercial solutions. Given our testing and customer data had shown the exact opposite, we asked for their test results – to which they repeatedly refused. They also repeatedly refused to provide the test data set so we attempt to verify any results they had shown in their labs.
I’m left to assume that the testing labs are biased in their testing, probably because they get their funding from the commercial vendors that pay them for testing. Their customers surely wouldn’t be happy if the testing labs claimed a free and open source solution was better.
Open source antivirus hasn’t gotten a fair shake, and all the meanwhile some commercial vendors escape with selling products so poor it should be considered a scam to sell them.
So what’s the AntiVirus Fightclub? Its a public & transparent test of the performance of many virus vendors on a real-world test set. This is not a 0-day virus test. This is not functionality comparison. It is a simple test of whether or not each vendor’s virus engine catches viruses that have been in the wild in wide distribution.
If you happen to keep old viruses around that you’ve found in the wild (or have been emailed to you), please submit them here. Submissions will be taken at the show as well, and all results and tests can be verified independently after the show.
edit: The results are in!
12 Responses on AntiVirus Fightclub!
This is a splendid idea, let me be the first to congratulate you on here.
I used to work in distribution, so we saw a lot of vendors, 3 in particular all converged on this space at the same time, a firewall vendor, a proxy vendor and a load balancing vendor, and they did the old tricks of turning up at each others events, bad mouthing each other, etc.
The numbers in the documentation slowly crept up during this whole time. Without any development as far as I know. This kind of thing happens all the time in the name of one-upmanship, the numbers slowly get higher and higher, product managers completely ignorant to the facts, thinking “oh, we’ll just code it in when someone wants it”.
The people who suffer are the customers of course, and it usually starts with a phone call to a bewildered pre-sales guy in distribution… “but that’s what it says in the documentation” doesn’t really cut it when they’re paying $xx,xxx for a future proof solution.
What were the results?!? Don’t keep us in suspense!!!
The results are in: http://blog.untangle.com/?p=96#more-96
enjoy!
You should really take a look at the results of testing labs that do this work day in and day out:
Test done by Av-Test.org – 606,901 malware samples:
http://www.pcmag.com/article2/0,1759,2135092,00.asp
http://www.av-comparatives.org/seiten/ergebnisse_2007_02.php
http://www.av-comparatives.org/seiten/ergebnisse/2ndgrouptest.pdf
The number of malware samples you used is a joke.
Cheers,
Eric S
Wow. A joke, huh?
Nice, constructive criticism.
This guy already explained he used a honeypot to harvest his samples.
Of the thousands of virii and variants, how many are ever found in the wild at all?
Interesting topic.
Anti-virus software comparison is not an easy task like comparing any other kinds of software. And it needs to update result for comparison from time to time since anti-virus software are updating from day to day.
Well, it is really good thought, its appreciative. Nice to have you, I also have one anti spyware software which will help you more.
Good Luck!
Appreciate your antivirus fightclub initiative.
An disadvange of using antivirus software I think is that the computer might slow down a bit.
Here’s info about viruses etc. at http://www.software-reports.com/virus-of-computer.shtml
Hi…
I fully agree with the comments made by lookingsoftware about anti-virus software comparisions. While there are many virus protection vendors in the marketplace each software has its pros and cons as to their effectiveness and I think the anti-virus software that keeps up with the latest threats by having their anti-virus database growing by each passiny day will be my favourite choice.
Keep up the good work !
Yeah i agree with Mike. After installing an anti virus software The Pc slow down
This site has a listing of antivirus programs and how they performed when tested with 100,000 viruses. You can also find the 2009 antivirus rankings to see which antivirus packages give you the best protection.
Chris
Thanks for the effort. Interesting results and product selection.
Leave a comment on AntiVirus Fightclub!
RSS feed for comments on this post · TrackBack URI