graph1_small.pngThe results are in! Congrats to Fortinet – today’s top performer. Fortinet blocked about 98% of porn sites in the test site. Others blocked about 94-98% of porn sites, so the competition was pretty close.

The methodology and process was fairly simple. We googled 50 pornographic search terms and tested each product against each search term 100 links deep (The test is documented more in the presentation). Each product was graded against the site list qualified by humans. Each missed between 2% and 6% of porn – enough for a determined user to easily get around. In addition, there was usually about a 1% false positive rate, an annoying side effect for users.

How does one interpret these results? It depends on your view on web filtering. All products performed within a small range, probably within the margin of error. If you are expecting to block all porn, all these products will not be effective, however if you’re looking to control and monitor porn usage as much as possible these products make great candidates. Given the similar performance on this test set other aspects should play a larger role in which to choose, like price, performance, and features.

The full test set with human and vendor scores is available here (30meg). The scripts used to run the full tests and live tests are available here.

Many asked why untangle was not included in the test. We felt including untangle itself in the test would not be fair, as we know the test set and methodology ahead of time. We also apologize to the vendors (netsweeper, besafe) who requested to be in the test but time didn’t allow. Other popular vendors did not have a offering that suitably fit our market (small business)

I hope this serves to educate the market as to what one can expect of web filtering and also provide a small help to web filter providers. Thanks to all those who joined us and the vendors!