How to Bypass Web Content Filters

Ahh the Internet, chock full of all kinds of content. Text, images, sounds, videos. Even though an inordinate amount of that content contains cats for some reason, there is still a whole lot to see, hear and interact with. There is certainly content that is inappropriate for specific ages and in specific environments, and content filters were created to keep that inappropriate content in check. What is inappropriate and when it is inappropriate can be argued, and it changes from culture to culture, but usually the person or organization who is paying the bill for the Internet circuit is who decides.

So we have providers who need to filter content, and we have those on the other side who want to access everything. Enter Filter Avoidance. Search “how do I get past my content filter”, and you will quickly understand just how many people are in the second of those two groups. The biggest issue for most filters is that they are only looking at HTTP traffic, good ole port 80. This becomes problematic because the people who want to break the filter know this as well.

In this post will cover the most popular ways used to bypass content filters including:

  • Search Engine tricks
  • HTTPS vs. HTTP
  • IP address vs. URL
  • Proxy Websites
  • Proxy Settings
    • USB Browsers
    • USB Computers
  • Filter Avoidance Applications

Search Engine Tricks

Most admins know that Google and other search engines cache pages. If someone wants to see things they shouldn’t, you need a tool that not only blocks access to the links, but also the cached pages. When you search for a term, there is a URL in Google under the result link. At the end of the URL, click the down arrow to view a cached version of the page. There are also “preview tools” that let you look at the page without actually clicking the link. Of course there are image results that show up as thumbnails that will load even if the page hosting the image is blocked

HTTPS

Asking for an encrypted version of a web page is usually one of the first methods attempted. Content filters that can’t actually decrypt an HTTPS web request can run into problems if they are only looking at the site certificate. Hosting companies like Akamai and others can host sites that don’t use their own certificates, making certificate inspection iffy at best. Full HTTPS decryption is hard; it’s resource intensive and can be difficult to deploy, so many filters don’t perform well when asked to actually look at the URL.

Using the IP Address Instead of the URL

Many filters are dependent on the actual URL to categorize content. Those who have some basic networking knowledge can request the IP addresses of blocked URLs and try entering that in the browser instead. Filters often do not have the ability to use DNS lookup on the fly to try and resolve the IP address, resulting in a fairly effective avoidance method.

Proxy Websites

Search “Proxy Websites 2014” in Google and bask in the glow of your monitor as it spits out hundreds of millions of results for you to try. There are groups that will post or email a dozen or more proxies every morning. You can visit proxy.org, where they update multiple times an hour sometimes. What does this mean? If your filter is dependent on a URL list only for protection, you have a problem. The number of filter sites will always out pace your URL library updates. Make sure your filter has logic in the code that can determine a proxy page even if it has never seen it before.

Proxy Settings

If you know how to search, you can learn how to change the proxy preferences in your browser. This causes a myriad of problems since the browser can use a non standard port (read: not port 80) to send and receive HTTP traffic. The lists I was referring to above also have entire pages devoted to showing users how to reconfigure their browsers. For the admins out there that have locked the ability of the users to change browser settings: good for you; however there is another way around. Keep reading.

USB Options

Visit portableapps.com, and you can download a full version of Firefox that runs entirely from a USB stick. So even if your permissions won’t let you change proxy preferences, you can do whatever you want at home or at the coffee shop down the road, then save those settings on the USB browser and use it where the content filter is keeping you from your desired sites. It will even support plugins, which gets really tough when you combine them with proxy applications like UltraSurf.

Filter Avoidance Applications

There are people out there who hate content filters. People who can code. They write applications like Ultrasurf, TOR, HotSpot shield. There are also P2P networks like gnutella and bittorrent. Most of these applications also have plugins for browsers, so even people with zero technical knowledge can simply install the file, and it will reconfigure the browser automatically

So what can you do?

To solve the problem, you need some specific features in your web filter. If you are reading this blog, you know that Untangle makes filtering products, and we indeed can solve the above issues. You need to be inline, in the flow of traffic, so the filter is not just seeing proxy or port 80/443 traffic. The filter needs to be able to fully decrypt HTTPS and cannot be 100% dependent on a list for URL detection. The filter needs to be able to do reverse IP lookups, understand multiple languages and enforce safe search features in browsers. It needs to be able to recognize proxy web site behavior in real time and use layer 7 application signatures so it can detect and stop things like Ultrasurf.

For more information on how Untangle products can help you stop filter avoidance, contact us at [email protected]

– Joe