vertner.net

Home Proxy Servers and Saving Bandwidth

We survived a recent interstate move relatively unscathed though unfortunately, we had to make one really huge change: our ISP (Internet Service Provider). This normally wouldn’t be that traumatic, except that we transitioned from an inexpensively-priced cable-based service to a slightly more expensive and much slower DSL-based service. To add insult to injury, they have data caps! Generous though they may seem, my family will surely put them to the test. The obvious answer: CACHING! Fortunately, the mighty 2007 Dell lives on to save the day once again! First: what is a proxy server and what is caching?

Better Living By Proxy

Simply put, a proxy server is a computer that stands between you and the internet. It can be used for a variety of purposes, including source IP address masking, content filtering, passive logging, caching, and more. All you have to do is tell a computer to use a proxy server and it will stop by there for whatever the proxy does before being passed along to the big bad internet.

I’m using my proxy for one simple thing: caching. Your computer already does this on its own; you visit your favorite site once and your browser cache will hang on to those files so it can pull from there instead of re-downloading them. A proxy server running a cache adds one more intermediary in the chain so now when a browser wants to download an image, first, it checks the browser cache, then it goes to the proxy server and checks its cache. If it still doesn’t find anything, then it goes to the internet and downloads.

Squid and Ubuntu Server

Since the Dell runs Ubuntu Server, the installation process and configuration is just an ssh away! First, let’s download and install Squid.

1
sudo aptitude install squid

This will download and install Squid and all of its dependencies. Next, it’s time to configure! Squid keeps all of its configuration files in one directory, but there’s only one we need to edit. First, make sure you make a copy of the default!

1
2
sudo cp /etc/squid3/squid.conf /etc/squid3/squid.conf.default
sudo nano /etc/squid3/squid.conf

As you can see, this is an extremely well-documented (and commented-out) configuration file. Rather than struggling to find, uncomment, and edit all of the appropriate lines, let’s just make some room at the top and add the following:

1
2
3
4
5
6
acl localnet src 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16

http_access allow localnet
icp_access allow localnet

cache_dir ufs /var/spool/squid3 30000 16 256

`

The ACL or Access Control List defines what IP addresses (in CIDR notation) can use the proxy server; I dumped in all of the common home IP ranges so if I don’t have to remember to update the proxy configuration if I change out networking hardware. Now that I have defined what my localnet looks like, I’ll give it http_access and icp_access to let it do its job. Finally, I need to set my caching directory and its size using the cache_dir variable. I left it as the default directory and set it to 30,000 megabytes (roughly 30GB). Note: I heartily recommend excluding your caching directory from any backups; it’s not necessary to retain.

Finally, restart Squid with the following commands:

1
2
sudo stop squid3
sudo start squid3

Implementation and Testing

You should now have Squid up and running smoothly… and it’s doing absolutely nothing for you until you tell the rest of the computers what to do with it. On the one hand, you can do some really clever stuff with automatic configuration files, custom DNS entries, and a local web server. On the other hand, you can just point each computer’s Proxy setting at the IP address and port of the proxy server and call it a day. The number of computers you have to configure and your desire to maintain an internal web server will probably dictate your choice.

On OS X, go to System Preferences, Network, and select the network interface you use to connect to the internet and select Advanced. Finally, select the Proxies tab, check Web Proxy (HTTP) and fill in your proxy server’s IP address and the port you set Squid to listen on (3128 is the default).

It just works... kind of.It just works… kind of.

On Windows 10, click on the Network icon in the System Tray and select Network Settings. From there you can select the Proxy tab and fill out the appropriate proxy server address and port under the Manual proxy setup.

Not much to see here.Not much to see here.

As for Linux, it’s complicated. It always is. That said, you can see that it’s a fairly simple process in the above two examples. You should have some luck finding what you need in some kind of network settings panel or using Google if you get desperate.

How do we know it’s working? Get back to your proxy server and run the following:

1
sudo tail -f /var/log/squid3/access.log

Now jump onto a couple of your newly-configured computers and start browsing to some unencrypted web sites. You should see the log start scrolling like crazy as the proxy starts intercepting requests. It’s working! Type Ctrl C whenever you’re done watching the excitement.

Extra Credit for Macs

If you have a lot of Apple systems running on your network (guilty), you can also save a ton of bandwidth with one more piece of software: the OS X Server. I don’t often recommend this software; many of its features are underpowered or offer nothing special to Apple users over what’s available elsewhere. One truly special feature is the Caching service, which is unfortunately named. Where the proxy caching server we just made will cache all unencrypted web content, this service will do the same any OS X or iOS updates, App Store downloads, and certain other Apple downloads, regardless of the user. It will not cache anything else, so use it alongside the Squid server you just built.

While limited, setting it up is a snap. Just open up your Server app, select Caching, select a directory to store the cache, pick a size, and turn it on. That’s it. All OS X and iOS systems on the network will automatically look there first.

No need to pay too much attention to the rest of that panel.No need to pay too much attention to the rest of that panel.

The Good, the Bad, and the Proxy

All of this is a great way of reducing your bandwidth costs and speeding up your connection, right? There’s one ugly little problem: the modern internet is made of a tremendous amount of encrypted dynamic content. As you watched the access log, you probably saw a lot of TCP_MISS entries and only an occasional TCP_IMS_HIT or TCP_MEM_HIT. Those TCP_MISS entries are all times where content still needed to be downloaded from the internet. The TCP_IMS_HIT is when content is found in the cache on the hard disk. The TCP_MEM_HIT is the same, except the content is still in the proxy server’s memory (even faster). Additionally, this type of proxy server won’t do anything with encrypted or streamed content. So much for speeding up Facebook or YouTube.

There are methods for caching certain encrypted content, but in case you haven’t noticed from my prior articles, security is very important. Most secure web proxies work by decrypting the content before it gets to your machine and enabling them makes you much more vulnerable to man-in-the-middle attacks.

So while I definitely recommend setting up a proxy caching server if you’re already running a server on your home network, don’t expect to be blown away by some sudden increase in download speeds. This will just trim some inefficiencies. That said, here’s some bonus stock photography of women using their new proxy caching servers.

THISTHIS ISIS SOSO AWESOMEAWESOME

Comments