[UPDATED 10-18-2009 - Numerous old typos fixed, several new typos added, syntaxes corrected, updates made for newer versions of stuff, better instructions, cooler errors, and even a little more attention to detail paid.]
[CREDIT for getlists.sh goes to Step By Step - www.maynidea.com. Thank you for this script, and sorry it took so long to put this credit in.]
This article is a revision of this post. It has been adapted for use on Kubuntu 8.04. I got a lot of info from this link here. Another excellent resource is here (PDF). As always, YMMV. This is a long and involved post – be prepared to take an afternoon, and to work on that degree from Google. But when you are done, you will have a powerful transparent-proxy-content-filter-porn-stomper. No charge.
1. Download the following (there may be newer versions, but definitely need db-2.7.7):
- db-2.7.7.tar.gz (http://download.oracle.com/berkeley-db/db-2.7.7.tar.gz)
- squid-2.6.STABLE5-20061110.tar.bz2 (http://www.squid-cache.org/)
- dansguardian-188.8.131.52.tar.gz (http://dansguardian.org/)
- squidGuard-1.2.0.tar.gz (http://www.squidguard.org/)
- A mail server – I use postfix (installed from Kubuntu repositories).
I checked these versions against the repositories, and except for the db-2.7.7, these are still fairly current. The version of iptables I am using is 1.3.8. For this, I prefer installing from tarballs, even though this means they will not get updates. The main advantages I see to this approach are that you can more directly control where they go in the file system (making them easier to troubleshoot and remove), and updates to packages might cause feature/config file breakage, whereas these ensure a static environment. Unfortunately, I cannot upload the actual tarballs for use, so either find these versions in an archive, or brace yourself for an adventure in configuration differences.
2. Unpack the downloaded files:
tar xvfz db-2.7.7.tar.gz
tar xvfj squid-2.6.STABLE5-20061110.tar.bz2
tar xvfz dansguardian-184.108.40.206.tar.gz
tar xvfz squidGuard-1.2.0.tar.gz
3. Check that you don’t already have squid, squidGuard, or dansguardian already installed, and that you have iptables installed. Adept Manager is an easy way to find out. Check that you do not already have a squid group and user. If you do not, then pick a group ID between 1 and 999 to use for the squid group:
more /etc/group | grep -i squid<is there a squid group?>
more /etc/passwd | grep -i squid<is there a squid user?>
more /etc/login.defs | grep -i UID_MIN<what is the lowest user ID? anything below this is a system account, and will not get a home directory by default, which is a good thing – so pick something lower than UID_MIN>
more /etc/group | grep <number below UID_MIN><is the group ID you picked already in use? If so, keep picking one until you find a number not in use.>
4. As root (sudo -s), make user and group. The “
groupadd -r squid” command is out – this would have made a system account. The new command syntax is shown below instead.
groupadd -g <number you picked> squid
useradd -u <number you picked> -g squid -d /var/spool/squid -s /bin/false -r squid
5. When making firewall rules (below), I kept getting the error “iptables: No chain/target/match by that name” until I discovered that I did not have the ipt_owner.ko module available to be loaded (on my current version of 220.127.116.11, it is called “xt_owner”). Issue an “
updatedb” command, followed by “
locate _owner.ko” to see if you have it for your kernel version. If you have it, see if it is loaded – “
lsmod | grep -i _owner“. I ended up compiling a new kernel from 18.104.22.168 to 22.214.171.124 (to get some other features I wanted, not just for the module), and ensuring the owner module was built (“
make oldconfig” and “
make menuconfig” steps of this post, under the networking section). Once I had that module, I was good to go with matching packets by owner.
Make menuconfig (need ncurses libraries installed: libncurses5-dev and libncursesw5-dev; helpful to have ncurses-term packages installed):
“Networking Support —>
Networking Options —>
Network Packet Filtering Framework (Netfilter) —>
Core Netfilter Configuration —>”
- (M) Netfilter connection tracking support (NF_CONNTRACK)
- (M) Transparent proxying support (EXPERIMENTAL) (NETFILTER_TPROXY)
- (M) “TPROXY” target support (EXPERIMENTAL) (NETFILTER_XT_TARGET_TPROXY)
- (M) “owner” match support (NETFILTER_XT_MATCH_OWNER)
REMEMBER: If you upgrade your kernel to a new version and use a proprietary video driver (ATI or NVIDIA), set your xorg.conf driver to “vesa” BEFORE you reboot. Reboot on the new kernel, log into the console (so as not to start any window manager or x session), and upgrade your video driver (update xorg.conf to reflect the new driver). Then either reboot, or just start your window manager normally.
6. Make BerkelyDB – must be 2.x version, not newer, not older:
7. Make squid v.2-6 (NOTE – To have SSL, I needed to install the libcurl4-openSSL-dev package. Otherwise, “
make” generated this error: “../include/md5.h:14:2: error: #error Cannot find OpenSSL headers” ):
./configure --enable-icmp --enable-delay-pools --enable-useragent-log --enable-referer-log --enable-kill-parent-hack --enable-cachemgr-hostname=hostname --enable-arp-acl --enable-htcp --enable-ssl --enable-forw-via-db --enable-cache-digests --enable-default-err-language=English --enable-err-languages=English --enable-linux-netfilter --disable-ident-lookups --disable-internal-dns
It is located in /usr/local/squid/.
8. Make squidGuard v.1.2:
Default install is in /usr/local/bin/.
9. Make dansguardian v.2.9.8:
./configure --prefix=/usr/local/dansguardian --with-proxyuser=squid --with-proxygroup=squid --enable-email=yes
- FOR EMBEDDED URL WEIGHTING AND OTHER FEATURES:
./configure --prefix=/usr/local/dansguardian --with-proxyuser=squid --with-proxygroup=squid --enable-email=yes --enable-pcre=yes(this last option is CPU intensive; turn on in dansguardianf1.conf)
It is located in
If you get an error during the configure part like this: “
configure: error: pcre-config not found!“, install the libpcre++-dev package.
When using GCC 4.3, I got errors of “
error: ‘strncpy’ was not declared in this scope“. The fix was found on GCC 4.3 Release Series – Porting to the New Tools. Basically, for each such error, go to the file referenced under the src folder and add the line
#include (cstring) (replace parentheses with angle brackets).
10. Make and configure squid directories:
chown -R squid:squid /usr/local/squid/var
chmod 0770 /usr/local/squid/var/cache
chmod 0770 /usr/local/squid/var/logs
11. Make and configure squidGuard directories (see getlists.sh for reference):
chown -R squid:squid /usr/local/squidGuard/log
chmod 0770 /usr/local/squidGuard/log
chown -R squid.squid /var/log/squidguard
chown -R squid:squid /var/lib/squidguard
12. Configure dansguardian directories:
chown -R squid:squid /usr/local/dansguardian/var/log
13. Edit and copy squid configs from respective source directories:
cp squid.conf /usr/local/squid/etc/squid.conf
- sample squid.conf settings:
- http_port 127.0.0.1:3128 transparent
- icp_port 0
- htcp_port 0
- redirect_program /usr/local/bin/squidGuard
- cache_effective_user squid
- cache_effective_group squid
- acl all src 0.0.0.0/0.0.0.0
- acl manager proto cache_object
- acl localhost src 127.0.0.1/255.255.255.255
- acl to_localhost dst 127.0.0.0/8
- acl allowed_hosts src 192.168.12.0/255.255.255.0
- acl SSL_ports port 443
- acl Safe_ports port 80 21 443 # http ftp https
- ##acl Safe_ports port 21 # ftp
- ##acl Safe_ports port 443 # https
- ##acl Safe_ports port 1025-65535 # unregistered ports
- acl CONNECT method CONNECT
- acl NUMCONN maxconn 5
- acl ACLTIME time SMTWHFA 7:00-21:00
- deny_info ERR_ACCESS_DENIED_TIME ACLTIME
- #http_access allow manager localhost
- #http_access deny manager
- http_access deny manager all
- http_access deny !Safe_ports
- http_access deny CONNECT !SSL_ports
- http_access allow localhost ACLTIME
- http_access deny NUMCONN localhost
- #http_access allow allowed_hosts
- http_access deny to_localhost
- http_access deny all
- http_reply_access allow all
- #icp_access allow allowed_hosts
- #icp_access allow all
- icp_access deny all
- visible_hostname localhost
Edit squid.conf and set up time based access, to prevent late night surfing (add the following lines):
acl ACLTIME time SMTWHFA 7:00-21:00 (add to the ACL section)
http_access allow localhost ACLTIME (add to the http_access section)
14. Edit and copy squidGuard configs from respective source directories:
cp squidGuard.conf /usr/local/squidGuard/squidGuard.conf
- change ip gateway address in squidGuard.conf
15. Edit and copy dansguardian configs from respective source directories:
cp dansguardia*.conf /usr/local/dansguardian/etc/dansguardian/
- sample dansguardian.conf settings:
- filterip = 127.0.0.1
- filterport = 8080
- proxyip = 127.0.0.1
- proxyport = 3128
- accessdeniedaddress = ‘http://YOURSERVER.YOURDOMAIN/cgi-bin/dansguardian.pl
- mailer = ‘/usr/sbin/sendmail -t’
- sample dansguardianf1.conf settings:
- groupmode = 1
- copy getlists.sh (it is posted as a PDF – copy the text to a shell script) to
[UPDATED 10-18-2009 with more current blacklist sites]
16. Make the firewall rules (iptables commands may appear wrapped in two lines):
iptables -t nat -A OUTPUT -s 127.0.0.1 -d 127.0.0.1 -p tcp --dport 3128 -j ACCEPT(without this rule, dansguardian may fail with the error: “Error connecting to parent proxy”)
iptables -t nat -A OUTPUT -p tcp --dport 80 -m owner --uid-owner squid -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 3128 -m owner --uid-owner squid -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 80 -m owner --uid-owner exemptuser -j ACCEPT (change exemptuser)
iptables -t nat -A OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 8080
iptables -t nat -A OUTPUT -p tcp --dport 3128 -j REDIRECT --to-ports 8080
It is a good idea to do this part *after* compiling and installing, as these rules will get in the way if you need to install a package (like libcurl4-openSSL-dev). If this happens, Adept Manager will abruptly crash (leaving you to find and remove the lock files), and
apt-get install will fail with a connection refused error. Just rerun the rules above, but replace the -A with a -D to delete them. Get your packages, install your software, and reapply the firewall rules.
17. Save and apply the firewall settings permanently (visit Iptables HowTo – Community Ubuntu Documentation for details):
sudo sh -c "iptables-save > /etc/iptables.rules"
sudo nano /etc/network/interfaces
pre-up iptables-restore < /etc/iptables.rules
post-down iptables-save -c > /etc/iptables.rules
18. Start or restart services as needed:
/usr/local/squid/sbin/squid -z(first-time config)
/usr/local/squid/sbin/squid -N -d 1 -D(test squid, kill when working fine)
/usr/local/squid/sbin/squid(this also runs squidGuard from “/usr/local/bin/squidGuard”)
/usr/local/bin/getlists.sh(you may have to kill this – it hangs after displaying the line “adult/usage”)
/usr/local/squid/sbin/squid -k reconfigure
The squid test revealed that I was missing a custom file: “errorTryLoadText: ‘/usr/local/squid/etc/errors/ERR_ACCESS_DENIED_TIME': (2) No such file or directory”. So, I copied it from “/usr/local/squid/etc/errors/English/ERR_ACCESS_DENIED”, and “edited” it in vi for a little access-denied humor. Never miss a chance to have a spot of fun! After that, squid worked fine.
Dansguardian kept failing with “Error connecting to parent proxy”, until I edited iptables with “
iptables -t nat -I OUTPUT 1 -s 127.0.0.1 -d 127.0.0.1 -p tcp --dport 3128 -j ACCEPT"
(to place it as the first output rule on the nat table). Then DG worked fine.
The script hung and had to be killed. I confirmed everything was finished by checking the last file date-time-stamp against the date-time-stamp it displays right after it is run. So if the DTS displayed was “20090214185211”, and the DTS returned with “ls -l /var/lib/squidguard/db/blacklists/porn/stats/20090214185211_stats” was more recent, say “2009-02-14 18:53″, then you can be sure it is finished. Or you can just use “lsof” and look for the getlists.sh process. That is probably smarter.
The script hung because a.) I could not download from the Norway site and b.) “squidguard -C all” from the getlists.sh script was not finding files and went into emergency mode, apparently a place it can hide and whimper silently. Forever. I ran instead “squidguard -d -C all” and discovered it was failing to find certain files, which I just created or copied into existence. This quieted squidguard down and let it finish. Almost – I also commented out the “not_ok” ACL block in the squidguard.conf file, since I am not using it. Details are on this article concerning the updated blacklist script “getlists.sh”: SquidGuard Blacklists…
19. Set up a mailer for notifications (here is a link for assistance):
- using postfix, point it to your mailserver.isp.domain
- postfix needs
- dansguardian.conf calls it with ‘
sendmail -t' command
- for non-authenticated use, do not set ‘by user = on’ in dansgaurdianf1.conf
20. Post-install testing and tweaking:
- Test with browser as different users – should be transparent proxy surfing now, works with lynx as well (“
su - <username>, lynx, G, http://www.playboy.com” should get either Playboy for an approved user or the dansguardian access denied page for a restricted user.)
- Check if your system emails you violations.
- Be sure to update your startup files (/etc/init.d/ or your rc.local) to ensure everything starts when the computer is booted.
- When you are ready, reboot, and check again with lynx as different users.
I have been working on this all day. I have not yet gotten email to work, and am not sure I need to – maybe I’ll just check the logs instead. So, hope this helps, and good luck.
Time for a beer.
Filed under: Apps, HowTo, Kernel, Kubuntu, Linux, Security, Tips…, Web Servers | Tagged: blacklist, content filter, dansguardian, db-2.7.7, filtering, firewall, HowTo, iptables, proxy, redirect, scripts, Security, shell, squid, squidGuard, transparent, web | 4 Comments »