More fun with Proxy

After disabling proxy on my server, I still see traffic on my server being high (My monthly quota might not exceed at this rate but takes up 50% of Bandwidth). Looking at access log it appears that requests hasn't stopped though they are getting 403 error. So requests coming to server and 403 response by itself is making up few GB worth a data every day. So decided to block these requests at IP level rather than proxy level. First I needed to get all unique IP addresses that needs to be blocked. That was easy to considering my log format being:

61.139.105.163 - - [17/Feb/2009:05:08:48 -0700] "GET http://ad.yieldmanager.com/imp?z=10&s=425858&u=http%3A%2F%2Fwww.popflashgames.com%2Findex.html HTTP/1.0" 403 388 "http://www.popflashgames.com/index.html" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0"


So I needed to get all the 403 message lines (access error's) and get IP (first field in the log) and get unique values of those IP's. Simple uniqx command can generate that(output redirected to tmp file):

more myserver.access_log | grep ' 403 ' | cut -d' ' -f1 | sort | uniq > /tmp/block.txt


Now that I have all the IP's that need to be blocked I wrote a simple script to block all the IP's in the tmp file (using iptables -A INPUT -s IPAddress -j DROP). Here is the script:

#!/bin/bash
# /tmp/blockspam.sh
# Drop all the spammers
SPAMIPS=$(egrep -v -E "^#|^$" /tmp/block.txt)
 
for spamip in $SPAMIPS
do
   iptables -A INPUT -s $spamip -j DROP   
done

To view all the blocked IP's use the command:
iptables -L -n