The Boy Who Harnessed the Wind

First watched this dude on John Stewart. Most fascinating thing I heard in a while and proof world is filled with Hope:

Bye Bye Spammers

It appears after the latest attempt to drop the connection from spammers at IP level, traffic has come to normal level.

To read about the issue and resolution follow these links:
Day 1
Day 2

Graph showing traffic returning to normal:

Someone remind google what year it is

Chrome 1.0.154.48. was released on Feb 3 2009.

Chrome home page seems to have the same issue:

More fun with Proxy

After disabling proxy on my server, I still see traffic on my server being high (My monthly quota might not exceed at this rate but takes up 50% of Bandwidth). Looking at access log it appears that requests hasn't stopped though they are getting 403 error. So requests coming to server and 403 response by itself is making up few GB worth a data every day. So decided to block these requests at IP level rather than proxy level. First I needed to get all unique IP addresses that needs to be blocked. That was easy to considering my log format being:

61.139.105.163 - - [17/Feb/2009:05:08:48 -0700] "GET http://ad.yieldmanager.com/imp?z=10&s=425858&u=http%3A%2F%2Fwww.popflashgames.com%2Findex.html HTTP/1.0" 403 388 "http://www.popflashgames.com/index.html" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0"


So I needed to get all the 403 message lines (access error's) and get IP (first field in the log) and get unique values of those IP's. Simple uniqx command can generate that(output redirected to tmp file):

more myserver.access_log | grep ' 403 ' | cut -d' ' -f1 | sort | uniq > /tmp/block.txt


Now that I have all the IP's that need to be blocked I wrote a simple script to block all the IP's in the tmp file (using iptables -A INPUT -s IPAddress -j DROP). Here is the script:

#!/bin/bash
# /tmp/blockspam.sh
# Drop all the spammers
SPAMIPS=$(egrep -v -E "^#|^$" /tmp/block.txt)
 
for spamip in $SPAMIPS
do
   iptables -A INPUT -s $spamip -j DROP   
done

To view all the blocked IP's use the command:
iptables -L -n

Proxy turn it off

Noticed something funny on my personal webserver. I was testing some proxy settings and left proxy setting on when done. Today morning when I tried to access my server, its unreachable. Checking on the log's there seem to be too many requests proxying through my web server. All the IP's seem to originate from China and destination seems to be Ad server:

60.173.11.121 - - [16/Feb/2009:12:07:27 -0700] "GET http://ad.spot200.com/imp?Z=728x90&s=533945&_salt=1697293642&B=12&m=2&u=http%3A%2F%2Fgifttiems.com%2F&r=1 HTTP/1.0" 302 - "http://ad.spot200.com/st?ad_type=iframe&ad_size=728x90§ion=533945" "Mozilla/3.01 (compatible;)"
59.53.48.207 - - [16/Feb/2009:12:07:27 -0700] "GET http://tag.contextweb.com/TAGPUBLISH/getad.aspx?tagver=1&if=0&ca=VIEWAD&cp=512141&ct=47581&cf=300X250&cn=1&cr=200&cw=300&ch=250&cads=0&cwu=http%3A%2F%2Fgoautoshop.com&mrnd=688840 HTTP/1.1" 200 438 "http://goautoshop.com" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)"
61.139.105.163 - - [16/Feb/2009:12:07:27 -0700] "GET http://ad.yieldmanager.com/imp?z=0&Z=0x0&s=494075&y=30 HTTP/1.1" 302 - "http%3A%2F%2Fwww.excellenceflash.com%2Findex.html" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0)"
61.139.105.166 - - [16/Feb/2009:12:07:28 -0700] "GET http://ad.yieldmanager.com/imp?z=0&Z=0x0&s=564462&y=30 HTTP/1.1" 302 - "http%3A%2F%2Fwww.flash-animation.net%2Findex.html" "Mozilla/4.0 (compatible; MSIE 4.5; Mac_PowerPC)"



Not sure what these requests are but seems like someone is trying to bump their ad revenue. Immediately disabled proxy on my machine.

Lesson to learn: Never forget to turn off proxy settings in Apache.

JSF and Nested Diagnostic Context

One of the important design criteria for any large concurrent system is to be able to audit and debug production logs. Any real-world web applications need to deal with multiple clients simultaneously. In a typical java web application implementation of such a system, different threads will handle different clients. A possible but discouraged approach to differentiate the logging output of one client from another consists of instantiating a new and separate logger for each client. This technique promotes the proliferation of loggers and considerably increases their management overhead.

Interleaved log output can still be meaningful if each log entry from different contexts had a distinctive stamp. This is where Nested Diagnostic Context or NDC in short come into play. Web App frameworks like JSF which provide well defined life cycle provide additional hooks to implement NDC.

The six phases of the JSF application lifecycle are as follows:
1. Restore view
2. Apply request values; process events
3. Process validations; process events
4. Update model values; process events
5. Invoke application; process events
6. Render response




JSF provides PhaseListner that can be implemented by objects that wish to be notified at the beginning and ending of processing for each standard phase of the request processing lifecycle. This can be used to add Context before Restoring View which can be removed after Response is Rendered.

Log4J provides a nested diagnostic contexts implementation using NDC class.


Lets see how these two can be used to add User Id to every log statement. Implement NDCLogger as follows:

package com.km.logging.util;

import javax.faces.context.FacesContext;
import javax.faces.event.PhaseEvent;
import javax.faces.event.PhaseId;
import javax.faces.event.PhaseListener;

import org.apache.log4j.NDC;

/**
 * Log4j NDC Logger to add Remote User in all the log entries.
 */
public class NDCLogger implements PhaseListener {
       

    public void beforePhase(PhaseEvent e) {
        PhaseId phaseId = phaseEvent.getPhaseId();
        if (phaseId == PhaseId.RESTORE_VIEW) {
            NDC.push((String) FacesContext.getCurrentInstance()
             .getExternalContext().getRemoteUser());
        }
    }

    public void afterPhase(PhaseEvent e) {
        PhaseId phaseId = phaseEvent.getPhaseId();
        if (phaseId == PhaseId.RENDER_RESPONSE) {
            NDC.pop();
            NDC.remove();
        }
    }
    
    public PhaseId getPhaseId() {           
        return PhaseId.ANY_PHASE;
    }
}

Add your PhaseListener to your faces config as follows:

   
  com.km.logging.util.NDCLogger

If configured to do so, PatternLayout and TTCCLayout instances automatically retrieve the nested diagnostic context for the current thread without any user intervention. Hence, even if a servlet is serving multiple clients simultaneously, the logs emanating from the same code (belonging to the same category) can still be distinguished because each client request will have a different NDC tag.

Microsoft and Silly bugs

Less than a fortnight back Zune30 leap year bug caused mass suicide of all Zune30 devices. Later on it was found to be simple looping bug in Zune's Clock Driver as shown here in the code to determine year part of the date:

year = ORIGINYEAR; /* = 1980 */

while (days > 365)
{
    if (IsLeapYear(year))
    {
        if (days > 366)
        {
            days -= 366;
            year += 1;
        }
    }
    else
    {
        days -= 365;
        year += 1;
    }
}

Now today an old bug that prevents IE setting cookies if domain attribute is in upper case and has odd number of characters surfaced. Obviously this is not an indication of quality at Microsoft but it is certainly fun watching world's largest software company having such silly bugs in multiple products.