I’ve been troubleshooting some possible problems on a mail server recently, and have been digging into TCP connections some. The ‘netstat’ command has a ‘-o’ option that displays some timers that are useful:
[mail]# netstat -on |grep 189.142.18.18 tcp 0 8 205.244.47.142:25 189.142.18.18:1256 ESTABLISHED on (17.00/4/0) tcp 0 452 205.244.47.142:25 189.142.18.18:2676 ESTABLISHED on (36.09/6/0)
This displays countdown timers for each TCP State. For example, if a connection is in FIN_WAIT and you run the command over and over with “watch”, you can watch the time count down to 0 and then go away. The man pages and documentation I could find didn’t explain the timers very well, so this is what I have learned by watching it. (read: this is not official).
When a connection is in the ESTABLISHED state, the timer can be either on or off. From what I can tell, the counter turns to ON when there is some kind of trouble with the connection. It looks like when a retransmission occurs, the timer is flipped ON, and then the countdown timer starts. The countdown timer has 3 numbers. The first is a countdown in seconds, the second is incremented for each retransmission, and the third one is always 0, so I’m not sure what it does
Now that I have a basic understanding of the output, I still have to figure out why these connections just hang. My guess at this point is that it is poorly written spamming software and maxed out bandwidth on all a whole much of compromised machines throughout the world that are hitting my mail servers.