Category: General (Page 18 of 25)

I’m not a fan of Qmail, and am in the process of migrating users off of it. As one of the steps in a long process, I’m first migrating users from one qmail server to another. The destination server uses a database-backed vpopmail installation to store some of the user information in, while the source server is still using the traditional file-based structure. Each email alias each had a file named .qmail-USERNAME which contains one forward per line. So a forward for brandon would be named .qmail-brandon and contain something like this:

&[email protected]
&[email protected]

There exists a utility named ‘vconvert‘ which converts the actual user accounts into the new, database format. But after a little searching, I was unable to find a similar utility to convert aliases. So I wrote up a quick one in perl.  I tried pasting it here, but WordPress mangles the syntax.  Instead you can view it separately or download it

I installed postgrey on several of the mail servers that I manage and have been impressed with the results. Greylisting works by temporarily blocking senders the first time that they attempt to send a message. Spammers will (hopefully) give up and move on to more susceptible targets, while legitimate mail servers will retry delivery a few minutes later.

Prior to installing this, our mail filters were identifying about 70-75% of the messages passing through it as spam (this is after rejecting invalid recipients, and using a couple IP-based blacklists). After installing postgrey, that number is down to around 50%. So, for us, a simple 10 minute installation of postgrey has reduced the amount of mail that we have to scan by about 35%.

I actually installed it on a few different machines, all around the same time and wrote up some instructions

Of course, I’ve spent a little more time tweaking the installation just a little. I changed the timeout to 4 minutes, instead of 5, so that if a legitimate mail server is set to retry every 5 minutes, it shouldn’t have a problem. I also customized the URL that it sends in the 450 response to one that points to our own website. Overall I’m very impressed and will recommend installing it on any mail system I’m involved with.

Update 2007-08-08

Here is a graph showing the drop in mail processed due to the greylisting. The drop has been significant and has helped our mail filtering service very much. Fewer spam messages bypass the filters, and the load on the servers has decreased so that we can handle more capacity as we need to.

The IPTrack application that I wrote saves a bunch of Netflow data generated by routers into a MySQL database for analysis and summarization. During peak usage times, it inserts about 50k rows per minute into a table. To keep the table at a manageable size, it then summarizes the useful data into a lower-volume table and deletes any Netflow data older than 10 minutes out of the table. This particular table usually has between 300k and 500k rows in it at any given time.

The MEMORY storage engine operates completely from Ram. Of course this has the advantage of being very quick compared to File based access. The downside is that any data in it is lost if the MySQL instance has to be restarted for any reason. Since my application just uses it for temporary data anyway, and it doesn’t matter all that much if I lose up to 10 minutes of data, this seemed perfectly acceptable.

So, I tried converting the table from MyISAM to MEMORY. For the first couple minutes it looked pretty promising. Disk IO and the machine’s load was pretty small and it was cruising along. But, after ten minutes, when it got to the point where it was purging old data from the table, it came to a grinding halt. The delete statement, which took only a few seconds using the MyISAM engine just kept going and going. It ended up taking about 9 MINUTES to complete. That, of course, is pretty unacceptable. The MEMORY storage engine was supposed to be faster than MyISAM, so this made no sense. The table was already pretty well indexed. I was deleting based on the timestamp column:

DELETE FROM flows WHERE timestamp < ?

and the timestamp column had an index. I had no idea what could make it take so long. I ran across a bug report of somebody else having the problem. The reply was that this was just the nature of hash indexes. The bug report was from 2004 though, so I posted my problem to the mysql forums in search of an answer.

I just noticed that somebody replied to my post and said to use btree indexes instead of hash indexes (the default for the MEMORY engine). Sure enough, I recreated my indexes using btree and it works perfectly now.

CREATE INDEX timestamp_btree USING BTREE ON flows (timestamp);

Thanks to KimSeong Loh on the MySQL forums for solving this for me

awstats is my favorite web statistics program. It provides quite a bit of interesting data about my websites, and I usually set it up for any site that I work on. One issue that I’ve had with it though, is that in a shared setup, by default, it uses the HTTP HOST header to check which domain to display statistics for. The config files and data files are all saved with the full hostname of the domain for the stats it contains. So it treats “www.brandonchecketts.com” different from “brandonchecketts.com”.

This has always been just a slight annoyance to me, and I have just remembered to make sure to put the ‘www’ on the URL when I’m looking at my statistics. Today, though, I thought I would dig into it a little bit to fix it permanently.

Turns out it was pretty easy to change this behavior so that it always just removes the ‘www’ from the domain name for any of the files it looks for. Simply add this on line 1160 of awstats.pl:

$SiteConfig =~ s/^www\\.//g;

It should go right after the $FileConfig= line, and right before the foreach loop. Since all of my database files were created with the ‘www’ in them, I had to go through and rename all of those database files to remove the www from them. Now I can hit either URL and get the same data.

I started a forum thread on the SourceForge forums to announce it to others and see if anybody else finds it useful.

I’ve recently setting up a bunch of hosts with a new Amanda backup server. I like to see, though, how much space each server that I’m backing up uses. Amanda stores a bunch of info in the ‘curinfo’ directory for each host and disk that is being backed up, but I haven’t found any good tools for querying or displaying that. So, I wrote my own. This script looks through all of the files in the ‘curinfo’ directory, and prints out a summary of how much space each disk/host is taking up:

#!/usr/bin/perl

## View Amanda disk space usage per host/disk
## Author:  Brandon Checketts
## Website: https://avazio.com/

$curinfo_dir = "/etc/amanda/avazio/curinfo";

opendir(DH, $curinfo_dir);
while($host = readdir(DH)) {
  next if($host =~ m/^./);
  if( -d "$curinfo_dir/$host") {
    opendir(DH2, "$curinfo_dir/$host");

    while($disk = readdir(DH2)) {
      next if($disk =~ m/^./);
      if( -f "$curinfo_dir/$host/$disk/info") {
        open(FH, "< $curinfo_dir/$host/$disk/info");
        while(my $line = ) {
          if($line =~ m/^history: ([0-9]+) ([0-9]+) ([0-9]+) ([0-9]+) ([0-9]+)/) {
            ## Example line: history: 1 2319760 2319760 1184766354 1121
            ## Line format:  history [lvl] [rawsize] [compsize] [timestamp] [unk?]
            $space->{$host}->{$disk}->{'rawsize'}  += ($2 / 1024);
            $space->{$host}->{$disk}->{'compsize'} += ($3 / 1024);

          }
        }
      }
    }
    closedir(DH2)
  }
}
closedir(DH);

$grandtotal_rawsize  = 0;
$grandtotal_compsize = 0;
foreach my $host (keys(%{$space})) {
  print "n$hostn";
  $thishost = $space->{$host};
  $thishost_rawsize  = 0;
  $thishost_compsize = 0;
  foreach my $disk (keys(%{$thishost})) {
    $thisdisk = $space->{$host}->{$disk};
    $thishost_rawsize    += $thisdisk->{'rawsize'};
    $thishost_compsize   += $thisdisk->{'compsize'};
    $grandtotal_rawsize  += $thisdisk->{'rawsize'};
    $grandtotal_compsize += $thisdisk->{'compsize'};
    $disk =~ s/_///g;
    printf("  %-40s %-6i Mb   %-6i Mb\\n", $disk, $thisdisk->{'rawsize'}, $thisdisk->{'compsize'});
  }
  printf("  TOTAL:                                   %-6i Mb  %-6i Mb\\n",  $thishost_rawsize, $thishost_compsize);
}

printf("GRAND TOTAL:                                 %-6i MB  %-6i MB\\n", $grandtotal_rawsize, $grandtotal_compsize);

The company I work at is having to change who their Dialup modem pool goes through. The new company blocks outgoing SMTP by default to prevent spamming. To enable port 25 for our mail servers, they we have to send them some attributes in the Radius ACCEPT packet during authentication. I haven’t really gotten into the Radius server that we use, because it has always worked and we’ve never had to change anything on it until now. So I began digging into FreeRadius, and found out that it is pretty useful.

Our server was configured to authenticate against a MySQL database that contains all of the configuration. I believe it is a pretty standard configuration. In the radius configuration, it defines certain queries to run for authentication, and also for attributes to send in the accept packet. Specifically, it’s was configured to do this query for the attributes:

authorize_reply_query = "
  SELECT id,UserName,Attribute,Value,op 
  FROM ${authreply_table} 
  WHERE Username = '%{SQL-User-Name}'
  ORDER BY id"

So, I can just put additional rows into the ${authreply_table} to add the attributes that this new provider requires. With the default setup, though, I would have to add the attributes for each user. That would get to be a mess, because I would have to modify the billing system which populates the radius database. Instead I modified the SQL query to include rows where the Username = ALLUSERS, like this:

authorize_reply_query = "
    SELECT id,UserName,Attribute,Value,op
    FROM ${authreply_table}
    WHERE (
        Username = '%{SQL-User-Name}' OR
        Username = 'ALLUSERS'
    ) 
   ORDER BY id"

Now, I can just insert rows for those attributes once, and no further modification is necessary. Pretty handy.

We’re installing the Amanda backup client on a bunch of servers that I administer. We’ve decided to tunnel Amanda connections through SSH though, and that feature isn’t available in the current RPM builds that are available.

We started out compiling it from scratch on each box, and then following a bunch of steps to add the appropriate users, create directories, setup SSH keys, etc. Obviously, that gets to be a tedious process pretty quick and is prone to errors and missing steps. The obvious solution is to create an RPM for this, since that is what they are for.

Creating an RPM is pretty easy. I found some useful instructions here and here. Basically, the buildrpm package goes through the standard configure; make; make install steps for you. You can create simple shell scripts to do whatever you want at each of those processes when building the RPM. Additionaly, you can also run commands before and after RPM installation, and before and after RPM removal. For this particular script, I added a bunch of stuff to the postinstall script. As an example, here is my .spec file:

Summary: Amanda Client with SSH
Name: amanda_client_ssh
Version: 2.5.2p1
Release: 2
License: GPL
Group: Amanda Backup
Source: amanda_client_ssh-2.5.2p1.tgz

BuildRoot: %{_builddir}/%{name}-root

%description
Amanda Client compiled with SSH authentication

%prep

%setup
./configure '--a-bunch' '--of options' --'can go here'

%build
make

%install
rm -rf $RPM_BUILD_ROOT
make DESTDIR=$RPM_BUILD_ROOT install
if [[ ! -d $RPM_BUILD_ROOT/etc/amanda ]]; then
    mkdir $RPM_BUILD_ROOT/etc/amanda
fi
cat >> $RPM_BUILD_ROOT/etc/amanda/amanda-client.conf < EOF
conf "MyConfig"
index_server "backup.mydomain.com"
tape_server "backup.mydomain.com"
auth "ssh"
ssh_keys "/root/.ssh/id_rsa_amrecover"
EOF

%pre

%post
useradd -M -n -g disk -o -r -d /var/lib/amanda -s /bin/bash
        -c "Amanda user" -u 33 amanda >/dev/null 2>&1 || :

if [[ ! -d /var/lib/amanda/.ssh ]]; then
    mkdir -p /var/lib/amanda/.ssh
fi
if [[ ! -d /var/log/amanda ]]; then
    mkdir -p /var/log/amanda
fi

touch /etc/amandates

cat >> /etc/amanda/exclude <
/var/spool/mail
/tmp
EOF
mkdir ~amanda/gnutar-lists
chown amanda:disk /etc/amandates /etc/amanda/exclude ~amanda/gnutar-lists /var/log/amanda/

if [[ ! -f /root/.ssh/id_rsa_amrecover ]]; then
    ssh-keygen -t rsa -N "" -f /root/.ssh/id_rsa_amrecover
fi

cat >> /var/lib/amanda/.ssh/authorized_keys <EOF
ssh-rsa abcdefgABigLongPublicKeyGoesHere== [email protected]
EOF

%preun
userdel amanda

%clean
rm -rf $RPM_BUILD_ROOT

%files
... a bunch of files listed here ...

%changelog

It took a couple revisions of trial and error to get everything correct. That part took the longest because I didn’t see an option to skip the configure and make steps, so anytime I changed the postinstall commands it had to rebuild the whole app. But now that I’ve got the RPM, installing it should be pretty straightforward on the rest of the boxes it needs to be installed on.