Quick Fix for Dovecot’s “save failed to INBOX: Timeout while waiting for lock”

I had a mail server lock up and die on me tonight. The data center replaced some hardware in order to get it to power back up. Everything started up fine, but I was unable to retrieve my own email. My mail client was complaining about timeouts on a single account. In looking at the log, I found some lines similar to these in the dovecot mail logs:

Jul 13 04:19:09 secure deliver(user@domain.com): msgid=<20110713105301.E4C2C3293DC@myserver.com>: save failed to INBOX: Timeout while waiting for lock
Jul 13 04:19:09 secure deliver(user@domain.com): msgid=<20110713105702.817483293E5@myserver.com>: save failed to INBOX: Timeout while waiting for lock
Jul 13 04:19:09 secure deliver(user@domain.com): msgid=<20110713105802.3F68E3293E7@myserver.com>: save failed to INBOX: Timeout while waiting for lock
Jul 13 04:19:09 secure deliver(user@domain.com): msgid=<20110713105402.B5DBA3293DE@myserver.com>: save failed to INBOX: Timeout while waiting for lock
Jul 13 04:19:09 secure deliver(user@domain.com): msgid=<20110713102202.6045E3293A6@myserver.com>: save failed to INBOX: Timeout while waiting for lock

Googling the error message didn’t find any quick results, so I attempted a more brute-force method. Inside the user’s maildir, I just moved some dovecot files out of the way:

[root@host /vmail/domain/user/]# mv dovecot-uidlist.lock dovecot.index dovecot.index.cache dovecot.index.log /tmp/badfiles

After moving those files out of the way, I restarted dovecot with

/etc/init.d/dovecot restart

and my client was able to connect without problems.

APISigning Now Works with Amazon Simple Email Service (SES)

APISigning.com has been signing Amazon Product Advertising requests for a couple of years now. Amazon recently announced their Simple Email Service that makes it easy to send emails via an API. The SES API requires that requests be authenticated using some cryptographic functions that are not easily available on all platforms or programming languages. In those cases, developers can use the APISigning SES Service to calculate the correct signature and perform the request on their behalf.

APISigning has free accounts that effectively allows 10k signing requests each month. Users who require additional requests can subscribe to a paid account with higher limits.

How to shrink a partition with unmovable files in Windows 7

For some reason Windows can’t figure out how to move some files around on disk. When attempting to shrink a volume, it will only allow you to shrink it to where the last immovable file is located. These are some things I found necessary to temporarily get rid of those immovable files so that an NTFS volume could be shrunk

1- Disable System Restore (Right click on Computer => Properties => System Protection => System Restore)
2- Disable Virtual Memory (Right click on Comptuer => Properties => Advanced System Settings => Performance (Settings) => Advanced Tab => Virtual Memory (Change) => No Paging File => Set.
3- Run Disk Cleanup to get rid of Thumbnails, Temporary Internet Files, and a bunch of other files that it makes no sense why they are immovable.
4- Restart the computer to have #1 and #2 take effect

Try to shrink the volume again. If it still is unreasonably large, you will then have to look at Event Viewer to find which file is at the boundary.
1- Right click on Computer => Manage => Event Viewer => Windows Logs => Application.
2- Click on Filter Current Log, and put ‘259’ for the Event ID
3- Click on the latest event and look through the detail to find the problematic file. You can then attempt to delete that file (or set of files) manually. You may have to restart into safe mode to delete some files

It took me about 5 loops of doing the above before I was finally able to shrink my volume to the size that I wanted. After successful, you can then re-enable the features that you want (namely System Restore and Virtual Memory)

KnitMeter.com Has Been Upgraded

KnitMeter Logo

KnitMeter Has Been Upgraded

KnitMeter.com was originally started over four years ago in December of 2007 as a small project that my wife thought would be useful. Since then, the site hasn’t changed much, but it has managed to grow to thousands of users who have knit nearly 20 thousand miles of yarn. I’ve received numerous requests and have finally gotten a chance to impliment what many of you have been requesting for a while now. New features on the site include:

  • Users can now add entries for knitting, crocheting, and spinning
  • Completely new and modernized design and logo
  • You can customize your widgets directly on KnitMeter.com rather than editing the code for the widget on your website
  • The website and the KnitMeter Facebook Application are now completely integrated. Entries added in one will be displayed and counted in the other
  • The Facebook application can (again) publish your entries to your news feed, but only when you tell it to
  • You can chose to make your profile public, which will display some of the most recent entries on the KnitMeter home page with a link to your website
  • Added several new timeframes, including specific calendar years (ie: I knit 4.3 miles in 2010)
  • Numerous technical changes that should make the site faster to use and make it easier to make future changes

These new features have been rolled out over the past couple of weeks. I appreciate the patience of those who have dealt with a few bugs over that time, and I believe that everything should be pretty bug-free now. I encourage you to check out the new site and to start adding up the mileage for your own projects. The next major milestone will be when we have gone through enough yarn to go around the earth (about 24,901 miles). At the present rate, we should hit that figure in about 3-5 months.

Happy Knitting, Crocheting, and Spinning,
Brandon Checketts
KnitMeter.com

Troubleshooting OpenVZ Out of Memory and Could Not Allocate Memory Errors

I will sometimes run into an error inside an OpenVZ guest where the system complains about running out of memory. This is usually seen as a message containing “Cannot allocate memory”. It may even occur when trying to enter into the container

[root@host ~]# vzctl enter 908
entered into CT 908
-bash: /root/.bash_profile: Cannot allocate memory

This particular error can be difficult to troubleshoot because OpenVZ has so many different limits. The “Cannot allocate Memory” error is a pretty generic error, so Googling it doesn’t necessarily yield any useful results.

I’ve found that the best way to track down the source of the problem is to examine the /proc/user_beancounter statistics on the host server. The far right column is labeled ‘failcnt’ and is a simple counter for the number of times that particular resource has been exhausted. Any non-zero numbers in that column may indicate a problem. In this example, you can see that the dcachesize parameter is too small and should be increased

[root@host ~]# cat /proc/user_beancounters
Version: 2.5
       uid  resource                     held              maxheld              barrier                limit              failcnt
      908:  kmemsize                  5170890              5776435             43118100             44370492                    0
            lockedpages                     0                    0                  256                  256                    0
            privvmpages                 53561                55089              1048576              1048576                    0
            shmpages                        2                    2                21504                21504                    0
            dummy                           0                    0                    0                    0                    0
            numproc                        30                   38                 2000                 2000                    0
            physpages                    6136                 7392                    0  9223372036854775807                    0
            vmguarpages                     0                    0                65536                65536                    0
            oomguarpages                 6136                 7392                26112  9223372036854775807                    0
            numtcpsock                      8                    8                  360                  360                    0
            numflock                        5                    6                  380                  420                    0
            numpty                          0                    1                   16                   16                    0
            numsiginfo                      0                    2                  256                  256                    0
            tcpsndbuf                  140032                    0             10321920             16220160                    0
            tcprcvbuf                  131072                    0              1720320              2703360                    0
            othersockbuf               151320               156864              4504320             16777216                    0
            dgramrcvbuf                     0                 8472               262144               262144                    0
            numothersock                  107                  109                 5000                 5000                    0
            dcachesize                3408570              3413265              3409920              3624960                   81
            numfile                       724                  957                18624                18624                    0
            dummy                           0                    0                    0                    0                    0
            dummy                           0                    0                    0                    0                    0
            dummy                           0                    0                    0                    0                    0
            numiptent                      10                   10                  128                  128                    0

You can do a little investigation to see what each particular limit does. I will usually not dig too deep, but just double the value and try again. You can set a new value using the vzctl command like this:

[root@host ~] vzctl set 908 --dcachesize=6819840:7249920 --save

Website Performance: Tables Versus CSS

Most website designers have been using CSS for page layout for several years now, but I occasionally see some websites that continue to use HTML tables for layout. As I’ve been focusing on website performance lately, I’ve found some references that modern browsers render sites using tables for layout slower than they do sites that use CSS. I decided to investigate and confirmed that there are many possible situations where sites using large tables will appear to load much slower than those using CSS. I put together two pages to confirm:

This page uses <div> elements for layout
and
This pages uses a large table for layout

On both pages I’ve added a 5-second sleep near the end of the page to show what might happen if the server was slow, if there were network problems, or any other number of things may have happened.

Notice that the page created using a table changes a lot after the delay. I’ve tried it in Firefox 3 which extends the main (yellow) content section all of the way to the right until it receives the rest of the document, at which point it has to shrink that part to make room for the section on the right. Internet Explorer behaves even worse. It leaves a blank white page until after the delay, at which point it draws the whole table.

By contrast, the page created with CSS positioning shows all of the content above the delay and has it in the correct position. When the rest of the document is sent it just fills in the appropriate content, but doesn’t have to re-arrange anything on the page.

Quick PHP Script to Generate a Barcode

This is a really quick script I came up with to generate a Code-39 Barcode. Many thanks to
Matthew Welch who created the Free 3 of 9 Barcode Font. PHP’s imagettftext function makes this pretty simple.

The $barcode_font referenced below is available from the link above ‘3 of 9’ font. The $plain_font is just a font file that I copied from /usr/share/fonts/default/Type1/ on a Linux box.

This script create a nice looking barcode with the text underneath it like this:

< ?php

$number = isset($_GET['number']) ? $_GET['number'] : '';

$barcode_font = dirname(__FILE__).'/fonts/FREE3OF9.TTF';
$plain_font   = dirname(__FILE__).'/fonts/plain.pfb';

$width = 200;
$height = 80;

$img = imagecreate($width, $height);

// First call to imagecolorallocate is the background color
$white = imagecolorallocate($img, 255, 255, 255);
$black = imagecolorallocate($img, 0, 0, 0);

// Reference for the imagettftext() function
// imagettftext($img, $fontsize, $angle, $xpos, $ypos, $color, $fontfile, $text);
imagettftext($img, 36, 0, 10, 50, $black, $barcode_font, $number);

imagettftext($img, 14, 0, 40, 70, $black, $plain_font, $number);

header('Content-type: image/png');

imagepng($img);
imagedestroy($img);

?>

UPS-PHP Patch to Log Requests and Responses

UPS doesn’t seem to be too big of a fan of Perl or PHP. They provide some powerful functionality through their API’s and their documentation is sufficient, but doesn’t contain examples of anything except for Visual Basic or Java. Fortunately their is an open source project called UPS-PHP that aims to fill that gap by providing some classes for interacting with UPS’s APIs. The UPS-PHP project seems to have lost steam though as the latest updates were over a year ago.

The latest version of the UPS shipping API requires the user to go through a process of creating test transactions, then voiding some transactions in a sandbox environment prior to allowing you access to the production environment. You have to email them every request and response that you send and received for the test transactions as well as various images and HTML documents that were created from their responses.

I added some logging ability to the UPS-PHP class responsible for sending and receiving the responses. A patch is available here if anybody else wants to try it. You basically call the new methods setDebugDir() and setTransaction() on the ‘ups’ object. Those tell it which directory to log to, and which filename to use respectively.

Usage would look something like this:

    $debug_dir = "{$_SERVER['DOCUMENT_ROOT']}/upsdebug/".date('Y')."/".date('m');
    // make sure that $debugdir exists... Create it if necessary
    $upsConnect->setDebugDir($debug_dir);
    $upsConnect->setTransaction(time());
    $upsConnect->setTemplatePath('../../xml/');
    $upsConnect->setTestingMode(1); // Change this to 0 for production
    $upsVoid = new upsVoid($upsConnect);
    $upsVoid->buildRequestXML($ShipmentIdentificationNumber);

At this point your $debugdir would have two files in it. One with the XML request and the other with the XML response, suitable for zipping up and sending to UPS for approval.

PROCEDURE can’t return a result set in the given context

I ran into a problem today when dealing with a very simple SQL Query. The query simply calls a stored procedure on the MySQL server. This is a trivial app, so I was using the very basic mysql_connect(), mysql_query() functions. The result wasn’t being returned an mysql_error() was saying that the error was:

PROCEDURE db.procedure_name can't return a result set in the given context

Of course ‘db.procedure_name’ was the actual name of the procedure I was calling. Googling for the error seemed to indicate that the MySQL client library was old, but this is on a fairly modern CentOS 5.5 server with the php-mysql package at version 5.1.6.

After a bit of experimenting, I found that I was able to change to using the mysql improved versions of the PHP functions and that worked fine

     $dbconn = mysql_connect($dbhost, $dbuser, $dbpass);
     $result = mysql_query("CALL db.procedure_name('arg1', 'arg2', 'arg3')", $dbconn)
     $row    = mysql_fetch_assoc($result);

Becomes

    $dbconn = mysqli_connect($dbhost, $dbuser, $dbpass);
    $result = mysqli_query($dbconn, "CALLdb.procedure_name('arg1', 'arg2', 'arg3')");
    $row    = mysqli_fetch_assoc($result);

Note that changing from mysql_query to myqli_query needs the parameters reversed.

After making that change I’m able to run the stored procedure correctly.