Brandon Checketts

Web Programming, Linux System Administation, and Entrepreneurship in Athens Georgia

Category: General (Page 10 of 25)

‘Maintenance’ Pages via Apache mod_rewrite

November 30, 2008 / / 0 Comments

Occasionally, I’ve found it useful to put up a maintenance page while performing some work on a website. It may be useful if you are debugging and want to ensure that regular visitors don’t see any application generated error messages or blank pages or anything.

This method uses mod_rewrite to redirect all requests to a maintenance page that you create. Since

First create maint.html with some message that you want to display to your users. Then add this to your Apache configuration to redirect users to that page. Obviously, you’ll need to substitute your own IP address. You can add multiple lines to include multiple users if necessary. The configuration essentially says requests not from your IP (notice the exclamation point) will be redirected to /maint.html and that is the last Rewrite rule that should be followed.

  ##### Maintenance section
  ## Uncomment and add your IP address for performing maintenance
  ## Add multiple addresses on multiple lines if necessary
  RewriteCond %{REMOTE_ADDR} !^11\.22\.33\.44$
  RewriteCond %{REMOTE_ADDR} !^1\.1\.1\.1$
  RewriteRule . /maint.html [L]
  ##### End Maintenance section

ddrescue Saves the Day

November 12, 2008 / / 0 Comments

I had a very bad thing happen to my regular work PC last week.   I use a Windows PC for my normal desktop machine, and when I turned it on one morning is refused to boot up.   After several attempts, it became obvious that the hard drive was dying and wouldn’t last much longer.   I have most of my irreplaceable files backed up to Amazon S3 via JungleDisk, but it is still a huge pain to try and reinstall an operating system, all of my applications, and try to get back to a working system

Fortunately, at a recent CALUG meeting, we had Barry Grundy give a presentation on Data Recovery.   Barry is the author of LinuxLEO – a pretty comprehensive document about Data Forensics  using open-source tools.  In his presentation he covered a number of open-source tools that are super-useful for recovering raw data and then in making sense of it.    The main tool that I found useful was GNU ddrescue which is a variant of dd specifically created to retrieve as much data as possible from a failing drive.’

ddrescue works by reading data from the drive.  When it encounters a bad sector it skips forward a ways and tries to read again.   If it is unsuccessful, it skips forward a larger amount and continues until it is able to read something successfully.  That process repeats until it has gone through the whole drive.    This retrieves as much good data from the drive as possible.   You can then run it again with some different parameters to go back and retry those error areas to retrieve all of the questionable areas.

The drive that I had failing was a 160 GB SATA drive that was over 4 years old.   The first round with ddrescue looked pretty bad – it had around 25% of the drive was bad or questionable.  After some experimenting to figure out the ideal parameters and a few passes through the entire drive I ended up recovering my entire drive minus just 110kb of bad sectors.

At that point I had almost all of my data, but I wasn’t able to boot off of the drive.  There were some problems with the master boot record and the NTFS volume was corrupt and wouldn’t mount cleanly.  I ended up attaching the drive to a working machine so that I could run chkdsk on it which solved the NTFS corruption problems.   I had to work around quite a few problems, but eventually was able to restore it to a point where I was able to boot just fine.

Jungle Disk Error getTotalSizeByType DB error (11)

November 10, 2008 / / 0 Comments

My hard drive recently died and I was able to restore most of the files without any incident. However, it seems that some of my cached JungleDisk data became corrupted. When trying to connect to a bucket, it generated this error:

getTotalSizeByType DB error (11) database disk image is malformed

Some quick google queries identified this is an SQLite error, but didn’t provide any method for fixing it. Removing and re-adding the bucket to the JungleDisk configuration didn’t resolve it.

To finally get it fixed, I just cleaned out the JD Cache manually. The cache directory is configurable from the JungleDisk Settings menu under "Application Settings" On a Windows machine, it is likely in C:\Documents and Settings\<Your Username>\Application Data\JungleDisk\cache. That directory contains sub directories for each bucket that you connect to. I closed JungleDisk, then simply deleted the cache directory for the bucket that was having the problem. Then when restarting JungleDisk back up, I was able to re-add the bucket and it recreated the cache and went on to work fine.

ClassPerformance.com Is Now Live

November 9, 2008 / / 0 Comments

I’ve spent the last little while working on ClassPerformance.com The site is for Brad Cohen who has Tourette’s Syndrome and has gone on to become an award-winning teacher, a motivational speaker, and author.  He has appeared on shows such as Oprah.   His life story has been made into a Hallmark Hall of Fame movie.  It will air on CBS on December 7th.  The website revamp is in anticipation of a spike of traffic around the time of the premier.

The blog, and most of the content for the site is through WordPress.   I wrote some additional custom functionality for the dynamic content, and integrated the HTML template.   Overall, I’m quite pleased with the result and am looking forward to the movie.

Funniest Mac ad so Far

October 23, 2008 / / 2 Comments

I just had to laugh for a while at this commercial. It is funny how these commercials have played out. Mac obviously did a great job with them, and then Microsoft came out with their “I’m a PC” commercials. This one is in response to that.

Trac error: “<username> is not a valid value for the owner field.”

October 21, 2008 / / 1 Comment

I stumbled around with this simple problem for longer than I care to admit.   Googling for a solution only found similar questions without any answers.   The problem occurs after adding a new user to the user database for trac (in my case, a simple .htpassword file), the user is unable to accept or to be assigned any tickets.  Attempts to do so just generate the error message:

"<username> is not a valid value for the owner field."

The solution is to log in as that user, then click on ‘Settings’ in the top right, and fill out the name and email address.   Evidently after adding that, it saves the user information to the trac database, and makes it possible to use that user in other places.

Fixing a Corrupt MySQL Relay Log

September 30, 2008 / / 4 Comments

As an extension of my post yesterday about skipping corrupt queries in the relay log, I found out that my problem is due to some network problems between the servers which triggers a MySQL bug.

The connection and replication errors in my MySQL log looks like this:

080930 12:26:52 [ERROR] Error reading packet from server: Lost connection to MySQL server 
  during query ( server_errno=2013)
080930 12:26:52 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, 
  log 'mysql-bin.000249' position 747239037
080930 12:26:53 [Note] Slave: connected to master 'replicate@mysqltunnel:13306',replication 
  resumed in log 'mysql-bin.000249' at position 747239037
080930 13:18:49 [ERROR] Error reading packet from server: Lost connection to MySQL server during
   query ( server_errno=2013)
080930 13:18:49 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 
  'mysql-bin.000249' position 783782486
080930 13:18:49 [ERROR] Slave: Error 'You have an error in your SQL syntax; check the manual 
  that corresponds to your MySQL server version for the right syntax to use near '!' at line 6' 
  on query. Default database: 'database'. Query: 'INSERT INTO `sometable`
            SET   somecol         = 3,
                    comeothercol  = 8,
                    othervalue      = NULL!', Error_code: 1064
080930 13:18:49 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and 
  restart the slave SQL thread with "SLAVE START". We stopped at log 'mysql-bin.000249' 
  position 783781942
080930 13:18:50 [Note] Slave: connected to master 'replicate@mysqltunnel:13306',
  replication resumed in log 'mysql-bin.000249' at position 783782486

When there are network problems between the server, there was some issue where the master didn’t properly detect and notify the slave of the failure. This resulted in parts of queries missing, duplicated, or replaced by random bits in the relay log on the slave. When the slave tries to execute the corrupt query, it will likely generate an error that begins with:

Error You have an error in your SQL syntax; check the manual that corresponds to 
  your MySQL server version for the right syntax to use near . .

This bug has been fixed in MySQL releases since February 2008, but still hasn’t made its way into the CentOS 5 repositories. Until then, that bug report contained a work-around which forces the slave to re-request the binary log from the master. Run ‘SHOW SLAVE STATUS’ and make note of the Master_Log_File and Exec_Master_Log_Pos columns. Then run ‘STOP SLAVE’ to suspend replication, and run this SQL:

CHANGE MASTER TO master_log_file='<Value from Relay_Master_Log_File>',
  master_log_pos=<Value from Exec_master_log_pos>;

After that, simply run ‘START SLAVE’ to have replication pick up from there again. That evidently has the slave re-request the rest of the master’s binary log, which it should (hopefully) get without becoming corrupt, and replication will continue as normal.

I guess the network connection between my servers is problematic lately. I’ve had to fix this several times in the past couple days. If that keeps up, I may add this fix to my Replication checking-script until I’m able to upgrade to a version of MySQL that contains this fix.

Fix MySQL Replication by Skipping The Problematic Query

September 29, 2008 / / 1 Comment

MySQL replication can be a bit fragile.  Most of the time replication fails when the slave tries to run an SQL statement that causes an error.   I’ve most frequently seen this when the slave is missing some table that I forgot to import when setting replication up.   Usually it is a simple enough job to create the table, then start the slave SQL thread again.

However, sometimes replication breaks for no particular reason at all. This morning, I had a slave stop because an invalid query somehow got written to the binary log.  I’m not quite sure how that happened, but fortunately I found a way to just skip the bad query.

mysql> stop slave;
Query OK, 0 rows affected (0.00 sec)

mysql> SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
Query OK, 0 rows affected (0.00 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

The ‘SQL_SLAVE_SKIP_COUNTER setting tells the slave SQL thread to skip that many queries when starting up. Note that you should really know what your database is doing and why it stopped before you just go running this command. It may cause your slave to get seriously out-of-sync with the master server.

« Older posts Newer posts »

© 2025 Brandon Checketts

Theme by Anders Noren — Up ↑