Backup, Recovery & Optimization Scripts

Published by marco on

Updated by marco on

This article was originally published on the Encodo Blogs. Browse on over to see more!

Perforce servers have a checkpointing mechanism in order to back up the server metadata. That checkpoint, along with the depot data and a journal file (for changes made since the checkpoint was taken) are enough to restore a Perforce database.

Here at Encodo, we take a checkpoint every evening right before the depot files are backed up. We retain a few days worth of these checkpoints, just in case. Since our server runs on Debian Etch (Linux), we have a Bash script to do all of this (included below). The comments in the script should be enough to get a gist of what’s going on; adjust the values of the variables at the top of the script to correspond to your installation.

#!/bin/bash

BACKUP_DIR=/home/p4/backup
DEPOT_DIR=/home/p4/srvroot
P4=/opt/p4/p4
HOST=p4:1666
USER=root
PASSWORD=password
DAYS_TO_KEEP=3

# Verify the depot
$P4 -p $HOST -u $USER -P $PASSWORD verify -q //… > /dev/null
$P4 -p $HOST -u $USER -P $PASSWORD verify -u -q //… > /dev/null 2> /dev/null

# Take a checkpoint
$P4 -p $HOST -u $USER -P $PASSWORD admin checkpoint > /dev/null

# Move all journal and checkpoint files to the backup folder
mv $DEPOT_DIR/journal.* $BACKUP_DIR
mv $DEPOT_DIR/checkpoint.* $BACKUP_DIR

# Remove old checkpoint and journal files
find $BACKUP_DIR/journal.* -mtime +$DAYS_TO_KEEP -exec rm -f {} \; > /dev/null 2> /dev/null
find $BACKUP_DIR/checkpoint.* -mtime +$DAYS_TO_KEEP -exec rm -f {} \; > /dev/null 2> /dev/null

Once you have regular checkpoints for backup, your data is safe. However, after a while, a Perforce server accumulates no small amount of cruft in its metadata, slowing down some common operations. A Perforce server can be optimized by replacing the database metadata with the contents of a checkpoint. When metadata is restored from a checkpoint, that metadata is imported in a pruned, balanced and optimized way, ensuring optimal performance.

In order to do this, you basically take a checkpoint, stop the server, restore the checkpoint and restart the server. Luckily for you, Encodo has a Bash script for that as well. The basic process is described in more detail in Perforce database maintenance by Rusty Jackson (p4 blog). The comments in the script should be enough to get a gist of what’s going on; adjust the values of the variables at the top of the script to correspond to your installation.

# This script describes a way of optimizing and cleaning the metadata for a Perforce server

# The following steps are copied from the following URL:
# − <http://blog.perforce.com/blog/?p=187>

# − Stop your server
# − Take a checkpoint
# − Move your existing db files to a save directory
# − Recover from the checkpoint
# − Check for errors during restore
# − Restart your server
# − Delete the files from the save directory

# Instead of deleting the old database files immediately after the process, we
# delete them during the next run instead.

BACKUP_DIR=/home/p4/backup
DEPOT_DIR=/home/p4/srvroot
P4D=/opt/p4/p4d
LOG_FILE=/var/log/p4d-error

# − Take a checkpoint
echo "Taking a checkpoint…"
/opt/scripts/p4d/makecheckpoint.sh

# − Stop your server
echo "Stopping the Perforce server…"
/etc/init.d/p4d stop

# − Delete the files from the save directory
echo "Backing up current database…"
rm -R $BACKUP_DIR/db

# − Move your existing db files to a save directory
mkdir $BACKUP_DIR/db
mv $DEPOT_DIR/db.* $BACKUP_DIR/db/

# − Recover from the checkpoint
CHECKPOINT_FILE=`ls -h –sort=time $BACKUP_DIR/checkpoint* | head -1`

echo "Do you wish to restore from checkpoint '$CHECKPOINT_FILE'?"
select yn in "Yes" "No"; do
  case $yn in
    Yes ) echo "Restoring checkpoint…"; su p4 -c "$P4D -r $DEPOT_DIR -L $LOG_FILE -jr $CHECKPOINT_FILE"; break;;
    No ) echo "Optimization aborted; restoring previous database…"; mv $BACKUP_DIR/db/db.* $DEPOT_DIR/; break;;
  esac
done

# − Restart your server
/etc/init.d/p4d start