Create local Centos and Debian mirrors for KVM guest install

Introduction

One of the bigger problems kvm solutions require is the install process. I prefer fully automated installations, since all machines follow a set of standards, and are alike to all previous machines. Since starting a cdrom based installation doesnt give the possibility to use a console in kvm (all machine provisions from my point of view should be automatic, and if needed I should be able to jump in and check status), I have the choice to go with PXE boot or network based installation.  I have done some initial research with PXE, but didnt find it fit for my needs. So I decided to go with network based installations.

Some prefer virt-manager as provisioning tool, but X sessions are a little overkill in my opinion, and if not mandatory should be avoided.

Specification

Two distributions are used at moment: Debian and Centos. A local mirror is required to have a quick install, reduce WAN useage (which is costly), and also guarantee all installed systems are same. As much as I love linux, there are times when broken packages or packages with unneeded dependencys are released, which can break running systems. A local mirror, which will only be updated when needed for bug fixes is a solution for such problems.

Solution

root@localhost:# cat update_mirrors.sh
#!/bin/bash

#####################
#       Update mirrors

usage()
{
cat << EOF
usage: $0 [-h] [-v]
Update local mirrors for debian and centos.

OPTIONS:
   -h                   Show this message
   -v                   Verbose
   -d or --debian       debian mirror
   -c or --centos       centos mirror
EOF
}
VERBOSE=0
stdout=/dev/null
debianlink="ftp.hu.debian.org::debian/dists/squeeze/main/"
debian=0
centoslink="ftp.freepark.org::linux/centos/6.2/os/x86_64/"
centos=0

TEMP=`getopt -o hvd::c:: --long --debian::,--centos:: -n 'update_mirror.sh' -- "$@"`

if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi

eval set -- "$TEMP"
while true; do
    case "$1" in
    -d|--debian)
        case "$2" in
            "")
              shift 2
              ;;
            *)
              debianlink=$2
              shift 2
            ;;
        esac
        debian=1
    ;;
    -c|--centos)
        case "$2" in
            "")
              shift 2
              ;;
            *)
              centoslink=$2
              shift 2
            ;;
        esac
        centos=1
    ;;
    -h)
      usage
      exit 1
          shift
      ;;
    -v)
       VERBOSE=1
       stdout=/dev/stdout
       echo "Lets be verbose" >$stdout 2>&1
       set -x
       shift
       ;;
    --)
       shift
       break
       ;;
    *)
      echo "Invalid option: -$1" >&2
      exit 1
  esac
done

if [ $debian -eq 1 ]
then
        echo "We need to do debian install" >$stdout
        rsync -v -r -a --include="**binary-amd64/" --include="**installer-amd64/" --include="debian-installer/" --exclude="/*" --delete $debianlink /var/www/debian/mirror/ >$stdout 2>&1

fi
if [ $centos -eq 1 ]
then
        echo "We need to do centos install" >$stdout
        rsync -v -r -a --delete $centoslink /var/www/centos/mirror/ >$stdout 2>&1
fi

The script is simple and straight forward. If h argument is given then the help message is displayed. With v it is verbose, printing output of rsync. C or centos argument is used for centos mirror, similiar d or debian is used for debian mirrors. One possibility is that after the option for example -clink or -dlink, the specified link will be used instead of the default mirror. This is usefully, if you have a mirror closer to you (you can always change the default one aswell) or the current mirror is down. In line 27 that is the reason why we have :: after some arguments.

Since only specific archs are needed, there is no point on mirroring the whole mirror. By debian they are listed under one directory, so include exludes were required to skip sparc and other platforms.

Centos has seperate folders for both platforms (x86_64 and i686), so the whole folder can be synced.

Thaughts

Disk useage is around 7 gigabytes, which is not much. If a minimal install is done usually that is around 500 megabytes, after about 14 installs the “investment” pays off.

Author: S4mur4i

Happy in the unhappy world.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s