Resizing LVM disks in KVM from host

Introduction

I have been doing research and trying to implement a home virtualisation solution based on kvm, and it is going nicely. Some concepts and ideas will be discussed in a different post. One question I was facing during setup and design was the idea to resize the disks according to needs. There were posts saying that rsync and dd works for them but I found these  just unsuitable and unacceptable. With doing rsync or dd I would need to duplicate the storage allocation, since to copy a 1Tb guest to a 1.3Tb guest would require a minimum of 2.3Tb of space and also a lot of IO bandwith. Since all machines are implemented from one kickstart or preseed installation, and all are according to my standards, a lot of information can be taken for sure and assumed.

Setup

There are two types of distros in use at moment. Debian and Centos. Both use two kinds of disc layout, either with no swap and everything on one partition, or one partition and a swap next to it. Since nagios monitors my disk usage I will always know and have scripts ready for cleanups. Swap is not required on smaller task based machines, as well as memory is controlled through ballooning, so it will expand. If a program goes rogue, then the estimated memory should be enough to spot it and stop it.The setup is easy. The guest device is on a LVM partition, but the inner guest is one partition (no LVM).

The host is a Debian machine with kernel: 2.6.32-5-amd64

Script

root@localhost # cat resize.sh
#!/bin/bash

### Disk resize on host for guest

usage()
{
cat << EOF
usage: $0 [-h] [-v] -s=VAR -n=VAR
This script is useable only with lvm devices and needs to be given path to /dev/mapper device

OPTIONS:
   -h           Show this message
   -s           Size of new disk (Gigs)
   -n           Lvm path
   -v           Verbose
EOF
}
SIZE=
NAME=
DISK=
VERBOSE=0
BLOCK=
stdout=/dev/null

while getopts "hvs:d:" opt; do
  case $opt in
    s)
          SIZE=$OPTARG
          BLOCK=$(($SIZE*1024*1024))
          SIZE=$SIZE"G"
      ;;
    d)
          kpartx -dv `readlink -f $OPTARG`
          NAME=$OPTARG
          DISK=`host_info.pl -n $NAME -disk`
      ;;
    h)
      usage
          exit 1
      ;;
        v)
          VERBOSE=1
          stdout=/dev/stdout
          echo "Let's be verbose" >$stdout 2>&1
          set -x
          ;;
    \?)
      echo "Invalid option: -$OPTARG" >&2
          exit 1
      ;;
  esac
done

if [[ -z $SIZE  ]] || [[ -z $DISK ]]
        then
        usage
        exit 1
fi

if [ ! -b $DISK ]
        then
        echo "It is not a disk."
        exit 1
fi
CBLOCK=`fdisk -s $DISK`
VDISK=(`kpartx -av $DISK | awk -v dir=\`dirname $DISK\` '/add/{print dir"/"$3}'`)
SWAP=`fdisk -l $DISK | awk '(/\//) && (/swap/)'`

if [[ ! -z $SWAP ]]; then
        echo "Swap detected" >$stdout 2>&1
        SWAPSIZE=`echo $SWAP | awk '{print $3-$2}'`
        fdisk $DISK >$stdout 2>&1  <<EOF
    d
        2
    w
EOF
fi

if [ $BLOCK -lt $CBLOCK ]; then
        echo "We need to shrink filesystem." >$stdout 2>&1
        fsck -n ${VDISK[0]} >$stdout 2>&1
        e2fsck -f ${VDISK[0]} >$stdout 2>&1
        tune2fs -O ^has_journal ${VDISK[0]} >$stdout 2>&1
        e2fsck -f ${VDISK[0]} >$stdout 2>&1
        CSIZE=`resize2fs -P ${VDISK[0]} 2>/dev/null | awk -v block=\`dumpe2fs ${VDISK[0]} 2>$stdout | awk '/Block size/ {print $3}'\` -v constant=\`fdisk -l $DISK | awk '/^Units =(.*[^=])=(\d*)/ {print $9}'\` '{print "+"($7*4)+int(constant/block)+1"K"}'`
        resize2fs -M ${VDISK[0]} >$stdout 2>&1
        kpartx -dv $DISK >$stdout 2>&1
        fdisk $DISK >$stdout 2>&1  <<EOF
        d
        n
        p
        1
        1
        $CSIZE
        a
        1
        w
EOF
        partprobe $DISK >$stdout 2>&1
        lvresize -f -L $SIZE $DISK >$stdout 2>&1
        ## Divert for swap
        if [[ ! -z $SWAP ]]; then
                MAX=`fdisk -l $DISK 2>/dev/null | awk /cylinders$/ | awk -v swap=$SWAPSIZE '{print $5-swap-1}'`
                SWAPMIN=`echo | awk -v max=$MAX '{print max+1}'`
                SWAPMAX=`echo | awk -v min=$SWAPMIN -v size=$SWAPSIZE '{print min+size}'`
                fdisk $DISK >$stdout 2>&1 <<EOF
                d
                n
                p
                1
                1
                $MAX
                a
                1
                n
                p
                2
                $SWAPMIN
                $SWAPMAX
                t
                2
                82
                w
EOF
        partprobe $DISK >$stdout 2>&1
        VDISK=(`kpartx -av $DISK | awk -v dir=\`dirname $DISK\` '/add/{print dir"/"$3}'`)
        mkswap -f ${VDISK[1]} >$stdout 2>&1
        else
                MAX=`fdisk -l $DISK | awk /cylinders$/ | awk '{print $5}'`
                fdisk $DISK >$stdout 2>&1 <<EOF
                d
                n
                p
                1
                1
                $MAX
                a
                1
                w
EOF
        partprobe $DISK >$stdout 2>&1
        VDISK=(`kpartx -av $DISK | awk -v dir=\`dirname $DISK\` '/add/{print dir"/"$3}'`)
        fi
        e2fsck -f ${VDISK[0]} >$stdout 2>&1
        resize2fs ${VDISK[0]} >$stdout 2>&1
        e2fsck -f ${VDISK[0]} >$stdout 2>&1
        fsck -n ${VDISK[0]} >$stdout 2>&1
        tune2fs -j ${VDISK[0]} >$stdout 2>&1
        kpartx -dv $DISK >$stdout 2>&1
        kpartx -dv `readlink -f $DISK` >$stdout 2>&1
elif [ $BLOCK -gt $CBLOCK ]; then
        echo "We need to expand the filesystem." >$stdout 2>&1
        lvresize -f -L $SIZE $DISK >$stdout 2>&1
        if [[ ! -z $SWAP ]]; then
                MAX=`fdisk -l $DISK 2>/dev/null | awk /cylinders$/ | awk -v swap=$SWAPSIZE '{print $5-swap-1}'`
        SWAPMIN=`echo | awk -v max=$MAX '{print max+1}'`
        SWAPMAX=`echo | awk -v min=$SWAPMIN -v size=$SWAPSIZE '{print min+size}'`
        fdisk $DISK >$stdout 2>&1 <<EOF
        d
        n
        p
        1
        1
        $MAX
        a
        1
        n
        p
        2
        $SWAPMIN
        $SWAPMAX
        t
        2
        82
        w
EOF
        partprobe $DISK >$stdout 2>&1
        VDISK=(`kpartx -av $DISK | awk -v dir=\`dirname $DISK\` '/add/{print dir"/"$3}'`)
        mkswap -f ${VDISK[1]} >$stdout 2>&1
        else
                MAX=`fdisk -l $DISK | awk /cylinders$/ | awk '{print $5}'`
                fdisk $DISK >$stdout 2>&1 <<EOF
                d
                n
                p
                1
                1
                $MAX
                a
                1
                w
EOF
                partprobe $DISK >$stdout 2>&1
        kpartx -av $DISK >$stdout 2>&1
        fi
    e2fsck -f ${VDISK[0]} >$stdout 2>&1
    resize2fs ${VDISK[0]} 2>&1 >$stdout
    e2fsck -f ${VDISK[0]} >$stdout 2>&1
    fsck -n ${VDISK[0]} >$stdout 2>&1
        tune2fs -j ${VDISK[0]} >$stdout 2>&1
    kpartx -dv $DISK >$stdout 2>&1
    kpartx -dv `readlink -f $DISK` >$stdout 2>&1
else
        echo "Size is same as present size."; >$stdout 2>&1
fi
#### END
if [ $VERBOSE ]
then
        set +x
fi

Explantion

So let’s go into some detail regarding the strategy and the usage of the script.

First disclaimer:Only use this script at your own risk.  I can not be held responsible for any data loss or problems in your enviroment!

It is strongly recommended that you understand the code and what it does.

You are advised to test it in a given enviroment and only if it works according to your expectation use it in any other enviroment.

Also please don’t use this while the virtual machine is running. The partition is shrunk to a minimal size, if there are any IOwrites performed they will get messed up and inconsistent. But the shutdown of the machine is not part of this script.

Explaining the code

The useage sub is only for a quick information on the script and how to use it. Getopts iterates through the arguments and sets the needed variables. Mandatory arguments are size and disk, since without them the script wouldn’t have any point. Also a check is done to see if these have been specified and if not print a help about the usage. For safety reasons it is checked whether the disk specified is a block device.

One might ask why wouldn’t the /dev/vgname/lvname combo work , basically because it’s just a symlink to /dev/dm*. Also I have experienced problems with kpartx in this directory.I haven’t yet investigated the root cause, but for some reasons sometimes it creates the device maps under /dev/dm* instead of folder of the lvm /dev/vgname which can cause problems that the script can’t handle.

One might also also ask /dev/mapper is also a symlink. Why not use the /dev/dm* straight away. Some initial tests where used resolved the symlink to /dev/dm* and to pass arguments to kpartx this way, but kpartx didn’t create the device maps under /dev/dm* but rather /dev/mapper/namep*.

If device maps are given under /dev/mapper then the further parts are created under /dev/mapper/name*, where * is a number. I found this easiest to query and to use, that’s why I kept with this setup.

Since swap doesn’t hold any information my approach was, that I can tear down the partition and at the end add a swap to it. The inner machine will feel no difference after it was started.

So the swap partition is detected, which -according to standards- is always the second partition and there can be no partition after it ( detection is done according to the partition table), and the commands to fdisk are piped with EOF. Tests were done with gparted and cfdisk but they werent as easy to script.

Then comes an if at line 81, with elsif at 153 and else at 205. This is the main part of the script I could say. The if at 81 is run if the requested size is less then the current size. Elsif is run if the size is bigger then the current requested size. Finnaly, it is also possible that the two sizes are the same, in this case we just exit because we don’t need to do anything.

Shrink

Starting off with the shrink since it needs more care and operation.

The theory of the whole process is that at first the filesystem needs to be shrunk to the minimum size then the partition is also shrunk to the minimum size , afterwards the logical volume shrunk to the required size to. Then the partition is expanded to the logical volume size (subtracting swap if there was one) and finally  the filesystem is expanded.The swap is a recomended size, that I usually keep for the machines needed, so there is no shrinking or expanding of them. Easy in theory and straight forward.

I use ext3 partitions on the devices and just convert them to ext2, and at the end add the journal back. There is multiple e2fsck going on, these are to make sure the filesystem stays consistent and if any errors arise they will be threated early. Also to remove journal first we need to mark filesystem clean for example and also to use resize2fs.

Also line 87 is a little tricky. It took me some time to figure it out and do it in oneliner but I managed and learned alot from awk during it. It could have been done in multiple lines to make the code cleaner and for maintanability, but as the saying goes “Challange accepted.” First the minimum size is obtained, then from dumpfs the current block size is returned, afterwards the part of the partition table unused or in other easier said the initial unused space is obtained. Math is done with them to get the minimum current size + 1block for safety. Then it is  resized to the smallest possible size and with fdisk the partition is shrunk to this size. Then comes the turning point of the lvm resize. From this point it gets simple.

If needed swap partition is added or simply the partition is expanded to the full lvm size. The filesystem is expanded and at the end the journal is added to make it an ext3 filesystem. Some final cleaning is done to have a nice and tidy computer.

Expand

Expanding is alot easier. We can start from the turning point of the shrinking concept, simply by expanding the lv, if needed adding the swap, expanding the partition and growing the filesystem.

Scipt in Action

Now that we know the concept behind the script let’s have a look.

root@muramasa:/script# ./resize.sh ?
usage: ./resize.sh [-h] [-v] -s=VAR -n=VAR
This script is useable only with lvm devices and needs to be given path to /dev/mapper device

OPTIONS:
   -h           Show this message
   -s           Size of new disk (Gigs)
   -n           Lvm path
   -v           Verbose

If we give the verbose argument we can track the program with every step it does, case we need to troubleshoot any problems.

Thoughts

In later milestones of the home virtualisation server I have further plans to implment lvm on lvm. I have done some smaller test to resize them, and as soon as I have working scripts I will post them with explanation. Also kvm parser and some useful scripts will be posted when finished, I am still doing tests with kickstart and integration into the createguest.pl, but that is going to be a different story.

Author: S4mur4i

Happy in the unhappy world.

4 thoughts on “Resizing LVM disks in KVM from host”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s