Backup concept for system device with Linux and Windows partition

The basic backup scenario looks very simple. If we have system disk device with mix type of partitions, the best approach will be using complete disk backup method. Several software offer better and better solution. Just few of them: Clonezilla, Norton Ghost, Linux dd command, etc. We are able to back up the whole disk or just each partition individually. Ok, it is simple but make our life little bit more complicated. What can we do if we do not want to make whole disk backup regularly. What about if Linux and Windows completely request different frequency to make a copy, but for some reason we defiantly need one image copy for whole disk for restore purpose?

The solution what I am going to introduce combine different technics to deliver the solution.

To sum up the backup-restore demand to don’t lose any requirements:

  • Windows system needs to back up every week once
  • Linux system needs to back up very often, sometimes more than once a day
  • Linux systems needs to restore as fast as possible, it may necessary more than once a day
  • Need to keep the two systems always ready in one disk image copy file (fast restore when hardware failure occurs or fast clone purpose)

The concept is based on dd command Linux image copy and somehow we keep the different partitions up-to-date. In the following steps I will write the details:

1.Create base disk image copy with simple dd Linux command

dd if=/dev/sda of=/mnt/backup/SandBox.img bs=4M



The image file is sitting in NFS server where in background I use ZFS file system with snapshot and compress option. This is the key file what we all the time modifies, but not all parts just that is necessary. One’s time Windows partition, another time Linux system area. I never delete the image file and replace a new one, just over-writes the necessary parts. And keep the previous image copies for a short retention time, I create snapshot before every overwrite operations.

2.The Windows magic part looks simple and it is also using dd as a Swiss Army knife:

dd if=/dev/sda of=/mnt/backup/SandBox.img bs=512 count=61646847 conv=notrunc


I need to comment this a little bit. Dou to the system disk first part stores the Windows partitions, simple I need to rewrite the first 61646847 disk sectors. Here the critical parts are:
– we have to know how many sectors (with what block size “bs=512”) need to rewrite (“count= 61646847”). This information comes from the system disk geometries data.

[root@SandBox ~]# fdisk -lu /dev/sda

Disk /dev/sda: 80.0 GB, 80026361856 bytes, 156301488 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0xe4b38f01

Device Boot     Start         End     Blocks   Id System

/dev/sda1   *       2048     206847     102400   7 HPFS/NTFS/exFAT

/dev/sda2         206848   61646847    30720000   7 HPFS/NTFS/exFAT

/dev/sda3       61646848   103589887   20971520   83 Linux

/dev/sda4       103589888   156301487   26355800  83 Linux

 

Here the -lu options help me the determinate the disk unit size and the last sector where the Windows partition is finished.

 

– we must use “notrunc” convert symbol which is do not truncate the output file (actually the Linux part)

3. The Linux systems also could work with some dd magic command, but because it is very often necessary to make back up – and we do not change the whole system every time –, I looked for an alternative incremental based solution. However the dd is really powerful, it is not able to solve this problem. For incremental Linux backup one of the basic standard the rsync command. Rsync is an administrator’s best friend, and it can be a wonderful tool for doing all kinds of things that admins need to do. One of the things that makes rsync unique is that it has a “delta transfer” capability that reduces the amount of data actually transferred. Only question is how to add the “delta transfer” into the image copy to keep the image consistence. The answer is simpler than many people think:
– using NFS mount we see the image file, and I can mount the part of the image (I mean just one partition from the image). We have few way how to do (e.g. using losetup command) I prefer the kpartx:

[root@SandBox ~]# kpartx -av /mnt/backup/Sandbox.img

add map loop2p1 (253:1): 0 204800 linear /dev/loop2 2048

add map loop2p2 (253:2): 0 61440000 linear /dev/loop2 206848

add map loop2p3 (253:3): 0 41943040 linear /dev/loop2 61646848

add map loop2p4 (253:4): 0 52711600 linear /dev/loop2 103589888

Here:
-a means: add partition mappings (e.g /dev/mapper/loop2p3)

-v means: operate verbosely

So we have new devices, it is easy to mount it and voilà:

[root@SandBox ~]# mount /dev/mapper/loop2p3 /mnt/sda3

[root@SandBox ~]# df -h /mnt/sda3/

Filesystem           Size Used Avail Use% Mounted on

/dev/mapper/loop2p3   20G 3.0G   16G 16% /mnt/sda3

Finally I need to execute rsync command to back up the Linux part:

rsync -aAXvSH –exclude={“/dev/*”,”/proc/*”,”/sys/*”,”/tmp/*”,”/run/*”,”/mnt/*”,”/media/*”,”/lost+found”} /* /mnt/sda3 > /tmp/rsync.log


I must comment the options:
– Using the -aAX set of options, the files are transferred in archive mode, ensuring that symbolic links, devices, permissions and ownerships, modification times, ACLs and extended attributes are preserved.
– Because I have sparse files, in this case Docker images I have to add the -S option.
– And finally I have many hard links I considered adding the -H option, which is turned off by default as memory expensive, but nowadays it should be no problem on most of modern machines.

Honestly, I do not have huge Linux backup knowledge, I come from Oracle database develop and admin background. So all of what I did here, is a concept which has been created to my personal Sand Box server requirements. I know it may not a only solution and maybe not a perfect one, probably I need to tune the rsync command. For example using the “–delete-after” option. The “–delete-after” is could be useful to use because otherwise, should the backup fail half-way, files that have been renamed since the last backup (meaning, old file is deleted, new file is created), are deleted before the new one is transferred. It’s best to first transfer the new file, and then delete the old one.

So if you have any comment or recommendation to me or other readers, please do not hesitate to write it.

Advertisements

Author: gyzszabo

IT Architect ...

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s