LinkedIn Sourceforge Twitter

Vincent's Blog

Pleasure in the job puts perfection in the work (Aristote)

An another backup script with versioning

Posted on 2018-10-11 19:44:00 from Vincent in Open Bsd

I'm spending lot of energy to apply best practices on my servers.

But, maybe like several persons, my desktops and laptops are just a big mess of different softwares, lot of different config files, ...

And when comes the question of the backup, most of the elements must not be take into account. So, in my mind it's not necessary to take a backup. (my important documents are synchronised in an automatic way via rsync on my NAS).

Nevertheless, some files and directories requires lot of attentions and must be correctly backuped like ~/.config for example.

This blog is about an another locale backup script which allow to precisely select which files to backup.

Moreover it allow a versioning. Some OSX's users call it "time machine".


Introduction

The script I will share must be placed in his exact context.

It's sure it will not satisfy all kind of situations, so, before re-using it, please takes the following assumptions into account.

My goal with this backup script is to:

  • be able to backup few folders and few files.
  • started by root, but easily modifiable with non root users
  • versioning. I need to keep 1 week of history with a version each day. To save some disk' space I use hard links.

So, knowing this context, usual backup tools were not matching my needs. And writing my own was not a difficult task.

My simple backup script

I provide you my scrip here.

In few words this script is doing the following:

  • look for a file called .backup in each HOMEDIR, in /root and in "/"
  • each .backup file lists files and folders to backup. To avoid bad actions, the files listed in this file will always be relative path. In other words putting "../folder1" or doing "folder1/../../etc/" will not work as you would imagine. All /../ or // are in fact removed.
  • such .backup file must contain one entry per line (file or folder).
  • comments (with char #) are allowed. Every text after the "#" will be discarded

Parameters of this scripts are (located on the top of the script):

  • DEST_DIR: The path where you want to store your backups.
  • DATE_TMPL: The template for the backup's folder name. This template must be compliant with the date command. In my case a daily backup is perfect, so my template is "+%Y%m%d", if you want to do an hourly backup I suggest to use the following template: +%Y%m%d%H
  • RSYNC: the full path name to rsync. On my OpenBSD machine, it's currently rsync-3.1.3
  • EXCLUDE: This is the exclude parameter of rsync. For example, to avoid to backup you Trash folder, I propose "-- exclude '.local/share/Trash/'. Please refer to the rsync man pages if you want to adapt it.
  • MAX_HISTORY: The number of backups you want to keep. In my case, since I'm doing a backup every day, I like to keep 7 iterations, so 1 week.

How to use it

I remind that I've written this script on OpenBSD, so on other systems, it should be adapted.

First, install it:

ftp http://www.vincentdelft.be/static/post/post_20181011/laptop_backup.sh
doas cp laptop_bacckup.sh /usr/local/bin
doas chmod +x /usr/local/bin/laptop_backup.sh

Then configure it:

  • Adapt at least DEST_DIR
  • Then create this DEST_DIR

    doas mkdir -p YOUR_DEST_DIR
    doas chmod 700 YOUR_DEST_DIR

Then you can run it ;-). But it must be triggered by root.

Schedule it

I propose to schedule it via my cron called vdcron.

In such case, the config to add in /etc/vdcron.conf is the following:

Always /usr/local/bin/laptop_backup.sh &

By doing so, each time you will execute vdcron, the backup will be triggered and the last adapted files will be taken into account. In my case, since DAT_TMPL is changing every day, the backup folder will be adapted each time the backup script is executed. On the next day, a new folder will be created. And so on.

An other smart option for laptops would be trigger this script via /etc/apm/suspend. In other words, the backup will be triggered every time you close the lid of your laptop (but I've not yet tested it)

Restore a file

A backup is only useful is you can restore your files.

To restore a file, you just have to copy it from your backup to where it should be.

To know how must backup's versions you have and which one to take, I propose to use the find command:

doas find DEST_DIR -name FILE_TO_RESTORE -ls

For example, in the following example we see that the file called vdcron.conf change every day because the inode number is always different.
So depending on how far in the past I would go, I will select a different path.

For the autostart.sh file, this is always the same, the inode number is always the same.

e5450:~$ doas find /home/bkp -user vi -name vdcron.conf -ls
7275648    4 -rw-------    1 vi       wheel         518 Oct  6 15:17 /home/bkp/20181006/home/vi/.vdcron/vdcron.conf
3663767    4 -rw-------    1 vi       wheel         517 Oct  7 11:30 /home/bkp/20181007/home/vi/.vdcron/vdcron.conf
3170104    4 -rw-------    1 vi       wheel         517 Oct  8 06:55 /home/bkp/20181008/home/vi/.vdcron/vdcron.conf
3507914    4 -rw-------    1 vi       wheel         517 Oct  9 19:06 /home/bkp/20181009/home/vi/.vdcron/vdcron.conf
3741785    4 -rw-------    1 vi       wheel         517 Oct  9 20:26 /home/bkp/20181010/home/vi/.vdcron/vdcron.conf
3118163    4 -rw-------    1 vi       wheel         517 Oct 11 06:38 /home/bkp/20181011/home/vi/.vdcron/vdcron.conf
e5450:~$ doas find /home/bkp -user vi -name autostart.sh -ls         
6989996    4 -rw-r--r--    6 vi       vi            460 Oct  5 18:17 /home/bkp/20181006/home/vi/.config/openbox/autostart.sh
6989996    4 -rw-r--r--    6 vi       vi            460 Oct  5 18:17 /home/bkp/20181007/home/vi/.config/openbox/autostart.sh
6989996    4 -rw-r--r--    6 vi       vi            460 Oct  5 18:17 /home/bkp/20181008/home/vi/.config/openbox/autostart.sh
6989996    4 -rw-r--r--    6 vi       vi            460 Oct  5 18:17 /home/bkp/20181009/home/vi/.config/openbox/autostart.sh
6989996    4 -rw-r--r--    6 vi       vi            460 Oct  5 18:17 /home/bkp/20181010/home/vi/.config/openbox/autostart.sh
6989996    4 -rw-r--r--    6 vi       vi            460 Oct  5 18:17 /home/bkp/20181011/home/vi/.config/openbox/autostart.sh

Conclusions

Even if some of my laptops are flagged "trash machine", where I do not plan to keep any files, I've discovered that some config are still important to keep.

Thanks to this simple script and thanks to rsync, and his hard link feature, I'm able to have several locale copies of each of my desktops/laptops.

Here the result on my dell machine.

e5450:~$ doas du -h -d1 /home/bkp
454M    /home/bkp/20181006
25.6M   /home/bkp/20181007
24.1M   /home/bkp/20181008
104M    /home/bkp/20181009
18.8M   /home/bkp/20181010
25.3M   /home/bkp/20181011
652M    /home/bkp


2, 1
displayed: 1090



What is the first letter of the word Python?