bak-git: tracking remote files in central git repository

If you are system administrator this will sound familiar: you have to quickly fix something, and you know that you should document it somewhere (or keep backup) but it's so much work. You could install one of existing source control management tools on each box, but they usually come with huge dependencies, and having all files in central location would be so useful to co-relate configuration changes. To add insult to injury, existing SCMs don't do good job in tracking just few files spread across file-system.

So, what would be perfect tool for keeping remote files in central git repository look like?

  • no dependency on non-standard tools on clients allowing easy deployment
  • track individual files and ignore rest
  • central repository, one directory per hostname

I tried to solve this problem several times, writing wrappers around subversion to handle sparse checkouts and installing subversion and ssh authentication all over the place. But, all this should be simpler... Like this:

  1. add new client to track:
    dpavlin@klin:~/klin/bak-git$ ./bak-git-server.pl ~/backup/ 10.60.0.92 --install brr
    install on brr
    # lot of output stripped
    
    This will do several steps:
    • create git repository in ~/backup/ if it doesn't exist already
    • install root ssh authentication to brr using ssh-copy-id
    • install bak shell helper which uses netcat to connect back to 10.60.0.92
    • install rsync on client and use it as root over ssh to sync files
  2. Now we can login into brr and start tracking our files:
    dpavlin@brr:~$ bak add /etc/cron.d/tun0 
    dpavlin@brr:~$ bak add /etc/network/interfaces
    dpavlin@brr:~$ bak commit
    dpavlin@brr:~$ bak log
    commit df09dc5e19ef1d47311d701b4c63f0859b0b81c1
    Author: Dobrica Pavlinusic 
    Date:   Thu Feb 18 19:04:21 2010 +0100
    
        brr [commit] /home/dpavlin/
    
     create mode 100644 brr/etc/cron.d/tun0
     create mode 100644 brr/etc/network/interfaces
    
  3. change some configuration and review changes
    dpavlin@brr:~$ bak diff
    diff --git a/brr/etc/network/interfaces b/brr/etc/network/interfaces
    index 806c08e..c52c646 100644
    --- a/brr/etc/network/interfaces
    +++ b/brr/etc/network/interfaces
    @@ -2,8 +2,6 @@
     # and how to activate them. For more information, see interfaces(5).
     
     # The loopback network interface
    -auto lo
    -iface lo inet loopback
     
     # The primary network interface
     #allow-hotplug eth0
    
  4. Uups!! Where did loopback disappeared?
    dpavlin@brr:~$ bak revert /etc/network/interfaces 
    dpavlin@brr:~$ bak diff
    
  5. If we are content with changes, we can also commit them:
    dpavlin@brr:~$ bak commit /etc/network/interfaces optional note
    
As you guessed by now, it's very similar to git usage (expect revert which is from subversion) but with easy deployment on clients. It implements reduced subset of git commands:
  • bak add /path
  • bak commit [/path [message]]
  • bak diff
  • bak status
  • bak log
  • bak - push all local changes to server (without commit!)
If you need anything more complex, you can use git directly on ~/backup repository (even to commit changes from multiple hosts in one go).

Whole solution seems like ftp protocol, with data channel using ssh and rsync. File transfer should be encrypted (since we are trying to manage configuration files with sensitive information) and if you want to be really secure, just run server on 127.0.0.1 and tunnel port using RemoteForward 9001 localhost:9001 in .ssh/config.