I'm working on Linux version of Sun storage machines, using commodity hardware, OpenVZ and Fuse-ZFS. I'm do have working system in my Sysadmin Cookbook so I might as well write a little bit of documentation about it.
My basic requirements are:
- Daily snapshots created with rsync on LVM snapshot from remote machines
- Snapshots are writable (using zfs clone) and can be converted back into running OpenVZ container (backuped machines are OpenVZ containers) for quick recovery or inspection of previous versions
- since disk space is limited (to 200Gb zfs mirror in my case) so I implemented snapshot expiatory
This makes it self-running system which won't fall over itself, so let's see how does it look:
root@opl:~# zpool status
pool: opl
state: ONLINE
scrub: resilver completed after 1h59m with 0 errors on Wed Jun 3 15:29:50 2009
config:
NAME STATE READ WRITE CKSUM
opl ONLINE 0 0 0
mirror ONLINE 0 0 0
sda2 ONLINE 0 0 0
sdb2 ONLINE 0 0 0
errors: No known data errors
root@opl:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
opl 183G 35.7G 21K /opl
opl/backup 180G 35.7G 22K /opl/backup
opl/backup/212052 76.1G 35.7G 8.12G /opl/backup/212052
opl/backup/212052@2009-05-01 5.69G - 7.50G -
opl/backup/212052@2009-05-10 5.69G - 7.67G -
opl/backup/212052@2009-05-15 5.57G - 7.49G -
opl/backup/212052@2009-05-22 3.54G - 7.74G -
opl/backup/212052@2009-05-25 3.99G - 8.38G -
opl/backup/212052@2009-05-26 3.99G - 8.38G -
...
opl/backup/212052@2009-06-05 3.72G - 8.09G -
opl/backup/212052@2009-06-06 0 - 8.12G -
opl/backup/212056 1.42G 35.7G 674M /opl/backup/212056
opl/backup/212056@2009-05-30 37.1M - 688M -
opl/backup/212056@2009-05-31 47.3M - 747M -
opl/backup/212056@2009-06-01 40.9M - 762M -
opl/backup/212056@2009-06-02 62.4M - 787M -
...
opl/backup/212056@2009-06-05 12.1M - 1.02G -
opl/backup/212056@2009-06-06 0 - 674M -
opl/backup/212226 103G 35.7G 26.8G /opl/backup/212226
opl/backup/212226@2009-05-05 4.29G - 26.7G -
opl/backup/212226@2009-05-10 4.04G - 26.6G -
opl/backup/212226@2009-05-15 4.19G - 26.6G -
opl/backup/212226@2009-05-22 4.12G - 26.7G -
opl/backup/212226@2009-05-23 4.12G - 26.7G -
opl/backup/212226@2009-05-24 4.09G - 26.6G -
opl/backup/212226@2009-05-25 4.14G - 26.7G -
opl/backup/212226@2009-05-26 4.13G - 26.7G -
...
opl/backup/212226@2009-06-05 4.20G - 26.8G -
opl/backup/212226@2009-06-06 0 - 26.8G -
opl/clone 719M 35.7G 25K /opl/clone
opl/clone/212056-60018 666M 35.7G 1.39G /opl/clone/212056-60018
opl/clone/212226-60017 53.0M 35.7G 26.7G /opl/clone/212226-60017
opl/vz 1.59G 35.7G 43.5K /opl/vz
opl/vz/private 1.59G 35.7G 22K /opl/vz/private
opl/vz/private/60014 869M 35.7G 869M /opl/vz/private/60014
opl/vz/private/60015 488M 35.7G 488M /opl/vz/private/60015
opl/vz/private/60016 275M 35.7G 275M /opl/vz/private/60016
There are several conventions here which are useful:
- pool is named same as machine (borrowing from Debian way of naming LVM volume groups) which makes it easy to export/import pools on different machines (I did run it with mirror over nbd for a while)
- snapshots names are dates of snapshot for easy overview
- clones (writable snapshots) are named using combination of backup and new container ID
There are several things which I wouldn't be able to get without zfs:
- clones can grows as much as they need
- data is compressed, which increase disk IO as result
- zfs and zpool commands are really nice and intuitive way to issue commands to filesystem
- zpool history is great idea of writing all filesystem operations to internal log
- ability to re-sliver (read/write all data on platters) together with checksums make it robust to disk errors