I have written about data migration from disk to disk before, but moving data off the laptop is really painful (at least for me). This time, I didn't have enough time to move files with filesystem copy since it was painfully slow and transferred only 20Gb from 80Gb of data in two hours. I seen 3M/s transfers because of seeking in filesystem, so something had to be done!
So, let's move filesystem images. Limiting factor should be speed of disk read. Have in mind that 100Mb/s full-duplex network (over direct cable) will bring you just 12Mb/s which is much less than normal laptop 5400 rpm disk can deliver (around 30Mb/s of sequential read -- you can check that with hdparm -t).
Since I'm coping data to RAID5 array, I can assume that any operation on images will be much quicker than seek times on 5400 rps disk which is in laptop.
Let's see which partition we have on laptop:
llin:~# mount -t ext3,reiserfs /dev/mapper/vg-root on / type ext3 (rw,noatime,errors=remount-ro) /dev/sda1 on /boot type ext3 (rw) /dev/mapper/vg-rest on /rest type reiserfs (rw,noatime,user_xattr)
First start netcat (on machine with RAID array) which will receive image. I'm using dd_rescue to display useful info while transfering data... If you don't have it, just remove it from pipe.
root@brr:/backup/llin-2008-01-12# nc -l -p 8888 | dd_rescue - - -y 0 > boot.img
Then start transfer on laptop:
llin:~# nc 192.168.2.20 8888 < /dev/sda1
Repeat that for all filesystems like this:
root@brr:/backup/llin-2008-01-12# nc -l -p 8888 | dd_rescue - - -y 0 > root.img llin:~# nc -w 1 192.168.2.20 8888 < /dev/vg/rootroot@brr:/backup/llin-2008-01-12# nc -l -p 8888 | dd_rescue - - -y 0 > rest.img
llin:~# nc -w 1 192.168.2.20 8888 < /dev/vg/rest
While you are waiting for copy to finish, you can use dstat, atop, vmstat or any similar command to monitor progress, but I came with this snippet:
root@brr:~# watch "df | egrep ' (/backup|/mnt/llin)' ; echo ; ls -al /backup/llin*/*.img"
which produce something like this:
Every 2.0s: df | egrep ' (/backup|/mnt/llin)' ; echo ; ls -al /bac... Mon Jan 12 23:30:21 2009125825276 112508688 13316588 90% /backup
82569904 22477768 60092136 28% /mnt/llin
25803068 24800808 740116 98% /mnt/llin.img/root-rw-r--r-- 1 root root 45843283968 2009-01-12 23:30 /backup/llin-2008-01-12/rest.img
-rw-r--r-- 1 root root 26843545600 2009-01-12 23:07 /backup/llin-2008-01-12/root.img
I decided to rsync files, since I already had 20Gb of them in file system. First you have to recover journal (since we where coping live devices, journal isn't finished and loopback device won't allow us to mount filesystem):
root@brr:/backup# e2fsck llin-2008-01-12/root.img e2fsck 1.41.3 (12-Oct-2008) llin-2008-01-12/root.img: recovering journal llin-2008-01-12/root.img: clean, 689967/3276800 files, 6303035/6553600 blocks (check in 4 mounts)root@brr:~# e2fsck /backup/llin-2008-01-12/rest.img
root@brr:~# reiserfsck /backup/llin-2008-01-12/rest.img
If you don't want to wait for reiserfs to finish check, you can abort it when it starts checking of filesystem because we only care about first step which will recover journal.
Next, we will mount images (read-only) using loopback:
root@brr:~# mkdir /mnt/llin.img root@brr:~# mount /backup/llin-2008-01-12/root.img /mnt/llin.img/ -o ro,loop root@brr:~# mount /backup/llin-2008-01-12/boot.img /mnt/llin.img/boot/ -o ro,loop root@brr:~# mount /backup/llin-2008-01-12/rest.img /mnt/llin.img/rest/ -o ro,loop
And finally rsync rest of changes to directory
root@brr:~# rsync -ravHS /mnt/llin.img/ /mnt/llin
After that, I used files to start OpenVZ virtual machine which will replace my laptop for some time (if you know good second-hand laptop for sale, drop me an e-mail).