I couldn’t help myself anymore. Newegg had 1 TB green drives on sale… again, and my old ~1 TB storage array was nearly full. Since my random computer parts are scattered about, I grabbed the only extra working machine I could find and threw the 4 drives into it. Unfortunately for me the machine was an old Athlon 64 3200+ so the RAID-5 syncing took some time (10+ hours, it was maxing out CPU at ~25MB/s).
Now came the process of migrating the data over from my file server to the new array. I didn’t want to fiddle with NFS or anything of that nature since I was booting off a Jaunty live CD. SSH was certainly possible but then there was the overhead. Rsync was another solution. But then I recalled a extremely fast way with tar & netcat. Combine that with pipeviewer and you’ll get a fast transfer complete with verbose information.
It’s very simple. With two computers, source and dest, you simply run the following:
On source (with IP 192.168.0.10):
tar -cf - /path | pv | nc -l -p 8888 -q 5
This will tar up /path and pipe it to netcat, which is listening on port 8888 for an incoming connection.
On dest:
nc 192.168.0.10 | pv | tar -xf -
Connect to source and pipe the output to tar for extraction.
Credit to Peteris’ wonderful blog entry on pv.