I’ve ran into some weird instance of this old RedHat Linux machine that is starting to die.. The old 30GB disk is clicking rather loudly…!
To make matters worse, the computer is not only in another country but on another continent. Â And no the users won’t let me take it down, ship it or let local techs work on it, they want it virtualized, and they want it to be done without taking it down…
All I have to work with is an old Windows 2003 server with a big external USB disk, and I don’t even have administrator privileges on the 2003 server so I can’t install anything ‘fun’ like VMWare, VirtualPC, VirtualBOX or anything else.
So I thought I’d go old school with this and use netcat.
Thankfully there is a win32 version of netcat, and the Linux PC already had it installed. Â So it really was trivial I must say knowing what to do once all the bits are in place. Â From the Windows machine I just ran:
nc -p 2222 -l |gzip -dc > disk0.raw
And on the Linux PC I ran:
cat /dev/hda | gzip -c | netcat 10.0.1.10 2222
So on the Windows side, netcat will listen on TCP port 2222 and then decompress the stream with gzip and save that into the file disk0.raw. Â On the Linux side, it’ll read the harddisk, compress the data with gzip and then using netcat redirect the stream to the Windows host.
It took about 30 minutes, I suppose if the disk wasn’t in a questionable state I’d have cleared out the empty space to speed it along, however I felt that doing a large amount of writing to a questionable disk is just asking for trouble.
Once the transfer was complete I was able to then fire up Qemu 0.15.0, and have it mount the raw disk image, and it worked perfectly! Â I added the modules for the AMDPCNet card (pcnet32) and I was able to use the user mode NAT via Qemu, and redirect a local TCP port into the VM’s sshd …. Something like this:
qemu -L pc-bios -m 256 -net nic,model=pcnet -net user -hda disk0.raw -no-reboot -redir tcp:3333::22
And that was that. Â Now we have a perfect copy of the machine!
what about remounting readonly some partitions (or parsing an output of the “lsof -b”), then cpio-ing && tar && gnufind (-mtime)? ( mount -o remount,ro )
It is extremely dangerous on a running system dd-ing in and out a raw fs because this process may produce inconsistent fs image ! remounting decreases the chance of this.
(under windows: afaik qemu doesn’t require admin privileges, and an average linux rescue live system taht contains nc, ftpd|sftpd|rsync should run in a vm, and a qcow hdd image could be created easily with this idea)
Well they didn’t want to take the box down, and at the same time it crashes.. a lot. Potential data loss is always a thing for this machine. Fsck cleared the disk in no time so it worked out fine. Naturally most people would reboot the machine into some kind of state that would allow you to capture the disk in a known good state, but sometimes you don’t get that luxury…..
They insist that while the machine does serve data, the data doesn’t change.. I know it doesn’t make sense, but sometimes users drive you to insanity, and sometimes it is easier to just give them what they asked for to get them off your back… 🙂
I agree 100%.
During my oddities with DG/UX machines (old-ass machines with ~15 years uptime [minus some electrical spikes] and no chance in hell of a reboot/downtime), I’ve also had to dd physical disks (local and on even older CLARiiON arrays), and those came out fine. I did try to keep it to off-peak hours, and fsck did help later on.
Bottom line is, yeah, sometimes you do not have much of a choice.
Great post.
Speaking of netcat, I’m providing my own compiled Windows version here, since I always had trouble downloading any other Win32 version thanks to either dead links, or braindead firewall/antivirus programs.
About cloning a live machine, I’ve done it several times with rsync – boot the target (virtual or physical) machine with something like SystemRescueCD, recreate the partition layout, rsync -aH (or -aAH if the source machine uses ACLs) twice (first time for initial sync, which can take a while, then to move over the changes that happened since), set up the bootloader on the target machine, then stop all services on the source machine and possibly remount read-only, do a final rsync, shutdown the source, and reboot the target.
that is a very good idea….! Esp if you have enough disk space you could keep a continuous mirror running of the source disk…!
I use rdiff-backup on a few machines – works similar to rsync, except it also keeps history of changes, so you can recover past state. Unfortunately, it’s been unmaintained for a while now, but at least for my purposes, it works well enough as-is (note that while it does support using a Windows machine as target, there’s a few known problems with this, so you should always use a *nix machine instead).
If the old system is starting to indicate disk problems it’s advisable to copy the whole image with ‘dd’ or ‘cat’ (with ‘dd’ you can specify a block size of course, which in some cases can be useful), instead of using ‘tar’ or ‘rsync’. I’ve had several cases where a disk is in too bad shape to even boot the system, but reading the disk sequentially worked OK and I could salvage the full content. Disks these days may do strange things behind the curtain so there’s no guarantee that a sequential ‘dd’ may actually be sequential in reality, but in practice the assumption that it’ll be sequential seems to work out well.
modern filesystems seem far more resistant to direct head/cylinder/offset type stuff… But for ancient things say like Xenix at least with Qemu you can specify the direct characteristics and get around it that way..
Until improvements in floppy emulation were made, that was the only way I knew of to migrate a Xenix install into a VM.. But having plenty of options is always a good thing.
If the disk is going bad, use ddrescue or dd_rescue, since they can quickly skip the unreadable portions of the drive, and then return to them later (this helps a lot when disk has several bad areas – you quickly get nearly all that’s readable normally, then the program retries the problematic areas sector-by-sector).
thankfully I got a clean read, but the more tools available to help others (or me down the road) the better!