Migrating to a new hard drive

My laptop hard drive started showing symptoms of potential failure, so I made the decision to migrate to a new hard drive. However, since much of the windows software comes pre-installed and without a good installation disk, I needed to do so without having to re-install software.

Additionally, my laptop does not have an internal CD/DVD drive, and won’t boot off an external one unless you get the (extra-priced) one blessed by the manufacturer. Furthermore, it’s a laptop, and does not have extra IDE slots to simply plug in the new drive to do a drive copy. And finally, I did not initially have a cable to enable me to plug in a drive as an external USB device. The second time I did this I used that cable, and it did simplify things a bit.

Next I’ll describe a few pre-requisites and assumptions, then give a summary of the solution. Afterwards I’ll go into details on a few of the steps.

Disclaimer: I am not a professional tech-supporter. This how-to is provided in the hopes it will be useful and helpful, but without any guarantee of success, that this information is correct, or that I will even respond to support inquiries. It is a description, after the fact, of how I solved this problem when I was in this situation.

First, I assume that the new drive you get to replace your old drive is bigger. This is not a requirement, but simply the most common case, since it’s probably been a few years since you got the first drive. If your replacement drive is an exact replica of the original drive, this how-to is more than you need. For that case it suffices to simply replicate the data on one drive to the other doing a drive copy. But if the new drive is bigger, and you want to be able to take advantage of this, then a simple drive-copy will not work, as it will over write the partition table of the new drive with the partition table of the old drive, rendering the extra space useless, as the drive will forget it has that space. This brings me to my next assumption — that you are not using the windows dynamic disk technology. Its layout is not well described, and thus this solution does not know how to handle it. Also, if you’re using LVM, then you would have to adapt this strategy for your purposes, I do not know details of what exactly needs to change. Finally, I assume that it is okay with you to take advantage of your additional space by adding new partitions of the disk with space, rather than resizing your original partitions. In many cases, resizing is an option, but it was not a concern of mine, so I can’t report on it.

I also should mention that the computer I did this with runs Linux and Windows. It should work with any OS which uses standard disk partitioning, but those are the only two I’ve tested it with. The solution involves using a Linux boot media (CD, floppy, etc), so I’ll assume you’re not terrified of Linux.

So, given those assumptions, the general solution is to create partitions on the new disk which are identical in size to the partitions on the old disk, then replicate the data within each partition from the old disk to the new disk.

Step 1: Record your partition information from your old disk. If you’re running linux, you can just do this from within, otherwise, boot your computer using a linux disk, such as a Knoppix LiveCD. Then run
fdisk -l /dev/XXX where XXX is the device name for your hard drive. Common device names are “hda” or “sda”. ‘h’ is commonly for IDE drives, while ‘s’ is commonly for SCSI drives. ‘a’ is for the first drive of that type — your drive could be ‘b’, or another letter. You’ll want to write down the whole partition table. An example couple of rows looks like this:

/dev/hda1 * 1 34 273073+ 83 Linux
/dev/hda2 35 788 6056505 5 Extended
/dev/hda5 35 315 2257101 83 Linux

The first column is the partition device name. The ‘*’ indicates whether the parition in marked bootable. The next two columns are the start and end cylinder numbers, followed by a some computation of size, a number representing the partition type, and finally a string representing the partition type. Later we’ll be reproducing this on the new drive.

Step 2: (optional, but recommended) Save the partitions from your old drive onto a temporary place, such as another drive you have in a computer with plenty of storage. You can store them as files in the filesystem. The main benefit of doing so is that if (like mine) your drive seems to be failing, you want to get a safe copy to work from without further damage. Now the “how” to do this varies depending on your setup. In my case, I had to boot my laptop from a boot disk, create a small ethernet network between my laptop and desktop (that topic beyond this howto), and use dd, netcat, and ssh to do the copy. ‘dd’ is a great tool for reading raw data from block devices. If you have the drive attached to another computer, you can do something like:
dd if=/dev/hdb1 of=/home/chip/drive-partition-1.img bs=32M
to copy the partition from the device hdb1 to a file in your home directory called drive-partitoin-1.img. The bs=32M tells it to copy 32 Megabyte blocks at a time, which has a significant improvement on speed over the default. However, in my case, I needed to copy the data to a remote computer attached by the network. So I left out the “of=…” clause, causing it to output the data to its standard output. Then I redirected that output to either ssh or netcat, which I was using to tunnel the data to the computer on the other end. For example:
dd if=/dev/hdb1 bs=32M | ssh chip@example.com dd of=/home/chip/drive-partition-1.img bs=32M

Does the copy over ssh—the input is sent over the ssh tunnel, and the remotely executing dd reads from the standard input and writes it to /home/chip/drive-partition-1.img.

Unfortunately, doing this copy over ssh is a little slow for my taste, because of the extra computation of encrypting and decrypting all the data. I tried compressing the data as well to speed things up, but the compression just took more CPU time, and the CPU was the bottleneck, not the I/O. The solution was to switch to use netcat. Netcat is a program which works much like ‘cat’, but splits it between two hosts talking over a TCP socket. To use netcat you have to first set up the listening one, using a command (on the desktop) like this:
nc -l -p 12345 > /home/chip/drive-partition-1.img

This tells it to start listening on port 12345, and copy anything sent to that port into the file /home/chip/drive-partition-1.img. We then modify the command earlier to be:
dd if=/dev/hda1 bs=32M | nc example.com 12345
which causes dd to send output to the nc program, which routes it over a socket to example.com with a port 12345. In my case, my linux boot disk did not have nc installed, so I had to compile a static version of it, and copy it over to the laptop. Using nc gained another factor of 3 in performance of the partition copy.

You should repeat this process for each partition.

Step 3: Replicate the partition table on the new drive. Since we’re trying to get it exactly the same, I suggest using the linux fdisk (one of the lowest level disk partitioners out there). Attach the new hard drive to a computer and boot it from a linux boot disk. Run fdisk on the drive (fdisk /dev/XXX). Create each drive in the partition table using the partition table you copied down earlier. Use ‘n’ to add each partition. Follow its prompts to partition it correctly, including typing in the starting and ending cylinder number for each partition. Make sure to toggle as bootable any partition that had a star, and go edit the partition types to match the original ones. If you need to figure out any command, use ‘m’ to see the help menu. After setting up what you think is correct, print the partition table using ‘p’. If its correct, you can write it to the disk and move on. If it’s not correct, delete partitions and try again.

Step 4: At this point, the new drive is ready for the data. You can now copy it there, either from the copy you made in step 2, or from the original drive. The same commands are used to copy the data to the new drive, but the “of=” will be a device this time instead of a file. If you are lucky enough to get both drives in the same computer, this can be as simple as
dd if=/dev/hda1 of=/dev/hdb1 bs=32M

for each partition. If you’re not that lucky, you can use ssh or netcat as described in Step 2.

Step 5: The last step is to make the new drive bootable. This varies for different OSes. If you’re an expert in master-boot-records, stop reading, because I am not. If you only run WinXP, you’ll want to boot from the windows install/rescue CD and run the “fixmbr” command from the repair console. If you run Linux, there’s a good chance you’ll use grub, so you can find a howto on grub mbr installation and setup. The boot menu from your old drive should work just fine, it’s just a matter of getting grub itself installed on the mbr. In my case, I mounted the new drive, and used ‘chroot’ to fool my shell that it was running under the new disk, and ran the grub commands from there. Your LiveCD may have grub on it natively.

Congrats! If everything went perfectly, you should now have a working new drive with extra space on it which can be partitioned and used once you get back into your running OS. If not, then hopefully you still have the copies made in step 2 so you can repeat the process or ask someone for help. Note that when I booted into WindowsXP, I had to give it some time to recognize the new drive before it was happy, including a few reboots including a couple into SafeMode.