Sunday, September 07, 2008

Virtualbox: Expanding A Disk Drive

I love Virtualbox, it's definitely my virtualization platform of choice for daily tasks* on a Linux-host, and it's quickly replacing VMware on Windows hosts as well.

However, recently I had to expand a virtual partition of a Windows guest. There are already lots of options and methods available to do this:

None of these actually offered a really good or quick solution, until I found this blog post. Tom's statement of hostility to copyright allows me to provide a summary... However, I've added some extra steps and tips for both Windows and Linux guests.

You’ll need:
  • A Gparted LiveCD.
  • Two virtual disk images: the old one (the one you're currently using, the small one), and a new one, which you can easily create in Virtualbox, make sure it's "raw" (unpartitioned).

1. Mount or burn the Gparted disk in Virtualbox so that it'll boot from it (make sure the "virtual bios" boots from it first).

2. Using VirtualBox, create a new VDI with a larger size that you want to expand the virtual machine to (see: "You'll need").

3. Leave your existing VDI as the primary IDE master. Set the new VDI to be the Primary IDE slave for the machine.

4. In Gparted: run fdisk -l or use Gparted GUI to view your partitions. You should get /dev/hda and /dev/hdb. /dev/hdb shouldn’t be partitioned at this point. /dev/hda should be your primary master (the original VDI that is too small for your needs), and /dev/hdb should be the new image (virtual primary IDE slave).

5. Do not mount those (yet). Instead, use dd to copy the old image to the new one:
dd if=/dev/hda of=/dev/hdb
Warning: don't despair, this may take some time, and the hard drive status icon of Virtualbox should indicate that something's happening.

Steps for Linux guests:

1. Launch GParted, and resize the partition on /dev/hdb (the new image) to take up the entire disk. This may take awhile. If you cannot move your partition because the swap partition is in the way, first remove the swap, then add a new swap partition add the end of the hard drive (linux-swap), and then extend the non-swap partition. The following screenshot shows how it should look after recreating the swap partition, right before extending the data-partition:

3. Extra step (update - thanks to comments): modern Linux distributions use disk IDs to identify disks instead of sda, sdb,... This might give you the following message in a few steps: "Waiting for device /dev/disk/by-id/scsi-SATA_VBOX_HARDDISK_VB... to appear", like the user in the comments of this post. So before you continue with the following steps, open a Terminal window.

I'm going to create two mount points for the old and new partition and mount them. You can use GParted to find out the exact device location (the new partition should probably be /dev/hdb1):
mkdir /mnt/old_disk
mkdir /mnt/new_disk
mount /dev/hda1 /mnt/old_disk
mount /dev/hdb1 /mnt/new_disk

Screenshot:

I'm now going to take a peek at the fstab-file on my old partition:
cat /mnt/old_disk/etc/fstab

With an Ubuntu partition, you get something like this:

Ubuntu uses UUIDs to identify partitions. We'll have to edit our new partition's fstab file with the new UUIDs. Your distribution could use:
  • Device locations, like /dev/sda1 (yes, you see this in my fstab file as well, but everything after a pound ('#') character is a comment, so it's not really using that). You can find out your new locations using GParted. Make sure you only change the numbers if you have to! So don't change sda to sdb or hdb! GParted might be showing you /dev/hdb, but when we'll make our new partition the master VDI, your Linux installation will see this as /dev/sda!
  • IDs, not common, you can find the new IDs by using ls /dev/disk/by-id -lah.
  • Paths, not common, you can find them by using ls /dev/disk/by-path -lah.
  • UUIDs (like me), more and more common. You can find your new UUIDs by using ls /dev/disk/by-uuid -lah. The new UUIDs will be listed next to ../../hdb-lines (remember: hda is our old hard disk).

Now let's edit our new fstab. Open a new Terminal window (so you can still see the UUIDs) and enter:
nano /mnt/new_disk/etc/fstab

Nano is a Terminal text editor. Just use it like Notepad, use CTRL+O to save your changes, and CTRL+X to exit. You can use it to edit your UUIDs.
Note: make sure you leave the proc-line intact! Also, note that the cdrom-line doesn't use an UUID but /dev/scd0.
Extra note: the data-partition will probably have the same UUID as before (we have only extended it). The swap partition has a new UUID (because I deleted and recreated it).

Following screenshot shows my edited fstab, I have removed the comments.

4. All done? Good. Poweroff with the GParted LiveCD.

5. Finally, set the new image (resized) as the primary master for the virtual machine. Uncheck the primary slave. Boot into the machine to verify that it worked. If everything works you may delete the small disk VDI.

Screwed? Then make the small image master and the large one slave again, and use GParted with nano to double check. Don't be afraid making a mistake, if you only edit the large VDI, you can always rollback to your old small one and start over.

Steps for Windows guests:

1. Do not use gparted or ntfsresize to resize the partition. I know you can, but I found it a tad risky. Instead, set the new image as the primary master for the virtual machine, uncheck the primary slave. Boot back into Windows.

2. You should be back into Windows, but still on a small partition. If you use Disk Management in Administrative Tools you should see that there is a lot of unallocated space now on the virtual hard disk, how can we expand our drive?

3. Open a cmd. Type diskpart.exe, this is a tool that comes built in with most Windows distributions.

Type extend and press enter. This will extend your partition. The full command is extend [size=n] [disk=n] [noerr], but we can leave all options to their defaults.

4. Once this is done, use chkdsk to make sure everything is okay.

*For server-oriented platforms, Xen is the undisputable champion.

21 comments:

BECK said...

hi
hello
how was your day?
i liked your blog
you are fantastic!!!

really nice blog
fabulous fantastic
bye
take care
see you

Anonymous said...

Hey there! Is seems that GParted can't recognize both partiotions. If i boot from the primary ide gparted only recognizes hda, else only hdb (the new unpartitioned one). Any ideas?

Spud said...

extending volumes doesnt work on system volumes or volumes containing page files...

http://support.microsoft.com/kb/325590

µacuyiko said...

@Anonymous: are you sure you are setting up both master and slave VDI's? (As in step 3?) And not using just one master?

@Spud: true, the KB says: "System or boot volumes may be blocked from being extended, and you may receive the following error". It did work in my case, but it may not work in yours. You can still try to use the *NIX ntfsresize tool, then, but save your old (small) VDI for when things don't work out!

Anonymous said...

Thank you for posting these instructions. I believe I followed them as described, but I have a problem when I try to boot off the new VDI. After I remove the original VDI and make the new VDI the Primary Master, the boot sequence starts, but it fails when
"Waiting for device /dev/disk/by-id/scsi-SATA_VBOX_HARDDISK_VB... to appear". It asks me if I want to fall back to the same disk ID. Answering Y or N results in the same failure, leaving me at a shell in the RAM disk. Is there anything else I need to do to make the new VDI bootable?

µacuyiko said...

@Anonymous: thanks for letting me know, I have updated the post with instructions to remedy this. I hope you read this, since I cannot contact you via mail. Recent Linux distributions use disk UUIDs in their /etc/fstab-configuration, which have changed in your new VDI. You have to edit /etc/fstab (in GParted for example, or the emergency shell) to recognize them.

adam shaw said...

for anyone who tried to boot their newly created partition, made it a primary master, and you get a black screen w/ cursor at virtualbox bootup, make sure your new partition has the 'boot' flag (can do this through the gparted gui)

Anonymous said...

@Spud:

you are correct. There exists freeware you can download that will do it for you though. I suggest

http://www.download.com/Easeus-Partition-Manager/3000-2248_4-10863346.html?tag=mncol

which did it for me in about 3 mins time including reboots.

Alan said...

Thanks for the tutorial. Couple of notes on the diskpart.exe:

- as others mentioned it cannot be used on the system partition. Simple solution: boot to your old VDI with the new disk image as secondary slave. Run extend on the new partition as a slave.

- Before typing "extend", I had to "select" the disk and partition. Disks are numbered starting at 0, and partitions starting at 1, so with the drive mounted as primary slave, the commands were:
select disk 1
select partition 1

It worked great!

BTW, while gparted is a great disc to have around, just about any linux liveCD image will work. They've all got dd on them.

Anonymous said...

None of the described uses of diskpart worked for me but Easeus-Partition-Manager did. Using dd to clone harddrive partitions is and old (but good) trick. I guess any disk imaging system should work provided that you do enough fiddling with the partitions later. Watch out for Windoze tricks for calling your slave partition c: and your master e:!

µacuyiko said...

Indeed, Easeus is another good alternative, and I would certainly recommend it to anyone using Windows. (It's one of the best partition managers around, forget Partition Magic.)

It's a pity they don't offer a free version for 64bit platforms though.

Mike VanHecke said...

I used system rescue (latest version). That is the way to go, but I would still recommend using GParted even for windows. You may have to run check disk from windows go back to a system rescue and run GParted to expand the partition. It worked great, where the winblooze utility diskparted was unable to expand the partition (kept erroring out). Needless to say I bow to the power of the Penguin!!!

The Rogue said...

I too found that the diskpart utility didn't work. I am using Windows XP Pro 32-bit I ran diskpart, entered "select disk 0", then "select partition 1" to pick my only disk/partition (Use "list disk" and "list partition" to see your numberings). When I tried to use "extend", it told me that it could not do it, and to make sure my partition was valid for extension. Looks like I'll be using gparted again... I've used it to resize a number of NTFS partitions successfully in the past, I just thought it would be a good idea to go the MS way. Guess I was wrong. Windows should run checkdisk automatically on boot after messing with it in gparted, but it should be ok after that.

sriram said...

works great - thank you. been wanting to do this for quite a while.

Jay said...

Thank you for this! I had one problem though... Tried to run the DD command (from /root of the RescueCD ISO) and got this error:

dd: opening '/dev/hda': No such file or directory

GPARTED shows the partitions as /dev/sda and /dev/sdb, so I changed the command to use that and it worked OK.

The next problem was the actual extending of the disk... DISKPART did not work, even when I made the new (larger) partition a slave. GPARTED would not extend it either. I ended up using VHD Resizer to create a larger VHD, and then Rescue CD's GPARTED to extend it. Now I am in business.

jimmy said...

Easy solution:

Create a new virtual disk, connect it as slave. Install programs to this drive :P

Ron Kurr said...

Your post was really helpful. I had a Xubuntu guest that ran out of space that really caused me some pain. It only took me an hour or so to get things right thanks to your post.

Boyan Boychev said...

Hi all,

If your root partition is lvm type and you are using LVM try this out:
--------------------------------------------------------------------------------------------------------------------------------------------
Use LiveCD with dd and fdisk utilities:
----------------------------------------
1) dd if=/dev/hda of=/dev/hdb # /dev/hda - old disk with smaller size, /dev/hdb - new disk with bigger size
2) fdisk -u /dev/hdb
3) Steps from 2) to 6) (incl.) described here - http://www.linuxquestions.org/questions/fedora-35/lvm-partition-resizing-666683/#post3281556
4) Shutdown the Virtual machine and remove the old disk and the LiveCD from the virtual CDROM

After the shutdown, boot the system normally (only with the new disk and without the LiveCD):
---------------------------------------------------------------------------------------------
5) pvdisplay -v # Checks the PV name (e.g. /dev/hda2)
6) pvresize --verbose /dev/hda2
7) vgdisplay -v # Checks the Free PE (e.g. 384)
8) lvextend -l+384 /dev/VolGroup00/LogVol00
9) resize2fs /dev/VolGroup00/LogVol00
10) Наздраве!
--------------------------------------------------------------------------------------------------------------------------------------------

Tested with VM:
CentOS release 5.4 (Final)
kernel-2.6.18-164.11.1.el5
lvm2-2.02.46-8.el5_4.2

Don't do that on production systems but better create a new partition with fdisk and a new PV, extend the existing VG and LV after that (see http://www.linuxquestions.org/questions/fedora-35/lvm-partition-resizing-666683/#post3281556).

Be sure that you have a backup!!!

Best Regards,

Boyan Boychev
Bulgaria

herana said...

It is so lucky to read your article, from this i can get some information that i didn’t know before. Your high quality articles are so great, and can we buy some ads from you? If you agree, just emial me the ad type and fee per month. If you own some other high quality related blogs, selling ads would be welcomed.
By Air Jordan shoes

Anonymous said...

Very useful, thank you very much!

eray said...

thnk you for sharing. travestitravesti

Post a Comment