In my case, Ubuntu 16 is the host OS, Windows 7 is the guest OS, with Virtualbox 5.0.40.
Before starting, ensure your Virtualbox is a VDI file. VMDK files can’t be resized–though they’re useful for other reasons, being compatible with other virtualization software like VMWare.
First, run the guest OS. Defrag the drive. Defragging crams all the files together with contiguous unused space at the end of the drive instead of scattered around. Then run: sdelete c: -z. You must run this from a command prompt having admin privileges. This step writes zeros across the unused space, which is critical because VirtualBox can’t compact the VDI unless the free space is full of zeroes. The sdelete command takes a long time after reporting 100%, appears to be hung. Be patient and let it finish.
Now shut down the guest OS. From the host OS command line, run: vboxmanage modifyhd file.vdi compact. Here, file.vdi is your VDI file.
Your’e done. The VDI file will be smaller. It will grow automatically as you use more of the guest OS disk space.
Note: don’t use the guest Windows 7 OS file management to shrink the volume. The free space it creates can’t be compacted by VirtualBox.