Creating an Azure VHD from Ubuntu Cloud Images on Mac OSX

We have many different ways of delivering the Lateral API to clients who would like to install it in their own environment. One of those is as an Azure VHD for deployment to Azure VMs. In this post I will cover how to create a VHD that is fully compatible with Azure from an Ubuntu Cloud Image base.

I found the Azure documentation to be a bit outdated. In fact in various places it didn’t work at all. So I thought I’d try to create a version that at least works for Mac OS X users. Well, that works until it itself becomes outdated and stops working. At which point I or hopefully someone else will take it upon themselves to write an updatedversion.

There were also quite a few gotchas that I will explain in due course. One that deserves an honorable mention is the fact that the size of the Azure VHD must be aligned to the nearest whole MB otherwise the image will be rejected after upload. I mean, seriouslyMicrosoft??

Dependencies

There are several dependencies required before we canstart:

VirtualBox with VBoxManage and VBoxHeadless on your path Go for the azure-vhd-utils-for-go Azure CLI for interacting with Azure from the command line qemu for qemu-img used in create-vhd script to resize and convert images coreutils for gtruncate used by the create-vm script cdrtools for mkisofs used by the create-vm script gawk used by the create-vhd script

You can use Homebrew to install the last 4 with the followingcommand:

brew install qemu coreutils cdrtools gawk

Next you need two scripts; create-vm and create-vhd . The create-vm script was modified to work with OS X from the example written by Scott Moser . Thanks Scott! Have a read through the comments of the two scripts to get an idea of what they do. Then download them to a location of your choosing and ensure they areexecutable :

create-vm create-vhd Creating the VM

Use the create-vm script to create a VM in VirtualBox from the Ubuntu 14.04image:

./create-vm my-trusty-vm

If you want to use a version of Ubuntu other than 14.04, just modify lines 8-9 of create-vm to point to the version youwant.

Once that has completed, you can start the VM in headlessmode:

VBoxHeadless --startvm my-trusty-vm

Now you can SSH into the instance from port 2222 :

ssh -p 2222 -A ubuntu@localhost

The password is set at line 35 of create-vm to ubuntu . We strongly recommend changing this to something secure or even better setting up key basedauthentication .

When you’re in the instance there’s a bit of setup required for it to work with Azure. See here for some more details about what’s being donehere:

sudo apt-get update sudo apt-get install -y linux-image-virtual-lts-xenial linux-lts-vivid-tools-common hv-kvp-daemon-init sudo reboot

After the system reboots, edit /etc/default/grub in a text editor, find the variable called GRUB_CMDLINE_LINUX_DEFAULT and changeto:

GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0,115200n8 earlyprintk=ttyS0,115200 rootdelay=300"

Then updategrub:

sudo update-grub

And install the azureagent:

sudo apt-get update sudo apt-get install walinuxagent

Now that’s all installed you can do whatever you need to get the VM up and running how you like it. Install packages, clone your code to the VM and set it up to start at boot,etc.

Preparing the VM for deployment

This step is important to get the system running successfully on Azure. When the instance is in a good state and you don’t want to make any more changes, run thefollowing:

sudo waagent -force -deprovision history -cw logout

And safely shutdown theVM:

VBoxManage controlvm my-trusty-vm poweroff soft

Now you can use the create-vhd script to create an Azure VHD from the VDI. Note this will require about 60 GB free diskspace:

./create-vhd my-trusty-vm

This will take a little while to complete. The script takes care of the resizing of the VHD to the nearest MB as required by Azure so you don’t have to worry aboutthat!

Uploading the VHD

Once completed we are ready to upload the VHD to Azure. The documentation from Microsoft says to use the azure-cli to upload the image. However, trying to use it caused this error for me. This response from March 2016 basically says it’s being looked in to and in the meantime we should use the azure-vhd-utils . Which does work! So to install thatrun:

go get github.com/Microsoft/azure-vhd-utils-for-go

And then you can run the tool to upload theimage:

azure-vhd-utils-for-go upload --localvhdpath my-trusty-vm.vhd --stgaccountname NAME --stgaccountkey KEY --blobname my-trusty-vm

Replacing NAME with the storage account name and KEY with the storage account key. You can get these from the Azure Portal. azure-vhd-utils-for-go is good because it doesn’t upload empty space. So if the VHD is 30 GB on your disk it will only upload the parts of the image that have data in. This speeds up the upload processgreatly.

Now you have to create an image from the Azure VHD stored in blobstorage:

azure vm image create --blob-url VHD_URL --os Linux my-trusty-vm '' Replacing VHD_URL with the URL to the Azure VHD in the storage account. You can get the full URL to the VHD by navigating to it using the Microsoft Azure Storage Explorer , right clicking the file and choosing “Copy URL to Clipboard”. The '' is required because the CLI is expecting a file to be uploaded there. If you leave it as an empty string it works and