A buddy of mine was thinking to move some of his Linux-based website to Azure. If you're running a Web Site that this node.js, ASP, ASP.NET, or PHP, it's easiest to use Azure Web Sites. I showed how to do setup Azure Web Sites in minutes with Git in this post. They hide the underlying OS from you, are automatically updated, scale easily, and share disks.
However, he likes VMs and the control they give him, plus he can run whatever he wants, move things around and generally control his world.
I'll be using the open source (GitHub) cross platform CLI tools (command line interface) for Azure. If you have node package manager you can "npm install azure-cli --global" then import your subscription. You can also get the command line tools by downloading and installing from www.windowsazure.com.
Setting up a Linux VM
We'll create a the initial VM using the Portal (I'll show you how to do it from the command line in a minute). This virtual machine will be for setting up a template VM image. I'm going to create an Ubuntu 13.04 server, then add Apache and PHP. Then I'll capture a generic image of my now-set-up machine and use it to create copies that I'll add to a farm. This image will show up later in "My Images" in the Azure Portal.
NOTE: There's an amazing community-driven category of prepped and ready Virtual Machine images at the Open Tech VM Depot. It's like the Azure's best kept secret. I'll do a post on it later but it's really deep and broad and worth checking out.
From within the Azure Portal I'll go New | Virtual Machine | and select Ubuntu Server 13.04.
The name doesn't matter, but I'll setup a user name and password (or use a SSH key):
Note I'll create a stand-alone Virtual Machine:
TIP: If I didn't want to use the Portal at all to make this VM, I could even find a VM image programmatically, then create a VM instance, all from the command line. Using "azure vm list" would get me the list of VMs to choose from. The Ubuntu one I wanted is "Ubuntu-13_04-amd64-server-20130501-en-us-30GB" (with a guid in the name as well) so I'd just "azure vm create MyDNSName ImageName [options]" and then proceed from there.
Once this Linux VM has started up, I'll SSH into it. You can see that Azure has mapped a random high number public port to the VM's internal SSH port 22.
I SSH in. I'm gonna add Apache, PHP, restart apache, then add a test.php that will show PHP is working as well as output the current IP address so I can tell which machine served the request.
sudo apt-get install apache2
sudo apt-get install libapache2-mod-php5 php5
I'll add a test.php
sudo nano /var/www/test.php
and put in
<?php echo gethostbyname(trim(`hostname`)); ?>
<?php phpinfo(); ?>
Then I'll exit SSH. VMs are locked down by default, so to test this I need to add an endpoint. I can do that via the Portal but I'd like to see what I can do from the Azure command line.
Run vm endpoint create to map external 80 to internal 80.
azure vm endpoint create mylinuxtemplate 80 80
At this point I should be able to hit mylinuxtemplate.cloudapp.net:80/test.php and see it work.
Cool. So I've got my Linux VM template the way I want it. Now I want to "capture it" as an image so I can stamp out more of them. This "waagent" on Linux is like "sysprep" on Windows.
From within a SSH session, run waagent -deprovision.
~$ sudo waagent -deprovision
scott@mylinuxtemplate:~$ sudo waagent -deprovision
WARNING! The waagent service will be stopped.
WARNING! All SSH host key pairs will be deleted.
WARNING! Nameserver configuration in /etc/resolvconf/resolv.conf.d/{tail,originial} will be deleted.
WARNING! Cached DHCP leases will be deleted.
WARNING! root password will be disabled. You will not be able to login as root.
Do you want to proceed (y/n)? y
WINDOWS PEOPLE: If you're a Windows person, you can setup your Windows VM just as you like it, then run %windir%\system32\sysprep.exe on it, capture an image of the VM and do everything described in this post as well!
I could shut it down and capture an image from the Portal, but again, it's command line fun today. Note that shutting down can take a little while.
azure vm shutdown mylinuxtemplate
azure vm capture mylinuxtemplate hanselmanlinuxwebfarmimage --delete
The capture command will DELETE the Virtual Machine. Remember that it was just a template. However, how I have a reusable image! I can see the images available to me, both user images and gallery images with "azure vm list."
NOTE: When you delete Virtual Machines, you're just deleting the "configuration" of the VM. You're not deleting the disk that was associated with it, as it's possible you might want to start that VM up again. If you're really trying to remove things, make sure you delete the instance of the VM and then delete the disk, too.
Creating a Linux VM Farm from the command line
Now I have an image sitting in my storage account that I can use to make "n" VMs from. I'll make one VM to start. I can watch it startup with "azure site list" after making it. When it's ready, I can make more! Make sure you use the --ssh switch or you will NOT be able to SSH into the machine!
C:\> azure vm create hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott MyPassword123 --location "West US" --ssh
info: Executing command vm create
+ Looking up image
+ Looking up cloud service
+ Creating cloud service
+ Creating VM
info: vm create command OK
C:\> azure vm list
info: Executing command vm list
+ Fetching VMs
data: DNS Name VM Name Status
data: ------------------------------- ------------------ ----------
data: hanselmanlinuxfarm.cloudapp.net hanselmanlinuxfarm CreatingVM
info: vm list command OK
C:\> azure vm list
info: Executing command vm list
+ Fetching VMs
data: DNS Name VM Name Status
data: ------------------------------- ------------------ ---------
data: hanselmanlinuxfarm.cloudapp.net hanselmanlinuxfarm ReadyRole
info: vm list command OK
WEIRD: Azure has a concept called a "Cloud Service" which is a lousy name, IMHO. For us, let's consider it a "container" for our VMs. It's a logical container that will hold and associate all our VMs (and other cloudy stuff) together. When you have one VM you have one cloud service associated with it, but you can't see it in the Portal but because it doesn't really provide value...yet. When you have TWO VMs in the same container, then you'll see that cloud service "container" appear in the Portal.
I've made a hanselmanlinuxfarm VM now so there's a hanselmanlinuxfarm cloud service 'container.' Now, I'm going to make a few more VMs but I'll connect them to the first VM. There's two ways to do this. First, the --connect option from the command line. Note that you don't have control over the name of your VM this way, if you care. If you have hundreds, you likely don't.
The command will find the existing cloud service (again, 'container') then make a new VM. I'm going to run this command twice so I'll have three VMs total.
SO IMPORTANT: It's the --connect used on this second call that is the key. It makes the second (and then n+1) VM and adds it to the same cloud service "container." It seems the VMs associated with each other. The name of the next VM will be whatever-2, then -3, etc but they will also use the same external name, like hanselmanlinuxfarm.cloudapp.net.
I'll do this twice, each time using a different high SSH port number that will map to 22 internally. If I don't want SSH expose externally, I can delete the public endpoint later.
C:\> azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott MyPassword --ssh 12345
info: Executing command vm create
+ Looking up image
+ Looking up cloud service
+ Getting cloud service properties
+ Looking up deployment
+ Creating VM
info: vm create command OK
When creating a Linux VM you MUST add a --ssh flag to the command line or you'll not be able to SSH into the box. Make sure to add a high port number so you'll get a mapping from that port to 22, so maybe 12346 -> 22, etc. If you make these VMs from the Portal, it will pick a random port for you. When you do it from the command line, you need to choose.
At this point, azure vm list says I have three. Two are ready and the last is being created now. You can tell these VMs are running in the same Cloud Service "container" because the DNS name is the same. These VMs are officially a "farm."
C:\> azure vm list
info: Executing command vm list
+ Fetching VMs
data: DNS Name VM Name Status
data: ------------------------------- -------------------- ----------
data: hanselmanlinuxfarm.cloudapp.net hanselmanlinuxfarm ReadyRole
data: hanselmanlinuxfarm.cloudapp.net hanselmanlinuxfarm-2 ReadyRole
data: hanselmanlinuxfarm.cloudapp.net hanselmanlinuxfarm-3 CreatingVM
info: vm list command OK
OK, now here's making a fourth VM from the Portal, just as an FYI.
Creating a Linux VM and adding to the Farm from the Portal
Just so you know, you can add VMs to your farm from the Portal also.
Give your new VM a name, then "connect it to an existing virtual machine." I don't like this phrasing, and I'm curious what you think. Basically it's "add this VM to this collection of VMs." It doesn't matter which one you select from this dropdown, as long as you pick one that's in the Cloud Service "container".
I won't click OK, but if I did, at this point I've would've a fourth VM, this one via the Portal.
Load Balancing my Linux VM Farm
I have three identical VMs running Apache and PHP and my test.php page.
C:\> azure vm list
info: Executing command vm list
+ Fetching VMs
data: DNS Name VM Name Status
data: ------------------------------- -------------------- ---------
data: hanselmanlinuxfarm.cloudapp.net hanselmanlinuxfarm ReadyRole
data: hanselmanlinuxfarm.cloudapp.net hanselmanlinuxfarm-2 ReadyRole
data: hanselmanlinuxfarm.cloudapp.net hanselmanlinuxfarm-3 ReadyRole
info: vm list command OK
Let me open up port 80 on all three. Since I want them load balanced I can't do this, as this is how you map single non-load-balanced ports.
azure vm endpoint create hanselmanlinuxfarm 80 80
azure vm endpoint create hanselmanlinuxfarm-2 80 80
azure vm endpoint create hanselmanlinuxfarm-3 80 80
For load balanced ports I need to use "create-multiple." Not only may I want to open multiple ports all at once, but also since I want load-balancing, I also may want a probe set up. For example, since I'm using HTTP, if there's a result other than 200 returned from test.php then I'll want that VM pulled out of the farm. It will also pull the VM out of rotation if it doesn't hear back in 30 seconds.
Here I'm creating those 80 to 80 maps, but also watching test.php for anything other than an HTTP 200.
azure vm endpoint create-multiple hanselmanlinuxfarm 80:80:HttpTrafficIn:http:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-2 80:80:HttpTrafficIn:http:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-3 80:80:HttpTrafficIn:http:80:/test.php
In this case "HttpTrafficIn" is what I am calling the Load Balancing Set Name.
NOTE: I'm doing a pull request now to add the ability to see the ProbePath from the endpoint command but for now you can see it with "azure vm show" like this:
C:\>azure vm show hanselmanlinuxfarm
info: Executing command vm show
+ Fetching VM
data: DNSName "hanselmanlinuxfarm.cloudapp.net"
data: VMName "hanselmanlinuxfarm"
data: IPAddress "100.68.xx.xx"
data: InstanceStatus "RoleStateUnknown"
data: InstanceSize "Small"
data: InstanceStateDetails ""
data: OSVersion ""
data: Image "hanselmanlinuxwebfarmimage"
data: DataDisks ""
data: Network Endpoints 0 LoadBalancedEndpointSetName "HttpTrafficIn"
data: Network Endpoints 0 LocalPort "80"
data: Network Endpoints 0 Name "endpname-80-80"
data: Network Endpoints 0 Port "80"
data: Network Endpoints 0 LoadBalancerProbe Path "/test.php"
data: Network Endpoints 0 LoadBalancerProbe Port "80"
data: Network Endpoints 0 LoadBalancerProbe Protocol "http"
data: Network Endpoints 0 Protocol "tcp"
data: Network Endpoints 0 Vip "137.135.xx.xx"
info: vm show command OK
Cool, so now let's see if I have a load-balanced farm.
PERF NOTE: In order to get the best performance from your Azure VMs (or any cloud VM) considering putting things like MySQL/PostgreS databases on a separate disk with different caching semantics. You want the OS disk and the Data Disks to be separate. For example. I have a Windows VM running MySQL. The OS is on a standard 30 gig disk, but the MySQL Database is alone on a 5 gig disk that's attached. It keeps things separate and tidy, plus it performs better
Checking on my new Farm
If I log into the Portal, I can look at each individual VM or I can look at the farm as if it's one 'cloud service.' Get it?
Making my Farm more reliable
I want to make sure my new VMs are all on different racks in the Azure Data Center. I know they are in "West US" because I put them there, but I'm not sure if they are together on the same rack or near each other or what. Since a rack is within a "fault domain" meaning that a Rack could, I dunno, spontaneously explode, then I want to tell Azure that all these VMs are part of an "availability set." This is a name I apply to the VMs that says "make these more available by keeping them apart."
From the Portal I'm going to pick the first VM and select Configure, then Create an Availability Set. I'll name it "hanselmanlinux" but it can be anything.
Adding an Availability Set can involve Azure moving my VM somewhere else within the Data Center and it may need to restart it if it does. Sometimes this is fast, other times it takes a minute or 10 as it's a big deal initially, so be aware. Once everyone's in the set, it's less of a big deal.
When it's done, head over to the other VMs and add them one at a time to the same availability set. The result looks like this in the Portal, and now I'm assured that my three VMs are all in different Fault Domains (and racks).
Hitting my Web Farm
Now I can hit hanselmanlinuxfarm.cloudapp.net/test.php and see the IP changing (as well as the CPU% changing in my portal!) or even watch HTOP over SSH and get a live view. Hey, I've got a little Linux farm!
Here's my SSH'ed into one of them, looking at htop (it's better than top!)
My Complete Script, Summarized
Here's my complete script. I used azure vm image list | find /I "13_04" to find an Ubuntu image. I could have done this with bash as well.
C:\>azure vm image list | find /i "13_04"
data: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-13_04-amd64-server-20130423-en-us-30GB
data: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-13_04-amd64-server-20130501-en-us-30GB
data: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130511-en-us-30GB
data: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130515-en-us-30GB
data: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130517-en-us-30GB
data: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130518-en-us-30GB
data: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-raring-13_04-amd64-server-20130521-en-us-30GB
Once I've found an image, I create my first VM from the command line in a location of my choice. Again, it's Linux, don't forget the -ssh.
azure vm create mylinuxtemplate b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-13_04-amd64-server-20130501-en-us-30GB scott MyPassword --location "West US" --ssh
Get it how you like it. SSH in, set it up, run the waagent to prep it. Shut it down and capture it.
azure vm shutdown mylinuxtemplate
azure vm capture mylinuxtemplate mylinuxfarmimage --delete
Finally, here's a basic batch file to make 5 VMs. Note the first command is different from the Nth. Of course, with bash you could make a script like "spinup 5" and automate to your heart's content. The HTTP probe is optional on the endpoint creation.
azure vm create hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --location "West US" --ssh
azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --ssh 12345
azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --ssh 12346
azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --ssh 12347
azure vm create --connect hanselmanlinuxfarm hanselmanlinuxwebfarmimage scott password --ssh 12348
azure vm endpoint create-multiple hanselmanlinuxfarm 80:80:HttpTrafficInLhttp:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-2 80:80:HttpTrafficInLhttp:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-3 80:80:HttpTrafficInLhttp:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-4 80:80:HttpTrafficInLhttp:80:/test.php
azure vm endpoint create-multiple hanselmanlinuxfarm-5 80:80:HttpTrafficInLhttp:80:/test.php
After it ran, I went in to the Portal and set up Availability Sets manually. That's only available in PowerShell today, but setting availability sets is coming soon to the cross-platform tools!
Next time, maybe I'll try"azure vm scale" to move these tiny VMs into 8 processor 56 gig monsters.
Sponsor: Big thanks to SoftFluent for sponsoring the feed this week! Less Plumbing, More Productivity! Generate rock-solid foundations for your .NET applications from Visual Studio and focus on what matters!
© 2013 Scott Hanselman. All rights reserved.