Written by: @samcogan
Edited by: @phrawzty
If you're a sysadmin who needs to deploy resources in Microsoft's Azure platform your choices have been limited to either the web portal or complex scripts. Today I want to highlight a set of new technologies that when combined together provide a way to move away from manual processes and complex scripts and instead produce a declarative, reusable, and easy to manage and maintain solution for deployment to Azure. Using these tools we will be able to stop focussing on defining "how" we deploy, and instead start defining recipes for “what” we want to deploy and let the technology take care of “how”. By the time you’ve finished this article you’ll see the power of these technologies and the agility they can bring to your deployment process.
Before we dive in, let’s take a step back and understand what the problem is we are trying to solve. Deploying resources to any cloud provider usually involves more than a few moving parts such as storage, virtual machines, websites, and databases -all with dependencies on each other. This gets even more complex when talking about test or development environments where there is a need to regularly create and destroy environments.
In this sort of world what is needed is a way to easily define an environment, automate the deployment and tear down of that environment, and then do it all again with the same results. Additionally you also want to apply software development practices such as testing, version control, patterns and practices to your deployment process - collectively, this is often referred to as "Infrastructure as Code" (IaC).
Up until recently this has been difficult (if not impossible) to do with Microsoft’s Azure platform. Sure you could create PowerShell scripts to deploy resources, but this all happened in a serial fashion with each resource having no knowledge or dependencies on other resources. This was more like "Infrastructure as scripts that failed as often as they worked". This lagged behind platforms such as AWS where solutions like Cloudformation offer a relatively straightforward way to represent your infrastructure as JSON.
This has all changed in the last year with the release of new tools and enhancement of existing ones to allow an Infrastructure as code approach to Azure deployments.
Azure Resource Manager
Azure Resource Manager (ARM) is the first new feature that enables moving to a code based deployment. ARM introduces a number of features, but the ones key to an IaC approach are:
Creation of resources inside a resource group so they can be managed collectively.
Declarative templates to define deployments.
Idempotent deployments that can be re-applied to ensure a consistent state.
Deployment of non-dependant resources in parallel to improve deployment times.
With these four components it is now possible to create recipes for your deployments using JSON templates. These templates define both the resources themselves and the relationships between them.
Once your templates are finished you can then deploy these to a resource group. If this is the first time you have deployed to this resource group it will create your resources; however, if you're running against an existing deployment it will validate the state of the deployment and amend or re-deploy resources as required.
Finally, if you want to delete your deployment, all you need to do is delete the resource group and that will delete and cleanup all the resources contained within. If you want to re-deploy, just run the deployment again.
Because your deployment is now in a declarative template you can store this in your version control tool, add it to your testing process, and use them as part of your build process to create test environments. You can also use Visual Studio to create your templates (ensure you have the latest Azure SDK and PowerShell tools for Visual Studio installed).
Templates are not limited solely to deploying top level resources! For example, a template that deploys an Azure Website can include a resource that downloads an MS Deploy package and deploys it to the website. Similarly, with a virtual machine you can use ARM to install various extension to provide things like anti virus protection, monitoring, or further configuration of the VM which we will discuss in the next section. Follow the link for the Azure Quick Start Templates page in the resources section to see examples of what can be achieved with Azure Resource Manager.
Below is an example of a ARM resource that deploys a website and downloads and applies an MS Deploy package for the content.
"resources": [
{
"apiVersion": "2014-06-01",
"type": "Microsoft.Web/serverfarms",
"name": "[parameters('hostingPlanName')]",
"location": "[resourceGroup().location]",
"properties": {
"name": "[parameters('hostingPlanName')]",
"sku": "[parameters('hostingPlanSku')]",
"workerSize": "0",
"numberOfWorkers": 1
}
},
{
"apiVersion": "2014-06-01",
"type": "Microsoft.Web/sites",
"name": "[parameters('siteName')]",
"location": "[resourceGroup().location]",
"tags": {
"environment": "test",
},
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', parameters('hostingPlanName'))]"
],
"properties": {
"name": "[parameters('siteName')]",
"serverFarm": "[parameters('hostingPlanName')]"
},
"resources": [
{
"apiVersion": "2014-06-01",
"type": "Extensions",
"name": "MSDeploy",
"properties": {
"packageUri": "https://storageaccount.blob.core.windows.net/packages/website.zip",
"dbType": "None",
"connectionString": "",
"setParameters": {
"Application Path": "[parameters('siteName')]"
}
}
}
]
}
]
It’s worth noting that ARM only works with V2 Azure resources, meaning V2 VMs and Storage, SQL V12, and so on. Some resources are not yet deployable using ARM such as Azure AD, Azure Service Bus, and Azure Remote App.
Powershell Desired State Configuration
Azure resource manager lets you deploy Azure resources and if you're solely deploying websites or SQL databases that may be enough. However, if you're deploying VMs, be aware that ARM won’t directly configure the VM operating system or install software; for this, you’ll need to use Azure Extensions. There are three Extensions that can be used to configure a VM and install software: Chef, Puppet, and PowerShell Desired State Configuration (DSC). We will focus on DSC for this article, but if you have existing Chef or Puppet deployments you can easily use these standalone or alongside DSC.
Where ARM allowed you to define Azure resources using JSON, DSC defines the configuration of your Windows VM resources using PowerShell. DSC provides hundreds of resources to configure Windows VMs as well as providing a way to write your own custom resources, or use custom scripts in your resources to do exactly what you want. Using the DSC extension then allows you to define the DSC file to be applied to any VM built using your ARM template.
Once you have defined your DSC files, Windows compiles these down to industry standard MOF files, and it's the MOF files that get applied to the VM. This means that you can easily integrate DSC with tools like Puppet and Chef if required and get the benefit of Windows specific resources.
Finally, the DSC engine will continuously apply the DSC configuration you have provided to prevent configuration drift and also apply any changes you make to the template over time. In this way your VM should remain in line with your configuration definition at all times. Windows will automatically test the machine against the configuration file at regular intervals and bring the machine back in-line if required. You can also manually trigger a test with the "Test-DSCConfiguration" command.
Below is an example PowerShell DSC configuration that will install IIS onto a VM, add the .net 4.5 role and then create a website.
configuration IISInstall
{
Import-DscResource -Module xWebAdministration
# Install the IIS role
WindowsFeature IIS
{
Ensure = "Present"
Name = "Web-Server"
}
# Install the ASP .NET 4.5 role
WindowsFeature AspNet45
{
Ensure = "Present"
Name = "Web-Asp-Net45"
}
# Stop the default website
xWebsite DefaultSite
{
Ensure = "Present"
Name = "Default Web Site"
State = "Stopped"
PhysicalPath = "C:\inetpub\wwwroot"
DependsOn = "[WindowsFeature]IIS"
}
# Copy the website content
File WebContent
{
Ensure = "Present"
SourcePath = "C:\Program Files\WindowsPowerShell\Modules\Website"
DestinationPath = "C:\inetpub\Website"
Recurse = $true
Type = "Directory"
DependsOn = "[WindowsFeature]AspNet45"
}
# Create a new website
xWebsite WebSite
{
Ensure = "Present"
Name = "Website"
State = "Started"
PhysicalPath = "C:\inetpub\Website"
DependsOn = "[File]WebContent"
}
}
Azure Automation
The combination of ARM and DSC is enough to go from zero to a fully configured and ready to use deployment; however triggering and maintaining the deployments is either a manual process, or will require an external orchestration tool. An alternative to this approach is to make use of Azure Automation.
By storing your ARM scripts in Azure Automation you can trigger deployments manually, on a schedule, or allow an external application or process to trigger them using webhooks. Azure Automation could allow you to build a self service way for your users to deploy and tear down their own environments.
Where Azure Automation really comes into its own is the recent update to use DSC and in particular to provide a DSC pull server. A pull server provides an alternative way to get your DSC files to the hosts. Instead of using the ARM script to apply the DSC extension and a particular DSC file to a host, the host registers with the DSC pull server and pulls its configuration from here. By using Azure Automation to act as a pull server you can maintain a centralised repository of DSC scripts which you can easily update and have all of your host pull the changes, rather than updating the scripts on each host. You can also easily see the status of each of your nodes and whether they are in compliance with your desired configuration.
Tips and Tricks
ARM and DSC are not without their issues and idiosyncrasies. Below are some tips when working with these technologies:
- Ensure you keep the version of the DSC Extension you are deploying in your ARM scripts up to date as new versions are released all the time, and older versions of the extension are occasionally disabled automatically. Consider using the "autoUpgradeMinorVersion" flag.
"properties": {
"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.8",
"autoUpgradeMinorVersion": "true"
…
}
If you are installing multiple VM extensions using ARM, configure your dependencies so that the extensions install one by one. Installing multiple extensions at the same time can cause errors in the deployment to occur.
ARM scripts can become large and difficult to navigate. Use Visual Studio and the built in JSON explorer to make navigation between objects and viewing hierarchy easier (requires the Azure SDK).
Keep your scripts and configurations in version control. This will let you keep a full history and audit trail of the changes made to configurations over time.
Bringing It All Together
We’ve discussed three different technologies in this article: ARM, DSC, and Azure Automation. Hopefully it’s clear now that each of these present part of the solution to an end to end deployment. Used together these solutions allow you build an automated solution to deploying and configuring resources in Azure and keeping these resources in line with your desired configuration throughout their lifetimes.
There is no one way that these tools can be used - they can be combined together to meet your needs. For example, if you're deploying a website with an SQL backend you’ll likely use ARM but never touch DSC. If you're already using tools like Chef and Puppet you may not use ARM, but you might choose to use DSC to give you the benefit of the Windows specific tools that still compile down to MOF files. If you're a System Centre user you may not need the automation and orchestration that Azure Automation provides, but still want to use the deployment tools available in ARM and DSC.
Hopefully this article has given you an insight into how these exciting new features in the Azure platform could help your deployment process become more agile and error free. Using these tools you can focus on "what" you want to deploy, and let PowerShell and Windows take care of “how”. Once we do that we can start reaping of benefits of more reusable, testable, economical and automated deployment processes.
If you're interested in finding out more about these technologies take a look at some of the links in the resources section, or drop me a line on email or Twitter.
Useful Resources
Azure Resource Manger Quick Start Templates - https://azure.microsoft.com/en-gb/documentation/templates/
PowerShell DSC Overview - https://msdn.microsoft.com/en-us/PowerShell/DSC/overview
PowerShell DSC Resources Gallery - https://www.powershellgallery.com/
Azure DS Extension - http://blogs.msdn.com/b/powershell/archive/2014/08/07/introducing-the-azure-powershell-dsc-desired-state-configuration-extension.aspx
Azure Automation and Powershell DSC - https://azure.microsoft.com/en-gb/documentation/articles/automation-dsc-overview/
Pester - PowerhShell Testing Framework - https://github.com/pester/Pester
Posts
No comments :
Post a Comment