Starting and Stopping Azure Virtual Machines in Parallel

The commands Start-AzureRmVM and Stop-AzureRmVM give us a simple way to start and stop VMs in Azure. The only thing is, they work synchronously, so we have to wait for the whole startup or deallocation process to finish before the command returns. If you have to start up anything more than a few virtual machines then using these commands is going to take a long time. If you are working at serious scale in the cloud and are starting up tens or even hundreds of VMs at a time then using these commands is not going to be a option. It would simply take too long to start or stop them one by one. What we need is a way to run these commands asynchronously and startup/shutdown many VMs in parallel.

I want to demonstrate a few ways we can use multi threading in Powershell to achieve this.

I previously posted a Powershell Multi Threaded Cookbook. Lets try and apply some of those methods to this problem.

Powershell Jobs

Powershell background jobs provide a generic mechanism to run scripts on a background thread. Here is my implementation of starting and stopping ARM VMs using background jobs.

The first thing to note is the parameter $ProfilePath. When a background job is run the background process will not be authenticated for Azure even if the main thread is. In other words, each background job has to authenticate into Azure itself. One way we can achieve this is by persisting our AzureRM profile to disk and then let the background job read it in. It seems a bit of a hack and it is. I believe there are plans to improve this scenario in Azure Powershell. Please see this thread for more details. See the section on Workflows for an alternative approach.

Once we have sorted out the authentication things go quite easily. Simply instantiate a background job for each VM you want to start or stop and pass in the correct arguments.

Finally we poll the jobs and wait for them to come out of the Running state. The usual rules around using Get-Job, Stop-Job and Receive-Job to manage Powershell background jobs will of course apply.

Note: this example spins up one background process for every VM you are starting/stopping.

Here is an example of how to call this command

Powershell Workflows

Workflows can give us another approach. Its Powershell but driven by Windows Workflow. Workflows provide out of the box support for parallel processing. Once again we need to authenticate to Azure in each background thread but this time we will pass a PSCredential into the script and use Add-AzureRmAccount to authenticate using this credential. The credential has to be a service principal or Azure Active Directory account. Add-AzureRmAccount does not work with LiveIds. Note the foreach -parallel call, that’s the out of the box support workflows give us for parallel processing.

Note: When using workflows and the -parallel switch we have no control on how many background processes Powershell spins up. That is handled for us behind the scenes.

Powershell Runspaces

Our third and final example uses runspaces to handle all the background threading. This method gives us the most control over the multithreading. I wrote about it in more detail in my Multithreading Cookbook.

Note how we can specify the number of background threads we want to use.

Wrapping Up

So that’s three very different ways to tackle the same problem. If you are working with classic VMs all the above scripts can be modified to use the classic cmdlets.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s