There are often times in the technical previews of Azure Stack where you will need to collect logs to send back to the product teams. Fortunately, in TP3 this previously tedious process has been consolidated into a single command, as per Charles Joy in the Azure Stack forums:

  • Command: Get-AzureStackLogs

    Instructions:

    1. From the Azure Stack POC HOST…
    2. Run the following to import the required PowerShell module:

      cd C:\CloudDeployment\AzureStackDiagnostics\Microsoft.AzureStack.Diagnostics.DataCollection
      Import-Module .\Microsoft.AzureStack.Diagnostics.DataCollection.psd1

    3. Run the Get-AzureStackLogs command, with optional parameters (examples below):

      # Example 1 : collect all logs for all roles
      Get-AzureStackLogs -OutputPath C:\AzureStackLogs

      # Example 2 : collect logs from BareMetal Role (this is the Role where DEPLOYMENT LOGS are collected)
      Get-AzureStackLogs -OutputPath C:\AzureStackLogs -FilterByRole BareMetal

      # Example 3 : collect logs from VirtualMachines and BareMetal Roles, with date filtering for log files for the past 8 hours
      Get-AzureStackLogs -OutputPath C:\AzureStackLogs -FilterByRole VirtualMachines,BareMetal -FromDate (Get-Date).AddHours(-8) -ToDate (Get-Date)

      # If FromDate and ToDate parameters are not specified, logs will be collected for the past 4 hours by default.

    Other Notes about the Command:

    • Note that the command is expected to take some time for log collection based on which roles logs are collected for and the time duration for log collection, and the numbers of nodes of the MASD environment.
    • After log collection completes, check the new folder created under the OutputPath specified in command input C:\AzureStackLogs in the examples above
    • A file with Name Get-AzureStackLogs_Output will be created in the folder containing the zip files, and will include the command output which can be used in troubleshooting any failures in log collection.
    • Each role will have the logs inside a zip file.

One of the wonderful new additions to Azure Stack in Technical Preview 3 is Marketplace Syndication.

The Azure Marketplace offers VM Images with pre-installed software/config, VM Extensions, SaaS Applications, Machine Learning services, and Data Services.

With Marketplace Syndication in TP3, we are now able to directly pull a subset of VM Images from Azure into Azure Stack for consumption by tenants. For anyone who built and deployed Gallery items in Azure Pack, this is just glorious.

The Public Azure Marketplace offers five pricing models:

 

  • BYOL model: Bring your own licence. You obtain outside of the Azure Marketplace the right to access or use the offering and are not charged Azure Marketplace fees for use of the offering in the Azure Marketplace.
  • Free: Free SKU. Customers are not charged Azure Marketplace fees for use of the offering.
  • Free Software Trial (try it now): Full-featured version of the offer that is promotionally free for a limited period of time. You will not be charged Azure Marketplace fees for use of the offering during a trial period. Upon expiration of the trial period, customers will automatically be charged based on the standard rates for use of the offering.
  • Usage-based: You are charged or billed based on the extent of your use of the offering. For Virtual Machines Images, you are charged an hourly Azure Marketplace fee. For Data Services, Developer services and APIs, you are charged per unit of measurement as defined by the offering.
  • Monthly Fee: You are charged or billed a fixed monthly fee for a subscription to the offering (from date of subscription start for that particular plan). The monthly fee is not prorated for mid-month cancellations or unused services.
    Offer specific pricing details can be found on the solution details page on /en-gb/marketplace/ or within the Microsoft Azure classic portal.

As of just now in TP3, BYOL is the only model available, and only for a small subset of offerings. That doesn’t matter though, we’re just proving the concept just now, so that being the case, enabling Marketplace Management was the very first thing I did once I’d fired up my TP3 portal.

 

Registering the Resource Provider

When you click through to the Marketplace Management resource provider, it presents you with a link to follow in order to register and activate the resource provider. It needs to be registered against an existing Public Azure subscription in order to pull marketplace items down from hyperscale to on-prem.

Marketplace Management + Add from Azure NAME PUBLISHER TYPE VERSIO... STATUS You need to register and activate before you can start syndicating Azure Marketplace content. Follow instructions here to register and acitivate

The documentation to do this is available at the following link:

https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-register

A PowerShell script is required in order to register the resource provider, available from GitHub:

https://github.com/Azure/AzureStack-Tools/blob/master/Registration/RegisterWithAzure.ps1

And of course you need the AuzreRM PowerShell module installed, via Install-Module AzureRM

When registering the RP you are prompted for an Azure subscription, and an Azure username and password. This can be a completely separate subscription and username to the one used for Azure Stack deployment. It cannot, however, be a CSP subscription.

Run the script to completion…

Administrator: Windows PowerSheII ISE Eile Edit yew Tools Debug Add-ons Help RegisterWithAzure.ps1 X Ln256 Col 130 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 38 39 The script will follow four steps : Confi gure local identity: confi gures local Azure Stack so that it can call to Azure via your Azure subscription Get regi stration request: get local environment informatnon to create a registration for this azure stack in azure Register with Azure: uses Azure powershell to create an "Azure Stack Registratnon" resource on your Azure subscription Activate Azure Stack: final step in connecting Azure Stack to be able to call out to Azure . PARAMETER azuresubscriptionld Azure subscri ption ID that you want to regi ster your Azure Stack with. This parameter is mandatory. . PARAMETER azureDi rectory Name of your AAD Tenant which your Azure subscri ption is a part of. This parameter is mandatory. . PARAMETER azureSubscriptionOwner Username for the owner of the azure subscription. This user must not be an MSA or 2FA account. This parameter is mandatory. . PARAMETER azureSubscriptionPassword Password for the Azure subscription. You will be prompted to type in this password. This parameter is mandatory. . PARAMETER marketpl aceSyndi cati on Flag (ON/OFF) whether to enable downloading items from the Azure marketplace on this environment. Defaults to "ON" . V EKBU5E : VERBOSE: VERBOSE: STARTING : VERBOSE: STARTING : VERBOSE : VERBOSE: WARNING: STARTING : VERBOSE: VERBOSE : VERBOSE : VERBOSE: VERBOSE: VERBOSE: yrogress T 1 1 e: yrogress . new«egl srr at onxequesr. xm 1 Log file: • strationRequest. 2017-03—01. 23—08—49.0. log Invoking action : NewRegi strationRequest Action NewRegi strationRequest ' Action: Running action plan 'NewRegistrationRequest' . Step I - Create regi stration request - 3/1/2017 PM Step: Running step 1 — Create regi stration request — 3/1/2017 11:08:49 PM Task: Running interface • NewRegistrationRequest• of role Attempt #1. - 3/1/2017 PM Task: Interface 'NewRegi strationRequest' is not a supported standard Interface. - 3/1/2017 PM Task - NewRegistrati onRequest Interface: Path to module: psm1 — 3/1/2017 11:08 PM Interface: Running interface NewRegistrationRequest psml, AzureBridge:NewRegistrationRequest) Runtime parameters are present, will use provided Bridge App configuration file — 3/1/2017 11:08:52 PM Bridge appliation object id : 22280345—84f4—469e-80dO-dcbb7f2ec91b - 3/1/2017 11 :08:52 Performing the operation "Create File" on target "Destination: json" - 3/1/2017 Registration request contents: { "Br i dgeAp$-onfi gOi d" "22280345-84f4-469e-80do-dcbb7f2ec91b" , ' RegionNames • " local " , "Identi tyProvi d er " : "Azur eAD" } - 3/1/2017 PM "Servi cePri ncipal Name" . azurestack. local /b20ffdea—f632—433c-b39d-6ba972192cac" , "Depl oymentldent i fi er" : • b20ffdea—f632 , "https : // azur e. - 3/1/2017 VERBOSE : VERBOSE : COMPL ETE : VERBOSE: COMPLETE : VERBOSE: VERBOSE : COMPL : VERBOSE: Registration request output file is at : json - 3/1/2017 11 Interface: Interface NewRegi strati onRequest completed. - 3/1/2017 11:08:52 PM Task - NewRegistrati onRequest Task: Task compl eted. - 3/1/2017 PM Step 1 - Create regi stration request Step: Status of step 'I — Create registration request' is 'Success' 3/1/2017 PM Action: Action plan 'NewRegistrati onRequest' completed. — 3/1/2017 11:08:52 PM Action NewRegi strationRequest ' New—RegistrationRequest. PSI : END on AS-HOST as AZURESTACK\azurestackadmin STEP 2: Registration request completed. Re—enter your Azure subscription credentials in the Running script/ selection. Press Ctrl* Break to stop. Press Ctrl—B to break into debugger, next step. Press Enter to conti nue. . 100%

' i dgeServi ce. Partiti onConnecti onStr i ng ZB/fs r oynNTJXm1 YBSW40; Pool i ng=Fa1 se" , 'Data Catalog—Microsoft. AzureStack.Connerce;User 'TokenRetri ever . Certifi cateThumbpri nt" : '9794C97694AF2B26ßFE264C14D39D9D2A5571838" , 'TokenRetri ever . Cl i entld" : ' ce8c5ed0-745 a-448c-82c1-117c62f7c348" , 'UsageUpIoader . GatewayUri " " https : //azstusage. tr affi cmanager. net/usage" , 8r i dgeJ obRunn er . Stor ageC1i entld" : " 6b5b2d0522f449d1b021afS35 30294cf" 'TokenRetri ever . ResourceUri • • 'https : //mi crosoft. onmi crosoft. com/azurestackusage" 3/1/2017 11:11 PM VERBOSE: VERBOSE: VERBOSE : VERBOSE : VERBOSE : VERBOSE: COMPL ETE : VERBOSE: COMPL ETE : VERBOSE : VERBOSE : COMPL ETE : VERBOSE: Creating remote power-shell session on MAS—WASOI - 3/1/2017 11 : 11:41 PM Initializing remote powershell session on MAS—WASOI with common functions. - 3/1/2017 11:11 PM Loading infra vm helpers PSI) to session on MAS-WASOI — 3/1/2017 11:11:41 PM Invoking command on remote session.. - 3/1/2017 11:11:41 PM Removing remote power-shell session on MAS—WASOI. 3/1/2017 PM Interface: Interface Configure completed. — 3/1/2017 11 PM Task cRi ngServi ces\UsageBri dge — Confi gure Task: Task compl eted. - 3/1/2017 11:11:42 Step 1 - Configure Usage Bridge Step: Status of step 'I — Configure Usage Bri dgeT is 'Success - 3/1/2017 11:11 PM Action: Acti on plan 'ConfigureUsageBridge' completed. - 3/1/2017 11:11 PM Action 'Confi gureUsage8ridge Activate—Bridge.psI : END on AS-HOST as AZURESTACK\azurestackadmin STEP 4: Activate Azure Stack compl eted Registration compl ete. You may now access Marketpl ace Management in the Admin UI PS dge»

… and all should be well! You can now refresh the Marketplace Management resource provider, to be presented with a new message and an ‘Add from Azure’ button. Yay!

Marketplace Management + Add from Azure NAME PUBLISHER TYPE VERSIO... STATUS You have no items downloaded to your Azure Stack marketplace yet. Click "Add from Azure" to add items.

The available list is currently quite small, but pretty much everything is useful, so kudos on the choices there Microsoft!

Simply select what you want to bring into your Azure Stack, and click download. One thing I noticed is that the transfers were pretty slow, even on our ridiculously fast connections. Pulling down a handful of gallery images had to be left running overnight.

Microsoft Azure Stack V Marketplace Management VERSIO... STATUS > Add from Azure asadmin@brightsolid.... >< Add from Azure NAME TYPE lick " Add from Azure" to add items. O (ț) (â) (Â) Remote Desktop Services (RDS) Basic Farm SQL Server 2014 SPI Express on Windows Server ; SQL Server 2016 RTM Developer on Windows Ser GitLab LAMP Magento Moodle Nginx ownCIoud Redmine Ruby WordPress Drupal PUBLISHER Microsoft Microsoft Microsoft Bitnami Bitnami Bitnami Bitnami Bitnami Bitnami Bitnami Bitnami Bitnami Bitnami TYPE Virtual Virtual Virtual Virtual Virtual Virtual Virtual Virtual Virtual Virtual Virtual Virtual Virtual Machine Machine M achine Ma chine Ma chine Ma chine Machine Machine Machine Machine Machine Machine Machine VERSIO... 1.0.0 1.0.0 1.0.0 8.9.61 5.6.270 2.1.20 3.1.22 1.10.14 9.1.11 3.3.10 2.3.15 4.6.14 8.1.90 SIZE 8.5G 18.3G 23.4G 30.OG 30.OG 30.OG 30.OG 30.OG 30.OG 30.OG 30.OG 30.OG 30.OG

Downloading… waiting… downloading…

Microsoft Azure Stack v Marketplace Management > Add from Azure > Drupal > Word Press c Marketplace Management + Add from Azure Search to filter items... NAME (4) Ruby WordPress Drupal PUBLISHER Bitnami Bitnami Bitnami TYPE Virtual Machine Virtual Machine Virtual Machine VERSIO... 2.3.15 4.6.14 8.1.90 STATUS Downloadi... Downloadi... Downloadi...

Five wonderful marketplace items added and ready for tenant consumption! Amazing.
Microsoft Azure Stack v Marketplace Management Marketplace Management Add from Azure Search to filter items„. LAMP Ruby Word Press e Remote Desktop Services (ROS) Basic Farm SQL Sewer 2014 SPI Express on Windows Server PUBLISHER Bitnami Bitnami Bitnami M icrosoft M icrosoft Virtual Machine Virtual Machine Virtual Machine Virtual Machine Virtual Machine VERSIO... 5.6270 2.3.15 4614 1.00 1.00 STATUS Succeeded Succeeded Succeeded Succeeded Succeeded

Tenants can now select these Marketplace items, and deploy them immediately. This is such a leap forward from Azure Pack, and I feel such joy in using this feature. How important this is cannot be overstated.

Microsoft Azure Stack New p Search the marketplace MARKETPLACE Tenant Offers + Plans Virtual Machines Data + Storage Networking Custom Security + Identity Developer Services Web + Mobile Management Media + CDN RECENT New > Media + CDN See all Media + CDN FEATURED APPS WordPress ee al The most popular and ready-to-go

… and here we are! A WordPress VM deployed using an image from Public Azure, all controlled and managed from within the Azure Stack web UI – no PowerShell, not building VM images, all just so simple. Phenomenal.
Microsoft Azure Stack WPI Search (CtH+,9 Ove rview Activty log Access control (IAM) Tags SETTINGS WPI Start Connect Essentials Resource group (c'.ge) wp-dev-rg Running Location Dundee Subscription name (change) Default Provider Subscription Subscription ID Restart Stop Delete ace2128b-1464-4gc2-8b1 d -oggc2ba420bd Computer name Operating system Linux Standard Al (1 core, 1.75 G8 memory) Public IP address/DNS name label 192.168.102.13/«none» Virtual network/subnet wp-dev-rg-vnet/default

This is a bit of a non-blog, as the TP3 deployment experience is utterly joyous. It deployed first try for me, taking around four hours from start to completion, with no errors logged. I happened to screenshot the process, so here it is in all its glory 🙂

In order to deploy the PoC, I followed the documentation at https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-deploy

Download and run the PoC Downloader. It’s highly recommended that you tick the ‘Download the Windows Server 2016 EVAL (ISO)’ box so you can get something added to the marketplace once it’s deployed.

-Z Azure Stack POC Downloader Download the Azure Stack Proof of Concept (POC) Notice Privacy Notice Microsoft automatically collects usage and performance data over the internet ("Data"). This feature is on by default. You can turn off this feature by following the instructions http://go_microsott.com/fwIink/?LinkID=699582_ For more information about Data collection please see Microsoft Azure Stack POC Privacy Statement at http://go.microsoft.com/fwIink/?LinkID=692023_ 1. Read the deployment prerequisites 2. Choose a build @ Technical Preview (release build) version 20170225.2 C) Technical Preview (in-development build) What's the difference? 3. Optional: Windows Server 2016 EVAL Download the Windows Server 2016 EVAL (ISO) Once Azure Stock has been deployed, this ISO can be used to add an image to the Azure Stack Marketplace. Note: This Windows Server Evaluation image may only be used with Microsoft Azure Stack Proof of Concept and is subject to the Microsoft Azure Stack Proof of Concept license terms. 4. Browse to where you want to save the build Space required: 15.07 Ga Download 5. Cancel

The PoC Downloader will download the PoC.

Azure Stack POC Downloader Download the Azure Stack Proof of Concept (POC) The download is in progress. Downloading Microsoft Azure Stack POC (release build) version 20170225.2. Transferred: 9.27 GB Remaining: 5.8 Ga Estimated time left: Cancel Details... Privacy Notice Pause

You will need to have at least 85GB of storage to extract the downloaded files, which you can extract by clicking the ‘Run’ button once download has completed.

Azure Stack POC Downloader Download the Azure Stack Proof of Concept (POC) Privacy Notice The download has completed and was saved here. Close the window to exit, or click Run to run the Azure Stack POC self-extractor. The Azure Stack POC self-extractor will guide you through the next steps of the installation process. Run

The extractor extracts a VHDX file from which we will boot and run a whole Azure Stack environment. One file to worry about – so simple, even I can’t mess it up.

Setup - Microsoft Azure Stack POC Please vvait while Setup extracts Microsoft Azure Stack POC on your computer. Extracting files. C : user a tor DesktopVvIicr o so ft Azure Stack POC CloudBuiIder. vhdx

Setup - Microsoft Azure Stack POC Completing the Microsoft Azure Stack POC Setup Wizard Setup has finished extracting Microsoft Azure Stack POC on your compu ter Click Finish to exit Setup.

Once extracted, copy the CloudBuilder.vhdx file to the root of your host’s C: drive.

The documented PowerShell below will download preparatory files you need, so blindly copy it into PowerShell ISE and run it 🙂

 

# Variables
$Uri = 'https://raw.githubusercontent.com/Azure/AzureStack-Tools/master/Deployment/'
$LocalPath = 'c:\AzureStack_SupportFiles'

# Create folder
New-Item $LocalPath -type directory

# Download files
( 'BootMenuNoKVM.ps1', 'PrepareBootFromVHD.ps1', 'Unattend.xml', 'unattend_NoKVM.xml') | foreach { Invoke-WebRequest ($uri + $_) -OutFile ($LocalPath + '\' + $_) }

The next step is the same as TP2, run the PrepareBootFromVHD PowerShell script to set the BCDBoot entry to allow the host to reboot into the CloudBuilder VHDX. Apply an Unattend file if you don’t have console access to the host. Or don’t, I’m not your boss.

Administrator: Windows PowerSheII PS C: DIR Di rectory: C: \ Azu restack_SupportFi les Mode LastWriteTime 01/03/2017 01/03/2017 01/03/2017 01/03/2017 17 17 17 17 Length 2611 8698 1684 3788 Name BootMenuNoKVM. PSI PrepareBootF romVHD. PSI Unattend . xml unattend NOKVM. xml PS . \prepareBootFromVHD. PSI -CloudBui1derDi skpath "c: oudBui1der. vhdx" -Applyunattend password for the local administrator account of the Azure Stack host. g password will be configured for the local administrator Requi res 6 characters minimum: account of the Azure Stack host: rea Ing new boot entry for Cloud8ui1der. vhdx Running command: bcdboot I : \windows Boot files successfully created. updating descri ption for the boot entry Descri pti on: Wi ndows Server 2016 Devi ce: partiti on—I: bcdedit /set " {default}" description "Azure Stack" The operation completed successfully. Confi rm Are you sure you want to perform this action? performing the operation "Enable the Local shutdown (AS-Base) " [Y] Yes [A] Yes to All [N] NO [L] NO to All access rights and Suspend [?] Help restart the (default is computer. " on target "local host

Once you’ve rebooted into the CloudBuilder VHDX and logged in using the password you provided when applying the Unattend file, run through the same steps as you would have in TP2.

If not using DHCP, set a static IP on the host.

If you’re anywhere other than UTC-8, set a time server.

Rename the host.

Reboot.

Disable all NICs other than the NIC that provides internet connectivity.

Actually I haven’t validated the last step – it was necessary in TP1 and TP2, but I’m pretty certain I saw the deployment script checking for the correct NIC to use while it was installing. Let’s check…

326 327 328 329 330 331 Car ray) Sn etwor kConfi gur ati on if (Sn etwor kConfi gur ati on . Count Get - NetlPConfi gur ation 1) -gt . NetAdapter . Status - EQ thr ow SLocaI i zedData. Mor eThanOneNi cEnabI ed

Yep, DeploySingleNode.ps1, lines 326 to 331 – only one NIC is allowed to be enabled still, so let’s disable all the other NICs.

Network Connections Control Panel Organize NICI Disabled Network and Internet Disabled Network Connections Search Network Connecti Network Intel(R) Gigabit X520/13iO rNDC NIC4 Disabled Intel(R) Gigabit X520/13iO rND... Intel(R) Ethernet IOG4PX520/13SO.„ SLOT 2 Di sabled Mellanox ConnectX-3 Pro Etherne... Intel(R) Ethernet IOG4PX520/13SO... SLOT 2 2 Disabled Mellanox ConnectX-3 Pro Etherne...

Ok! So in this environment I’ve not got DHCP available so we need to set a Static IP, for this lab I’m using 10.20.39.124. Here are the steps to kick off deployment from an elevated PowerShell window. NOTE: Do not use PowerShell ISE for this – if you do, it may lead to fuckery.

cd C:\CloudDeployment\Configuration

$adminpass = ConvertTo-SecureString “Local Admin Password” -AsPlainText -Force

$aadpass = ConvertTo-SecureString “Azure AD Account Password” -AsPlainText -Force

$aadcred = New-Object System.Management.Automation.PSCredential (“AAD account email address“, $aadpass)

.\InstallAzureStackPOC.ps1 -AdminPassword $adminpass -InfraAzureDirectoryTenantAdminCredential $aadcred -NatIPv4Subnet 10.20.39.0/24 -NatIPv4Address 10.20.39.124 -NatIPv4DefaultGateway 10.20.39.1 -TimeServer sometimeserver

This is a slight change from TP2, with -AADCredential being renamed to -InfraAzureDirectoryTenantAdminCredential, which just rolls off the tongue :/

Deployment kicks off, and you pretty much wait for four hours. This is also a slight change from TP1 and TP2, with the ‘Cross Fingers and Pray to the Old Gods and the New’ step now being notably absent as everything just works.

Administrator: Windows PowerSheII RARNING: T e names o some 1 mporte comman s rom t e 110 u •e Fa r 1 cR1ngApp11cat10ns Inc •u e unapprove ver s t at ight make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with Action ' Depl oyment ' Running action plan.. Phase O - Confi gure physical machine and external networking Step O - Deploy and confi gure physical machines, 8GP and NAT Task Cloud - Depl oyment- Phas eo- Depl oy8ar etal And8GPAndNAT Running action Action ' Depl oyment-PhaseO-DepI And8GPAndNAT ' Running action plan. 1000000000000 (DE?) Validate Physical Machi nes step 0.12 - Validating the hardware and OS confi guration on the physical nodes . Task CI - Validate Running interface t e Ver ose parameter. For a 1 st approve ver s , type Get-Ver . 3 1 2017 ARNING: The names of some imported commands from the module 'ACS8cdr' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module connand again with the Verbose parameter. For a list of approved verbs, type Get-Verb. - 3/1/2017 ARNING: The names of some imported commands from the module 'ACS' include unapproved verbs that might make then less discoverable. To find the connands with unapproved verbs, run the Import-Module conmand again with the Verbose parameter. For a list of approved verbs, type Get-Verb. - 3/1/2017 ARNING: The names of some imported connands from the module '800tArtifactsHeIper' include unapproved verbs that might ake them less discoverable. To find the commands with unapproved verbs, run the Import-Module connand again with the erbose parameter. For a list of approved verbs, type Get-Verb. — 3/1/2017 5 : 44: 46 ARNING: The names of some imported commands from the module 'JeaHeIper' include unapproved verbs that might make them less discoverable. To find the connands with unapproved verbs, run the Import-Module conmand again with the Verbose parameter. For a list of approved verbs, type Get-Verb. — 3/1/2017 5 : 44: 46 ARNING: The names of some imported connands from the module 'UpdatePhysicaIMachineHeIper' include unapproved verbs hat might make then less discoverable. To find the commands with unapproved verbs, run the Import-Module connand again with the Verbose parameter. For a list of approved verbs, type Get-Verb. — 3/1/2017 5 : 44: 46 ARNING: The names of some imported commands from the module 'UpdateNC' include unapproved verbs that might make them less discoverable. To find the connands with unapproved verbs, run the Import-Module conmand again with the Verbose parameter. For a list of approved verbs, type Get-Verb. — 3/1/2017 5 : 44: 46 ARNING: The names of some imported connands from the module 'PhysicalMachines' include unapproved verbs that might ake them less discoverable. To find the commands with unapproved verbs, run the Import-Module connand again with the erbose parameter. For a list of approved verbs, type Get-Verb. — 3/1/2017 5 : 44: 46 ER80SE : ER80SE : ER80SE : ER80SE : ARNING: ER80SE : ER80SE : Setting IP addresses. — 3/1/2017 5 : 44: 46 Normalizing MAC address '24-6E-96-02-46-2C' . — 3/1/2017 5 : 44: 46 Choosing an address from 'Management' in range '192.168. 200. 65/24' . — 3/1/2017 5 : 44: 46 Find out which NICs are able to connect on each node. - 3/1/2017 Ping to 192.168.100.4 failed Status: TimedOut - 3/1/2017 PM - AS-HOST storagel - 3/1/2017 PM + AS-FOST Deployment - 3/1/2017 5:45 PM
Action ' Depl oyment ' Running action plan. . Step O - phase O - Configure physical machine and external networking Deploy and configure physical machines, BGP and NAT Task Cloud Dep 1 oyment -phas eO-Dep 1 oyBareMeta1 AndBGPAndNAT Running action Action ' Depl oyment-phaseO-Dep1 oyBareMeta1AndBGPAndNAT ' Running action plan. Coooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo Step 0.20 (S TO) Configure Storage Cluster Create storage cluster, create a storage pool and file server. Task Cloud \ Infrastructure\storage Depl oyment Runni ng NARNING: The names of NARNING: The names of Import-Module command NARNING: The names of NARNING: The names of NARNING: The names of NARNING: The names of Import-Module command NARNING: The names of Import-Module command NARNING: The names of NARNING: The names of interface some imported commands from the module 'FabricRingApp1ications ' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 PM some imported commands from the module 'ACSB10b ' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 PM some imported commands from the module 'FabricRingApp1ications ' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 6:27 PM some imported commands from the module 'FabricRingApp1ications ' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 6:27 PM some imported commands from the module 'FabricRingApp1ications ' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 6:27 PM some imported commands from the module 'ACSBcdr' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 6:27 PM some imported commands from the module 'ACS' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 6:27 PM some imported commands from the module 'BootArtifactsHe1per' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 PM some imported commands from the module 'JeaHe1per' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 6:27 PM NARNING: The names of some imported commands from the module 'Updatephysica1MachineHe1per' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 6:27 PM NARNING: The names of some imported commands from the module 'UpdateNC' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 6:27 PM NARNING: The names of some imported commands from the module 'physicalMachines ' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs , run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-verb. 3/1/2017 6:27 PM COMPL : COMPL : STARTING : STARTING : NARNING: NARNING: NARNING: COMPL : COMPL : STARTING : STARTING : COMPL : COMPL : STARTING : STARTING : NARNING: NARNING: NARNING: Task Confi gure Step 0.17 (DEP) Confi gure physi cal Machi ne Step 0.18 - (DEP) Confi gure physi cal Machi ne Task [AS-HOST] : [AS-HOST] : [AS-HOST] : Task updateHostComponent C CPDTVMSwi tch] publ i cswi tch] C CPDTVMSwi tch] publ i cswi tch] C CPDTVMSwi tch] publ i cswi tch] updateHostComponent Net Net Net adapters adapters adapters are are are down : down : down : 3/1/2017 PM 3/1/2017 6: PM 3/1/2017 PM Step 0.18 - (DEP) Confi gure physi cal Machi ne Step 0.19 - (FBI) Confi gure powershell JEA for Storage. Task Cloud \ Fabri c\JEA — Confi gure Task - Confi gure Step 0.19 - (FBI) Confi gure powershell JEA for Storage. Step 0.20 - (S TO) Confi gure Storage Cluster Task Depl oyment Cluster validation completed, but had a few tests either unselected/ cancelled/ deemed not applicable. Refer to the validation report for more information 3/1/2017 PM There were issues while creating the clustered role that may prevent it from starting. For more information view the report file below. 3/1/2017 PM Report file location: C: Cluster Wizard S—cluster on 2017 .03. 01 At 18. 33. 06. htm 3/1/2017 PM
Administrator: Windows PowerSheII VERBOSE : Inter ace: Runmng 1 nter ace Ml grate c asses ECEsee RI nq ECEsee RI ng .psml, ECEsee R 1 rig : Ml grate 3 1 2017 PM ARNING: The names of some imported commands from the module 'ECE incl ude unapproved verbs that mi ght make them less di scoverable. Import-Module command again with the verbose parameter. For a list of approved verbs, type Get-verb. - 3/1/2017 PM TO find the commands with unapproved verbs , run the ERBOSE : ERBOSE : ERBOSE : ERBOSE : ERBOSE : ERBOSE : ERBOSE : ERBOSE : ERBOSE : ERBOSE : ARNING : ERBOSE : ERBOSE: ERBOSE : ERBOSE : ARNING : ERBOSE: ERBOSE : ERBOSE : ERBOSE : ERBOSE: ERBOSE : ERBOSE : ERBOSE : ERBOSE: ERBOSE : ERBOSE : ERBOSE : ERBOSE: ERBOSE : ERBOSE : ERBOSE : ERBOSE: COMPLETE : ERBOSE : COMPLETE : ERBOSE : STARTING : ERBOSE • ERBOSE : STARTING : ERBOSE : ERBOSE : ERBOSE : ERBOSE : ERBOSE : Loading module from path oyment\ECEngi oudEngine.cmd1ets.d11 ' - 3/1/2017 PM Importing cmdlet 'set-Ecesecret' . - 3/1/2017 PM Importing cmdlet 'Get-Ececonfi guration' . - 3/1/2017 PM Importing cmdlet 'Get-Actionprogress ' - 3/1/2017 PM Importing cmdlet 'Get-JsonTemp1ate' . - 3/1/2017 PM Importing cmdlet 'Invoke-EceAction ' - 3/1/2017 PM Importing cmdlet ' Joi n-R01eTemp1ate ' - 3/1/2017 PM Importing cmdlet 'Import-Ececustome rconfiguration ' - 3/1/2017 PM - 3/1/2017 PM Importing cmdlet 'set-RoleDefinition ' Attempting to retrieve BareMeta1Admin credential ... - 3/1/2017 10:24:55 PM unable to retri eve BareMeta1Admin credential . It may not exist. - 3/1/2017 PM Exception calling "Getcredential " with "1" argument (s) : contains no elements' sequence Attempting to retrieve AADAdmin credential .. - 3/1/2017 Attempting to retrieve AADAzureT0ken credential ... - 3/1/2017 PM Attempting to retrieve AzureBri dgeuser credential ... - 3/1/2017 PM unable to retri eve AzureBridgeuser credential . It may not exist. - 3/1/2017 10:24:55 PM Exception calling "Getcredential " with "1" argument (s) : sequence contains no elements' Attempting to retrieve LocalAdmin credential ... - 3/1/2017 10:24:55 PM Attempting to retrieve MgmtLoca1Admin credential ... - 3/1/2017 10:24:55 PM Attempting to retrieve CAcertifi cateuser credential . - 3/1/2017 10:24:55 PM - 3)i/2017 10:24:55 PM Attempting to retrieve Domai nAdmin credential ... Attempting to retrieve Fabric credential ... - 3/1/2017 10:24:55 PM Attempting to retrieve sq 1 service credential ... - 3/1/2017 10:24:55 PM Migrating cloudDefinition to ECEservi ce... - 3/1/2017 10:24:55 PM - 3/1/2017 PM - 3/1/2017 10:24:55 PM Initializing remote powershell session on MAS-ERCS01.AzureStack.Loca1 with common functions. - 3/1/2017 10:24:57 PM Loading infra vm helpers . PSI) to session on MAS-ERCS01.AzureStack.Loca1 - 3/1/2017 10:24:57 PM Migration of cl oudDefinition to ECEservnce completed successfully. - 3/1/2017 10:25:10 PM Migrating ECELite to AD VMS. . . - 3/1/2017 10:25:10 PM copying cloudDep10yment Files to AD VM... - 3/1/2017 10:25:10 PM comed cloudDep10yment Files to AD VM. - 3/1/2017 10:25:31 PM Hydrating ECELite with runtime values... - 3/1/2017 10:25:31 PM Migration of ECELite to AD VMS completed succesfully! - 3/1/2017 10:25:33 PM Interface: Interface Mi grate completed. - 3/1/2017 10:25:33 PM Task cl oud\Fabri c\seedRi ngservi ces\ECEseedRi ng Mi grate Task: Task completed. - 3/1/2017 10:25:33 PM PHASE. 3.1 -C FBI) Migrate confi guration to ECE service on seedRing step 241 - step: Status of step '241 - PHASE. 3.1 -(FBI) Migrate configuration to ECE service on seedRing' is 'success ' prepare for future host reboots step 251 - prepare for future host reboots - 3/1/2017 10:25:33 PM Running step 251 - - 3/1/2017 10:25:33 PM Running interface 'startup' of role 'cl . Attempt #1. Task cl - startup Interface: path to module: C: psml - 3/1/2017 10:25:33 PM Interface: Running interface startup psml, poc:startup) - 3/1/2017 10:25:33 PM Deleting onstartup scheduled task. - 3/1/2017 10:25:42 PM - 3/1/2017 10:25:33 PM setting restart callback as: Import—Module c: oudDepI oyment\ECEngi ne\Ente rpri secl oudEngi ne . psdI Invoke-EceAction -Rolepath Cloud -ActionType Startup —verbose -ErrorAction conti nue Invoke-EceAction -Rolepath Cloud -Action Type Startup —verbose -ErrorAction conti nue Invoke-EceAction -Rolepath Cloud -ActionType Startup —verbose -ErrorAction conti nue ' 3/1/2017 10:25:43 PM Reaistering the callback for powershell . exe with argument: '-Executionpolicy Remotesigned -NOEXit -command Import-Module .\CIoudDepIoyment .psdl Import-M0duTe c: oudDepI pers . psmI Import-Module c: oudDepI oyment\ECEngi ne\Enterpri secl oudEngi ne . psdI Invoke-EceAction -Rolepath Cloud -Act non Type Startup —verbose -ErrorAction conti nue Invoke-EceAction -Rolepath Cloud -ActionType Startup —verbose -ErrorAction conti nue Invoke-EceAction -Rolepath Cloud -ActionType Startup —verbose -ErrorAction conti nue ' Registering the scheduled task named ' coldstartMachine under the user ' AzurestackAdmin' . ERBOSE : ERBOSE : COMPLETE : ERBOSE : COMPLETE : ERBOSE : ERBOSE : COMPLETE : - 3/1/2017 10:25:43 PM - 3/1/2017 10:25:43 PM Interface: Interface Startup completed . - 3/1/2017 10:25:43 PM Task cl - startup Task: Task completed. - 3/1/2017 10:25:43 PM prepare for future host reboots step 251 - Step: Status of step '251 — prepare for future host reboots' is 'success ' - 3/1/2017 10:25:43 PM Action: Action plan Action ' Depl oyment ' ' Depl oyment' compl eted . - 3/1/2017 10:25:43 PM ps c: oudDep1 PS c: oudDepI ps c: oudDepI PS c: oudDepI 10:27 PM 3/1/2017

Change the default password expiry to 180 days as per the documentation:

 

Set-ADDefaultDomainPasswordPolicy -MaxPasswordAge 180.00:00:00 -Identity azurestack.local
And that's it! Azure Stack TP3 deployed and ready to rock and roll!

Edit dashboard New Region Management Plans U pd ates Provider Settings Locations Marketplace Subscriptions Resource Explorer Portal settings Virtual machine scale s... Availability sets Storage accounts Images Marketplace Managem... Import/export jobs Snapshots More services > Dashboard V + New dashboard Get started Share Fullscreen Clone Updates Delete Feedback Marketplace Alerts Critical Warning Virtual Machines Provision Windows and Linux virtual machines in minutes App Service Create web and mobile apps for any platform and device SQL Database Managed relational database-as-a-service Storage Durable, highly available and massively scalable storage Resource Providers NAME Key Vault Network Capacity Storage Updates Compute STATUS SQL Region Management f) REGION local CRITICAL WARNING CURRENT VERSION 1.0.170225.2 LAST CHECKED PM STATE U pToDate ALERTS Unknown Unknown Unknown Unknown 10:54 PM 3/1/2017

 

 

Well Azure Stack TP3 has landed, and with it a whole host of excitement and capability! Jeffrey Snover has laid out the improvements and roadmap for TP3 through to GA in this blog, so I thought I’d note some thoughts about a few of the listed capabilities while my CloudBuilder VHDX extracts 🙂

What’s new in Azure Stack TP3

  • Deploy with ADFS for disconnected scenarios

I wonder if the only use case here is in disconnected scenarios as noted, or rather also perhaps for scenarios where the customer is comfortable with their Azure Stack having internet access, but not with their identity being synchronised to Azure Active Directory (AAD).

  • Start using Azure Virtual Machine Scale Sets for scale out workloads

These are self-explanatory and I look forward to trying this functionality – the ability to scale a set of identical IaaS VMs in or out with ease, just like Azure.

  • Syndicate content from the Azure Marketplace to make available in Azure Stack

This is hugely exciting to me, even if it is just limited to Bring Your Own License (BYOL) scenarios at this time. One of the major pains of Azure Pack was having to custom build all of the gallery items you wanted to deploy to your tenants. Now with marketplace syndication we can pull relevant gallery items directly from Azure into our Azure stack environments and make them available to our tenants. For me, this is the biggest new feature in TP3 😀

  • Use Azure D-Series VM sizes

But probably not the higher spec ones for those using 96GB or 128GB RAM hosts… 🙂 I should be able to deploy up to Standard_D14 in size, all things being equal.

  • Deploy and create templates with Temp Disks that are consistent with Azure

This is interesting, as in Azure a temp disk is non-permanent and exists on the same host as the VM, rather than in Azure Storage. If the host reboots, the contents of the temp disk are lost. In Azure Stack the underlying storage is hyperconverged Storage Spaces Direct rather than Azure Storage. Performance of temporary disks typically has better IOPS and latency than data disks, this is presumably not the case in Azure Stack though as it’s all the same storage on the same hosts. Will temp disk storage be wiped if a host reboots? Are the use cases the same as in Azure? Or is this purely a consistency item? Stay tuned and find out!

  • Take comfort in the enhanced security of an isolated administrator portal

I feel the comfort like a warm fuzzy blanket enveloping me, letting us split out the tenant and admin portals into separate security zones in the same way as Azure Pack 🙂

  • Take advantage of improvements to IaaS and PaaS functionality

In the short term I’ll be spending the next few days testing and documenting changes/improvements in the IaaS portion of TP3, and look forward to the PaaS services landing so I can go through them with rigor as well!

  • Use enhanced infrastructure management functionality, such as improved alerting

Alerting and monitoring are very important, I’d like to be able to gather data into any or all of SCOM, OMS, Power BI, or Grafana. I’m also very excited to see if Usage and Rate Card data are now available from TP3, as that functionality was broken (at least for me!) in TP2.

Well the CloudBuilder.vhdx is just about finished extracting, so time to document the deployment process – fingers crossed!

There’s some speculation in here, but first up, some concrete notes learned from deploying SDN 2016:

  • MTU needs to be set to 1702 in the physical network for the tenant traffic, to support max ethernet frames of 1542 plus the required EncapOverhead of 160.
  • Make sure your NIC drivers are up to date to ensure that they can make use of EncapOverhead (if supported) to auto-configure MTU in the server infrastructure.
  • If enabled, Sequence Randomisation needs to be disabled within your physical firewall, or return paths won’t go back through the ASA and sequence numbers can’t be restored back to original value.

Ok, so when Azure Stack ships it will be as a turnkey appliance – a black box of wonder that arrives at your datacentre, gets racked, powered on and immediately delivers the glory of Azure to you locally. Right?

Not exactly.

As with any appliance, there are backend integrations which will need to be done – identity, billing and chargeback, integration into existing network infrastructure, that sort of thing. It’s on the latter which we’ve been working on recently, and for which we have some useful lessons learned to share.

While multi-node Azure Stack infrastructures aren’t available for most to work on this integration piece yet and the 1-node Azure Stack implementation hides behind a BGPNAT VM, we do still have options for making sure we’re as well prepared as possible.

Specifically, the Software Defined Networking implementation in Azure Stack is the Azure-inspired SDN which is also in Windows Server 2016, meaning that if we can deploy the end to end SDN stack in a Hyper-V 2016 cluster, in theory much of the required physical network config from above the TOR level should be identical in Azure Stack.

Regardless, Hyper-V 2016 has a long and illustrious future ahead of it as a resilient, cost-effective, and ultra-secure IaaS platform which can be managed through the Azure Stack portal in its own right, so having consistent SDN implemented in Hyper-V 2016 isn’t a nice to have, for me it’s an absolute necessity.

This blog doesn’t seek to document the step by step process for deploying SDN, but rather to showcase a few of the lessons learned encountered along the way which can help inform the integration of Azure Stack into an existing physical infrastructure.

Useful Resources

Software Defined Networking Overview

There is a plethora of documentation available for SDN, and while it can at first glance be a little overwhelming, it’s really important to really read and understand the entire documentation set before proceeding with deployment.

Set up a Software Defined Network Infrastructure in the VMM Fabric

This is the documentation set we have followed in order to deploy SDN successfully on a Hyper-V 2016 cluster managed by VMM 2016.

Set up SDN on One Single Physical Host using VMM

While we’re deploying SDN in a clustered environment, this implementation is really useful to read through as it has step by step screenshots which are extremely useful to reference through deployment.

Troubleshoot SDN

This is one of the most important pages to reference in an SDN deployment. Pretty much every single cmdlet and test documented therein is valuable in understanding the status of your SDN deployment and figuring out where any errors lie.

SDN GitHub

There are a number of useful resources in the SDN GitHub repo, in particular the example SwitchConfigExamples and the Diagnostics scripts are invaluable.

Physical Network Setup

Lesson the first…

SDN NC Planning

The SDN documentation includes a good amount of information about how to configure to a TOR level and how traffic flows within there, how you integrate this into your existing physical network will vary significantly though depending on what hardware you have and how it’s set up.

Image result for "azure stack" network topology

Public Azure Stack documentation to date uses the above image to show how separate cluster fault domains will connect through their TORs and AGGs, but naturally doesn’t go into any detail above that level.

Typically above this level we would find a set of hardware firewalls, and from there a series of core through to edge network devices. We questioned early on though the firewall placement in this scenario, the thought process being that tenant traffic would benefit from bypassing the physical firewall and making use of the SDN distributed firewall. This removes typical firewall bottlenecks, and enables the full automation power of the SDN infrastructure from Hyper-V switch to edge.

Management traffic is still critical to traditionally secure however, so our implementation splits management and tenant traffic out via VRFs.

Per host, Management and SDN traffic are run through a pair of 10Gbps Mellanox cards, while SMB/RDMA storage traffic is split out onto separate Chelsio NICs. Expanding the above image, it starts then to look more like this – yikes! This is not a pretty image, but it’s accurate.

Public VIPs route via core network over the Tenant VRF, while Private VIPs route via AGG/Firewalls, and all works joyously. BGP is in place from RRAS/SLB to ToR, then OSPF for Tenant traffic to the core and out to the edge.

Is this how it’ll be in Azure Stack? I don’t know! One thing’s for sure, learning lessons on how to integrate SDN 2016 into your physical network now can only benefit your Azure Stack deployment in the future.

 

If you’ve ever sat and watched an Azure Stack deployment end to end you’ll have seen a few typos here and there. I had to look this one up though, and it is indeed a word! Today I learned 🙂

http://www.dictionary.com/browse/configurate

Brief Overview

The App Service Resource Provider in Azure Stack offers the same deployment and management experience for Web, Mobile, and API apps as is available in Public Azure, while also extending the provider with capabilities which are unique to Azure Stack.

In addition to the expected Azure consistent capabilities, the App Service in Azure Stack allows customisation of shared and dedicated web worker VMs which host tenant applications, as well as the associated pricing SKUs, in order to most efficiently meet the needs of on-premises and hosted customers, both computationally and financially.

Worker Tiers 
MgmtSvcCIoud 
Add Refresh 
Shared Worker Tiers 
NAME 
Shared 
CORES 
MEMORY 
1792 
MEMORY 
1024 
2048 
4096 
8192 
QUANTITY 
Windows Se 
Dedicated Worker Tiers 
NAME 
Small 
Medium 
Large 
Supreme 
CORES 
2 
4 
8 
Wi dows 
Wi n dows 
Wi n dows 
Wi n dows 
Server 2012 R2 
Server 2012 
Server 2012 R2 
Server 2012 R2 
o 
o 
10 
AVA' LABLE 
AVAILABLE 
o 
10

Azure Stack administrators can deploy multiple shared worker instances, and define and deploy multiple different tiers of worker which do not exist in Public Azure. Different worker instances can have different core and memory counts, different operating systems, and custom software available.

The ability to deploy custom software is unique to Azure Stack and is not available within Public Azure, where you are limited to pre-defined options. Custom software can be made available for deployment via MSI, Zip, Exe, or DLL, and is packaged for consumption within custom worker tiers.

Create New Custom So... 
X Discard 
* Product Id 
p H pNuke 
Title 0 
Version O 
Installer Type O 
Installer (Msi) 
Package (Zip) 
Executable (Exe) 
Lib ra (D Il) 
Target Directory O 
Install Executable O 
Install Arguments O

In true PaaS fashion, these capabilities and the underlying compute resources are abstracted away from tenants and presented as easy to digest SKUs, which are again fully customisable to an Azure Stack admin’s customer requirements.

SKUs 
Mg mtSvcCIoud 
Add 
Free 
Shared 
Sta n d a rd 
High 
X 
3 
High 
SKU 
Discard 
* Name O 
Color Card O 
Green 
Compute Mode O 
Dedicated 
Worker Tiers O 
Large 
Features 
Qu otas 
X 
Refresh 
MODE 
Shared 
Shared 
Dedicated 
Dedicated 
Disable 
COLOR CARD 
Orange 
Orchid 
Blue 
Gree n 
COLOR CARD 
SKU Features 
X Discard 
Web Sockets O 
Custom Domains 
Off 
Ip Based SSI Mode O 
Off 
SNI 
SNI & IPv4 SSL 
SNI & IPv6 SSL SNI & IP SSL 
Worker process 64 bit as default 
Disabled SKUs 
MODE 
No SKUs defined. 
On 
Off 
Worker Process 64 bit Enabled O 
Off 
App Concurrent Request Limit O 
5000 
Site Cpu Percentage Limit O 
Site Memory Lit-nit (VIE) O 
1024 
Site Memory Working Set Limit (MB) O 
1024 
Site Idle Timeout (Minutes) O 
10080

Notes from Testing in Azure Stack TP2 Refresh 1

Hardware in operation:

Dell R630 13G Server

2x 10 Core E5-2650 Processors

384GB RAM

2x 800GB SSD, 6x 1.2TB 15k HDD

Deployment of Web Workers

  • Ten Large Web Workers were deployed concurrently to test deployment process and time to complete. While the deployment ultimately succeeded, the technical preview limitations of (SSD backed) Azure Stack storage for more than a single operation became apparent through this process, with deployment taking over 62 hours to complete.
  • After completing, the Status of the Web Worker Instances can occasionally change from Ready to Installing – consoling onto the VMs shows that this is due to installation of Windows Updates. Due to the insanely slow storage access with this many VMs running, in the Roles with only a single instance, this had the effect of taking that instance offline on a regular basis as the tier becomes unavailable due to insufficient Ready Instances.

Large Web Worker Instances 
Repair All 
NAME 
10.02.13 
10.0.2.18 
10.0.2.10 
10.0.2.11 
10.0.2.19 
10.02.15 
10.02.12 
10.0.2.14 
10.0.2.17 
10.0.2.16 
PLATFORM VERSION 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
STATUS 
Ready 
Installing 
Installing 
Installing 
Ready 
Ready 
Ready 
Ready 
Ready 
Installing
Shared Web Worker Instances 
Repair All 
NAME 
10.0.2.4 
PLATFORM VERSION 
57.0.10696.7 
STATUS 
Installing
Updating your system (96%)

  • Following one Windows update, the Shared Instance Web Worker VM came back up without a NIC, however its VM is definitely configured with a NIC.

Not connected - No connections are available 
1/23/2017
Network Connections 
Network and Internet Network Connections 
Organize 
This folder is empty.
Settings for e9038d1 b-bec9-426d-b9cf 
e9038d1 b- bec9-426d-b9cf-e851 bd6551 v 
Hardware 
r Add Hardware 
BIOS 
Boot from CD 
Security 
Key Storage Drive disabled 
Memory 
1792 Ma 
æ_] Processor 
1 Virtual processor 
E] IDE Controller O 
Hard Drive 
IDE Controller 1 
DVD Drive 
None 
SCSI controller 
00 IDD8B71C06 
SdnS',Nitch 
Hardware Acceleration 
Advanced Features 
e851 bd655138 on ASHOST 
Netvvork Adapter 
Specify the configuration of the netw•ork adapter or remove the netv•ork adapter. 
Virtual switch: 
SdnSwitch 
Enable virtual LAN identfication 
The VLAN identfier specifies the virtual LAN that this virtual machine will use for all 
net',xork communications through this net',xork adapter 
Bandwidth Management 
Enable bandwidth management 
Specify ho•A' this network adapter utilizes network band'"idth, Both Minimum 
aand'A'idth and Maximum aand',vidth are measured in Megabits per second. 
Minimum bandwidth : 
Maximum bandh'idth: 
O 
O 
Mbps 
Mbps

  • Running a Repair All on the Instance from within the Azure Stack portal did not resolve this.
  • Leaving the VM for >24 hours did not resolve this.
  • Manually rebooting the VM did resolve this.

 

One of the ten Web Workers (WW3) took significantly longer than the others to deploy – around 25 minutes extra, which is probably not an issue but just noting it for rigour The first nine VMs deployed in 45 minutes, with WW3 finishing after 70 minutes.
Operation details 
RESOURCE 
WW4-VM/OnStart 
WW2-VM/OnStart 
WW6-VM/OnStart 
WW7-VM/OnStart 
WW1 -VM/OnStart 
WW5-VM/OnStart 
WW9-VM/OnStart 
WW8-VM/OnStart 
WWIO-VM/OnStart 
WW3-VM 
WW2-VM 
WW6-VM 
TYPE 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
Microsoft.Compute... 
STATUS 
Created 
Created 
Created 
Created 
Created 
OK 
OK 
OK 
OK 
Created 
OK 
OK 
TIMESTAMP 
2017-01-20T21 
2017-01-20T21 
2017-01-20T21 
2017-01-20T21 
2017-01-20T21
Specs for a Large Web Worker are 4 Core, 4 GB RAM, however all Large Web Workers deployed at Shared instance specs of 1 Core, 1.75GB RAM. This repeated when deploying a custom Web Worker of 8 Cores, 8GB RAM. All Web Workers in TP2 Refresh 1 deploy at Shared tier spec for now it seems.
08 
08 
08 
08 
08 
08 
08 
•••!1e4n6uu03 
srueas 
80€L ZZZ 
OBSO LOZ 
LOZ 
LOZ 
fi LB LOZ 
LEGSOOZ 
9760 LOZ 
awgdn 
Z6LL 
ZSLL 
Z6LL 
ZSLL 
Z6LL 
an ZSLL 
Z6LL 
ZSLL 
Z6LL 
ZSLL 
Z6LL 
Lo Luan pau6!ssv 
BLI!uur1H 
5u1uunH 
BLI!uurre 
BLI!uurrH 
BLI!uurrH 
5u1uurrH 
ôuluure 
aEesn n dû 
L ïq-sesv-ttsz-ss LLZûtS 
"'te•08L7-SLSL-LZS8LOLt 
L -OZSZOOE 
• •ytve-G067-ape-9-somqsa 
- es26-00h-8€zq-eqp€m€q 
•••q-eot7-tesz-zLGLqqae 
--0L6-6q•osqa-9L9L602È 
a LueN 
lenu!A

  • Deleting a Web Worker Role from the Azure Stack Portal does not currently remove the underlying Virtual Machine from the Infrastructure.

Role Repair

  • Repair of all roles works as expected, other than on the controller instance which fails with the following error:

    • Code: 400, Message: Role object is not present in the request body.
  • I am currently unaware of a fix for this, or indeed even if one is needed or available.

Controller Instances 
Repair All 
NAME 
10.0.2.6 
O Repair instances 
x 
10:30 PM 
Repair 1 instance(s) failed. Reason: Client error occured. Code: 
400, Message: Role object is not present in the request body. 
PLATFORM VERSION 
57.0.10696.7 
STATUS 
Ready

Deployment of a WebApp

Most WebApp deployments succeed as expected, however there are instances where it has failed.

  • During the below deployment of a new WebApp to a dedicated instance, an error was thrown – Conflict error: Not enough available reserved instance servers to satisfy this request. Currently 0 instances are available.

Microsoft.WebSite08f9395c-9acd 
Deployment 
Delete Cancel Refresh 
Lt.] Redeploy 
Failed. Click here for details 
"Code": "Conflict" 
X 
View template 
RELATED 
Outputs 
NO DEPLOYMENT OUTPUTS 
Inputs 
NAME 
HOSTINGPLANNAME 
HOSTINGENVIRONMENT 
LOCATION 
SKU 
WORKERSIZE 
SERVERFARMRESOURCEGROUP 
SUBSCRIPTIONID 
Operation details 
RESOURCE 
dedicated 
Events 
dedicated 
local 
Standard 
2 
dedicated 
8145fe3d%27e4c12Æb39S7878275b703 
Operation details 
OPERATION ID 
TRACKING ID 
STATUS 
PROVISIONING STATE 
TIMESTAMP 
DURATION 
TYPE 
RESOURCE ID 
STATUSMESSAGE 
TYPE 
Microsoft.Web/serv... 
STATUS 
Conflict 
TIMESTAMP 
2017 
938E6474AB3EF62A 
3e87c883-1 c36-43c6-a5ae-4275a40c2988 
Conflict 
Failed 
1/23/2017 2:30:55 PM 
7 seconds 
Microsoft. Web/serverfarms 
/subscriptions/8145fe3d-b27e-4c12-8b39-57878275b703/reso... 
"Code": "Conflict", 
"Message": "Not enough available reserved instance servers to 
satisw this request. Currently O instances are available. If you are 
changing instance size you can reserve up to O instances at this 
moment. If you are increasing instance count then you can add 
extra O instances at this moment." 
"Target": null, 
"Details": 
"Message": "Not enough available reserved instance setvers 
to satisfy this request. Currently O instances are available. If you 
are changing instance size you can reserve up to O instances at 
this moment. If you are increasing instance count then you can 
add extra O instances at this moment." 
"ErrorEntity": { 
"Code": "Conflict", 
"Message": "Not enough available reserved instance servers 
to satisfy this request. Currently O instances are available. If you

  • Two thoughts as to why this could happen were:

    • All Web Worker Instances were undergoing updates at the same time
    • All Web Worker Instances had sites running on them, and as they are dedicated there are none free

Only one Instance was updating at the time, deleting it to bring all Instances to a Ready state had no impact, the deployment still failed.

Large Web Worker Instances 
Repair All 
NAME 
10.0.2.13 
10.02.19 
10.0.2.15 
10.0.2.12 
10.0.2.14 
10.0.2.17 
PLATFORM VERSION 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
STATUS 
Ready 
Ready 
Installing 
Ready 
Ready 
Ready

The Admin portal reports that none of the Instances have any running sites on them, which should make them available for placement…

Large Web Worker Instances 
x 
Repair All 
NAME 
10.0.2.13 
10.0.2.19 
10.0.2.12 
10.0.2.14 
10.0.2.17 
STATUS 
Ready 
Ready 
Ready 
Ready 
Ready 
10.0.2.17 
Large Web Worker 
Repair 
Essentials 
Narne 
10.0.2.17 
Worker Tier 
Large 
Running Sites 
o 
Custom Software 
Not defined 
Instance 
1:45 PM 
CPU 
Http 
Stop 
Logs 
PLATFORM VERSION 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
57.0.10696.7 
Delete 
2 PM 
Role 
Web Worker 
Compute Mode 
Dedicated 
Allocated Dedicated Worker 
No 
2-15 PM 
DISK QUEUE LENGTH 
Add tiles 
2:30 PM 
HI-rp QUEUE LENGTH 
146 
MEMORY PERCENTAGE

… however, there are in fact currently two webapps deployed in the Dedicated Large tier, and another in the Shared tier, and not a single instance is reporting any WebApps deployed into them.

Conclusion: The Admin Portal does not accurately report the number of running WebApps in an instance yet, and all of the dedicated instances were indeed in use at this time.

Backup of WebApp

When configuring the Storage for backup of a WebApp, I accidentally left the storage endpoint as core.windows.net rather than specifying the local Azure Stack endpoint. This naturally caused the backup to fail. After correcting the issue, backups still failed for ~10 minutes as it believed an existing backup is in progress.

Once the infrastructure timed out the initial backup, subsequent backups to Azure Stack storage were able to proceed as expected, and completed successfully.

Backup Configuration Summary 
Backup configured properly with a Schedule to backup your site every 1 days. Last 
backup happened on Monday, January 23, 2017 1:14:47 PM. 
L. 
Schedule: Configured 
Storage: Configured 
All backups 
STATUS 
••/ Succeeded 
BACKUP TIME 
1/23/2017 1:14 PM 
SIZE (MB) 
0.12 
LAST RESTORED

Scale Up of an App Service Plan

Initial testing of Scale Up of an App Service Plan failed with a non-specific error.

Microsoft Azure Stack 
Failed to update App Service plan 
Failed to update App Service plan 
> 
Detail 
klowe@bl 
X 
DESCRIPTION 
STATUS 
TIME 
CORRELATION IDS 
EVENT 
Failed to update App Ser... 
Failed to update App Service plan klappspl: 
error has occurred. 
Error 
Monday, January 23, 2017 1 - 
.09:16 PM 
4e1be96d 0826 40b3-a852 21 164552d8eb 
Detail 
TITLE 
DESCRIPTION 
STATUS 
TIMESTAMP 
UTC TIMESTAMP 
CORRELATION IDS 
Failed to update App Service plan 
Failed to update App Service plan klappspl: {"Message"•"An 
error has occurred." ) 
Error 
Mon Jan 23 2017 GMT+OOOO (GMT Standard Time) 
Mon, 23 Jan 2017 GMT 
421 be96d-082640b3-a852-21164552d8eb 
LEVEL 
O Error 
STATUS 
Error 
TIME 
Jus...

Deploying a new WebApp and testing scale up of it worked as expected, so cause of the initial failure is currently unknown.

Scale Out of an App Service

Initial test of Scale Out of an App Service fails with an error ‘Failed to save scale settings’.

Saving scale settings 
DESCRIPTION 
STATUS 
TIME 
CORRELATION IDS 
EVENT 
Saving scale settings 
Failed to save scale settings. 
Error 
Monday, January 23, 2017 1:33 PM 
clientNotification Idb7feOa a66d 4bbe a47b 
dd2e956bf02e 
LEVEL 
O Error 
STATUS 
Error 
TIME 
Jus...

As with Scale Up, creating a new WebApp resolved this as well and Scale Out reported as completing successfully.

All other tests/integrations currently tried worked as expected. For completeness, these are:

  • Deploy the App Service RP
  • Add the App Service RP to Azure Stack
  • Create custom SKUs
  • Configure Source Control Providers (GitHub only)
  • Custom DNS Integration
  • Create an App Service Plan
  • Create an Empty Website (using custom SKU utilising custom Web Worker tier)
  • Deploy an existing Web App into App Service

    • For this I used a Bot for Dynamics CRM which I had previously created using the Microsoft Bot Framework. As this Azure Stack instance doesn’t have an externally available URL/IP I wasn’t able to test it connected through to Dynamics CRM, however deployment succeeded as expected.
    • Note: Microsoft should have called the Bot Framework the Bot Net Framework.

DISCLAIMER: The below represents my personal views.

 

The announcement by VMware and AWS that they are partnering to deliver VMware services from Amazon datacentres has already started to generate quite the flurry of excitement around the interwebs, and quite rightly so – it’s a big announcement from two of the biggest global players in Cloud (AWS) and traditional virtualisation (VMware) today. The timings are interesting, with the announcement coinciding with the launch of Windows Server 2016 and the delivery timeframe the same as that of Azure Stack. A cynic might call it reactionary and say that it lends great credence to Microsoft’s ‘all in on hybrid’ strategy. A cynic might.

 

I want to really delve into some of the messaging they’ve delivered, what it means, what’s changed in our world now vs yesterday, really try to understand where this industry of ours is heading, and what it all means as a service provider.

 

First up, this would seem to sound the final death knell for vCloud Air, which was unceremoniously downplayed and pretty much put out to pasture at VMworld in recent weeks. Effectively, VMware are deploying a similar model as vCloud Air into AWS datacentres, extending it with enhanced hardware-level elasticity and resiliency, and integrating AWS services and the AWS name and reputation in Cloud into the mix.

 

As these AWS services will not be deployable on-premises or within service provider datacentres, this is effectively the same hybrid story as previously with vCloud Air, with lower latency and less of an air gap between the off-prem VMware instances and AWS services, and a better management story.

 

As an Enterprise, I could well see this as a great way to bridge the gulf between on-premises virtualisation and born-in-the-cloud capabilities of AWS with a view to eventually moving to all AWS. In a world inexorably moving away from IaaS and into the world of PaaS, I cannot see the long-term here for VMware beyond buying some more time.

 

I say inexorably, because as IT Pros, loathe though we may sometimes be to admit it, the platforms we build and support are driven not by our wants and needs, but by those of the applications we deploy and the developers who create them. Innovation doesn’t follow IT, innovation in our world follows developers – from mainframes to PCs to the web to app stores and now on into cloud services, it’s a truism that we build to support that created by others.

 

Even today, a majority of new players entering the market fresh from university have been taught, guess what, either Azure or AWS. It’s so easy for a university to spin up and down developer resources in these true cloud platforms, and both throw huge numbers of free credits at educators in order to upskill the forthcoming workforce in their respective technologies, that snowballing and default adoption of them is all but inevitable.

 

Indeed I equate the time we’re living in now to the OS ‘wars’ of the 80s and 90s, where many many operating systems vied for attention, be it RiscOS, DrDOS, OS2, BeOS, or a plethora of others. Now twenty plus years on, the majority of these are dead and buried and we’re left with two main players – Windows, and ‘the *nixes’. Two distinct and core development platforms which development talent settled into naturally as each hit a critical mass that just couldn’t be stopped. If you wanted to develop an application which had good market penetration, you would target Linux or Windows, not BeOS. Through this natural attrition, all the other snowflake operating systems effectively died off or became very niche plays.

 

I believe this is what is happening in the world of Cloud today. The market is being hit by an influx of new talent coming in which expects just to be able to take advantage of Cloud-native PaaS services, because that’s what they’ve been taught. Containerisation is a nice compromise and half-way house in some cases here as it removes IT as a point of conflict and friction (if the container works on developer’s laptop it’ll work on IT department’s container host), but it’s only a tiny bit of the journey to true PaaS and cloud-native development. It’s long been my view that developer need will naturally kill off snowflake Clouds over time, and if we believe that AWS and Azure are the analogs of Windows and the *nixes from the OS wars in the Cloud era, we have to believe that they’ll emerge as the de-facto hyperscale offerings. I believe that if you’re ploughing money into building a Cloud which is not designed to dovetail into AWS or Azure consistency in time, what you’re building is the RiscOS of today. I loved RiscOS, but it didn’t attract sufficient development talent, and it ultimately fell into the hobbyist niche.

 

This being the case then, what does tying VMware into AWS actually bring to the table here? It offers an easy path for existing VMware houses to get more comfortable running their workloads in Amazon datacentres, which is wonderful for AWS as that’s a massive market for them to capitalise on. It gives VMware a hybrid story that actually includes Cloud-native development services from a true Cloud player for the first time. It doesn’t in my opinion give a true hybrid story, as those services still cannot run on-premises.

 

When you boil it down to its core and examine the differences between this offering and Azure Stack, fundamentally the value prop here is in bringing you from your premises and into AWS to leverage their Cloud services, using tools and technologies you already know to ease the way. The value prop of Azure Stack is in bringing the Cloud services from Azure back to your datacentre, and allowing you to take advantage of developer capabilities that are wanted and expected, but on your own terms and where you want and need them.

 

Regardless of how this deal plays out, I believe that AWS and Azure becoming the de-facto hyperscale Clouds is all but inevitable now, and in the best possible way, so that’s the world I architect towards. Other than as an on-ramp to AWS and a way for VMware houses to keep running virtualisation on-prem while giving developers Cloud capabilities in AWS in a reasonably credibly hybridised way, I don’t see the long-term play for VMware here.

 

While I may paint a picture here of hyperscale ruling all, there are some fundamental factors which absolutely necessitate the existence of Azure Stack and the ability to run true Cloud on your own premises, or from a regional service provider.

 

The most basic one is the speed of light. We can’t break it, and latency matters. It particularly matters in the world where IoT is starting to explode, and data needs to be captured and managed not up at the hyperscale cloud level, but at the edge in regional facilities. Not just for IoT, but for any big-data generating application or toolset – the less distance you have to send your data, the better – more and more often, sub-millisecond responses matter. Believe in the future of Bots and the BotFramework? You’d better believe latency matters.

 

There are of course always varying regional compliance and audit requirements that necessitate running things on-prem or in local providers, and even sometimes public-sector mandates to use local services as much as possible from a local economics perspective – this is one area I can’t see changing any time soon.

 

In many more rural areas, the connectivity just doesn’t exist to reliably make use of hyperscale cloud services, and this will be the case for many years to come in many countries. With all the innovation happening in AWS and Azure and all the new services being offered by them pretty much day by day, up until this point organisations in this situation have had no choice but to work in a more traditional IT modus, regardless of their desire to take advantage of Cloud. Azure Stack fixes this. Virtualisation doesn’t. AWS doesn’t.

 

I appreciate that a number of things I say above are contentious and that a lot of people disagree with them. I always enjoy lively discourse around this topic, so please do weigh in.

 

I believe that faster, more productive, and more innovative development work can only happen best in a Cloud-native world. I believe that we need to be able to run Cloud-native services on-premises and in regional datacentres, with exact hyperscale consistency. I believe that a developer should be able to develop once, and run their app consistently using the same services in our facilities as in the hyperscale cloud.

 

I believe that Azure Stack is the only product which fulfils these needs.

 

I believe that this deal is VMware’s best chance at staying relevant in the world of Cloud where they have no Cloud story of their own.

Chapter 1: Prologue

Chapter 2: Shielded VMs

Chapter 3: Storage Replica

Chapter 4: Speaking at Ignite

Chapter 5: The True Value of Ignite

Chapter 6: The Parties

Chapter 7: Key Learnings

Chapter 8: Thoughts to the Future

 

Chapter 2

So having noted in my previous post that I was asked to speak about my experience in operationalising a Guarded Fabric which hosts Shielded VMs, the question which you, dear reader, may have, is “what the blazes is a Shielded VM and why should I care?”

 

Well it’s a good question, so well done for asking it. Before I dive into what a Shielded VM is and what it aims to achieve though, first I want to take a little detour through human psychology.

tfas

We humans are inherently built for pattern-matching and auto-association – there’s a wonderful book called ‘Thinking, Fast and Slow‘ which dives deep into this trait and which I heartily recommend reading. Wherever our brains can, they look for the familiar, correlate, and file as neatly as they can – in many cases we have no conscious control over this.

 

If you read the word ‘Library’, you have no choice but to automatically understand it and process it. You simply cannot make your brain incapable of instantly recognising, reading, and understanding the word. This is a wonderful evolutionary trait which lets us make decisions in split seconds and understand relatively new concepts which relate to existing knowledge with ease. This is a very base definition of ‘Thinking Fast’.

 

If I ask you to solve 1463 * 432 in your head, then unless you are a mathematical savant you will need to engage a series of conscious processes and steps to work through the multiplication to find the answer. This is a very crude way of explaining ‘Thinking Slow’.

 

Thinking Fast and Thinking Slow are not complementary processes which work in harmony, they compete and conflict, and can cause significant angst and annoyance, or even danger. Everyone who drives knows well the feeling of getting in their car, setting off to work, arriving at work and wondering how the hell they got there as there’s no memory of the drive. The fast thinking portion of the brain has taken over and is almost automatically driving a route it knows well while other parts of the brain engage elsewhere to focus on other thoughts. If the route was ever unchanging and always the same every day this would be an extremely valuable trait, however other drivers are unpredictable and conditions change, so we’ve also all experienced being jolted into alertness as another driver swerves in front, or roadworks get in the way.

 

All of the above is contextually relevant, because without fail every time I start to explain Shielded VMs to someone, they immediately drop into a fast thinking modality, relate it to basic encryption, nod and file it away in their mind as just that. They could not be further from the truth. So before I explain what assurances VM Shielding provides, first empty your mind of preconceptions about existing security technologies, and believe me when I say that this capability is completely unique to Hyper-V 2016 today.

shldvm

Shielded VMs as a concept came about because of a growing attack vector which Microsoft Research identified around four years ago. This attack vector covers many different methods of infiltrating and exfiltrating data from virtualised environments, but the one commonality across them all was the leveraging of fabric-level admin credentials to conduct malicious activities.

 

Today, a virtualisation admin is the god-emperor of all that he or she has purview over – there is no effective way to protect the data and secrets contained within a virtual machine from the fabric on which it runs. Whether it’s taking a copy of a VHDX/VMDK on USB and attacking it or booting it off-site, consoling on to a VM, capturing live migration or state traffic, attaching debuggers to VM processes, or a plethora of other routes, the contents of virtual machines today are inherently open to the admin credentials of the fabric on which they run.

 

I say admin credentials, because more than a few people I’ve spoken to about this have taken significant umbrage at the implication that they may be in some way untrustworthy or capable of malicious activity. ‘IT is an position of trust, we’re given these services to manage and it’s our duty to protect them.’ is a common line of rebuttal, and I agree wholeheartedly.

 

There’s no avoiding the fact though that today the majority of data breaches which occur are caused by insider attacks, be it due to malware, compromised credentials, coercion, or, yes, malicious administrators. This isn’t a slur against sysadmins, it’s a recognition of three routes to insider attack which can occur even to the nicest sysadmin, and that in all large-scale industries it’s inevitable that bad actors and disgruntled employees will at some time exist.

 

What Shielded VMs aim to do is completely fill in this hole in the security model that we currently have patched together with process, audit, and trust, and completely remove it as a valid attack vector. It does this through a series of very clever technologies which come together to form a cryptographically trusted platform on which we can run Shielded VMs, in a model we call a Guarded Fabric.

 

I’m not going to go into any great detail here about the methods in which this is achieved, for that I strongly recommend watching this Dive into Shielded VMs with Windows Server 2016 Hyper-V session from Ignite 2016. All I want to achieve here is an understanding of what the feature seeks to and manages to achieve.

 

  • VM Disks are encrypted at the VM level using strong cyphers and the most secure method of key release we have today, TPM.
  • VMs are only allowed to boot/migrate on cryptographically provably healthy hosts. i.e. hosts which have not been compromised.
  • PowerShell Direct does not work to a Shielded VM.
  • Console connections to a Shielded VM will not work.
  • RemoteFX is disabled.
  • Specific WMI calls are disabled (screenshot, thumbnail, keyboard, mouse)
  • Guest File copy IC is disallowed.
  • IMC registry hive injection is disallowed.
  • Some virtual devices like debug devices, synthetic keyboard, synthetic mouse, serial device are removed.
  • Shielded VMs run inside a protected process which will not allow debuggers to be attached.
  • A whitelisted Code Integrity Policy is enforced on hosts to ensure that only known trusted binaries can run on them.
  • All live migration traffic is encrypted.
  • All VM states are encrypted.
  • VM access is available to owners only, via certificate signed RDP or key-trusted SSH.
  • For all the encryption items above, the VM owner owns and maintains the encryption keys, not the fabric admin.

 

For a hosting service provider like ourselves, this allows us to guarantee to tenants that their VM secrets are completely hidden and protected from us the fabric admins. For tenants, they can run workloads which have stringent compliance and regulatory requirements in a multi-tenanted environment. Enterprise admins can enforce strong separation between Hyper-V administrators and sensitive workloads. I would argue that this is a huge benefit to enterprise sysadmins, who can use VM Shielding to indemnify themselves in the event of data breach, as the data remains completely encrypted and secure.

 

The absolute most important point here is that all of the above trust is rooted in hardware through TPMv2 based attestation. In a fully operationalised and working Guarded Fabric there isn’t a locked down subset of admins who can still compromise the security model, the trust is rooted in cryptography, and in the hardware.

 

Hopefully the benefits here are now obvious, and why we as a hosting provider are so excited about the capabilities that VM Shielding allows us to bring to our customers.

 

Hopefully it’s also obvious that VM Shielding represents a leap step forward in virtualisation security that goes way beyond anything else available in the market today.

Chapter 1: Prologue

Chapter 2: Shielded VMs

Chapter 3: Storage Replica

Chapter 4: Speaking at Ignite

Chapter 5: The True Value of Ignite

Chapter 6: The Parties

Chapter 7: Key Learnings

Chapter 8: Thoughts to the Future

 

Chapter 1

 

I spent last week with Senior Systems Engineer Craig Dalrymple at the Microsoft Ignite conference in Atlanta, one of the largest IT conferences in the world with 20,000+ attendees, which is around half the number of people who work on the Redmond campus.

A lot of times at conferences like these there’s a lot of hot air and future announcement designed to show thought leadership and keeping pace with competitors. Not so this year at Ignite, with the launch of Windows Server 2016 and System Center 2016 scheduled for the same week (a one in four year event, basically an IT world cup), and Azure Stack technical preview 2 being released to the masses. These are hugely impactful products to the IT landscape, so it was vital we attend and immerse.

I was originally signed up as an attendee, but a few weeks before the conference I was asked by Dean Wells and Ned Pyle if I’d join them on stage to talk about their products – Shielded VMs, and Storage Replica respectively. After ploughing the last year and a half or so of my life into making sure that brightsolid could be among the first in the world to bring these and other valuable features to market at the launch of Windows Server 2016, naturally I said ‘hell yeah!’

We’ve hammered the bejesus out of Storage Replica and Shielded VMs along with the new SDN stack over the past year and a half or so, through various technical previews and engagements with Microsoft. We’ve worked fervently to operationalise and learn them intimately so that now, at general availability, we can immediately start bringing the value they represent to our customers. With Shielded VMs in particular, there’s a huge amount of preparation required to take it from installed to fully ready to sell to customers, and as that is knowledge that’s pretty unique to brightsolid, it was that aspect we focused that session on.

Beyond those speaking slots, the conference as a whole was a smorgasbord of knowledge, networking, and opportunity, and the energy and buzz running through the venue was palpable. What I’d like to do through this series of posts is capture some of that knowledge, enthusiasm, and energy, and send it on to those who couldn’t be there, because it is ridiculously and utterly infectious.