Managing Microsoft Azure Arc-Enabled Infrastructure from the Azure Portal

1. Executive Summary

An Azure Arc-enabled infrastructure is a cloud infrastructure that is managed and monitored by Microsoft Azure. It includes features such as Azure Resource Manager, Azure Monitor, and Azure Security Center. The Azure portal is a web-based management tool that provides a unified experience for managing all Azure resources. The Azure portal allows you to create, manage, and monitor Azure resources in a single, unified console. Many are managing their Microsoft Azure Arc-enabled infrastructure from an Azure portal.

Azure Stack HCI is a hybrid cloud solution that merges the power of the Azure control plane with hyperconverged infrastructure (HCI) and enables organizations to advance digital modernization goals with cost-effective technology.

Hybrid cloud solutions play an integral role in the digital transformation of enterprise IT. Organizations that fail to leverage technology that aids in the transition of legacy, on-premises workloads into the cloud face multiple challenges:

• Multiple support models for both on-premises and public cloud infrastructure
• Mixed cost model overhead with CapEx infrastructure and cloud OpEx
• Added lifecycle and redundancy costs

Azure Stack HCI is an opportunity for organizations to keep high performance workloads or workloads tied to data sovereignty on-premises while using Azure’s control plane to administer and monitor those environments. Further, Azure Stack HCI supports Azure Kubernetes Service (AKS) enabled by Azure Arc and Azure Virtual Desktop (AVD), which provide a very simple way to run these solutions on-premises.

Hybrid cloud management is only one facet of the Azure Stack HCI solution, however. Hyperconverged infrastructure provides benefits over traditional infrastructure deployments, including:

• Robust scalability
• Reliability
• Improved performance
• Simplified deployments and management
• Software-defined infrastructure and reduced infrastructure costs

A primary element of digital modernization for any organization is software-defined infrastructure (SDI) implementation. Aiding automation, resiliency, security, and redundancy, Azure Stack HCI offers software-defined networking as an available feature and software-defined storage as a core technology.

In our testing, Azure Stack HCI showed an advantage in each activity completed. This reflects the product’s advantage in providing the benefits of a unified infrastructure management across clouds for forward-looking IT organizations. Of particular note is the familiarity current administrators and engineers in the Microsoft Windows space will find with much of the tooling, leveraging existing resource expertise.

Our testing aligned numeric values to score the level of effort and skill needed to complete common activities in Azure Stack HCI compared to a typical IT environment without it. Azure Stack HCI needed fewer resources with fewer skill sets to accomplish the same work as current or legacy on-premises infrastructure administration. Additionally, Azure Stack HCI provides the only way to operate Azure Virtual Desktop on-premises and managed from the same portal as in-cloud deployments.

We hope this report is informative and helpful in reviewing the capabilities of Azure Stack HCI and presenting the time-to-value total cost of ownership (TCO).

2. Organizational Readiness

While moving to cloud continues to be a defining aspect of digital modernization in enterprise IT, many organizations are migrating from traditional three-tier architecture in their datacenters to hyperconverged infrastructure. HCI platforms provide an experience that can be akin to utilizing cloud service providers but without the need to shift workloads away from on-premises.

Azure Stack HCI offers the benefits of the established Azure control plane along with the benefits of the established Windows ecosystem. Integrating this Microsoft offering with existing environments reveals many existing tools and components are already compatible.

At its core, Azure Stack HCI leverages some of the Windows operating system capabilities, utilizing the existing technology of Hyper-V, Storage Spaces Direct, and Microsoft’s Software Defined Networking solution. Windows Admin Center can be used via local installation or the Azure portal as a familiar toolbox for cluster, VM, and server administration. VMs can be deployed to the HCI cluster using existing solutions, such as SCVMM.

For enterprises leveraging Windows Server and/or Hyper-V on-premises, Azure Stack HCI’s  core components will be familiar to use and customize. Likewise, organizations with an Azure presence will enjoy the flexibility of managing even their on-premises workloads and infrastructure from the Azure portal, integrated with their existing public cloud resources.

As reported in GigaOm’s December 2021 Radar Report for Hyperconverged Infrastructures: Enterprise Deployments, “An important feature is the ability to run AKS enabled by Azure Arc directly on Azure Stack HCI, helping to bring Kubernetes to the edge. The AKS control plane resides on the local HCI system, enabling disconnected use without continuous access to Azure Cloud. Additional Azure Cloud services will be added to Azure Stack HCI over time. System administrators can enable users to create and manage VMs and containers in a self-service fashion.”

3. Testing Overview

Environment Specifications

The test environment for this benchmark report was based on the M50CYP family from Intel. This server system supports third-generation Intel Xeon Scalable processors with two sockets for up to 80 cores per server. Up to 12 TB of system memory is supported with 3,200 MT/s of bandwidth. For network connectivity, 100 gigabit RDMA Ethernet is supported with an OCP 3.0 module or add-in NICs. A combination of front-mounted, hot-swap drives are supported on this hardware, including up to 24 2.5” SAS/SATA/NVME SSD drives on a 2U chassis. Additionally, up to six full-height and two half-height PCIe 4.0 cards are supported via riser cards.

Specifically, the hardware used for this test was implemented with the M50CYP2SB2U board, 2x Intel Xeon Gold 6348 @ 2.60 GHz processors (56 total cores and 112 logical processors). Each node of the four-node cluster was configured with 256 GB of memory DRAM + 1 TB of Intel Optane persistent memory. The installed OS was Microsoft Azure Stack HCI 21H2 – Build 20348.

The Xeon Gold 6348 represents one of the best in the Intel Xeon Scalable processor family. Mainstream customers will likely find this series an excellent choice in terms of performance for cost, second only to Intel’s top-end Platinum series. The Intel Xeon Gold 6300 processors are optimized for multicloud compute, storage, and network workloads with socket scalability (up to four), workload acceleration, and hardware acceleration.

A significant benefit of this hardware system is the support for Intel Optane Persistent Memory (PM) 200 Series. Intel presents its persistent memory technology as a tiered architecture for memory and storage by offering large capacity and persistence. This provides an opportunity for system architects and developers to address workload challenges by leveraging the speed of technology closest to the CPU and still have access to higher overall capacity.

Intel Optane PMem 200 Series can function as persistent or volatile, depending on the need. For example, large databases are supported entirely in-memory so that workloads with significantly sized data sets can work without the need to repeatedly load and store data locally. Intel PMem also offers increased capacity per socket over DRAM in the case of virtualized infrastructures, allowing for some use cases that may have historically required bare metal to be virtualized.

On the provided hardware, we assembled a lab consisting of:

• 4 domain controllers
• 8 SQL database servers
• 4 load-generating servers
• 20 VDI session hosts

Testing Structure

The tests below reflect the effort involved in completing a series of deployments or configurations commonly used in enterprise IT. This testing revealed that Azure has unified and simplified many administrative capabilities. This allows one engineer to deploy configurations or solutions where, in our alternate scenarios, it would be typical to see multiple individuals or teams engaged to complete the task, with specialized expertise often required. In short, managing on-premises infrastructure from the Azure portal can be done with fewer technical resources due to simplified processes.

Furthermore, organizations pursuing digital modernization through public cloud resources can reduce the need for technical resources by leveraging a common administrative interface and framework for compatible infrastructure both on-premises and in the public cloud.

The effort columns below denote a factor of complexity and expected skill area to complete the steps in the test zone. An effort noted as one (1) is the best score due to clear documentation, simplicity of interface, and/or the level of experience needed by the engineer(s). By contrast, a score of five (5) indicates that the task is very complex due to the number of sub-steps involved, the complexity of the interface, and/or the required experience needed by the engineer(s). Such activity would likely be completed by multiple resources, coordinated by a project manager with an architectural resource involved.

Between these, we have two (2) through four (4). A two (2) scoring represents a slightly complex or advanced task due to variables unaccounted for in the documentation or an increased number of sub steps. While a separate skill set may not be required, the activity may typically be completed by a different resource at some organizations. Scoring a three (3) indicates that in most medium to large organizations, multiple skill sets or resources would be involved in executing the activity. A four (4) scoring reflects an activity that many organizations would coordinate centrally with a predefined process or coordinator resource to execute with the involvement of multiple teams or engineers.

4. Tests

For each test, we combined the effort required for each step and displayed a sum for a comparison of each solution. The lower total, based on the scoring system described above, is judged to have an advantage from the perspective of engineering and administrative effort.

To evaluate the effort for various capabilities common and important within IT, we selected five tests covering five categories. These tests demonstrate the capabilities of Azure Stack HCI that are managed from the Azure portal in comparison with alternative solutions for on-premises infrastructure.

• Test 1 – Client virtualization: Azure Virtual Desktop vs. Remote Desktop Gateway
• Test 2 – Monitoring: Heartbeat Alert and Notification with Azure Stack HCI vs. Without
• Test 3 – Security: Update Management with Azure Stack HCI vs. Without
• Test 4 – Inventory: Change Tracking with Azure Stack HCI vs. Without
• Test 5 – Workload Protection: Noisy Neighbor with Azure Stack HCI vs. Without

Test 1. Client Virtualization – Azure Virtual Desktop Deployment vs. Remote Desktop Gateway

This test evaluated the effort involved in setting up a deployment of Azure Virtual Desktop (AVD), which is currently in public preview. Users can expect some of the tasks performed here to be even simpler to execute in the GA version. We compared the configuration of the pre-release AVD on-premises solution via an Azure Stack HCI cluster with a standard deployment of a remote desktop gateway and session hosts. This test assumes certain prerequisites are already in place, such as an Azure Stack HCI cluster, Azure Active Directory (AD) Connect—a tool that helps organizations connect their on-premises Active Directory to Azure Active Directory—and Session Hosts present on the cluster. Likewise, for the alternate test scenario, on-premises infrastructure, including a Windows OS server and Session Hosts licensed for Remote Desktop are assumed.

Table 1. Client Virtualization: Azure Virtual Desktop (preview) vs. Remote Desktop Gateway

A key capability of Azure Stack HCI is the ability to manage and access an on-premises deployment of Azure Virtual Desktop (currently in public preview) via the Azure portal. We followed the configuration steps in Azure and connected our on-premises desktop VMs to Azure via the required agent installations. This deployment process required no additional on-premises infrastructure besides the HCI cluster, desktops, and network access to the required Azure URLs. Without Azure Stack HCI, a traditional remote desktop solution accessed via Remote Desktop Gateway may be chosen. This would require the additional infrastructure to provide remote access, and a user assignment process that is less intuitive than the infrastructure available in Azure.

VDI workloads are both CPU and memory intensive. In this test, the on-premises Azure Virtual Desktops benefitted from the additional memory provided by Intel Optane persistent memory. The additional memory enables the same size Azure Stack HCI cluster to support more VDI users, or the same number of users can be supported on a smaller cluster to reduce TCO. Alternatively, additional virtual memory can be assigned to each user to support the broad range of more demanding workloads in use by today’s distributed workforce.

Test 2. Monitoring – Heartbeat Alert Configuration

The Heartbeat Alert refers to a common configuration in infrastructure monitoring where an alert triggers when an agent on a target device has not contacted the monitoring service in a certain period or set intervals. In this test, we not only configured the alert to trigger in the monitoring console but also to notify an admin via email.

This test assumed certain configurations had been completed prior to the test. For the Azure-based test, the prerequisites included an automation account and related workspace, as well as the Azure Arc agent installed on the VMs, which would be configured for alerts.

Prerequisites for the “without Azure Stack HCI” scenario included Operations Manager, supporting infrastructure, and an agent installed on target devices.

Table 2. Monitoring: Heartbeat Alert and Notification with Azure Stack HCI vs. Without

The non-Azure test for Heartbeat alert had the advantage of a built-in mechanism that did not require an initial setup. However, we found setting up the notification configuration to generate an email was more complex than desired compared with Azure’s Alert Rules and Alert Groups. Additionally, using the Arc data stream and logging to create queries for alerts allows more flexibility than currently available with Operations Manager.

Test 3. Security – Update Management Configuration

The Update Management Configuration test focused on the ability to manage the security patches of a Microsoft Windows OS device via scheduled deployment. Prerequisites for the Azure test portion included an Automation account and related workspace, as well as a Windows VM with the Arc agent installed.

Without Azure Stack HCI, the scenario required the prerequisite of Windows Server Update Services (WSUS) and a Windows device to update. It’s worth noting that the Azure Update Management offering does integrate with WSUS and Microsoft Endpoint Configuration Manager. Azure Stack HCI and Intel hardware-integrated security technologies together support a range of more sophisticated security functionality. For this test, however, we remained within basic update scheduling capabilities.

Table 3. Update Management Configuration with Azure Stack HCI vs. Without

Our Update Management test successfully showed that a step-by-step scheduler for standard Windows security patches was simple and quick to set up. Our “without” test required, at a minimum, the infrastructure of WSUS to coordinate delivery and installation on target devices, along with cumbersome GPOs to complete the scheduling. It is worth noting that the Update Manager can also integrate with WSUS and Configuration Manager for third-party patching deployments.

Test 4. Inventory – Change Tracking Functionality

Change tracking refers to the ability to determine what changes were made to a system (in our case, a system running Windows OS) in terms of files, registry, and software.

Our prerequisites for the Azure test once again required only an Automation account, related workspace, and an Arc enabled VM. For the alternate test, a Configuration Manager deployment along with Reporting Services and related components were required.

Table 4. Inventory: Change Tracking with Azure Stack HCI vs. Without

Our change tracking test showed that Azure Arc’s data stream simplifies change tracking on a per-VM basis. In contrast to the typical “without” scenario, the Azure portal offered a single pane of glass to manage all related aspects.

Our test with Azure Stack HCI was as simple as enabling change tracking on our automation account, confirming our desired configuration, such as frequency of service collection, and installing a sample piece of software (Putty). The expected result was confirmed—Putty appeared as installed in the change tracking dashboard shortly after installation. The contrast test required more technical experience and infrastructure to achieve a similar result, but as a report and not a live dashboard.

Test 5. Workload Protection – Noisy Neighbor Scenario

This test was designed to evaluate the Azure Stack HCI solution’s ability to protect a designated workload from other workloads on the same cluster and even the same node. This was accomplished through utilization of the auto-balance function of the cluster, followed by single node tests with storage QOS policies and CPU reservations. This test depended on the prerequisite of an Azure Stack HCI cluster, along with a SQL Server VM and Desktop VMs running IOMeter to generate simulated user load. HammerDB measured transactions per minute on the SQL database.

The advantage of the Azure Stack HCI cluster in this situation is the ability to properly implement QOS policies for VM disks residing on clustered volume storage, including maximum, minimum, dedicated, and aggregate settings.

The contrasted solution, without clustered storage, is limited to a per-VM application of the QOS policies and settings.

Table 5. Workload Protection: Noisy Neighbor with Azure Stack HCI vs. Without

In our test above, we baselined a SQL instance on a near empty node at 1.1 million average TPM during a TPC-C test. This SQL Server VM had 32vCPU and 64 GB of memory assigned. Our “user” workload consisted of 10 4vCPU, 16 GB memory Windows, and 11 multi-session VMs running IOMeter as a load generator.

When introducing the user-simulated load, the TPM average of the SQL server was reduced by 40% until Auto Balance (every 30 minutes per default setting) relocated VMs to reduce resource contention on the specific node.

To protect the SQL workload from contention on one node, we disabled the Auto Balance feature to ensure VMs could not be relocated across the cluster. Then, storage QOS policies were assigned to both the VM to protect and the VMs to protect against. A minimum IOPS initiator setting of 7,500 with a maximum of 8,200 was applied to the SQL VM. The user VMs were set to a maximum of 500. With this configuration, IOPS for the SQL instance averaged 6,000 and the TPM via HammerDB averaged 1,000,000. This was sustained for the duration of the test.

The simplicity of a few PowerShell commands to set up these policies along with the out-of-the-box capability of Azure Stack HCI was shown to be superior to traditional testing using trial-by-error and per-VM settings via Hyper-V only.

Table 6. QOS Policy and Performance Impact

5. Conclusion

Azure Stack HCI showed an overall advantage in each of the tests, scoring 35, while the test “without” scored 50 (where lower is better). The results illustrate the skill and effort advantage that Azure Stack HCI provides over a combination of common tasks and capabilities.

In general terms, Azure Stack HCI demonstrated capabilities that required less experience and effort than typical on-premises solutions without it. Enterprises looking to simplify management of combined on-premises private and public cloud infrastructure, or reduce costs during their transition to public cloud, have a capable option in this solution.

Of particular note is the ability to use Azure Stack HCI to deploy and manage Azure Virtual Desktops on-premises as another host pool location in conjunction with virtual desktops and apps hosted in Azure public cloud. Further, Azure Stack HCI uniquely offers Windows 10 and 11 multi-session as an operating system licensed for on-premises usage. VDI teams can benefit greatly from a unified console to maintain a multi-location VDI farm with familiar tooling, as well as from Intel Optane persistent memory to support more VDI users, lower TCO to support the current users, or affordably increase the resources assigned to each VDI user.

While IT organizations strive to move their workloads to a cloud operating model, the fact is that on-premises infrastructure will remain a critical component of the global IT landscape for the foreseeable future. Azure Stack Hyperconverged Infrastructure offers the benefits of cloud-based computing without duplicating engineering resources or sacrificing business needs that dictate on-premises workloads. Azure Arc provides a unified experience for managing resources in Azure. Employing a consistent hardware and software infrastructure, based on Azure Arc-enabled infrastructure and the underlying Intel technologies, across environments provides the agility and flexibility required by today’s cloud operating models.

6. Appendix

Assessment Methodology

In this appendix, we explain our assessment methodology and how we scored Azure Stack HCI compared to a typical on-premises deployment without Azure Stack HCI. The methodology and scoring used a rubric that we developed to evaluate the capabilities and effort requirements for hybrid cloud solutions. We also provide the results from a questionnaire to several customers currently using the solution in their data centers.

Scoring

To score the outcome of the selected test capabilities, we gave each step a score between 1 and 5 to represent effort and required skill set. These scores were then totaled to give a summary determination of the results. A lower total is better.

• Basic admin/engineer skill required and low effort to complete – 1
• Intermediate engineer skill required or significant effort – 2
• Multiple skill sets required and/or advanced effort – 3
• Project coordination or architectural advisement required – 4
• Architecture skill set and multiple specialized roles required – 5

Customer Surveys

We surveyed three enterprises currently using Azure Stack HCI about their experience with its implementation and use for their respective infrastructure and workload hosting needs. These surveys did not affect scoring but offer insight into different use case scenarios for managing Microsoft Azure Arc-enabled infrastructure from the Azure portal. Note that planned and in-progress updates to the solution stack may address some of the customer feedback.

Customer 1
Our first customer, a US-based food manufacturer, has integrated Azure Stack HCI into its factory locations to have onsite hosting for their manufacturing workloads. It found that the solution addressed the original challenge and was easy to maintain with updates from the vendor. The continuing evolution of their solution somewhat negatively impacted the initial setup and installation by Microsoft, which still has many features in a preview state.

The company indicated that the Microsoft engineers and support technicians assisting with the installation were very knowledgeable. For future improvements, the customer would like to be able to use Windows Admin Center as a fully functional central management console.

Customer 2
The second surveyed customer, a manufacturing company in Belgium, chose Azure Stack HCI as part of a data center renewal effort. Although it considered another HCI solution, the key criteria for the selection of Azure Stack HCI was its integration with Azure and its ease of use. While there were hurdles migrating from the existing Hyper-V farm, it found the storage capabilities along with Windows Admin Center integration compelling features in daily usage.

This customer is growing its Azure Stack HCI usage and increasing the size of the cluster, with main use cases being workload expansion and AKS enabled by Azure Arc services. Still, it would like further development of the product to include more access to advanced Hyper-V and Storage Spaces Direct via the Admin Center.

Customer 3
The final customer we engaged is a large retailer from the United Kingdom. Some key aspects that led to its decision to select Azure Stack HCI included: ease of management from the Azure portal, flexible licensing model, consistent support and updates, and stretch clustering for automatic DR capabilities.

Resiliency and efficiency were key drivers in its decision-making. A central decision point was the flexibility of locale for virtual servers while retaining a single management console. This customer also expects to benefit from storage resiliency and long-term support by the manufacturer.

This customer’s implementation of the Azure Stack HCI cluster required an older version of the Azure Stack HCI OS to facilitate migration from its existing Hyper-V infrastructure. The customer would like to see improved migration tooling and storage replica capabilities.

7. About William McKnight

William McKnight is a former Fortune 50 technology executive and database engineer. An Ernst & Young Entrepreneur of the Year finalist and frequent best practices judge, he helps enterprise clients with action plans, architectures, strategies, and technology tools to manage information.

Currently, William is an analyst for GigaOm Research who takes corporate information and turns it into a bottom-line-enhancing asset. He has worked with Dong Energy, France Telecom, Pfizer, Samba Bank, ScotiaBank, Teva Pharmaceuticals, and Verizon, among many others. William focuses on delivering business value and solving business problems utilizing proven approaches in information management.

8. About GigaOm

GigaOm provides technical, operational, and business advice for IT’s strategic digital enterprise and business initiatives. Enterprise business leaders, CIOs, and technology organizations partner with GigaOm for practical, actionable, strategic, and visionary advice for modernizing and transforming their business. GigaOm’s advice empowers enterprises to successfully compete in an increasingly complicated business atmosphere that requires a solid understanding of constantly changing customer demands.

GigaOm works directly with enterprises both inside and outside of the IT organization to apply proven research and methodologies designed to avoid pitfalls and roadblocks while balancing risk and innovation. Research methodologies include but are not limited to adoption and benchmarking surveys, use cases, interviews, ROI/TCO, market landscapes, strategic trends, and technical benchmarks. Our analysts possess 20+ years of experience advising a spectrum of clients from early adopters to mainstream enterprises.

GigaOm’s perspective is that of the unbiased enterprise practitioner. Through this perspective, GigaOm connects with engaged and loyal subscribers on a deep and meaningful level.

9. Copyright

© Knowingly, Inc. 2022 "Managing Microsoft Azure Arc-Enabled Infrastructure from the Azure Portal" is a trademark of Knowingly, Inc. For permission to reproduce this report, please contact sales@gigaom.com.