20 min read
Created on
Updated on

Azure Local – Design the infrastructure


I have seen many designs of Azure Local stacks that have encountered issues that could have been prevented if the design was changed to follow some best practices. I wanted to share some of the things I have come across that should never have been built the way it was. I have also included some drawings of minimal designs based on my experiences.

The examples below focus on cloud-connected, hyperconverged deployments using Storage Spaces Direct and external DNS. SAN-backed and disconnected deployments need their own validated designs. My preferences are not all platform requirements; use the linked Microsoft Learn guidance for support constraints and check the version that matches your deployment.

Pre-deployment

Validate the complete solution before ordering

I would not order servers based only on CPU, memory, disk capacity, and a familiar NIC brand. Validate the exact OEM/catalog configuration, the NIC’s qualification for its intended traffic role, and the supported driver, firmware, and Solution Builder Extension (SBE) combination.

An adapter qualified for management or compute traffic is not automatically qualified for storage traffic. The host network requirements also rule out relying on inbox Microsoft NIC drivers. Get the server and network teams to agree on the supported configuration before procurement.

Break-glass accounts

Azure Local hardens the built-in Administrator account during cluster creation. It renames the account and changes the password as part of its security hardening procedure — and the new credentials are not documented anywhere. If you later lose network connectivity to the domain (for example during an update where a NIC driver fails), you will not be able to log in locally because the original credentials no longer work.

For emergency access, I recommend creating at least two additional local administrator accounts on each node immediately after OS installation, before the cluster is deployed. Test them, document them, and store the credentials in a password vault. For these additional break-glass accounts, both names and passwords should be unique across nodes.

net user <nodename>BreakGlass1 "YourSecurePassword14+" /add
net localgroup Administrators <nodename>BreakGlass1 /add

net user <nodename>BreakGlass2 "AnotherSecurePassword14+" /add
net localgroup Administrators <nodename>BreakGlass2 /add

Do this before joining the domain or starting the cluster deployment. Without break-glass accounts, a lockout scenario can result in a full rebuild.

These emergency accounts are not the deployment/management identity. In a Local Identity with Key Vault deployment, Microsoft requires a separate local administrator account, not the built-in Administrator, with the same credentials across all nodes. Keep those credentials available for node addition and repair, and plan their rotation and expiration. Do not apply my per-node break-glass naming/password pattern to that shared management account.

Password requirements

The current Active Directory preparation guidance calls for passwords of at least 14 characters, including uppercase, lowercase, numbers, and special characters. Do not design around the older “three of four categories” wording. Check the current limits for your chosen identity/deployment method and validate the local administrator and deployment user credentials before starting.

Active Directory

Active Directory is now a design choice, not the only deployment model. Local Identity with Azure Key Vault is generally available and supports deploying without an Active Directory domain. It still needs its documented DNS, static IP, credential, and Key Vault prerequisites. The Active Directory examples below remain relevant when that is the model you choose.

Joined Azure Local to production domain

I see it all the time. Azure Local is been joined to the same domain as production where all kinds of member servers and clients also are joined to (client devices should not be in any domain, they should be joined to Entra ID and only connected to domain member servers via Cloud Kerberos Trust). This is not a recommended approach. Yes, if your domain is fully configured with LAPS, Tiering, Kerberos armoring, disabling of older protocols e.g., you would have a “safe” domain that you could join your Azure Local nodes into. However, I would still not do it. I will always prefer a dedicated management Active Directory for Azure Local. This is because I can then segment this management domain completely from all other areas of my environment. In some industries this is referred to as “out-off-band”. This management Active Directory should only be reached from Azure Local nodes management network and from a secured instance of Azure Bastion.

That dedicated management domain is my isolation preference, not a Microsoft requirement for a separate forest. For an AD-based deployment, prepare the required dedicated OU, deployment user permissions, and blocked GPO inheritance. Enforced GPOs can still apply despite inheritance blocking, so inspect the effective policies rather than assuming the OU is isolated. Moving deployed node computer accounts to another OU is not supported. Validate this with the AD team before deployment using the Active Directory preparation requirements.

Active Directory for Azure Local running on top of stack

I believe this speaks for itself, but having the Active Directory that Azure Local is joined to, running on top of the same stack, is prone to fail at some point. That is why I like running management Active Directory in Azure. No need for extra hardware for management domain, just running a few small VMs in Azure.

Active Directory DNS Servers

Treat the DNS addresses supplied for the Azure Local infrastructure and Arc resource bridge as long-lived infrastructure. Changing infrastructure DNS servers after deployment is currently unsupported; this is different from changing DNS settings inside a guest or on a workload network.

For the external-DNS designs in this article, at least one configured DNS server must remain available outside this Azure Local cluster, including during startup, maintenance, and recovery. Hosting all required DNS servers on the stack creates the same circular dependency as hosting its only domain controllers there.

I prefer stable resolver/proxy addresses that outlive individual DNS servers. Depending on the design, that could involve Azure DNS Private Resolver, Azure Firewall DNS Proxy, or another supported forwarding arrangement. When using AD DNS server addresses directly, preserve those addresses through server replacement. Whichever design we choose, it must resolve the required on-premises names and public Azure/Arc endpoints. Check the DNS requirements in the network planning guidance.

Networking

One Azure Local stack per management VLAN

You may run into serious trouble if you choose to deploy multiple Azure Local Stacks on the same management VLAN. Azure Local stacks must always have separate VLANs for storage if using the storage configuration with a storage switch between nodes.

Compute VLANs can be shared between stacks.

And you would think that sharing a large management network for all your stacks would not cause any issues as long as no IP overlapping is present.

Security wise it is recommended to have a separate VLAN for each Azure Local management network, but there is another issue; Installing Azure Local on one stack, and then later you want to install a new Azure Local stack on one of the nodes (maybe due to issues that require total redeployment using a new stack deployment on the first node and later move workloads and reinstall other nodes into new stack), the MOC ARB can reuse the same MAC address, causes unstable network and lose of access from Azure to the Azure Local stacks.

Azure Local MOC starts MAC address assignment using these 4 letters first: 02-ec Rest of the MAC address is calculated, and should ensure no MAC address reuse is possible.

However, I have seen real life example of that not being true if one performs the scenario I described earlier in this section.

Conclusion: use one management VLAN per Azure Local stack. Use subnetting to have segmented your network into smaller ranges.

You can start with a 10.0.0.0/8 IP range.

If you have an Azure Local stack with 2 nodes you must have the following amount of IP addresses:

  • 1 for network address (standard first reserved IP)
  • 1 for default gateway
  • 1 for broadcast traffic (standard last reserved IP)
  • 1 per node
  • At least 6 consecutive available addresses for Azure Local infrastructure, separate from the node addresses

Consider leaving IP addresses in spare. A /28 is an example for this small two-node design, not a universal sizing rule. Account for growth and all other addresses your design consumes before choosing the subnet size.

So my first subnet would be:

  • 10.0.0.0/28

And my next subnets would be:

  • 10.0.0.16/28
  • 10.0.0.32/28
  • 10.0.0.48/28
  • 10.0.0.64/28

And so on.

Plan infrastructure and workload address ranges together

Reserve the infrastructure pool explicitly; do not let node addresses, DHCP, or another allocation consume it. Define explicit, non-overlapping IP pools for workload logical networks as well, and check the networks and services the instance must reach in both the datacenter and Azure.

Arc resource bridge (ARB) and AKS also use internal address ranges. The current network planning guidance distinguishes them:

Component / deploymentService CIDRPod CIDR
ARB on a new deployment from release 2610, using ARB 1.8.010.96.0.0/2410.244.0.0/16
ARB deployed before 2610, including after an upgrade10.96.0.0/1210.244.0.0/16
AKS, including on 2610 and later10.96.0.0/1210.244.0.0/16 by default; customizable at creation

Upgrading an existing deployment does not shrink its ARB service range, and AKS does not inherit ARB’s smaller range. Check infrastructure addresses, workload networks, and required DNS/proxy service addresses against the applicable internal ranges. This is about avoiding conflicts along required connectivity paths, not declaring every unrelated subnet in the datacenter unusable.

RDMA-capable storage adapters

RDMA (Remote Direct Memory Access) support on network adapter cards is essential when working with Storage Spaces Direct (S2D) in Azure Stack HCI or Azure Local environments because it enables low-latency, high-throughput communication between nodes with minimal CPU overhead. RDMA allows data to be transferred directly between the memory of different servers without involving the CPU, significantly improving performance for storage traffic. This is critical for S2D, which relies heavily on east-west network traffic to mirror and distribute storage across cluster nodes. Without RDMA, performance suffers due to increased latency and CPU load, reducing the efficiency and scalability of the system.

RDMA protocol: iWARP vs RoCE

For RDMA storage traffic, choose a supported protocol and use the same protocol on both ends:

  • RoCEv2 (RDMA over Converged Ethernet) — UDP-based. Requires end-to-end Data Center Bridging (DCB) configuration, including Priority Flow Control (PFC) and Enhanced Transmission Selection (ETS). QoS planning is critical during high network activity such as failovers and backup/restore operations.
  • iWARP — TCP-based. More forgiving on switch configuration since TCP handles congestion control natively. Easier to deploy but may have slightly different performance characteristics.

Validate the complete NIC, driver, firmware, and switch design with the OEM before ordering network equipment. iWARP does not require the lossless DCB fabric that RoCEv2 requires; it is not a matter of buying a switch that “supports iWARP.” Do not assume that falling back to non-RDMA traffic makes an unqualified storage adapter supported. Use the documented host network requirements.

Lack of RSS support (Mandatory)

RSS (Receive Side Scaling) support on network adapter cards is important when using Storage Spaces Direct (S2D) in Azure Local because it ensures efficient processing of high-volume network traffic across multiple CPU cores. S2D generates substantial east-west traffic between cluster nodes, and without RSS, all incoming traffic would be handled by a single core, creating a bottleneck. RSS distributes network processing across multiple cores, improving throughput and reducing latency. This parallel processing is essential for maintaining consistent performance and maximizing the benefits of S2D in high-performance, hyper-converged infrastructure environments.

My go-to network adapter card

I have often worked with Mellanox/NVIDIA adapters, but the brand is not a purchasing specification. Choose the exact adapter SKU qualified for the required Azure Local traffic roles in the OEM solution, with its supported driver and firmware. I would not use older ConnectX examples as a current shopping list.

Switchless networking for storage

In some setups I have storage intent without a switch. This is referred to as switchless and is supported for up to 4 nodes (please look at the requirement if aiming for 4 node support, because it requires 6 storage NICs per node).

However, as of writing this article, if we choose 2 node setup to begin with in a switchless setup, Microsoft currently does not support expanding to 3 or 4 nodes. This could be a key point for choosing to include a supported storage switch to begin with, because we can then go for 2 or 4 storage NICs per node and have the capability to expand to more nodes in the future.

Compute intent with DHCP IP addresses

I often see that there is DHCP enabled on compute network VLANs. That can be enabled for a valid reason for workloads running on top of the Azure Local stack. However, no need to assign any static or DHCP IP addresses (you could do a closed untagged VLAN on the compute intent with static IP addresses on each node just to enable the failover cluster to recon the network as compute, but not required) on the untagged VLAN going go the NICs on the compute intent. I like to configure only tagged VLANs on compute intent because it is only used for workloads running on top of the Azure Local stack.

Static IP addresses on management network

Azure Local cluster validation requires static IP addresses on the management network. DHCP will cause deployment failures. Configure static IPs using SCONFIG (Option 8) during the initial OS setup on each node.

While configuring networking, also make sure to disable all disconnected network adapters. Enabled but disconnected NICs can cause validation failures during cluster deployment. If you have physical ports that are not cabled, disable the adapter in the OS before starting the deployment.

Software-Defined Networking (SDN)

SDN enables zero-trust micro-segmentation using Network Security Groups on Azure Local. It is a powerful feature, but it adds significant operational complexity.

There is one critical design constraint to be aware of: SDN is a one-way door. You can choose to enable SDN after the initial deployment — it does not need to be part of day one. But once deployed, you cannot remove it. There is no supported way to uninstall SDN from an Azure Local cluster.

My recommendation is to defer SDN unless the organization is ready for the additional operational overhead. Adding it later depends on the network intents you choose now. For SDN enabled by Azure Arc, current unsupported configurations include more than three intents, combined compute/storage intents, a standalone compute intent on a single-node deployment, and three-intent configurations on two-node or three-node switchless deployments.

A management+compute intent with a separate storage intent is therefore a useful starting pattern when Arc-managed SDN is on the roadmap. Check the SDN enabled by Arc limitations for your exact topology; do not apply legacy on-premises-managed SDN assumptions to the Arc-managed model.

HINT

If you plan to run AKS Arc (Kubernetes) on the cluster, keep its logical network and AKS workload VM NICs free of NSGs. Both placements are unsupported; moving the NSG from the logical network to the NIC is not a workaround. Use a dedicated NSG-free logical network for AKS and design any additional controls using supported AKS networking features.

Decide the outbound path before deployment

Choose the supported public-path or private-path connectivity model, and validate firewall, proxy, DNS, and endpoint requirements together. Arc Gateway is an option for supported public-path traffic and is required for the documented private-path model; it does not carry every Azure Local dependency.

I would test the required destinations from the actual deployment network, not just from an administrator’s laptop. See the network planning guidance and my Arc Gateway overview for the distinction between the gateway, local proxy, and remaining direct dependencies.

Storage

Disk controller configuration

Storage Spaces Direct requires data disks to be presented in JBOD, HBA, or pass-through mode — not hardware RAID. If the data disks are behind a RAID controller, S2D will not be able to manage them and deployment will fail.

However, the OS disk should have RAID 1 protection for redundancy. This means you typically need two separate controllers:

  • Controller 1: OS disk in RAID 1 mode (using a BOSS card or a dedicated RAID controller)
  • Controller 2: Data disks in HBA/JBOD mode for Storage Spaces Direct

You can verify that data disks are correctly presented by checking BusType:

Get-PhysicalDisk | Select-Object FriendlyName, BusType, MediaType, Size

If BusType shows RAID, Storage Spaces Direct will not work with those disks. It should show SAS, SATA, or NVMe.

HINT This is primarily relevant for non-certified hardware or test rigs. All certified Azure Local hardware configurations include BOSS cards or equivalent solutions for the OS disk out of the box.

Cluster Shared Volumes

Microsoft recommends us to configure 1 CSV per node in the cluster and divide workloads across these CSVs. This is also default when deploying a new stack. So if our stack consists of 4 nodes, we should have 4 CSVs created and in production mode, we should ensure that each node is owner of 1 CSV to spread ownership of CSVs across nodes.

Cache and capacity storage

Some customers goes for all flash SSD, but we can gain more speed and better price to space ratio, if we use use tiered model.

We can use a combination of NVMe, SSD, and HDD in Azure Local (Storage Spaces Direct)—this is actually a common and recommended approach to build a hybrid storage configuration that balances performance and capacity.

Here’s how it typically works:

  • NVMe drives are used as the cache tier (also called performance tier).
  • SSD and/or HDD drives are used as the capacity tier (also called capacity devices).

Benefits:

  • NVMe Cache Tier: Accelerates read/write performance by buffering data before it’s written to slower disks.
  • HDD Capacity Tier: Offers large storage capacity at lower cost per TB.
  • SSD Capacity Tier (optional): Used when you want faster capacity storage but without the high cost of all-NVMe.

Key Considerations:

  • All nodes in the cluster should have a consistent drive layout.
  • NVMe cache is especially beneficial in write-intensive or latency-sensitive workloads.
  • S2D automatically manages tiering between cache and capacity layers.

Typical Example:

A 2-tier configuration might look like:

  • 2 × NVMe drives (cache)
  • 4 × HDDs or SATA SSDs (capacity)

This setup is fully supported and optimized in Azure Local for performance and efficiency.

Raw capacity is not usable capacity

Size the storage pool after accounting for the selected resiliency, repair reserve, infrastructure volumes, and workload growth. Three-way mirroring, for example, consumes three times the usable volume capacity. For two-node production designs, evaluate nested resiliency before deployment rather than discovering the trade-off after filling the pool.

Microsoft’s volume planning guidance recommends leaving the equivalent of one capacity drive per server, up to four drives, unallocated for in-place repair. With both SSD and HDD capacity tiers, it recommends that reserve for each drive type. That reserve is not free space we should count toward workload growth.

Do the same exercise for CPU and memory: the surviving nodes must be able to carry the required workloads during maintenance or a failure. Include infrastructure services and AKS overhead, and agree on an expansion plan before the cluster is nearly full.

Cluster sizing

Single-node clusters

Azure Local supports single-node clusters, but they are not recommended for production workloads. Azure Local is designed with the assumption of 24/7 uptime and redundancy — similar to how Azure datacenters operate. A single-node cluster breaks this model:

  • No redundancy during updates or hardware failures
  • No failover capability for virtual machines
  • The Azure Portal experience degrades during maintenance windows since the only node is being serviced

Single-node clusters are suitable for testing, labs, and proof-of-concept environments. For production workloads, deploy a minimum of 2 nodes to maintain service availability during updates and failures.

Azure resource design

Subscription and resource group strategy

Azure Local VM management uses one custom location per Azure Local instance. The component documentation does not impose a blanket “one custom location per subscription” limit. The Azure Local FAQ also states that moving Azure Local resources between subscriptions or resource groups is unsupported; changing that placement requires redeployment.

My recommendation is:

  • One platform subscription per Azure Local cluster — my preference for separating infrastructure ownership and billing, not a platform limit
  • One resource group per cluster — contains all cluster resources

This approach gives clean separation between sites, simplified billing per location, and easy decommissioning — if a cluster is retired, you can clean up the entire resource group or subscription without affecting other environments.

The key takeaway is to plan your resource group and subscription strategy upfront. Reorganizing later is not supported.

For deeper guidance on how to fit Azure Local into your Azure landing zone design and align with the new Local management group, see Azure Local - Landing zone design with the Local management group.

Review inherited policies and privileged operations

Check the policies inherited from the landing zone before onboarding the hosts. Ordinary Arc-server patching assumptions must not push out-of-band updates onto Azure Local hosts. Use the Azure Local solution update workflow, not the individual Arc machine’s update pane or a generic server patching policy.

Separate deployment permissions from day-to-day operations and use time-bound elevation, such as PIM, where appropriate. Treat permission to deploy host extensions as privileged access: Arc extensions execute as Local System on Windows or root on Linux. Agree on extension approval, policy exemptions where needed, and operational ownership before granting broad contributor access.

Designs

Design 1: Two-node setup with switched storage

Two Azure Local nodes connect through separate management/compute and RDMA storage ports to the same redundant MLAG ToR switch pair, with external DNS and an outbound path to Azure.

For a switched hyperconverged design, Microsoft now recommends using the same MLAG-configured ToR pair for management, compute, and storage, rather than a separate storage-only switch pair. Separate pairs can introduce asymmetric connectivity failures. This is current design guidance, not a claim that every alternative topology is automatically unsupported.

This example keeps separate physical NIC ports for management+compute and storage while sharing the switches. Management+compute is one intent; storage is a second intent with its own VLANs and RDMA configuration. It is a topology sketch, not a replacement for the OEM’s validated port, VLAN, bandwidth, and QoS configuration. DNS and, where used, management AD remain available outside the cluster; their location and connectivity are separate design decisions.

Design 2: Two-node setup with switchless storage

Two Azure Local nodes connect management and compute ports to a redundant MLAG ToR switch pair, while two dedicated RDMA storage links connect the nodes directly without a storage switch.

Only storage is switchless: each node’s S1 and S2 ports connect directly to the corresponding ports on the other node. Management and compute still use the redundant ToR pair. External DNS and identity follow the same design considerations as Design 1.

This example uses a management+compute intent and a separate storage intent, rather than the three separate intents in the previous drawing. It avoids the three-intent combination that is unsupported for Arc-managed SDN on a two-node switchless deployment; check the other SDN limitations before enabling it.

Further reading

Microsoft’s Azure Local: 10 mistakes that can derail your deployment before it starts is a useful complementary field checklist, covering procurement, planning, lifecycle, and Day-2 operations. I used it to revisit this article, rather than duplicate its ten-mistake structure.

For hard support constraints and release-specific details, follow Microsoft Learn. In particular, the current network planning documentation dates the smaller ARB service CIDR to new 2610 deployments; do not infer that range from the blog post’s earlier date or from an existing cluster having been upgraded.