AZ - 700 - 03 - Design and implement Azure ExpressRoute

Introduction

Azure ExpressRoute is a private, enterprise-grade connection that enables secure and reliable network connectivity between your on-premises infrastructure and Microsoft’s global network. It bypasses the public internet and provides high throughput, low latency, and consistent performance for mission-critical workloads.

Architecture Overview

Azure ExpressRoute uses a hub-and-spoke model involving:

• A physical on-premises site(on the left)

• Private fiber connectivity to Microsoft’s edge

• Microsoft Secure Edge routers

• Azure VNets (private address space)

• Microsoft public services (e.g., Microsoft 365, Azure SQL, Storage)

A Meet-Me Room (MMR) is a secure physical location within a colocation data center where multiple telecommunications carriers, cloud providers (like Microsoft), and enterprises interconnect their networks.

Connection Models

1. ExpressRoute via Service Provider Model (Layer 3)

• Connectivity is managed through a partner or a colocation facility.

• The partner establishes the cross-connect to Microsoft’s edge via a Meet Me Room (MMR).

• Suitable for standard enterprise workloads with high availability (two connections) and moderate control.

2. ExpressRoute Direct (Layer 2)

• Provides a dedicated port pair (10 Gbps or 100 Gbps) directly to Microsoft.

• Enables direct monitoring (e.g., signal light levels, propagation).

• Offers granular control and is ideal for enterprises needing extreme performance and flexibility.

• Requires customer setup and responsibility for all routing and cabling.

ExpressRoute Circuit

A circuit is the logical container for the ExpressRoute connection. Once provisioned:

• You can attach it to virtual networks.

• Use it for both public and private Azure services.

• Choose bandwidth, location, SKU, and routing preferences.

A Meet-Me Room (MMR) is a secure physical location within a colocation data center where multiple telecommunications carriers, cloud providers (like Microsoft), and enterprises interconnect their networks.

in the image below (express direct) we have a 100Gbps fiber connection between on-prem and microsoft secure edge, with two differnt circut one 40 Gbps to m365 services and one 5 Gbps to azure services.

• ExpressRoute is the overall Azure service that enables private connectivity between your on-premises network and Microsoft cloud services.

• SKU Options refer to the tiers and configurations you choose when provisioning an ExpressRoute circuit — essentially, SKUs are part of the circuit configuration.

SKU Options

• Standard: Access to Azure services within a geopolitical region(e.g. US east, US West).

• Local: Restricts access to one (sometimes two) Azure region. Lower cost.

• Premium: Enables global access across regions and increases route limits.

Each SKU allows varying capabilities depending on network size, access needs, and resiliency requirements.

Edge and Region Mapping

• ExpressRoute peering is done at Microsoft Edge locations.

• These are globally distributed and not limited to Azure regions.

• For example, a customer in Perth can peer through a local Edge site to reach multiple paired Azure regions.

Billing Models

Two billing options are available:

1. Unlimited data: Fixed monthly fee with unrestricted traffic.

2. Metered data: Pay-per-GB model; more cost-effective for lighter usage.

The Azure pricing calculator helps determine the best model based on expected throughput.

Deployment Process in Azure Portal

The complete setup may take several weeks, but we can start by reviewing how the provisioning process works in the Azure portal, assuming the cabling is correctly positioned.

1. Create an ExpressRoute circuit.

2. Choose deployment region and provider.

3. Select SKU, bandwidth and billing method

if we choose provider model depends on the region that we select we have different provider:

Next, configure the gateway to land traffic—this process will be familiar if you’ve set up a VPN gateway before(we just select express route instead of VPN).

1. Provision a virtual network gateway in Azure.

2. Attach the gateway to the ExpressRoute circuit.

3. Complete provider-side setup or configure Direct ports.

Express Route Peering Types:

What actually goes across our express route circuit?

1. Private peering

2. Microsoft peering

Here one connection across an Expressroute can carry both Microsoft peering traffic and private peering traffic, you can use one or the other or both:

So one connection through express route can carry both microsoft and private peering traffic.

Private Peering in Azure ExpressRoute

Private peering is used to access Azure Virtual Networks (VNets) through private IP address space. It enables connectivity between your on-premises network and Azure resources via ExpressRoute.

Configuration requirements:

• IP subnets: Two subnets not part of any VNet address space—one for the primary link, one for the secondary.

• VLAN ID: A valid on-premises VLAN ID to establish the peering.

• ASN: An Autonomous System Number for BGP peering.

• BGP session: Configure your on-prem Edge router to advertise routes to Azure via BGP.

• MD5 hash (optional): Use for BGP session authentication if required.

VLAN ID is used in ExpressRoute private peering to separate traffic types at the data link layer between your router and Microsoft’s edge (MSEE). VNets don’t use VLANs because they operate at Layer 3, using subnets, route tables, and security rules for isolation.

Now, once you have this private pairing established, we just need to go through and actually do the connection itself.

If you have a provisioned ExpressRoute circuit, go to the Connections tab and select Add. Then, choose the virtual network you want to connect. You’ll need a gateway subnet in the VNet, along with an ExpressRoute gateway—either already provisioned or created during this step. Once the gateway is in place, you can configure additional settings such as routing weight for traffic distribution.

Microsoft Peering in Azure ExpressRoute

Microsoft peering enables private connectivity to Microsoft public services such as Microsoft 365, Azure Storage, and Azure SQL via ExpressRoute.

Configuration requirements:

• Public IP prefixes: Must be owned by your organization and registered with a recognized Routing Internet Registry (RIR/IRR).

• Subnets: A pair of registered subnets—one for the primary link, one for the secondary.

• ASN: Autonomous System Number used for BGP peering.

• VLAN ID: Valid on-premises VLAN ID for establishing the peering session.

• Advertised prefixes: A list of all public IPs your organization plans to advertise over the BGP session.

• Routing Registry Name: The registry (e.g., ARIN, RIPE) that confirms ownership of the advertised prefixes.

• BGP configuration: To establish routing between your network and Microsoft.

• Route filters: Define which Microsoft services and regions are accessible over the peering session.

Route Filters and BGP Communities

• Route filters control which services we want to carry over Microsoft Peering.

• BGP communities identify specific Azure services and regions.

• You can selectively include services like Storage or SQL while excluding services like Exchange or SharePoint, which are optimized for internet routing.

Consider this scenario: you’re a customer consuming Azure services hosted in Australia East, and you’re specifically interested in accessing storage services. Instead of routing all Microsoft traffic through your ExpressRoute circuit, you can apply a route filter to select a BGP community that corresponds only to Azure Storage in Australia East. This allows you to target exactly the services you need.

You’re not required to select all traffic types. In fact, services like Microsoft 365 (e.g., Exchange Online, SharePoint) are designed and optimized for internet routing, so using ExpressRoute for them is typically unnecessary unless you have a specialized scenario. Using route filters gives you precise control over what traffic flows through your ExpressRoute circuit.

Resiliency and High Availability

In the context of “Resiliency and High Availability” for Azure ExpressRoute, resiliency refers to the system’s ability to recover quickly from failures and continue operating with minimal disruption. Here’s how it breaks down:

Resiliency ensures your network connection can withstand failures (e.g. link failure, hardware fault) without going down completely.

High Availability complements this by ensuring constant uptime—you always have at least one active path for traffic.

• ExpressRoute circuits are active-active by default.

• Each circuit has dual connections and subnets.

• Use Bidirectional Forwarding Detection (BFD) for rapid failure detection.(BFD is able to detect if an express route goes down very quickly)

• BFD is enabled by default and helps reduce failover time from minutes to seconds.

Multi-Region Redundancy

• Enterprises can provision ExpressRoute circuits in multiple Azure regions.

• Weighted BGP routing enables active-passive or load-balanced setups.

• Multiple circuits can serve as failover paths.

Hybrid Redundancy with VPN (as a secondry failover for express route)

• A VPN can serve as a backup for ExpressRoute in case of circuit failure.

• This creates two diverse paths:

  • Private fiber (ExpressRoute)

  • Public internet (VPN)

• The same Azure VPN gateway can serve dual purposes (primary for VPN, failover for ExpressRoute).

Encryption over ExpressRoute

By default, ExpressRoute traffic is private but not encrypted. Two encryption options exist:

1. Overlay VPN (IPsec): Create a VPN tunnel over the ExpressRoute connection.

2. MACsec: Layer 2 encryption available with ExpressRoute Direct.

Encryption should be planned during initial deployment to meet compliance requirements.

ExpressRoute Global Reach

• Enables connectivity between two on-premises sites through Microsoft’s global network.

• Bypasses Azure VNets entirely.

• Use case: connecting San Francisco and London data centers via existing ExpressRoute circuits.

This is configured by linking the two circuits and enabling route propagation between them.

FastPath

• Improves performance for high-throughput scenarios.

• Bypasses the ExpressRoute gateway after initial setup (control plane) and sends traffic directly on the data plane.

• Requires Ultra Performance Gateway SKU (Gateway v3).

• Ideal for scenarios like remote desktop sessions, where low latency is critical.

Configure ExpressRoute FastPath:

Updating an existing connection to enable FastPath:

$connection = Get-AzVirtualNetworkGatewayConnection -Name "MyConnection" -ResourceGroupName "MyRG"
$connection.ExpressRouteGatewayBypass = $True
Set-AzVirtualNetworkGatewayConnection -VirtualNetworkGatewayConnection $connection

Monitoring and Troubleshooting ExpressRoute Connection

Verify circuit provisioning and state through the Azure portal

Get-AzExpressRouteCircuit

• Shows all the backend data for current express route that is provisioned including: provisioning state, bandwidth, provider, location, etc.

Provider Status: Provisioned → The service provider has completed their setup.

Circuit Status: Enabled → You (the customer) have finished your side of the configuration.

These two indicators help you quickly understand where the setup stands and who is responsible for the next step.

Reset a failed circuit:

Connect-AzAccount 
Get-AzSubscription 
Select-AzSubscription -SubscriptionName "Replace_with_your_subscription_name" 
$ckt = Get-AzExpressRouteCircuit -Name "ExpressRouteARMCircuit" -ResourceGroupName "ExpressRouteResourceGroup" 
Set-AzExpressRouteCircuit -ExpressRouteCircuit $ckt

Set-AzExpressRouteCircuit

• Refreshes a circuit even without changes. Useful for resetting failed states.

Validate Peering Configuration

ExpressRoute circuit is up and running, but peering isn’t functioning properly (we’re not seeing private or Microsoft traffic flow across the connection)

• Check connection and provisioning status under the circuit’s Peering tab.

• Ensure subnets and peerings are configured and advertised correctly.

Validate ARP: Layer 2 Diagnostics

• Use ARP tables to validate device connections and MAC-level communication.

Network Performance Testing

• Use the Azure Connectivity Toolkit (Azure Cwrt) to simulate and test:

  • Throughput

  • Latency

  • Packet behavior

• Also supports synthetic traffic testing with different packet sizes and patterns.

• 

Summary: Service Provider Model vs. Direct

Final Considerations

• Choose the right SKU and model (Direct vs Provider) based on:

  • Bandwidth needs

  • Regional access

  • Security/compliance requirements

• Configure peering, encryption, and redundancy early to avoid operational complexity later.

• Monitor proactively using built-in tools and routing visibility.