🔗 Azure ExpressRoute: Complete Design & Implementation Guide

📋 Table of Contents

ExpressRoute Connectivity Models
SKU and Tier Selection
Traffic Flow Architecture
Implementation with Azure CLI
Configuration Command Sequence

🌐 ExpressRoute Connectivity Models

1. CloudExchange Co-location

graph TB A[On-Premises Network] --> B[Customer Edge Router] B --> C[Cloud Exchange Provider] C --> D[Microsoft Edge Router] D --> E[Azure Virtual Network] subgraph "Co-location Facility" C F[Other Cloud Providers] G[Internet Exchange] end C --> F C --> G style A fill:#ff9999 style E fill:#99ccff style C fill:#99ff99

CloudExchange Co-location Model: Your infrastructure is physically co-located in the same facility as the cloud exchange provider. This model offers the highest bandwidth options (up to 100 Gbps) and lowest latency. The customer manages their own equipment in the co-location facility and connects directly to Microsoft's edge routers through the exchange provider's infrastructure.

2. Ethernet Point-to-Point

graph LR A[On-Premises Network] --> B[Customer Edge Router] B --> C[Ethernet Provider Network] C --> D[Microsoft Edge Router] D --> E[Azure Virtual Network] subgraph "Service Provider Network" C F[MPLS Backbone] G[Metro Ethernet] end C --> F F --> G style A fill:#ff9999 style E fill:#99ccff style C fill:#ffcc99

Ethernet Point-to-Point Model: A dedicated Ethernet connection from your premises to Microsoft through a connectivity provider. This model provides predictable bandwidth and latency, with options from 50 Mbps to 10 Gbps. The service provider manages the network path from your location to Microsoft's edge.

3. Any-to-Any (IPVPN)

graph TB A[Branch Office 1] --> C[IPVPN Provider Network] B[Branch Office 2] --> C D[Headquarters] --> C E[Data Center] --> C C --> F[Microsoft Edge Router] F --> G[Azure Virtual Network] subgraph "Service Provider MPLS/IPVPN" C H[PE Router 1] I[PE Router 2] J[Core MPLS Network] end C --> H C --> I H --> J I --> J style A fill:#ff9999 style B fill:#ff9999 style D fill:#ff9999 style E fill:#ff9999 style G fill:#99ccff style C fill:#cc99ff

Any-to-Any (IPVPN) Model: Integrates Azure into your existing IPVPN network, allowing all sites to communicate with Azure as if it were another branch office. This model leverages your existing WAN infrastructure and provides seamless connectivity across all locations. Bandwidth typically ranges from 10 Mbps to 10 Gbps per connection.

⚙️ ExpressRoute SKU and Tier Selection

ExpressRoute SKU Comparison Table

SKU	Bandwidth	VNet Links (Standard)	VNet Links (Premium)	BGP Routes	Use Case
Standard	50 Mbps - 10 Gbps	10	N/A	4,000	Regional connectivity
Premium	50 Mbps - 10 Gbps	10	100	10,000	Global connectivity
Local	50 Mbps - 10 Gbps	2	N/A	4,000	Single metro area
Direct	10 Gbps - 100 Gbps	Unlimited	Unlimited	Unlimited	Massive scale

SKU Selection Guidelines:

Local: Choose when all resources are in the same metropolitan area as the ExpressRoute location
Standard: For regional deployments within the same geopolitical region
Premium: For global deployments requiring cross-region connectivity
Direct: For enterprise-scale deployments requiring 10+ Gbps bandwidth

🔄 Traffic Flow Architecture

ExpressRoute Traffic Flow - Standard Configuration

flowchart TD A[On-Premises Network
10.0.0.0/16] --> B[Customer Edge Router
BGP AS 65001] B --> C[Primary Circuit
Microsoft Peering] B --> D[Secondary Circuit
Microsoft Peering] C --> E[Microsoft Edge Router 1
BGP AS 12076] D --> F[Microsoft Edge Router 2
BGP AS 12076] E --> G{ExpressRoute Gateway
Virtual Network Gateway} F --> G G --> H[Azure Virtual Network
172.16.0.0/16] H --> I[Subnet 1
172.16.1.0/24] H --> J[Subnet 2
172.16.2.0/24] subgraph "Azure Region" G H I J K[Route Table] L[Network Security Groups] end G --> K I --> L J --> L style A fill:#ff9999 style H fill:#99ccff style C fill:#99ff99 style D fill:#99ff99

Standard Traffic Flow: Shows the complete path from on-premises to Azure VNet. Traffic flows through redundant circuits for high availability. The ExpressRoute Gateway acts as the entry point into the Azure Virtual Network, with routing managed through Azure route tables and security enforced via Network Security Groups. BGP is used for dynamic routing between customer and Microsoft edge routers.

Multi-VNet Hub-Spoke with ExpressRoute

graph TB A[On-Premises Network] --> B[ExpressRoute Circuit] B --> C[Hub VNet
ExpressRoute Gateway] C --> D[Spoke VNet 1
Production] C --> E[Spoke VNet 2
Development] C --> F[Spoke VNet 3
DMZ] D --> G[Application Tier] D --> H[Database Tier] E --> I[Dev Resources] F --> J[Public Services] subgraph "Hub-Spoke Architecture" C D E F K[Azure Firewall] L[VPN Gateway] M[Route Server] end C --> K C --> L C --> M K --> D K --> E K --> F style A fill:#ff9999 style C fill:#ffcc99 style D fill:#99ccff style E fill:#99ccff style F fill:#99ccff

Hub-Spoke with ExpressRoute: Demonstrates a centralized connectivity model where the ExpressRoute Gateway resides in a hub VNet, providing connectivity to multiple spoke VNets. This architecture enables centralized security policies through Azure Firewall, shared services in the hub, and simplified routing. The Route Server can be used for advanced routing scenarios with Network Virtual Appliances.

ExpressRoute Global Reach

graph LR A[Office - New York
10.1.0.0/16] --> B[ExpressRoute
US East] C[Office - London
10.2.0.0/16] --> D[ExpressRoute
UK South] E[Office - Singapore
10.3.0.0/16] --> F[ExpressRoute
Southeast Asia] B --> G[Azure US East
172.16.0.0/16] D --> H[Azure UK South
172.17.0.0/16] F --> I[Azure Southeast Asia
172.18.0.0/16] subgraph "Global Reach Connections" B -.-> D D -.-> F B -.-> F end G --> J[Application Services] H --> K[Data Services] I --> L[Analytics Services] style A fill:#ff9999 style C fill:#ff9999 style E fill:#ff9999 style G fill:#99ccff style H fill:#99ccff style I fill:#99ccff

ExpressRoute Global Reach: Enables direct connectivity between on-premises locations through Microsoft's global network backbone. This eliminates the need for traffic to transit through Azure VNets when communicating between offices. Global Reach connections (shown as dotted lines) create a full mesh of connectivity between ExpressRoute locations, providing optimal routing for office-to-office communication while maintaining private connectivity to Azure services.

🚀 Implementation with Azure CLI

Step 1: Prerequisites and Resource Group

# Create resource group for ExpressRoute resources
az group create \
    --name "rg-expressroute-prod" \
    --location "East US" \
    --tags Environment=Production Project=Networking

Resource Group Creation: This creates a dedicated resource group for all ExpressRoute-related resources. The --tags parameter helps with resource organization and cost tracking.

Parameters:

--name: Resource group name following Azure naming conventions
--location: Azure region where metadata will be stored
--tags: Key-value pairs for resource management

Alternative locations: "West US 2", "Central US", "North Europe", "West Europe"

Step 2: Create ExpressRoute Circuit

# Create ExpressRoute circuit
az network express-route create \
    --resource-group "rg-expressroute-prod" \
    --name "er-circuit-primary" \
    --peering-location "Washington DC" \
    --bandwidth 1000 \
    --provider "Equinix" \
    --sku-family "MeteredData" \
    --sku-tier "Standard" \
    --tags Circuit=Primary Environment=Production

ExpressRoute Circuit Creation: This is the foundational component that represents your dedicated connection to Microsoft. The circuit must be provisioned by your connectivity provider after creation.

Key Parameters:

--peering-location: Physical location where you connect to Microsoft (must match provider capabilities)
--bandwidth: Circuit bandwidth in Mbps (50, 100, 200, 500, 1000, 2000, 5000, 10000)
--provider: Your connectivity provider (Equinix, AT&T, Verizon, etc.)
--sku-family: MeteredData (pay per GB) or UnlimitedData (flat rate)
--sku-tier: Standard, Premium, or Local

Next Step: Provide the Service Key to your connectivity provider for circuit provisioning.

Step 3: Configure BGP Peering

# Configure private peering for Azure VNet connectivity
az network express-route peering create \
    --resource-group "rg-expressroute-prod" \
    --express-route-circuit-name "er-circuit-primary" \
    --peering-type "AzurePrivatePeering" \
    --peer-asn 65001 \
    --primary-peer-subnet "192.168.1.0/30" \
    --secondary-peer-subnet "192.168.1.4/30" \
    --vlan-id 100 \
    --shared-key "MySecureSharedKey123!"

Private Peering Configuration: Sets up BGP peering for private connectivity to Azure VNets. This is required for VM-to-VM communication between on-premises and Azure.

Critical Parameters:

--peer-asn: Your BGP Autonomous System Number (must be private: 64512-65534)
--primary-peer-subnet: /30 subnet for primary BGP session
--secondary-peer-subnet: /30 subnet for secondary BGP session (redundancy)
--vlan-id: VLAN tag for traffic separation (coordinate with provider)
--shared-key: Optional MD5 authentication key

Subnet Requirements: Must use /30 subnets from private IP ranges not used elsewhere in your network.

# Configure Microsoft peering for Office 365 and Azure PaaS services
az network express-route peering create \
    --resource-group "rg-expressroute-prod" \
    --express-route-circuit-name "er-circuit-primary" \
    --peering-type "MicrosoftPeering" \
    --peer-asn 65001 \
    --primary-peer-subnet "192.168.2.0/30" \
    --secondary-peer-subnet "192.168.2.4/30" \
    --vlan-id 200 \
    --advertised-public-prefixes "203.0.113.0/24" \
    --customer-asn 65001 \
    --routing-registry-name "ARIN"

Microsoft Peering Configuration: Enables private connectivity to Microsoft cloud services (Office 365, Azure PaaS) using your public IP addresses. Required for accessing services like Azure Storage, SQL Database over ExpressRoute.

Unique Parameters:

--advertised-public-prefixes: Your registered public IP prefixes
--customer-asn: ASN registered with routing registry
--routing-registry-name: ARIN, RIPE, APNIC, etc.

Prerequisites: Must own public IP addresses and have them registered in a routing registry.

Step 4: Create Virtual Network and Gateway Subnet

# Create virtual network for ExpressRoute connectivity
az network vnet create \
    --resource-group "rg-expressroute-prod" \
    --name "vnet-hub-prod" \
    --address-prefix "172.16.0.0/16" \
    --subnet-name "default" \
    --subnet-prefix "172.16.1.0/24" \
    --location "East US" \
    --tags VNetType=Hub Environment=Production

Virtual Network Creation: Creates the Azure VNet that will connect to your on-premises network via ExpressRoute. The address space must not overlap with on-premises networks.

Planning Considerations:

--address-prefix: Choose RFC 1918 private addresses that don't conflict with on-premises
--subnet-prefix: Initial subnet for resources
Location must support ExpressRoute gateways

# Create dedicated gateway subnet (required for ExpressRoute Gateway)
az network vnet subnet create \
    --resource-group "rg-expressroute-prod" \
    --vnet-name "vnet-hub-prod" \
    --name "GatewaySubnet" \
    --address-prefix "172.16.0.0/27"

Gateway Subnet Creation: The GatewaySubnet is mandatory for ExpressRoute Gateways and must be named exactly "GatewaySubnet".

Sizing Guidelines:

Minimum: /29 (8 addresses)
Recommended: /27 (32 addresses) for future growth
High-performance gateways: /26 or larger

Note: This subnet cannot contain any other resources and cannot have NSGs applied.

Step 5: Create Public IP for ExpressRoute Gateway

# Create public IP for ExpressRoute Gateway
az network public-ip create \
    --resource-group "rg-expressroute-prod" \
    --name "pip-er-gateway" \
    --allocation-method "Static" \
    --sku "Standard" \
    --version "IPv4" \
    --zone 1 2 3 \
    --tags Component=Gateway Environment=Production

Public IP for Gateway: ExpressRoute Gateways require a public IP address for management and monitoring, though data traffic flows privately.

Configuration Options:

--allocation-method: Must be "Static" for gateways
--sku: Standard (recommended) or Basic
--zone: Availability zones for high availability

Security Note: This IP is used for Azure management only; customer traffic remains private.

Step 6: Create ExpressRoute Gateway

# Create ExpressRoute Virtual Network Gateway
az network vnet-gateway create \
    --resource-group "rg-expressroute-prod" \
    --name "vgw-expressroute-hub" \
    --public-ip-address "pip-er-gateway" \
    --vnet "vnet-hub-prod" \
    --gateway-type "ExpressRoute" \
    --sku "Standard" \
    --vpn-type "RouteBased" \
    --no-wait

ExpressRoute Gateway Creation: Creates the gateway that terminates ExpressRoute circuits in your VNet. This process takes 20-45 minutes to complete.

SKU Options and Performance:

Standard: 1 Gbps, 1,000 routes
HighPerformance: 2 Gbps, 4,000 routes
UltraPerformance: 10 Gbps, 4,000 routes
ErGw1Az: 1 Gbps, zone-redundant
ErGw2Az: 2 Gbps, zone-redundant
ErGw3Az: 10 Gbps, zone-redundant

Next: Gateway deployment continues in background due to --no-wait flag.

Step 7: Create Connection Between Gateway and Circuit

# Create connection between ExpressRoute Gateway and Circuit
az network vpn-connection create \
    --resource-group "rg-expressroute-prod" \
    --name "connection-er-primary" \
    --vnet-gateway1 "vgw-expressroute-hub" \
    --express-route-circuit2 "er-circuit-primary" \
    --connection-type "ExpressRoute" \
    --routing-weight 100

Gateway-Circuit Connection: Links the ExpressRoute Gateway to the ExpressRoute Circuit, enabling traffic flow between on-premises and Azure VNet.

Parameters:

--routing-weight: Priority for multiple connections (0-32000, higher = preferred)
--connection-type: Must be "ExpressRoute" for ER connections

Prerequisites: Circuit must be provisioned by provider and in "Enabled" state.

📊 Configuration Command Sequence Diagram

flowchart TD A["🏗️ Step 1: Resource Group
az group create"] --> B["🔌 Step 2: ExpressRoute Circuit
az network express-route create"] B --> C["🤝 Step 3A: Private Peering
az network express-route peering create
(AzurePrivatePeering)"] B --> D["🌐 Step 3B: Microsoft Peering
az network express-route peering create
(MicrosoftPeering)"] C --> E["🏠 Step 4A: Virtual Network
az network vnet create"] D --> E E --> F["🚪 Step 4B: Gateway Subnet
az network vnet subnet create
(GatewaySubnet)"] F --> G["📍 Step 5: Public IP
az network public-ip create"] G --> H["🌉 Step 6: ExpressRoute Gateway
az network vnet-gateway create
(20-45 minutes)"] H --> I["🔗 Step 7: Connection
az network vpn-connection create"] subgraph "Provider Actions" J["📋 Service Key
Provided to ISP"] K["⚙️ Circuit Provisioning
By Connectivity Provider"] L["✅ Circuit Status: Enabled
Required before Step 7"] end B --> J J --> K K --> L L --> I subgraph "Parallel Configuration" M["🔄 Route Tables
az network route-table create"] N["🛡️ Network Security Groups
az network nsg create"] O["🔥 Azure Firewall
az network firewall create"] end E --> M E --> N E --> O style A fill:#e1f5fe style B fill:#fff3e0 style C fill:#f3e5f5 style D fill:#f3e5f5 style E fill:#e8f5e8 style F fill:#e8f5e8 style G fill:#fff8e1 style H fill:#fce4ec style I fill:#e0f2f1 style J fill:#ffebee style K fill:#ffebee style L fill:#ffebee

Configuration Sequence Flow: This diagram shows the correct order for executing Azure CLI commands and external provider actions. Steps 1-2 can be completed immediately, but Step 7 (creating the connection) requires the circuit to be provisioned by your connectivity provider first. Steps 3A and 3B (peerings) can be configured in parallel. The gateway creation (Step 6) is the longest operation, taking 20-45 minutes. Parallel configurations like route tables and NSGs can be set up while waiting for the gateway deployment to complete.

Advanced Configuration Examples

Route Tables and Custom Routes

# Create route table for custom routing
az network route-table create \
    --resource-group "rg-expressroute-prod" \
    --name "rt-hub-custom" \
    --location "East US" \
    --disable-bgp-route-propagation false

Route Table Creation: Manages custom routing in your VNet. BGP route propagation should typically remain enabled for ExpressRoute scenarios.

Key Parameter:

--disable-bgp-route-propagation: false allows ExpressRoute routes to propagate automatically

# Add custom route for on-premises traffic
az network route-table route create \
    --resource-group "rg-expressroute-prod" \
    --route-table-name "rt-hub-custom" \
    --name "route-to-onprem" \
    --address-prefix "10.0.0.0/8" \
    --next-hop-type "VirtualNetworkGateway"

Custom Route Creation: Forces specific traffic through the ExpressRoute Gateway. Useful for traffic engineering or security requirements.

Next-Hop Types for ExpressRoute:

VirtualNetworkGateway: Through ExpressRoute Gateway
VirtualAppliance: Through Network Virtual Appliance
None: Drop traffic (blackhole route)

⚠️ Important Notes:

ExpressRoute circuits must be provisioned by your connectivity provider before creating connections
Gateway creation takes 20-45 minutes - plan accordingly
Ensure IP address ranges don't overlap between on-premises and Azure
Test connectivity thoroughly before moving production workloads
Monitor circuit utilization and adjust bandwidth as needed

💡 Best Practices:

Use zone-redundant gateway SKUs for high availability
Implement dual ExpressRoute circuits for redundancy
Monitor BGP route advertisements and filtering
Use Network Security Groups for micro-segmentation
Enable ExpressRoute monitoring and alerting
Document your network architecture and IP addressing scheme

Verification and Monitoring Commands

# Check ExpressRoute circuit status
az network express-route show \
    --resource-group "rg-expressroute-prod" \
    --name "er-circuit-primary" \
    --query "{Name:name, Status:serviceProviderProvisioningState, Circuit:circuitProvisioningState}" \
    --output table

# View BGP peering status
az network express-route peering list \
    --resource-group "rg-expressroute-prod" \
    --express-route-circuit-name "er-circuit-primary" \
    --output table

# Check gateway connection status
az network vpn-connection show \
    --resource-group "rg-expressroute-prod" \
    --name "connection-er-primary" \
    --query "{Name:name, Status:connectionStatus, Type:connectionType}" \
    --output table

Verification Commands: These commands help you verify that your ExpressRoute configuration is working correctly. Regular monitoring ensures optimal performance and early detection of issues.