🌐 Azure ExpressRoute Implementation Guide

📋 Table of Contents

🔍 Azure ExpressRoute Overview

Azure ExpressRoute enables you to create private connections between Azure datacenters and infrastructure on your premises or in a colocation environment. ExpressRoute connections don't go over the public Internet, offering more reliability, faster speeds, consistent latencies, and higher security than typical connections over the Internet.

graph TB subgraph "On-Premises" CP[Customer Premises] CR[Customer Router] end subgraph "Connectivity Provider" PP[Provider Edge] MPLS[MPLS Network] end subgraph "Microsoft Edge" MSEE[Microsoft Enterprise Edge] PE[Provider Edge Router] end subgraph "Azure" VNG[Virtual Network Gateway] VNET[Virtual Network] VM[Virtual Machines] end CP --> CR CR --> PP PP --> MPLS MPLS --> PE PE --> MSEE MSEE --> VNG VNG --> VNET VNET --> VM style CP fill:#e1f5fe style MSEE fill:#fff3e0 style VNET fill:#f3e5f5
ExpressRoute Connection Overview: This diagram shows the complete ExpressRoute connection path from customer premises through the connectivity provider's network to Microsoft's edge routers and into Azure virtual networks. The connection bypasses the public internet entirely, providing dedicated bandwidth and enhanced security.

⚡ ExpressRoute Options

🌍 ExpressRoute Global Reach

ExpressRoute Global Reach enables you to connect your on-premises networks together through ExpressRoute circuits. When you have multiple ExpressRoute circuits, Global Reach allows traffic to flow between your on-premises networks through Microsoft's global network.

graph LR subgraph "Location A" CP1[Customer Premises A] ER1[ExpressRoute Circuit A] end subgraph "Location B" CP2[Customer Premises B] ER2[ExpressRoute Circuit B] end subgraph "Microsoft Global Network" MGN[Microsoft Backbone] AZ1[Azure Region 1] AZ2[Azure Region 2] end CP1 --> ER1 CP2 --> ER2 ER1 --> MGN ER2 --> MGN MGN --> AZ1 MGN --> AZ2 ER1 -.->|Global Reach| ER2 style MGN fill:#fff3e0 style ER1 fill:#e8f5e8 style ER2 fill:#e8f5e8
ExpressRoute Global Reach: This diagram illustrates how Global Reach creates a direct connection between two ExpressRoute circuits, allowing on-premises locations to communicate with each other through Microsoft's global network backbone without routing traffic through Azure virtual networks.

🚀 ExpressRoute FastPath

ExpressRoute FastPath is designed to improve the data path performance between your on-premises network and your virtual network. When enabled, FastPath sends network traffic directly to virtual machines in the virtual network, bypassing the gateway.

graph TB subgraph "On-Premises" OnPrem[On-Premises Network] end subgraph "ExpressRoute" ER[ExpressRoute Circuit] MSEE[Microsoft Enterprise Edge] end subgraph "Azure Virtual Network" ERGW[ExpressRoute Gateway] VM1[Virtual Machine 1] VM2[Virtual Machine 2] VM3[Virtual Machine 3] end OnPrem --> ER ER --> MSEE MSEE --> ERGW ERGW --> VM1 ERGW --> VM2 ERGW --> VM3 MSEE -.->|FastPath| VM1 MSEE -.->|FastPath| VM2 MSEE -.->|FastPath| VM3 style MSEE fill:#fff3e0 style ERGW fill:#f3e5f5 style VM1 fill:#e1f5fe style VM2 fill:#e1f5fe style VM3 fill:#e1f5fe
ExpressRoute FastPath: The solid lines show traditional traffic flow through the ExpressRoute Gateway, while the dotted lines show FastPath traffic that bypasses the gateway for improved performance. FastPath reduces latency and increases throughput by eliminating the gateway as a bottleneck.

🔌 ExpressRoute Direct

ExpressRoute Direct provides the ability to connect directly into Microsoft's global network at peering locations strategically distributed around the world. ExpressRoute Direct provides dual 100 Gbps or 10 Gbps connectivity.

graph TB subgraph "Customer" CR[Customer Router] CPE[Customer Premises Equipment] end subgraph "Peering Location" CC[Cross Connect] ERD[ExpressRoute Direct Port] end subgraph "Microsoft Network" MSEE[Microsoft Enterprise Edge] MGN[Microsoft Global Network] end subgraph "Multiple Azure Regions" AZ1[Azure Region 1] AZ2[Azure Region 2] AZ3[Azure Region 3] end CR --> CPE CPE --> CC CC --> ERD ERD --> MSEE MSEE --> MGN MGN --> AZ1 MGN --> AZ2 MGN --> AZ3 style ERD fill:#fff3e0 style MGN fill:#e8f5e8 style MSEE fill:#f3e5f5
ExpressRoute Direct: This shows a direct physical connection to Microsoft's network infrastructure. ExpressRoute Direct eliminates the need for a connectivity provider and gives you direct access to Microsoft's global network with dedicated bandwidth of 10 Gbps or 100 Gbps.

🔄 Peering Types

🔒 Azure Private Peering

Azure private peering connects to Azure services deployed within virtual networks using private IP addresses.

graph LR subgraph "On-Premises" OnPrem[10.1.0.0/16] end subgraph "ExpressRoute" PP[Private Peering
192.168.1.0/30] end subgraph "Azure Virtual Network" VNet[VNet: 10.2.0.0/16] Subnet1[Subnet 1: 10.2.1.0/24] Subnet2[Subnet 2: 10.2.2.0/24] end OnPrem --> PP PP --> VNet VNet --> Subnet1 VNet --> Subnet2 style PP fill:#e8f5e8 style VNet fill:#f3e5f5
Azure Private Peering: Private peering enables connectivity to Azure IaaS services like Virtual Machines, Cloud Services, and Application Gateways using private IP addresses. Traffic flows directly between on-premises and Azure private resources without traversing the public internet.

🌐 Microsoft Peering

Microsoft peering provides connectivity to Microsoft cloud services that use public IP addresses, such as Office 365, Dynamics 365, and Azure PaaS services.

graph TB subgraph "On-Premises" OnPrem[Customer Network
Public IPs] end subgraph "ExpressRoute" MP[Microsoft Peering
Public IP Range] end subgraph "Microsoft Cloud Services" O365[Office 365] D365[Dynamics 365] Storage[Azure Storage] SQL[Azure SQL] Cosmos[Cosmos DB] end OnPrem --> MP MP --> O365 MP --> D365 MP --> Storage MP --> SQL MP --> Cosmos style MP fill:#fff3e0 style O365 fill:#e1f5fe style D365 fill:#e1f5fe style Storage fill:#e1f5fe style SQL fill:#e1f5fe style Cosmos fill:#e1f5fe
Microsoft Peering: Microsoft peering allows access to Microsoft cloud services using public IP addresses. This includes Office 365, Azure PaaS services, and other Microsoft cloud offerings. The connection uses BGP routing to advertise and receive public IP prefixes.

🔄 Both Peering Types Combined

You can configure both Azure private peering and Microsoft peering on the same ExpressRoute circuit to access both private Azure resources and Microsoft cloud services.

graph TB subgraph "On-Premises Network" OnPrem[Customer Router
10.1.0.0/16] end subgraph "ExpressRoute Circuit" ER[ExpressRoute Circuit] PP[Private Peering
192.168.1.0/30] MP[Microsoft Peering
Public IP Range] end subgraph "Azure Private Resources" VNet[Virtual Network
10.2.0.0/16] VM[Virtual Machines] end subgraph "Microsoft Cloud Services" O365[Office 365] Storage[Azure Storage] end OnPrem --> ER ER --> PP ER --> MP PP --> VNet VNet --> VM MP --> O365 MP --> Storage style ER fill:#e8f5e8 style PP fill:#f3e5f5 style MP fill:#fff3e0
Combined Peering Configuration: This diagram shows how a single ExpressRoute circuit can support both private peering (for Azure IaaS services) and Microsoft peering (for Microsoft cloud services) simultaneously. Each peering type uses separate BGP sessions and routing tables.

🛠️ Implementation Steps

📋 Command Execution Order

1
Create Resource Group
2
Create ExpressRoute Circuit
3
Get Circuit Service Key
4
Configure Peering (Private/Microsoft/Both)
5
Create Virtual Network Gateway
6
Link Circuit to VNet Gateway
7
Configure Global Reach (Optional)
8
Enable FastPath (Optional)

💻 Azure CLI Commands

1️⃣ Create Resource Group

az group create \
  --name "rg-expressroute-prod" \
  --location "East US 2"
Parameters Explained: Purpose: Creates a logical container for all ExpressRoute-related resources. This should be done first as all subsequent resources will be created within this resource group.

2️⃣ Create ExpressRoute Circuit

az network express-route create \
  --resource-group "rg-expressroute-prod" \
  --name "er-circuit-primary" \
  --peering-location "Silicon Valley" \
  --bandwidth 1000 \
  --provider "Equinix" \
  --sku-family "MeteredData" \
  --sku-tier "Standard" \
  --allow-global-reach true \
  --tags Environment=Production Project=NetworkInfra
Parameters Explained: Purpose: Creates the ExpressRoute circuit which represents the logical connection between your network and Microsoft. The circuit must be provisioned by your connectivity provider before it becomes active.

3️⃣ Get Circuit Service Key

az network express-route show \
  --resource-group "rg-expressroute-prod" \
  --name "er-circuit-primary" \
  --query "serviceKey" \
  --output tsv
Parameters Explained: Purpose: Retrieves the service key that you provide to your connectivity provider for circuit provisioning. This key uniquely identifies your circuit to Microsoft.

4️⃣ Configure Azure Private Peering

az network express-route peering create \
  --resource-group "rg-expressroute-prod" \
  --circuit-name "er-circuit-primary" \
  --peering-type AzurePrivatePeering \
  --peer-asn 65001 \
  --primary-peer-subnet "192.168.1.0/30" \
  --secondary-peer-subnet "192.168.1.4/30" \
  --vlan-id 100 \
  --shared-key "MySharedKey123!"
Parameters Explained: Purpose: Configures private peering to enable connectivity between your on-premises network and Azure virtual networks using private IP addresses. Both primary and secondary subnets ensure high availability.

5️⃣ Configure Microsoft Peering

az network express-route peering create \
  --resource-group "rg-expressroute-prod" \
  --circuit-name "er-circuit-primary" \
  --peering-type MicrosoftPeering \
  --peer-asn 65001 \
  --primary-peer-subnet "203.0.113.0/30" \
  --secondary-peer-subnet "203.0.113.4/30" \
  --vlan-id 200 \
  --advertised-public-prefixes "203.0.113.8/29" \
  --customer-asn 65001 \
  --routing-registry-name "ARIN"
Parameters Explained: Purpose: Configures Microsoft peering to access Office 365, Azure PaaS services, and other Microsoft cloud services using public IP addresses. Requires ownership validation of public IP prefixes.

6️⃣ Create Virtual Network and Gateway Subnet

az network vnet create \
  --resource-group "rg-expressroute-prod" \
  --name "vnet-expressroute-hub" \
  --address-prefix "10.2.0.0/16" \
  --subnet-name "GatewaySubnet" \
  --subnet-prefix "10.2.255.0/24" \
  --location "East US 2"
Parameters Explained: Purpose: Creates the virtual network that will host the ExpressRoute gateway. The GatewaySubnet is specifically reserved for Azure gateways and must be named exactly "GatewaySubnet".

7️⃣ Create Public IP for Gateway

az network public-ip create \
  --resource-group "rg-expressroute-prod" \
  --name "pip-expressroute-gateway" \
  --allocation-method Static \
  --sku Standard \
  --zone 1 2 3
Parameters Explained: Purpose: Creates a static public IP address that will be assigned to the ExpressRoute gateway for management and control plane connectivity.

8️⃣ Create ExpressRoute Gateway

az network vnet-gateway create \
  --resource-group "rg-expressroute-prod" \
  --name "ergw-expressroute-primary" \
  --public-ip-address "pip-expressroute-gateway" \
  --vnet "vnet-expressroute-hub" \
  --gateway-type ExpressRoute \
  --sku ErGw1AZ \
  --no-wait
Parameters Explained: Purpose: Creates the ExpressRoute virtual network gateway that connects your virtual network to the ExpressRoute circuit. This is the entry point for ExpressRoute traffic into your Azure virtual network.
⚠️ Important: Gateway creation typically takes 20-45 minutes. Use --no-wait to avoid CLI timeout and monitor progress with az network vnet-gateway show.

9️⃣ Connect Circuit to Virtual Network Gateway

az network vpn-connection create \
  --resource-group "rg-expressroute-prod" \
  --name "conn-expressroute-to-hub" \
  --vnet-gateway1 "ergw-expressroute-primary" \
  --express-route-circuit2 "/subscriptions/{subscription-id}/resourceGroups/rg-expressroute-prod/providers/Microsoft.Network/expressRouteCircuits/er-circuit-primary" \
  --connection-type ExpressRoute \
  --routing-weight 100
Parameters Explained: Purpose: Creates the logical connection between your ExpressRoute circuit and the virtual network gateway, enabling traffic flow between on-premises and Azure.

🔟 Configure Global Reach (Optional)

az network express-route peering connection create \
  --resource-group "rg-expressroute-prod" \
  --circuit-name "er-circuit-primary" \
  --peering-name "AzurePrivatePeering" \
  --name "globalreach-to-secondary-circuit" \
  --peer-circuit "/subscriptions/{subscription-id}/resourceGroups/rg-expressroute-prod/providers/Microsoft.Network/expressRouteCircuits/er-circuit-secondary" \
  --address-prefix "192.168.10.0/29" \
  --authorization-key "your-authorization-key-here"
Parameters Explained: Purpose: Enables Global Reach to connect two ExpressRoute circuits, allowing direct communication between on-premises locations through Microsoft's backbone network.

1️⃣1️⃣ Enable FastPath (Optional)

az network vpn-connection update \
  --resource-group "rg-expressroute-prod" \
  --name "conn-expressroute-to-hub" \
  --express-route-gateway-bypass true
Parameters Explained: Purpose: Enables FastPath on the connection, allowing network traffic to bypass the ExpressRoute gateway for better performance. Requires Ultra Performance or ErGw3AZ gateway SKU.
📝 FastPath Requirements:

1️⃣2️⃣ Create ExpressRoute Direct (Optional)

az network express-route port create \
  --resource-group "rg-expressroute-prod" \
  --name "er-direct-port" \
  --peering-location "Equinix-Seattle-SE2" \
  --bandwidth-in-gbps 100 \
  --encapsulation QinQ \
  --location "West US 2"
Parameters Explained: Purpose: Creates ExpressRoute Direct port for direct connection to Microsoft network. Provides dedicated 10 Gbps or 100 Gbps connectivity without going through a service provider.

🔄 Traffic Flow Diagrams

Complete ExpressRoute Architecture

graph TB subgraph "Branch Office A" BOA[Branch Router A
10.1.0.0/16] end subgraph "Branch Office B" BOB[Branch Router B
10.3.0.0/16] end subgraph "Data Center" DC[DC Router
10.4.0.0/16] FW[Firewall] end subgraph "ExpressRoute Circuits" ER1[Circuit A
1 Gbps] ER2[Circuit B
1 Gbps] GR[Global Reach
Connection] end subgraph "Microsoft Network" MSEE1[MS Edge Router 1] MSEE2[MS Edge Router 2] MGN[Microsoft Global Network] end subgraph "Azure Hub VNet" HubVNet[Hub VNet
10.2.0.0/16] ERGW[ExpressRoute Gateway
ErGw3AZ] AzFW[Azure Firewall] FP[FastPath Enabled] end subgraph "Azure Spoke VNets" Spoke1[Spoke VNet 1
10.5.0.0/16] Spoke2[Spoke VNet 2
10.6.0.0/16] VM1[Production VMs] VM2[Development VMs] end subgraph "Microsoft Services" O365[Office 365] Storage[Azure Storage] SQL[Azure SQL] end BOA --> ER1 BOB --> ER2 DC --> FW FW --> ER1 ER1 --> MSEE1 ER2 --> MSEE2 MSEE1 --> MGN MSEE2 --> MGN ER1 -.->|Global Reach| ER2 MGN --> ERGW ERGW --> HubVNet HubVNet --> AzFW MSEE1 -.->|FastPath| VM1 MSEE2 -.->|FastPath| VM2 HubVNet --> Spoke1 HubVNet --> Spoke2 Spoke1 --> VM1 Spoke2 --> VM2 MGN --> O365 MGN --> Storage MGN --> SQL style ER1 fill:#e8f5e8 style ER2 fill:#e8f5e8 style MGN fill:#fff3e0 style ERGW fill:#f3e5f5 style FP fill:#ffeb3b
Complete ExpressRoute Architecture: This comprehensive diagram shows a hub-and-spoke network topology with multiple ExpressRoute circuits. Key features include Global Reach connecting branch offices directly, FastPath bypassing the gateway for improved performance, and both private peering (for Azure resources) and Microsoft peering (for Microsoft cloud services). The hub VNet contains shared services like Azure Firewall, while spoke VNets contain workload-specific resources.

Traffic Flow with Route Priorities

graph TB subgraph "On-Premises" OnPrem[Customer Network
BGP AS 65001] Routes[Advertised Routes:
10.1.0.0/16
10.4.0.0/16] end subgraph "ExpressRoute" Primary[Primary Circuit
Weight: 100
Local Pref: 100] Secondary[Secondary Circuit
Weight: 50
Local Pref: 100] end subgraph "Azure" VNet1[VNet 1: 10.2.0.0/16
Advertised via Primary] VNet2[VNet 2: 10.3.0.0/16
Advertised via Secondary] UDR[User Defined Routes
Override BGP] end subgraph "Route Selection" Decision{Route Selection
Priority Order} UDRPrio[1. User Defined Routes] BGPPrio[2. BGP Routes] DefaultPrio[3. System Routes] end OnPrem --> Routes Routes --> Primary Routes --> Secondary Primary --> VNet1 Secondary --> VNet2 VNet1 --> Decision VNet2 --> Decision UDR --> Decision Decision --> UDRPrio Decision --> BGPPrio Decision --> DefaultPrio style Primary fill:#e8f5e8 style Secondary fill:#fff3cd style Decision fill:#f3e5f5 style UDRPrio fill:#d4edda
Route Selection and Traffic Flow: This diagram illustrates how Azure selects routes based on priority. User Defined Routes (UDRs) have the highest priority and can override BGP routes from ExpressRoute. BGP routes learned via ExpressRoute have the second-highest priority. Route weights and local preferences influence path selection when multiple ExpressRoute circuits are available.

FastPath Performance Comparison

graph LR subgraph "Traditional Path" T1[On-Premises] --> T2[ExpressRoute Circuit] T2 --> T3[MS Edge Router] T3 --> T4[ExpressRoute Gateway
Processing Overhead] T4 --> T5[Azure VM
Higher Latency] end subgraph "FastPath" F1[On-Premises] --> F2[ExpressRoute Circuit] F2 --> F3[MS Edge Router] F3 -.->|Bypass Gateway| F4[Azure VM
Lower Latency] end subgraph "Performance Metrics" PM[Traditional: ~2-5ms additional latency
FastPath: Direct path to VMs
Throughput: Up to 10 Gbps per VM] end style T4 fill:#ffcdd2 style F4 fill:#c8e6c9 style F3 fill:#fff3e0 style PM fill:#e1f5fe
FastPath Performance Benefits: The traditional path routes traffic through the ExpressRoute Gateway, adding processing overhead and latency. FastPath allows traffic to bypass the gateway and flow directly from Microsoft Edge routers to Azure VMs, reducing latency by 2-5 milliseconds and improving throughput. This is particularly beneficial for latency-sensitive applications and high-throughput workloads.

✅ Best Practices

🔒 Security Best Practices

⚡ Performance Optimization

💰 Cost Optimization

📊 Monitoring and Troubleshooting

# Check ExpressRoute circuit status
az network express-route show \
  --resource-group "rg-expressroute-prod" \
  --name "er-circuit-primary" \
  --query "{CircuitProvisioningState:circuitProvisioningState,ServiceProviderProvisioningState:serviceProviderProvisioningState,BandwidthInMbps:serviceProviderProperties.bandwidthInMbps}"

# Monitor BGP session status  
az network express-route peering show \
  --resource-group "rg-expressroute-prod" \
  --circuit-name "er-circuit-primary" \
  --name "AzurePrivatePeering" \
  --query "{State:state,PrimaryPeerAddressPrefix:primaryPeerAddressPrefix,SecondaryPeerAddressPrefix:secondaryPeerAddressPrefix}"

# Check gateway connection status
az network vpn-connection show \
  --resource-group "rg-expressroute-prod" \
  --name "conn-expressroute-to-hub" \
  --query "{ConnectionStatus:connectionStatus,EgressBytesTransferred:egressBytesTransferred,IngressBytesTransferred:ingressBytesTransferred}"
Monitoring Commands: These commands help you monitor the health and status of your ExpressRoute implementation. Regular monitoring ensures optimal performance and helps identify issues before they impact users.

🎯 Key Takeaways

ExpressRoute provides enterprise-grade connectivity to Azure with multiple configuration options: