Table of Contents
Overview
Azure ExpressRoute provides multiple encryption options to secure data in transit between your on-premises network and Azure. This guide covers the implementation of MACsec (Layer 2 encryption) and IPsec (Layer 3 encryption) over ExpressRoute connections.
Important: MACsec is ONLY available with ExpressRoute Direct connections. Regular ExpressRoute circuits (through service providers) do NOT support MACsec and can only use IPsec encryption.
graph TB
subgraph "ExpressRoute Direct"
A1[On-Premises] --> B1[Customer Edge]
B1 --> C1[Direct Physical Connection]
C1 --> D1[Microsoft Edge]
D1 --> E1[Azure VNet]
C1 -.->|"MACsec Available"| F1[Layer 2 Encryption]
E1 -.->|"IPsec Available"| G1[Layer 3 Encryption]
end
subgraph "Regular ExpressRoute Circuit"
A2[On-Premises] --> B2[Customer Edge]
B2 --> C2[Service Provider]
C2 --> D2[Microsoft Edge]
D2 --> E2[Azure VNet]
C2 -.->|"MACsec NOT Available"| F2[❌ Layer 2]
E2 -.->|"IPsec Available"| G2[Layer 3 Encryption]
end
style F1 fill:#4caf50
style G1 fill:#4caf50
style G2 fill:#4caf50
style F2 fill:#f44336
Encryption Types and Availability
Connection Type Comparison:
- ExpressRoute Direct: Supports both MACsec (Layer 2) and IPsec (Layer 3) encryption
- Regular ExpressRoute Circuit: Supports only IPsec (Layer 3) encryption
MACsec (Media Access Control Security) - ExpressRoute Direct Only
MACsec provides Layer 2 encryption between your premises and Microsoft's edge routers. It offers:
- Hardware-based encryption at line speed
- No impact on application performance
- Protection against Layer 2 attacks
- Requires ExpressRoute Direct connection
- Requires compatible hardware
IPsec VPN over ExpressRoute - Available for All ExpressRoute Types
IPsec provides Layer 3 encryption end-to-end between your network and Azure VNets:
- Software-based encryption
- Complete end-to-end encryption
- Works with any hardware
- Available for both ExpressRoute Direct and regular circuits
- May impact performance based on throughput
graph TB
subgraph "ExpressRoute Direct - MACsec Coverage"
A[Customer Edge] --> B[Microsoft Edge]
B -.->|"Direct Physical Link"| A
end
subgraph "Both Connection Types - IPsec Coverage"
C[On-Premises] --> D[Azure VNet]
D -.->|"End-to-End Encryption"| C
end
B --> E[Azure Services]
D --> E
style A fill:#4caf50
style B fill:#4caf50
style C fill:#2196f3
style D fill:#2196f3
style E fill:#ff9800
Prerequisites
Before You Begin:
- Azure subscription with appropriate permissions
- ExpressRoute circuit or ExpressRoute Direct connection already provisioned
- Azure CLI installed and configured
- For MACsec: ExpressRoute Direct connection required
- For MACsec: Compatible hardware for Layer 2 encryption
- For IPsec: Works with any ExpressRoute connection type
Connection Type Check: Verify your ExpressRoute connection type before proceeding. Use the command below to determine if you have ExpressRoute Direct or a regular circuit.
Check Your ExpressRoute Connection Type
# Check if you have ExpressRoute Direct
az network express-route port list --query "[].{Name:name, Location:location, ResourceGroup:resourceGroup}"
# Check regular ExpressRoute circuits
az network express-route list --query "[].{Name:name, ServiceProvider:serviceProviderProperties.serviceProviderName, Location:location}"
Required Azure CLI Extensions
# Install required extensions
az extension add --name express-route
az extension add --name network
# Login to Azure
az login
# Set subscription
az account set --subscription "your-subscription-id"
MACsec Configuration (ExpressRoute Direct Only)
ExpressRoute Direct Required: The following configuration only applies to ExpressRoute Direct connections. If you have a regular ExpressRoute circuit through a service provider, skip this section and proceed to the IPsec configuration.
1Create ExpressRoute Direct Resource
# Create resource group
az group create \
--name rg-expressroute-encryption \
--location eastus
# Create ExpressRoute Direct (not a regular circuit)
az network express-route port create \
--resource-group rg-expressroute-encryption \
--name er-direct-ports \
--peering-location "Silicon Valley" \
--bandwidth 10 \
--encapsulation QinQ
2Create Circuit on ExpressRoute Direct
# Create circuit on the ExpressRoute Direct resource
az network express-route create \
--resource-group rg-expressroute-encryption \
--name er-circuit-on-direct \
--peering-location "Silicon Valley" \
--bandwidth 1000 \
--provider "ExpressRoute Direct" \
--sku-family MeteredData \
--sku-tier Standard \
--express-route-port "/subscriptions/{subscription-id}/resourceGroups/rg-expressroute-encryption/providers/Microsoft.Network/expressRoutePorts/er-direct-ports"
Key Parameters Explained:
Parameter | Description | Example Value |
---|---|---|
--express-route-port | Resource ID of the ExpressRoute Direct port | /subscriptions/.../expressRoutePorts/er-direct-ports |
--provider | Must be "ExpressRoute Direct" for direct connections | ExpressRoute Direct |
--encapsulation | VLAN tagging method (QinQ or Dot1Q) | QinQ |
--bandwidth | Port bandwidth in Gbps (10 or 100) | 10 |
3Configure MACsec on ExpressRoute Direct
# Create Azure Key Vault for MACsec keys
az keyvault create \
--resource-group rg-expressroute-encryption \
--name kv-macsec-keys-$(date +%s) \
--location eastus \
--enable-soft-delete true
# Create user-assigned managed identity
az identity create \
--resource-group rg-expressroute-encryption \
--name id-macsec-access
# Configure MACsec on ExpressRoute Direct port
az network express-route port update \
--resource-group rg-expressroute-encryption \
--name er-direct-ports \
--macsec-config cipher=GcmAes128 \
--macsec-config pre-shared-key-secret-identifier="https://your-keyvault.vault.azure.net/secrets/macsec-key/version"
MACsec Configuration Parameters:
Parameter | Description | Possible Values |
---|---|---|
cipher | Encryption algorithm | GcmAes128, GcmAes256, GcmAesXpn128, GcmAesXpn256 |
pre-shared-key-secret-identifier | Azure Key Vault secret reference | https://vault.vault.azure.net/secrets/key/version |
sci-state | Secure Channel Identifier state | Enabled, Disabled |
Best Practice: Use GcmAesXpn128 or GcmAesXpn256 ciphers for links with 40 Gbps or greater bandwidth to prevent sporadic session failures.
sequenceDiagram
participant CE as Customer Edge
participant Direct as ExpressRoute Direct
participant ME as Microsoft Edge
participant Azure as Azure Services
CE->>Direct: MACsec Encrypted Frame
Note over CE,Direct: Layer 2 Encryption
Direct->>ME: Encrypted Transport
ME->>Azure: Decrypted Data
Azure->>ME: Response Data
ME->>Direct: Encrypted Transport
Direct->>CE: MACsec Encrypted Frame
IPsec over ExpressRoute
1Create Virtual Network Gateway
# Create virtual network
az network vnet create \
--resource-group rg-expressroute-encryption \
--name vnet-hub \
--address-prefix 10.0.0.0/16 \
--subnet-name GatewaySubnet \
--subnet-prefix 10.0.1.0/24
# Create public IP for VPN gateway
az network public-ip create \
--resource-group rg-expressroute-encryption \
--name pip-vpn-gateway \
--allocation-method Static \
--sku Standard
# Create VPN gateway
az network vnet-gateway create \
--resource-group rg-expressroute-encryption \
--name vpn-gateway-hub \
--public-ip-address pip-vpn-gateway \
--vnet vnet-hub \
--gateway-type Vpn \
--vpn-type RouteBased \
--sku VpnGw2 \
--no-wait
Gateway SKU Options:
SKU | Max Tunnels | Aggregate Throughput | BGP Support |
---|---|---|---|
VpnGw1 | 30 | 650 Mbps | Yes |
VpnGw2 | 30 | 1 Gbps | Yes |
VpnGw3 | 30 | 1.25 Gbps | Yes |
2Create Local Network Gateway
# Create local network gateway representing on-premises
az network local-gateway create \
--resource-group rg-expressroute-encryption \
--name lng-onpremises \
--gateway-ip-address 203.0.113.1 \
--local-address-prefixes 192.168.0.0/16 \
--asn 65001 \
--bgp-peering-address 192.168.1.1 \
--peer-weight 0
Local Gateway Parameters:
Parameter | Description | Example |
---|---|---|
--gateway-ip-address | Public IP of on-premises VPN device | 203.0.113.1 |
--local-address-prefixes | On-premises network address spaces | 192.168.0.0/16 |
--asn | BGP ASN for on-premises device | 65001 |
--bgp-peering-address | BGP peer IP address | 192.168.1.1 |
3Create VPN Connection
# Create site-to-site VPN connection
az network vpn-connection create \
--resource-group rg-expressroute-encryption \
--name vpn-connection-s2s \
--vnet-gateway1 vpn-gateway-hub \
--local-gateway2 lng-onpremises \
--connection-type IPsec \
--shared-key "YourSharedKeyHere123!" \
--enable-bgp true \
--ipsec-policy sa-life-time-seconds=3600 \
--ipsec-policy sa-data-size-kilobytes=102400000 \
--ipsec-policy ipsec-encryption=AES256 \
--ipsec-policy ipsec-integrity=SHA256 \
--ipsec-policy ike-encryption=AES256 \
--ipsec-policy ike-integrity=SHA256 \
--ipsec-policy dh-group=DHGroup2 \
--ipsec-policy pfs-group=PFS2
IPsec Policy Parameters:
Parameter | Description | Recommended Value |
---|---|---|
ipsec-encryption | IPsec encryption algorithm | AES256 |
ipsec-integrity | IPsec integrity algorithm | SHA256 |
ike-encryption | IKE encryption algorithm | AES256 |
ike-integrity | IKE integrity algorithm | SHA256 |
dh-group | Diffie-Hellman group for IKE | DHGroup2 |
pfs-group | Perfect Forward Secrecy group | PFS2 |
graph TB
subgraph "On-Premises"
A[VPN Device] --> B[Local Network
192.168.0.0/16] end subgraph "Azure" C[VPN Gateway] --> D[Virtual Network
10.0.0.0/16] end A -.->|"IPsec Tunnel
AES256 Encryption"| C subgraph "ExpressRoute" E[ExpressRoute Circuit] end A --> E E --> C style A fill:#ff9800 style C fill:#4caf50 style E fill:#2196f3
192.168.0.0/16] end subgraph "Azure" C[VPN Gateway] --> D[Virtual Network
10.0.0.0/16] end A -.->|"IPsec Tunnel
AES256 Encryption"| C subgraph "ExpressRoute" E[ExpressRoute Circuit] end A --> E E --> C style A fill:#ff9800 style C fill:#4caf50 style E fill:#2196f3
VPN Gateway Configuration
1Configure Route-Based VPN
# Update VPN gateway configuration
az network vnet-gateway update \
--resource-group rg-expressroute-encryption \
--name vpn-gateway-hub \
--enable-bgp true \
--asn 65000 \
--bgp-peering-address 10.0.1.4 \
--peer-weight 0
# Configure custom BGP addresses if needed
az network vnet-gateway update \
--resource-group rg-expressroute-encryption \
--name vpn-gateway-hub \
--custom-routes 192.168.100.0/24
2Configure Connection Routing
# Create route table for VPN traffic
az network route-table create \
--resource-group rg-expressroute-encryption \
--name rt-vpn-traffic
# Add route for on-premises traffic via VPN
az network route-table route create \
--resource-group rg-expressroute-encryption \
--route-table-name rt-vpn-traffic \
--name route-onprem-vpn \
--address-prefix 192.168.0.0/16 \
--next-hop-type VirtualNetworkGateway
# Create subnet for workloads
az network vnet subnet create \
--resource-group rg-expressroute-encryption \
--vnet-name vnet-hub \
--name subnet-workloads \
--address-prefix 10.0.2.0/24 \
--route-table rt-vpn-traffic
Important: When using both ExpressRoute and VPN Gateway in the same VNet, ensure proper routing configuration to avoid asymmetric routing issues.
Monitoring and Troubleshooting
Connection Status Monitoring
# Check ExpressRoute circuit status
az network express-route show \
--resource-group rg-expressroute-encryption \
--name er-circuit-macsec \
--query "circuitProvisioningState"
# Check VPN connection status
az network vpn-connection show \
--resource-group rg-expressroute-encryption \
--name vpn-connection-s2s \
--query "connectionStatus"
# Monitor BGP peers
az network vnet-gateway list-bgp-peer-status \
--resource-group rg-expressroute-encryption \
--name vpn-gateway-hub
Diagnostic Commands
# Get VPN connection statistics
az network vpn-connection show \
--resource-group rg-expressroute-encryption \
--name vpn-connection-s2s \
--query "{Status: connectionStatus, IngressBytes: ingressBytesTransferred, EgressBytes: egressBytesTransferred}"
# Test connectivity
az network vnet-gateway vpn-connection show-shared-key \
--resource-group rg-expressroute-encryption \
--name vpn-connection-s2s
# Download VPN device configuration
az network vpn-connection get-shared-key \
--resource-group rg-expressroute-encryption \
--name vpn-connection-s2s
Common Troubleshooting Steps
Troubleshooting Checklist:
- Verify shared keys match on both ends
- Check firewall rules allow IPsec traffic (UDP 500, 4500)
- Validate BGP configuration if using dynamic routing
- Confirm network security groups allow required traffic
- Check route tables for conflicts
flowchart TD
A[Connection Issue] --> B{Check Basic Connectivity}
B -->|Pass| C{Verify Authentication}
B -->|Fail| D[Check Physical Connectivity]
C -->|Pass| E{Check Routing}
C -->|Fail| F[Verify Shared Keys]
E -->|Pass| G{Check Security Groups}
E -->|Fail| H[Review Route Tables]
G -->|Pass| I[Connection Working]
G -->|Fail| J[Update NSG Rules]
style A fill:#ff5722
style I fill:#4caf50
style D fill:#ff9800
style F fill:#ff9800
style H fill:#ff9800
style J fill:#ff9800