Contents
- Introduction and Overview
- Multipoint Generic Routing Encapsulation (mGRE)
- Next Hop Resolution Protocol(NHRP)
- Verification
- Optimization
Introduction and Overview
DMVPN is a Cisco developed technology used in enterprise networks to create Virtual Private Networks (VPNs) to interconnect an enterprise’s various geographically separated offices over a public network such as the Internet.
DMVPN is a point-to-multipoint layer 3 overlay VPN providing full-meshed connectivity from an initial logical hub and spoke topology. Direct spoke-to-spoke traffic is supported. DMVPN's only requirement is for IP connectivity between the local and remote site. The routing policy is not dictated by the service provider; as such, it does not have the restrictions of MPLS L3VPNs such as the routing protocol that the service provider supports for the L3VPN.
DMVPN is highly scalable when properly designed.
How DMVPN Works
DMVPN relies on two technologies i.e. Multipoint Generic Routing Encapsulation (mGRE) and Next Hop Resolution Protocol (NHRP). To configure DMVPN, the following steps should be followed;
- Step 1: Configuration of mGRE tunnels.
- Step 2: Configuration of Next Hop Resolution Protocol (NHRP).
- Step 3: Securing the DMVPN Tunnel with IPSec(covered here).
DMVPN has three modes of operation; Phase 1, Phase 2, and Phase 3. Each phase incrementally adds features that improve the operation of DMVPN. DMVPN reduces the need for n × (n-1) ⁄ 2 static tunnel configurations. This is made possible through the use of one mGRE interface for all connections.
DMVPN maintains tunnels based on traffic patterns. Spoke-to-spoke tunnel is on-demand and this tunnel's lifetime is based on traffic utilization of the tunnel.
DMVPN supports IPv4 and IPv6 protocols for both passenger (overlay) and transport (underlay) traffic.
DMVPN Features
DMVPN's characteristics make its operation flexible and scalable:
-
DMVPN provides full-meshed connectivity with simple configuration of hub and spoke.
DMVPN creates on-demand tunnels between nodes: the initial tunnel-mesh is hub-and-spoke(always on) topology. Traffic patterns trigger spoke-to-spoke tunnels thereby creating a topology that dynamic shifts anywhere between a hub and spoke to full mesh topology. This solves management scalability problems.
-
It supports dynamically addressed spokes i.e. where spokes have their IP addresses issued through DHCP or dynamic NAT and these IP addresses are not consistent over time. DMVPN supports spoke routers behind dynamic NAT and hub routers behind static NAT.
-
DMVPN facilitates zero-touch configuration on the hub for addition of new spokes i.e., no configuration on the hub when adding a new spoke.
DMVPN features automatic IPsec triggering for building IPsec protected VPN tunnels. However, it remains usable with or without IPsec encryption.
Supports IP unicast and IP multicast traffic enabling the support of interior routing protocols such as OSPF, EIGRP. Multicast packets are encapsulated by GRE in unicast packets.
DMVPN supports the configuration of one mGRE interface on each spoke which supports multipoint VPN connectivity. On the hub, DMVPN configuration is accomplished with fewer configuration commands and one tunnel interface for all spokes.
Dynamic tunnel destination simplifies support for dynamically addressed spokes.
- Spoke-to-spoke data-plane traffic is through the hub (phase 1) or direct (phase 2 and 3). However, control-plane and multicast traffic is only exchanged with the hub regardless of DMVPN phase in operation. This results in the spokes forming layer 3 adjacencies only with the hub. There are no spoke-to-spoke layer 3 adjacencies. So routing protocol updates come only from the hub.
Connectivity
As previously indicated, DMVPN topologies start initially with a hub to spoke topology. This dynamically changes to full mesh topologies depending on traffic patterns.
Hub to Spokes
The two main components of a hub and spoke connection are:
- DMVPN hub/NHRP server (NHS).
- DMVPN spokes / NHCs:
- Spokes manually specify the hub's address and register with the hub/server.
- The Registration message from the spokes is sent via NHRP Registration Request message type.
- The hub dynamically learns the spoke's VPN address and NBMA address.
- Spokes establish tunnels to the hub through which exchange of IGP routing information takes place.
Spoke to Spoke
With the spoke-to-spoke tunnel formed, the hub is only used for control-plane information exchange. The spoke-to-spoke tunnel data-plane may flow through the hub initially but subsequent traffic is spoke-to-spoke.
Multipoint Generic Routing Encapsulation Tunnels (mGRE)
mGRE creates Virtual Private Network (VPN) tunnels between routers across a network such as the Internet. A hub router is the central connection point of what is a star topology. DMVPN has the capability to create topologies that contain more than one hub. For purposes of understanding DMVPN, we will implement a single-hub DMVPN topology.
GRE is used as a transport mechanism for protocols such as IP by encapsulating the IP packets. GRE uses IP protocol number 47. Dynamic routing protocols such as OSPF, EIGRP are supported due to multicast support provided by GRE. Whereas GRE supports unicast packets, the default configuration of most dynamic routing protocols (without optimization) is to use multicast packets. Spokes can be configured to send encapsulated multicast packets in unicast GRE packets to the hub.
The tunnels created by mGRE are usually referred to as the overlay network. The existing transport network is referred to as the underlay network.
When network traffic is to be routed through the overlay network, a new GRE header is added to the existing IP packet. The GRE header encapsulates the tunnel IP header and data. This encapsulated packet may be a multicast or unicast packet. This packet is then routed through the underlay network using the underlay IP destination address. The router at the end of the tunnel, strips the GRE header and routes the packet based on the previously encapsulated IP header destination address (for destination-based routing).
GRE tunnels support IPv4 or IPv6 addresses as underlay and/or the overlay network. By default, GRE does not support encryption. However, tunnel encryption can be added through IPSec.
The following output is a Wireshark packet capture of a GRE tunneled ICMP reply message:
Frame 57: 138 bytes on wire (1104 bits), 138 bytes captured (1104 bits) on interface -, id 0
Ethernet II, Src: ca:06:06:9e:00:08 (ca:06:06:9e:00:08), Dst: ca:07:06:ad:00:1c (ca:07:06:ad:00:1c)
Internet Protocol Version 4, Src: 99.255.30.1, Dst: 99.255.60.6
Generic Routing Encapsulation (IP)
Flags and Version: 0x0000
0... .... .... .... = Checksum Bit: No
.0.. .... .... .... = Routing Bit: No
..0. .... .... .... = Key Bit: No
...0 .... .... .... = Sequence Number Bit: No
.... 0... .... .... = Strict Source Route Bit: No
.... .000 .... .... = Recursion control: 0
.... .... 0000 0... = Flags (Reserved): 0
.... .... .... .000 = Version: GRE (0)
Protocol Type: IP (0x0800)
Internet Protocol Version 4, Src: 10.2.10.1, Dst: 172.30.1.7
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 100
Identification: 0x0013 (19)
000. .... = Flags: 0x0
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..0. .... = More fragments: Not set
...0 0000 0000 0000 = Fragment Offset: 0
Time to Live: 255
Protocol: ICMP (1)
Header Checksum: 0xfa5d [validation disabled]
[Header checksum status: Unverified]
Source Address: 10.2.10.1
Destination Address: 172.30.1.7
Internet Control Message Protocol
Type: 0 (Echo (ping) reply)
Code: 0
Checksum: 0xb995 [correct]
[Checksum Status: Good]
Identifier (BE): 3 (0x0003)
Identifier (LE): 768 (0x0300)
Sequence Number (BE): 4 (0x0004)
Sequence Number (LE): 1024 (0x0400)
[Request frame: 56]
[Response time: 52.565 ms]
Data (72 bytes)
Configuration of GRE Tunnels
Control-plane neighborships only happen between spoke and hub. Spokes are not able to form dynamic routing protocol neighbor relationships with each other. All spoke to spoke communication transits the hub because the hub sets itself as the next-hop in control-plane traffic. All data plane traffic transits via the hub as a result.
mGRE requires the configuration of the hub and spokes. Usually the hub takes many more commands than those of the spokes.
In the configuration of DMVPN, the following topology will be used for reference:
Step 1: Create the tunnel interface
The global configuration command interface tunnel <tunnel-number> is used.
HUB#configure terminal
HUB(config)#interface tunnel 0
Step 2: Identify the tunnel source
A tunnel source is the interface whose IP address will be used as the source IP
in the IP packets sent through the underlay network.
It is usually a good idea to specify a logical interface such as a loopback interface.
A loopback interface is always up and if a router has more than one Internet
connection through different router interfaces, then any one of the other "up" interfaces can
be used to route the tunnel traffic if the primary interface through which the
VPN tunnel has been created goes down. If a loopback
interface is used, remember to advertise the configured network on this interface.
The configuration command is tunnel source <ip-address | interface>.
HUB(config-if)#tunnel source g3/0
Hub Configuration
The following set of commands are implemented on the hub router.
Step 3: Configure the mode of the tunnel to multipoint GRE
The tunnel multipoint configuration supports the creation of multiple logical tunnels through one tunnel interface. DMVPN is therefore able to dynamically support a topology that dynamically transitions from point-to-point to point-to-multipoint.
HUB(config-if)#tunnel mode gre multipoint
Step 4: Allocate an IP address to the tunnel interface
The subnet should have a sufficient number of IP addresses to be able to accommodate the planned number of spokes and hubs in the network. Proper planning during the design of the network should cater for this. For this example, we will accommodate a combined maximum of 254 hubs and clients so a subnet mask of 255.255.255.0 (/24) is used.
HUB(config-if)#ip address 172.30.1.1 255.255.255.0
Step 5: (Optional) Define the tunnel bandwidth
The tunnel bandwidth is usually configured for routing purposes. In the tunnel
interface configuration mode, the command is bandwidth <1 – 10000000>
The value is in kbps. The default bandwidth is 100Kbps. If the bandwidth value
is not configured, then the tunnel priority will be low.
HUB(config-if)#bandwidth 100000
Step 6: (Optional) Specify the GRE tunnel keepalive
Keepalives ensure that bi-directional communication between tunnel end-points
remain active. In tunnel interface mode, the command is keepalive <seconds> <retries>.
The default keepalive time is 10 seconds and 3 retries.
HUB(config-if)#keepalive 10 5
Step 7: (Optional) Define the MTU
By default, the MTU for tunnel interfaces is 1476. The rationale behind setting
the MTU value to 1476 is that the GRE header, with a minimum size of 24 bytes, is
added to the IP packet. Together, the new MTU size of the IP packet should total
1500. This size is supported by many network device interfaces as the maximum
MTU. It is recommended to specify an MTU value of 1400. In tunnel configuration
mode, the command is ip mtu <value>.
HUB(config-if)#ip mtu 1400
Step 8: Define the Maximum Segment Size (MSS)
This ensures that the router modifies the payload of a three-way handshake if the MSS exceeds the configured value. With DMVPN, this value is typically 1360. During tunnel configuration, configure MTU to 1400, MSS to 1360 to ensure fragmentation is implemented by hosts. The general idea is to ensure that fragmentation is implemented by the hosts and not the routers as it is memory intensive.
HUB(config-if)#ip tcp adjust-mss 1360
Spoke configuration (DMVPN Phase 1)
Configuration of the Spoke in DMVPN phase 1 is similar to the configuration of the hub. The only exception is that with the spoke, GRE is configured instead of mGRE. In the configuration of the hub, replace steps 3 and 4.
Step 3: Configure the destination of the tunnel
The destination IP address should be the address of the underlay/transport interface of the hub;
SPOKE7(config-if)#tunnel destination 99.255.10.2
Step 4: Allocate an IP address to the tunnel interface
This IP address should be in the subnet of the tunnel of the Hub
SPOKE7(config-if)#ip address 172.30.1.7 255.255.255.0
Next Hop Resolution Protocol (NHRP)
NHRP is used for mapping tunnel (overlay) addresses to transport (underlay) addresses. It is defined in RFC 2332. NHRP operates using the client-server model with a hub as the server and the spokes as the client. The clients are referred to as Next Hop Clients (NHC) and the server as Next Hop Server (NHS). Spokes are configured with the IP address of the hub so they can register their tunnel and NBMA (transport) address with the hub. No manual registration of clients is required on the hub i.e., zero-touch configuration. This reduces administrative workload and improves scalability with addition of new spokes.
NHRP Message Types
NHRP messages are encapsulated by GRE between devices.
There are the various types of messages NHCs and NHSs exchange with each other:
- NHRP Registration(Request and Reply)
- NHRP Resolution(Request and Reply)
- NHRP Redirect
- NHRP Purge
- NHRP Error
NHRP Registration
NHRP Registration Request is sent from NHCs to NHS to register its NHRP information. Spokes dynamically register the mapping of their NBMA and VPN IP address to the NHS. This is required to build the spoke-to-spoke tunnels. NHRP Registration supports spokes with dynamic NBMA addresses or NAT.
NHRP registration packets are sent every NHRP timeout period. If no registration
reply is received, a new registration packet is sent with a delay of 1 second,
second packet is delayed for two seconds and third four seconds. NHS is declared
down after third attempt. Use the command show dmvpn to confirm.
Periodic NHRP registration requests by spokes refresh the NHRP timeout period. In spoke to spoke tunnels, if the tunnel is not used within two minutes of expiry time, it is torn down.
Probe state: NHCs attempting to register with a down NHS start sending probe packets periodically to determine the availability of the NHS. The delay in sending of the probe packets is 1, 2, 4, 8, 16, 32, 64. After 64 seconds, no more probe packets are sent. This can be verified by debugging messages;
. . .
*Feb 14 21:55:12.391: NHRP-RATE: Sending initial Registration Request for 10.100.100.1, reqid 2
*Feb 14 21:55:14.303: NHRP: Setting retrans delay to 4 for nhs dst 10.100.100.1
*Feb 14 21:55:14.307: NHRP-ATTR: Requester Ext Len: Total ext_len with NHRP attribute VPE 40
*Feb 14 21:55:14.307: NHRP: Attempting to send packet via DEST 10.100.100.1
*Feb 14 21:55:14.311: NHRP: NHRP successfully mapped '10.100.100.1' to NBMA 10.10.10.10
*Feb 14 21:55:14.315: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: 10.10.10.10
*Feb 14 21:55:14.319: NHRP: Send Registration Request via Tunnel0 vrf 0, packet size: 92
*Feb 14 21:55:14.323: src: 10.100.100.9, dst: 10.100.100.1
*Feb 14 21:55:14.327: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Feb 14 21:55:14.327: shtl: 4(NSAP), sstl: 0(NSAP)
*Feb 14 21:55:14.331: pktsz: 92 extoff: 52
*Feb 14 21:55:14.331: (M) flags: "unique nat ", reqid: 2
*Feb 14 21:55:14.331: src NBMA: 9.9.9.9
*Feb 14 21:55:14.335: src protocol: 10.100.100.9, dst protocol: 10.100.100.1
*Feb 14 21:55:14.339: (C-1) code: no error(0)
*Feb 14 21:55:14.339: prefix: 32, mtu: 17912, hd_time: 7200
*Feb 14 21:55:14.343: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0
*Feb 14 21:55:14.343: Responder Address Extension(3):
*Feb 14 21:55:14.343: Forward Transit NHS Record Extension(4):
*Feb 14 21:55:14.347: Reverse Transit NHS Record Extension(5):
*Feb 14 21:55:14.347: NAT address Extension(9):
*Feb 14 21:55:14.347: (C-1) code: no error(0)
*Feb 14 21:55:14.351: prefix: 32, mtu: 17912, hd_time: 0
*Feb 14 21:55:14.351: addr_len: 4(NSAP), subaddr_len: 0(NSAP), proto_len: 4, pref: 0
*Feb 14 21:55:14.351: client NBMA: 10.10.10.10
*Feb 14 21:55:14.355: client protocol: 10.100.100.1
*Feb 14 21:55:14.355: NHRP: 120 bytes out Tunnel0
*Feb 14 21:55:14.359: NHRP-RATE: Retransmitting Registration Request for 10.100.100.1, reqid 2, (retrans ivl 4 sec)
*Feb 14 21:55:18.171: NHRP: NHS-DOWN: 10.100.100.1
*Feb 14 21:55:18.187: NHRP: Already pending Registration Request for NHS: 10.100.100.1
*Feb 14 21:55:18.191: NHRP: NHS 10.100.100.1 Tunnel0 vrf 0 Cluster 0 Priority 0 Transitioned to 'E' from 'E'
*Feb 14 21:55:18.191: NHRP: Setting retrans delay to 8 for nhs dst 10.100.100.1
*Feb 14 21:55:18.195: NHRP-ATTR: Requester Ext Len: Total ext_len with NHRP attribute VPE 40
. . .
*Feb 14 21:55:18.251: NHRP-RATE: Retransmitting Registration Request for 10.100.100.1, reqid 2, (retrans ivl 8 sec)
*Feb 14 21:55:25.567: NHRP: Setting retrans delay to 16 for nhs dst 10.100.100.1
*Feb 14 21:55:25.571: NHRP-ATTR: Requester Ext Len: Total ext_len with NHRP attribute VPE 40
. . .
*Feb 14 21:55:25.623: NHRP-RATE: Retransmitting Registration Request for 10.100.100.1, reqid 2, (retrans ivl 16 sec)
*Feb 14 21:55:39.495: NHRP: Setting retrans delay to 32 for nhs dst 10.100.100.1,
. . .
*Feb 14 21:55:39.551: NHRP-RATE: Retransmitting Registration Request for 10.100.100.1, reqid 2, (retrans ivl 32 sec)
*Feb 14 21:56:09.879: NHRP: Setting retrans delay to 64 for nhs dst 10.100.100.1
*Feb 14 21:56:09.883: NHRP-ATTR: Requester Ext Len: Total ext_len with NHRP attribute VPE 40
The following output is a Wireshark packet capture of an NHRP Registration Request packet:
Next Hop Resolution Protocol (NHRP Registration Request)
NHRP Fixed Header
Address Family Number: IPv4 (0x0001)
Protocol Type (short form): IPv4 (0x0800)
Protocol Type (long form): 0000000000
Hop Count: 255
Packet Length: 92
NHRP Packet Checksum: 0xa772 [correct]
[NHRP Packet Checksum Status: Good]
Extension Offset: 52
Version: 1 (NHRP - rfc2332)
NHRP Packet Type: NHRP Registration Request (3)
Source Address Type/Len: NSAP format/4
.0.. .... = Type: NSAP format (0)
..00 0100 = Length: 4
Source SubAddress Type/Len: NSAP format/0
.0.. .... = Type: NSAP format (0)
..00 0000 = Length: 0
NHRP Mandatory Part
Source Protocol Len: 4
Destination Protocol Len: 4
Flags: 0x8002, Uniqueness Bit, Cisco NAT Supported
Request ID: 0x00010003 (65539)
Source NBMA Address: 99.255.60.6
Source Protocol Address: 172.30.1.7
Destination Protocol Address: 172.30.1.1
Client Information Entry
Code: Success (0)
Prefix Length: 32
Unused: 0
Max Transmission Unit: 17916
Holding Time (s): 7200
Client Address Type/Len: NSAP format/0
.0.. .... = Type: NSAP format (0)
..00 0000 = Length: 0
Client Sub Address Type/Len: NSAP format/0
.0.. .... = Type: NSAP format (0)
..00 0000 = Length: 0
Client Protocol Length: 0
CIE Preference Value: 0
Responder Address Extension
1... .... .... .... = Compulsory Flag: True
..00 0000 0000 0011 = Extension Type: 0x0003
Extension length: 0
Forward Transit NHS Record Extension
1... .... .... .... = Compulsory Flag: True
..00 0000 0000 0100 = Extension Type: 0x0004
Extension length: 0
Reverse Transit NHS Record Extension
1... .... .... .... = Compulsory Flag: True
..00 0000 0000 0101 = Extension Type: 0x0005
Extension length: 0
Cisco NAT Address Extension
0... .... .... .... = Compulsory Flag: False
..00 0000 0000 1001 = Extension Type: 0x0009
Extension length: 20
Client Information Entry
Code: Success (0)
Prefix Length: 32
Unused: 0
Max Transmission Unit: 17916
Holding Time (s): 0
Client Address Type/Len: NSAP format/4
.0.. .... = Type: NSAP format (0)
..00 0100 = Length: 4
Client Sub Address Type/Len: NSAP format/0
.0.. .... = Type: NSAP format (0)
..00 0000 = Length: 0
Client Protocol Length: 4
CIE Preference Value: 0
Client NBMA Address: 99.255.10.2
Client Protocol Address: 172.30.1.1
End of Extension
1... .... .... .... = Compulsory Flag: True
..00 0000 0000 0000 = Extension Type: 0x0000
NHRP Resolution Message
Resolution request messages are sent by NHC to NHS to request address resolution information of a remote NHC. Resolution Reply provides the tunnel IP address and the NBMA of the remote spoke.
NHRP Redirect
NHRP redirect messages are sent by an intermediate router (hub) to notify the spoke
originating traffic that a specific destination network can be reached through a more optimal
path (spoke to spoke) (Phase 3) rather than through the hub. This makes it possible
for DMVPN to support building dynamic spoke-to-spoke tunnels. NHRP Redirect
suppression messages can be sent by spoke to indicate that the redirected path is not feasible.
This is similar to IP redirects, when packet in/out interface is the same.
Redirect messages are sent by hubs that are configured with the tunnel command
ip nhrp redirect in DMVPN phase 3 operation.
NHRP Purge
Purge messages are sent to remove a cached NHRP entry for example after the loss of a route by NHS. They are sent by the hub to spokes which previously requested for NBMA-overlay address mapping.
NHRP Error
Used to notify the sender of an NHRP packet that an error has occurred.
NHRP Extension Messages
These messages are included as an extension to an NHRP packet and contain additional information;
- Responder Address: To determine the address of the responding node for replies.
- Forward-Transit NHS Record and Reverse-Transit NHS Record: List of NHSs that an NHRP request and reply may have traversed respectively.
- Authentication: conveys authentication information between a pair of NHRP speakers in plain text.
- Vendor Private: vendor private information between speakers.
- NAT Address: DMVPN operates in NAT environment. It detects inside local and inside global addresses.
All listed NHRP packets contain a mandatory header section that must include:
- Source NBMA address
- Source protocol address
- Destination protocol address
- NHRP message type
NHRP Table
All spokes statically define the hub in their configuration. The NHS creates a mapping of NBMA to tunnel IP address for all NHCs that register with it. This mapping is known as the NHRP table. Information in this table is shared by the NHS to NHC so that when a spoke wishes to communicate with a remote NHC, when it sends a resolution request to the NHS, the NHS shares the NHRP mapping with the NHC.
Configuration of NHRP
Hub Configuration
NHRP configuration takes place under the tunnel interface. Continuing from the previous stages on the configuration of a hub using mGRE;
Step 9: Enable NHRP on the tunnel interface
NHRP is enabled on the tunnel using the tunnel mode command
ip nhrp network-id <1 – 4294967295>.
Network-ID is locally significant but is recommended to match on all routers.
It identifies the DMVPN cloud on a router. One router can participate in many
DMVPN clouds. This ID uniquely identifies each of these clouds.
HUB(config-if)#ip nhrp network-id 200
Step 10: (Optional) Define the tunnel key
The tunnel key is defined using the tunnel interface configuration command
tunnel key <0 – 4294967295>. The tunnel key
identifies the interface in the DMVPN cloud. It provides some limited control
over the registration of NHCs with NHSs. If configured on the NHS, the tunnel key
must match on all the connecting NHCs. This provides some control over
misconfigurations of NHCs registering with the wrong NHS.
HUB(config-if)#tunnel key 200
Configuration of the tunnel key adds an additional 4-bytes to the DMVPN header.
Step 11: (Optional) Enable multicast support for NHRP
HUB(config-if)#ip nhrp map multicast dynamic
This enables the hub to accept dynamic registrations from spokes.
This is particularly important if Interior Gateway Protocols such as OSPF and EIGRP are to be used for routing. IGPs use multicast for sending and receiving some packet types such as Hello packets to dynamically-learned spokes.
Step 12: (Optional) Authentication
To require spokes to authenticate with the NHS before registration or resolution requests can be made. The password used with NHRP authentication is stored in plain text.
HUB(config-if)#ip nhrp authentication simplesimple123
This command helps prevent misconfiguration such as one NHC registering with the wrong NHS. If authentication is configured on the NHS, it must be configured on all NHCs with the same password. For security of DMVPN tunnels, IPSec is used instead.
DMVPN Phase 1: Point to Point
DMVPN Phase 1 creates point to point tunnels between the hub and each of the spokes resulting in a more traditional hub and spoke topology. For each NHC to communicate with any other NHC, all traffic must be sent through the hub.
In DMVPN phase 1, all traffic is sent through the hub regardless of whether the spokes are in the same broadcast domain. The hub places itself as the next-hop in all routing updates.
In Phase 1, no NHRP requests are sent. Additionally, route summarization at the hub is possible in phase 1 and 3 but not in DMVPN phase 2.
The default behaviour of EIGRP in DMVPN phase 1 a router makes itself as the next-hop by default.
If using OSPF, DMPVN phase 1 is implemented if the network type is point-to-multipoint. This is because the hub automatically sets itself as the next-hop in routing updates. If the interface is broadcast, then phase 2 or 3 is implemented because OSPF does not modify the next-hop. It keeps it unchanged.
Spoke Configuration
Continuing the earlier configuration of mGRE on the spoke;
Step 5: Enable NHRP on the tunnel interface
NHRP is enabled using the command ip nhrp network-id <1 – 4294967295>.
The Network ID identifies the DMVPN cloud on a router. It is locally
significant and does not have to be the same between spoke an hub but is recommended
to match on all routers participant in the same DMVPN cloud.
SPOKE7(config-if)#ip nhrp network-id 200
Step 6: (Optional) Define the tunnel key
The command tunnel key <0 – 4294967295> is used to configure the tunnel
key. The tunnel
key identifies the interface in the DMVPN cloud. It provides some limited control
over the registration of NHCs with NHSs. If configured on the NHS, the tunnel key
must match on all the connecting NHCs. This provides some control over misconfigurations of
NHCs registering with the wrong NHS.
SPOKE7(config-if)#tunnel key 200
Step 7: Specify the NHS NBMA address and multicast mapping
The NHS NBMA address and multicast mapping is accomplished using the command
ip nhrp nhs <nhs_tunnel_address> nbma <nbma_address> multicast.
SPOKE7(config-if)#ip nhrp nhs 172.30.1.1 nbma 99.255.10.2 multicast
Alternative NHRP mapping commands are needed only where a static unicast or multicast map is needed for a node (NHC).
#ip nhrp nhs <nhs_tunnel_address>
#ip nhrp map <nhs_tunnel_address> <nhs_nbma_addr>
#ip nhrp map multicast <nhs_nbma_addr | dynamic>
SPOKE7(config-if)#ip nhrp nhs 172.30.1.1
SPOKE7(config-if)#ip nhrp map 172.30.1.1 99.255.10.2
SPOKE7(config-if)#ip nhrp map multicast 99.255.10.2
The value of the NBMA address of the NHS must be the same as the IP address configured as tunnel source IP address or specifying the tunnel source interface during configuring of the tunnel interface on the NHS. Configuring any other IP address on the NHS even if the NHC has a route to that interface will not activate the tunnel.
Verification
traceroute
The NHS (HUB) should always be one hop away from the NHC through the VPN tunnel. This can confirmed through a traceroute to the NHS from any of the NHCs.
SPOKE2#traceroute 172.30.1.7
Type escape sequence to abort.
Tracing the route to 172.30.1.7
VRF info: (vrf in name/id, vrf out name/id)
1 172.30.1.1 24 msec 56 msec 52 msec
2 172.30.1.7 176 msec 144 msec 140 msec
SPOKE2#
The NHCs should be two hops away from each other with the first hop being the NHS.
SPOKE7#traceroute 172.30.1.2
Type escape sequence to abort.
Tracing the route to 172.30.1.2
VRF info: (vrf in name/id, vrf out name/id)
1 172.30.1.1 76 msec 76 msec 24 msec
2 172.30.1.2 132 msec 84 msec 84 msec
SPOKE7#
show dmvpn detail
HUB#show dmvpn detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface Tunnel0 is up/up, Addr. is 172.30.1.1, VRF ""
Tunnel Src./Dest. addr: 99.255.10.2/MGRE, Tunnel VRF ""
Protocol/Transport: "multi-GRE/IP", Protect ""
Interface State Control: Disabled
nhrp event-publisher : Disabled
Type:Hub, Total NBMA Peers (v4/v6): 2
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
1 99.255.30.1 172.30.1.2 UP 01:18:17 D 172.30.1.2/32
1 99.255.60.6 172.30.1.7 UP 00:49:31 D 172.30.1.7/32
Crypto Session Details:
--------------------------------------------------------------------------------
Pending DMVPN Sessions:
show ip nhrp
View the NHRP table on the hub and the spoke. The records in the NHRP table expire after 5 minutes if not used. On spokes, before communicating with any other remote spoke, only the hub information is contained in the NHRP table.
The NHRP table can be viewed as shown below:
HUB#show ip nhrp detail
172.30.1.2/32 via 172.30.1.2
Tunnel0 created 05:05:46, expire 01:39:22
Type: dynamic, Flags: unique registered used
NBMA address: 99.255.30.1
172.30.1.3/32 via 172.30.1.3
Tunnel0 created 04:32:30, expire 01:54:17
Type: dynamic, Flags: unique registered used
NBMA address: 99.255.40.6
172.30.1.7/32 via 172.30.1.7
Tunnel0 created 05:03:10, expire 01:21:34
Type: dynamic, Flags: unique registered used
NBMA address: 99.255.60.6
172.30.1.8/32 via 172.30.1.8
Tunnel0 created 05:00:34, expire 01:54:43
Type: dynamic, Flags: unique registered used
NBMA address: 99.255.60.2
172.30.1.100/32 via 172.30.1.100
Tunnel0 created 05:06:18, expire 01:39:06
Type: dynamic, Flags: unique registered used
NBMA address: 99.255.20.2
HUB#
DMVPN Phase 2
In DMVPN phase 2 operation, all spoke to spoke data traffic is routed directly without transiting the hub. All routers i.e. NHCs and NHS implement multi-point GRE. A dynamic NHRP mapping entry appears in the NHRP table for communication with a remote spoke. When a spoke initiates communication with a remote spoke, it sends an NHRP request to the hub about the remote NBMA IP address. The hub sends an NHRP forward to the spoke (destination) indicating the source spoke NBMA and tunnel addresses. The destination spoke adds this mapping entry into its NHRP table /cache. The destination spoke then sends the source spoke an NHRP reply (unicast) with its NBMA and tunnel addresses.
Spokes are statically configured with the hub. The hub does not modify the next-hop in routing updates. If a spoke X receives a routing update from the hub and this update has an unchanged next-hop, the NHRP table is consulted for the next-hop NBMA address. If the next-hop NBMA address is not in the NHRP table, an NHRP request is sent to the hub requesting for the underlay-overlay IP address mapping of the next-hop. The hub will forward with request to the spoke that is the next-hop.
Phase 2 doesn’t support summarisation (at the hub). This is the primary difference between phase 2 and phase 3. If the hub summarizes the routes, then spoke-to-spoke communication will not happen directly and DMVPN reverts to phase 1 where all data-plane traffic will transit the hub.
Spoke Configuration
In configuration of a phase 2 DMVPN tunnel, replace tunnel destination command
with tunnel mode gre multipoint.
Replace:
SPOKE7(config-if)#tunnel destination 99.255.10.2
with
SPOKE7(config-if)#tunnel mode gre multipoint
Next-Hop
If EIGRP is configured as the overlay IGP, the hub does not modify the next hop
attribute of remote prefixes that it advertises.
So the next-hop-self feature of EIGRP on the tunnel interface
can be disabled on the NHS. Additionally, as the hub is connected to the spokes
through the same interface, split-horizon for EIGRP should also be disabled on
the tunnel interface otherwise EIGRP will not advertise routes that are learned
via the mGRE interface back out that interface;
HUB(config)#interface tunnel0
HUB(config-if)#no ip next-hop-self eigrp 100
HUB(config-if)#no ip split-horizon eigrp 100
With OSPF, the interface type should be changed to BROADCAST from point-to-multipoint.
HUB(config)#interface tunnel0
HUB(config-if)#ip ospf network broadcast
Like OSPF, the tunnel interface in IS-IS should be configured as a BROADCAST. However, database synchronisation and convergence need to be completed first. With eBGP, third party next-hops can be used. IBGP does not modify the next-hop by default.
Dynamic Peers
The mapping of tunnel to NBMA addresses is dynamic on the NHS. Each NHC can register with the NHC without any prior preconfigurations on the NHS. This results in the NHCs being dynamic peers. The number of spokes may grow exponentially. Dynamic peering makes the registration of this increasing number of NHCs scalable.
Peering is automatic under the following conditions:
- Correct authentication
- Proper phase 1, phase 2 matching.
The hub has open configuration accepting NHCs with the correct configuration. The NHCs are configured with the IP address of the hub.
The spoke to spoke tunnel information is temporary. The address of the hub is permanent. However, for spoke information,the tunnel is torn down after the communication is completed and the expiry of the tunnel session timer.
OSPF
In DMVPN phase 2, if OSPF is configured as the routing protocol for the overlay network, the following should be considered:
- Configure the tunnel interface on the hub with the highest priority
ip ospf priority 255. This ensures that the hub becomes the designated router (DR) - The spokes should be eliminated from DR/BDR election:
ip ospf priority 0. - The tunnel interface should be set to BROADCAST:
ip ospf network broadcast.
DMVPN Phase 3
Spoke Configuration
In DMVPN Phase 3 configuration;
NHS
HUB(config)#interface tunnel0
HUB(config-if)#ip nhrp redirect
NHC;
SPOKE7(config)#interface tunnel0
SPOKE7(config-if)#ip nhrp shortcut
The command ip nhrp redirect on the NHS checks the flow of packets on the
physical interface and sends a redirect message to the source spoke router when
it detects packets hairpinning out of the DMVPN cloud. Hairpinning is when
traffic is received and sent out of an interface in the same DMVPN cloud (identified
by the NHRP network ID). Hairpinning of traffic triggers redirect messages.
When NHC X wants to communicate with NHC Y:
- NHC X sends traffic to the NHS.
- NHS sends NHRP messages to NHC Y informing NHC Y that NHC X would like to communicate with it.
- NHC Y replies with its particulars to NHS.
- NHS detects that the reply from NHC Y was received through the same interface that it received the resolution request from NHC X (hairpining). It sends a traffic indication (redirect) message to NHC Y informing NHC Y that NHC X can be reached through a direct route than the route through the NHS.
- NHS also sends a redirect message to NHC X indicating that NHC Y can be reached through a better direct route than the route through NHS.
- The redirect from the NHS is known as traffic indication.
- For NHC X and NHC Y to utilise these redirect messages from NHS, they need to
have the command
ip nhrp shortcutconfigured on their tunnel interface.
SPOKE8#traceroute 10.2.12.1
Type escape sequence to abort.
Tracing the route to 10.2.12.1
VRF info: (vrf in name/id, vrf out name/id)
1 172.30.1.1 96 msec 60 msec 40 msec
2 172.30.1.2 68 msec 56 msec 52 msec
SPOKE8#
SPOKE8#traceroute 10.2.12.1
Type escape sequence to abort.
Tracing the route to 10.2.12.1
VRF info: (vrf in name/id, vrf out name/id)
1 172.30.1.2 16 msec 72 msec 28 msec
SPOKE8#
Verification of traffic indication
SPOKE7#debug dmvpn all all
SPOKE7#ping 10.2.10.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.2.10.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 16/82/256 ms
SPOKE7#
*Apr 7 04:30:18.035: NHRP: NHRP could not map 172.30.1.2 to NBMA, cache entry not found
*Apr 7 04:30:18.039: NHRP: MACADDR: if_in null netid-in 0 if_out Tunnel0 netid-out 1000
*Apr 7 04:30:18.039: NHRP: Sending packet to NHS 172.30.1.1 on Tunnel0
*Apr 7 04:30:18.039: NHRP: NHRP successfully mapped '172.30.1.1' to NBMA 99.255.10.2
*Apr 7 04:30:18.043: NHRP: Checking for delayed event NULL/172.30.1.2 on list (Tunnel0).
*Apr 7 04:30:18.043: NHRP: No node found.
*Apr 7 04:30:18.043: NHRP: No cache for forwarding(flags:0x0) to 172.30.1.2
*Apr 7 04:30:18.043: NHRP: Updating with NHS cache for dst:172.30.1.2
*Apr 7 04:30:18.043: NHRP: Tunnel0: Cache add for target 172.30.1.2/32 next-hop 172.30.1.2
*Apr 7 04:30:18.043: 99.255.10.2
*Apr 7 04:30:18.043: NHRP: Adding Tunnel Endpoints (VPN: 172.30.1.2, NBMA: 99.255.10.2)
*Apr 7 04:30:18.055: NHRP: Successfully attached NHRP subblock for Tunnel Endpoints (VPN: 172.30.1.2, NBMA: 99.255.10.2)
*Apr 7 04:30:18.059
SPOKE7#: NHRP: Inserted subblock node for cache: Target Inserted subblock node for cache: Target 172.30.1.2/32nhop 172.30.1.2
*Apr 7 04:30:18.063: NHRP: Converted internal dynamic cache entry for 172.30.1.2/32 interface Tunnel0 to external
*Apr 7 04:30:18.067: NHRP: Enqueued NHRP Resolution Request for destination: 172.30.1.2
*Apr 7 04:30:18.083: NHRP: Checking for delayed event NULL/172.30.1.2 on list (Tunnel0).
*Apr 7 04:30:18.083: NHRP: No node found.
*Apr 7 04:30:18.087: NHRP-ATTR: Requester Ext Len: Total ext_len with NHRP attribute VPE 20
*Apr 7 04:30:18.091: NHRP: Sending NHRP Resolution Request for dest: 172.30.1.2 to nexthop: 172.30.1.2 using our src: 172.30.1.7
*Apr 7 04:30:18.091: NHRP: Attempting to send packet via DEST 172.30.1.2
*Apr 7 04:30:18.095: NHRP: Setting 'used' flag on cache entry with nhop: 172.30.1.2
*Apr 7 04:30:18.099: NHRP: NHRP successfully mapped '172.30.1.2' to NBMA 99.255.10.2
*Apr 7 04:30:18.103: NHRP: Encapsulation succeeded. Sending
SPOKE7# NHRP Control Packet NBMA Address: 99.255.10.2
*Apr 7 04:30:18.107: NHRP: Send Resolution Request via Tunnel0 vrf 0, packet size: 72
*Apr 7 04:30:18.111: src: 172.30.1.7, dst: 172.30.1.2
*Apr 7 04:30:18.111: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Apr 7 04:30:18.115: shtl: 4(NSAP), sstl: 0(NSAP)
*Apr 7 04:30:18.115: pktsz: 72 extoff: 52
*Apr 7 04:30:18.115: (M) flags: "router auth src-stable nat ", reqid: 3
*Apr 7 04:30:18.119: src NBMA: 99.255.60.6
*Apr 7 04:30:18.119: src protocol: 172.30.1.7, dst protocol: 172.30.1.2
*Apr 7 04:30:18.123: (C-1) code: no error(0)
*Apr 7 04:30:18.127: prefix: 32, mtu: 17916, hd_time: 7200
*Apr 7 04:30:18.127: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0
*Apr 7 04:30:18.127: Responder Address Extension(3):
*Apr 7 04:30:18.127: Forward Transit NHS Record Extension(4):
*Apr 7 04:30:18.127: Reverse Transit NHS Record Extension(5):
*Apr 7 04:30:18.127: NAT a
SPOKE7#ddress Extension(9):
*Apr 7 04:30:18.127: NHRP: 96 bytes out Tunnel0
*Apr 7 04:30:18.127: NHRP-RATE: Sending initial Resolution Request for 172.30.1.2, reqid 3
*Apr 7 04:30:18.171: NHRP-ATTR: ext_type: 32772, ext_len : 0
*Apr 7 04:30:18.171: NHRP-ATTR: ext_type: 32773, ext_len : 0
*Apr 7 04:30:18.175: NHRP-ATTR: ext_type: 9, ext_len : 0
*Apr 7 04:30:18.175: NHRP-ATTR: ext_type: 32768, ext_len : 0
*Apr 7 04:30:18.179: NHRP: Receive Traffic Indication via Tunnel0 vrf 0, packet size: 84
*Apr 7 04:30:18.183: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Apr 7 04:30:18.183: shtl: 4(NSAP), sstl: 0(NSAP)
*Apr 7 04:30:18.187: pktsz: 84 extoff: 68
*Apr 7 04:30:18.187: (M) traffic code: redirect(0)
*Apr 7 04:30:18.187: src NBMA: 99.255.10.2
*Apr 7 04:30:18.187: src protocol: 172.30.1.1, dst protocol: 172.30.1.7
*Apr 7 04:30:18.187: Contents of nhrp traffic indication packet:
*Apr 7 04:30:18.187: 45 00 00 64 00 0
SPOKE7#5 00 00 FE 01 FB 6B AC 1E 01 07
*Apr 7 04:30:18.187: 0A 02 0A 01 08 00 A8 4B 00 01 00
*Apr 7 04:30:18.187: Forward Transit NHS Record Extension(4):
*Apr 7 04:30:18.187: Reverse Transit NHS Record Extension(5):
*Apr 7 04:30:18.187: NAT address Extension(9):
*Apr 7 04:30:18.187: NHRP: netid_in = 1000, to_us = 1
*Apr 7 04:30:18.187: NHRP: netid_out 0, netid_in 1000
*Apr 7 04:30:18.187: NHRP: Parsing NHRP Traffic Indication
*Apr 7 04:30:18.259: NHRP-ATTR: ext_type: 32771, ext_len : 20
*Apr 7 04:30:18.263: NHRP-ATTR: ext_type: 32772, ext_len : 20
*Apr 7 04:30:18.263: NHRP-ATTR: ext_type: 32773, ext_len : 0
*Apr 7 04:30:18.267: NHRP-ATTR: ext_type: 9, ext_len : 0
*Apr 7 04:30:18.267: NHRP-ATTR: ext_type: 32768, ext_len : 0
*Apr 7 04:30:18.271: NHRP: Receive Resolution Reply via Tunnel0 vrf 0, packet size: 120
*Apr 7 04:30:18.271: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Apr 7 04:30:18.271: shtl: 4(NSAP), sstl: 0(NSAP)
SPOKE7#*Apr 7 04:30:18.271: pktsz: 120 extoff: 60
*Apr 7 04:30:18.271: (M) flags: "router auth dst-stable unique src-stable nat ", reqid: 3
*Apr 7 04:30:18.271: src NBMA: 99.255.60.6
*Apr 7 04:30:18.271: src protocol: 172.30.1.7, dst protocol: 172.30.1.2
*Apr 7 04:30:18.271: (C-1) code: no error(0)
*Apr 7 04:30:18.271: prefix: 32, mtu: 17916, hd_time: 7200
*Apr 7 04:30:18.271: addr_len: 4(NSAP), subaddr_len: 0(NSAP), proto_len: 4, pref: 0
*Apr 7 04:30:18.275: client NBMA: 99.255.30.1
*Apr 7 04:30:18.275: client protocol: 172.30.1.2
*Apr 7 04:30:18.275: Responder Address Extension(3):
*Apr 7 04:30:18.275: (C) code: no error(0)
*Apr 7 04:30:18.275: prefix: 32, mtu: 17916, hd_time: 7200
*Apr 7 04:30:18.275: addr_len: 4(NSAP), subaddr_len: 0(NSAP), proto_len: 4, pref: 0
*Apr 7 04:30:18.275: client NBMA: 99.255.30.1
*Apr 7 04:30:18.275: client protocol: 172.30.1.2
*Apr 7 04:30:18.275: Forward
SPOKE7# Transit NHS Record Extension(4):
*Apr 7 04:30:18.275: (C-1) code: no error(0)
*Apr 7 04:30:18.275: prefix: 32, mtu: 17916, hd_time: 7200
*Apr 7 04:30:18.275: addr_len: 4(NSAP), subaddr_len: 0(NSAP), proto_len: 4, pref: 0
*Apr 7 04:30:18.275: client NBMA: 99.255.10.2
*Apr 7 04:30:18.275: client protocol: 172.30.1.1
*Apr 7 04:30:18.275: Reverse Transit NHS Record Extension(5):
*Apr 7 04:30:18.275: NAT address Extension(9):
*Apr 7 04:30:18.275: NHRP: netid_in = 0, to_us = 1
*Apr 7 04:30:18.275: NHRP: nhrp_pak_reply_from_target: answer_nbma: 99.255.30.1, responder_nbma: 99.255.30.1
*Apr 7 04:30:18.275: NHRP: Tunnels gave us pak src: 99.255.30.1
*Apr 7 04:30:18.275: NHRP: Checking for delayed event NULL/172.30.1.2 on list (Tunnel0).
*Apr 7 04:30:18.275: NHRP: No node found.
*Apr 7 04:30:18.279: NHRP: No need to delay processing of resolution event nbma src:99.255.60.6 nbma dst:99.255.30.1
*Apr 7 04:30:18.279: NHRP-ATTR: In nhrp_cach
SPOKE7#e_pak LINE: 1422
*Apr 7 04:30:18.279: NHRP: Tunnel0: Cache update for target 172.30.1.2/32 next-hop 172.30.1.2
*Apr 7 04:30:18.279: 99.255.30.1
*Apr 7 04:30:18.283: NHRP: Deleted subbblock node associated with Cache: Prefix: Deleted subbblock node associated with Cache: Prefix: 172.30.1.2/32nhop 17
2.30.1.2
*Apr 7 04:30:18.283: NHRP: Deleted subbblock node associated with Cache: Prefix: Deleted subbblock node associated with Cache: Prefix: 172.30.1.2/32nhop 17
2.30.1.2
*Apr 7 04:30:18.283: NHRP: Adding Tunnel Endpoints (VPN: 172.30.1.2, NBMA: 99.255.30.1)
*Apr 7 04:30:18.283: NHRP: Cleanup up 0 stale cache entries
*Apr 7 04:30:18.287: NHRP: Successfully attached NHRP subblock for Tunnel Endpoints (VPN: 172.30.1.2, NBMA: 99.255.30.1)
*Apr 7 04:30:18.287: NHRP: Inserted subblock node for cache: Target Inserted subblock node for cache: Target 172.30.1.2/32nhop 172.30.1.2
*Apr 7 04:30:18.295: NHRP: Setting 'used' flag on cache entry with nhop: 172.30.1.2
SPOKE7#*Apr 7 04:30:18.295: NHRP: NHRP successfully mapped '172.30.1.2' to NBMA 99.255.30.1
*Apr 7 04:30:18.347: NHRP: NHRP successfully mapped '172.30.1.2' to NBMA 99.255.30.1
*Apr 7 04:30:18.367: NHRP: NHRP successfully mapped '172.30.1.2' to NBMA 99.255.30.1
*Apr 7 04:30:18.411: NHRP: NHRP successfully mapped '172.30.1.2' to NBMA 99.255.30.1
SPOKE7#
On the NHS:
HUB#debug dmvpn all all
HUB#
*Apr 7 04:57:28.563: NHRP: Tunnels gave us remote_nbma: 99.255.60.6 for Redirect
*Apr 7 04:57:28.563: NHRP: Attempting to Redirect, remote_nbma: 99.255.60.6
*Apr 7 04:57:28.567: NHRP: inserting (99.255.60.6/10.2.10.1) in redirect table
*Apr 7 04:57:28.571: NHRP-ATTR: Requester Ext Len: Total ext_len with NHRP attribute VPE 16
*Apr 7 04:57:28.575: NHRP: Attempting to send packet via DEST 172.30.1.7
*Apr 7 04:57:28.579: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: 99.255.60.6
*Apr 7 04:57:28.583: NHRP: Send Traffic Indication via Tunnel0 vrf 0, packet size: 84
*Apr 7 04:57:28.587: src: 172.30.1.1, dst: 172.30.1.7
*Apr 7 04:57:28.591: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Apr 7 04:57:28.591: shtl: 4(NSAP), sstl: 0(NSAP)
*Apr 7 04:57:28.591: pktsz: 84 extoff: 68
*Apr 7 04:57:28.595: (M) traffic code: redirect(0)
*Apr 7 04:57:28.595: src NBMA: 99.255.10.2
*Apr 7 04:57:28.595: src protocol: 1
HUB#72.30.1.1, dst protocol: 172.30.1.7
*Apr 7 04:57:28.595: Contents of nhrp traffic indication packet:
*Apr 7 04:57:28.595: 45 00 00 64 00 05 00 00 FE 01 FB 6B AC 1E 01 07
*Apr 7 04:57:28.595: 0A 02 0A 01 08 00 A8 4B 00 01 00
*Apr 7 04:57:28.595: Forward Transit NHS Record Extension(4):
*Apr 7 04:57:28.595: Reverse Transit NHS Record Extension(5):
*Apr 7 04:57:28.595: NAT address Extension(9):
*Apr 7 04:57:28.595: NHRP: 108 bytes out Tunnel0
*Apr 7 04:57:28.599: NHRP-ATTR: ext_type: 32771, ext_len : 0
*Apr 7 04:57:28.599: NHRP-ATTR: ext_type: 32772, ext_len : 0
*Apr 7 04:57:28.599: NHRP-ATTR: ext_type: 32773, ext_len : 0
*Apr 7 04:57:28.599: NHRP-ATTR: ext_type: 9, ext_len : 0
*Apr 7 04:57:28.599: NHRP-ATTR: ext_type: 32768, ext_len : 0
*Apr 7 04:57:28.599: NHRP: Receive Resolution Request via Tunnel0 vrf 0, packet size: 72
*Apr 7 04:57:28.599: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Apr 7 04:57:28.599:
HUB# shtl: 4(NSAP), sstl: 0(NSAP)
*Apr 7 04:57:28.599: pktsz: 72 extoff: 52
*Apr 7 04:57:28.599: (M) flags: "router auth src-stable nat ", reqid: 3
*Apr 7 04:57:28.599: src NBMA: 99.255.60.6
*Apr 7 04:57:28.599: src protocol: 172.30.1.7, dst protocol: 172.30.1.2
*Apr 7 04:57:28.599: (C-1) code: no error(0)
*Apr 7 04:57:28.599: prefix: 32, mtu: 17916, hd_time: 7200
*Apr 7 04:57:28.599: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0
*Apr 7 04:57:28.599: Responder Address Extension(3):
*Apr 7 04:57:28.599: Forward Transit NHS Record Extension(4):
*Apr 7 04:57:28.599: Reverse Transit NHS Record Extension(5):
*Apr 7 04:57:28.599: NAT address Extension(9):
*Apr 7 04:57:28.599: NHRP: netid_in = 1000, to_us = 0
*Apr 7 04:57:28.599: NHRP: nhrp_rtlookup for destination on 172.30.1.2 yielded interface Tunnel0, prefixlen 24
*Apr 7 04:57:28.603: NHRP: netid_out 1000, netid_in 1000
*Apr 7 04:57:28.603: NHRP: Forwarding requ
HUB#est due to authoritative request.
*Apr 7 04:57:28.603: NHRP-ATTR: In nhrp_recv_resolution_request NHRP Resolution Request packet is forwarded to 172.30.1.2
*Apr 7 04:57:28.603: NHRP: Attempting to forward to destination: 172.30.1.2
*Apr 7 04:57:28.603: NHRP: Forwarding: NHRP SAS picked source: 172.30.1.1 for destination: 172.30.1.2
*Apr 7 04:57:28.603: NHRP: Attempting to send packet via DEST 172.30.1.2
*Apr 7 04:57:28.603: NHRP: NHRP successfully mapped '172.30.1.2' to NBMA 99.255.30.1
*Apr 7 04:57:28.607: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: 99.255.30.1
*Apr 7 04:57:28.607: NHRP: Forwarding Resolution Request via Tunnel0 vrf 0, packet size: 92
*Apr 7 04:57:28.607: src: 172.30.1.1, dst: 172.30.1.2
*Apr 7 04:57:28.607: (F) afn: AF_IP(1), type: IP(800), hop: 254, ver: 1
*Apr 7 04:57:28.607: shtl: 4(NSAP), sstl: 0(NSAP)
*Apr 7 04:57:28.607: pktsz: 92 extoff: 52
*Apr 7 04:57:28.607: (M) flags: "router auth src-s
HUB#table nat ", reqid: 3
*Apr 7 04:57:28.607: src NBMA: 99.255.60.6
*Apr 7 04:57:28.607: src protocol: 172.30.1.7, dst protocol: 172.30.1.2
*Apr 7 04:57:28.607: (C-1) code: no error(0)
*Apr 7 04:57:28.607: prefix: 32, mtu: 17916, hd_time: 7200
*Apr 7 04:57:28.607: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0
*Apr 7 04:57:28.607: Responder Address Extension(3):
*Apr 7 04:57:28.607: Forward Transit NHS Record Extension(4):
*Apr 7 04:57:28.607: (C-1) code: no error(0)
*Apr 7 04:57:28.607: prefix: 32, mtu: 17916, hd_time: 7200
*Apr 7 04:57:28.607: addr_len: 4(NSAP), subaddr_len: 0(NSAP), proto_len: 4, pref: 0
*Apr 7 04:57:28.607: client NBMA: 99.255.10.2
*Apr 7 04:57:28.607: client protocol: 172.30.1.1
*Apr 7 04:57:28.607: Reverse Transit NHS Record Extension(5):
*Apr 7 04:57:28.607: NAT address Extension(9):
*Apr 7 04:57:28.607: NHRP: 116 bytes out Tunnel0
*Apr 7 04:57:28.679: NHRP: Tun
HUB#nels gave us remote_nbma: 99.255.30.1 for Redirect
*Apr 7 04:57:28.679: NHRP: Attempting to Redirect, remote_nbma: 99.255.30.1
*Apr 7 04:57:28.683: NHRP: inserting (99.255.30.1/172.30.1.7) in redirect table
*Apr 7 04:57:28.687: NHRP-ATTR: Requester Ext Len: Total ext_len with NHRP attribute VPE 16
*Apr 7 04:57:28.691: NHRP: Attempting to send packet via DEST 10.2.10.1
*Apr 7 04:57:28.695: NHRP: Encapsulation succeeded. Sending NHRP Control Packet NBMA Address: 99.255.30.1
*Apr 7 04:57:28.695: NHRP: Send Traffic Indication via Tunnel0 vrf 0, packet size: 84
*Apr 7 04:57:28.699: src: 172.30.1.1, dst: 10.2.10.1
*Apr 7 04:57:28.699: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Apr 7 04:57:28.699: shtl: 4(NSAP), sstl: 0(NSAP)
*Apr 7 04:57:28.699: pktsz: 84 extoff: 68
*Apr 7 04:57:28.699: (M) traffic code: redirect(0)
*Apr 7 04:57:28.699: src NBMA: 99.255.10.2
*Apr 7 04:57:28.699: src protocol: 172.30.1.1, dst protocol: 10.2.10.1
HUB#
*Apr 7 04:57:28.699: Contents of nhrp traffic indication packet:
*Apr 7 04:57:28.699: 45 00 00 64 00 05 00 00 FE 01 FB 6B 0A 02 0A 01
*Apr 7 04:57:28.699: AC 1E 01 07 00 00 B0 4B 00 01 00
*Apr 7 04:57:28.699: Forward Transit NHS Record Extension(4):
*Apr 7 04:57:28.699: Reverse Transit NHS Record Extension(5):
*Apr 7 04:57:28.699: NAT address Extension(9):
*Apr 7 04:57:28.699: NHRP: 108 bytes out Tunnel0
The following is a Wireshark packet capture of an NHRP Traffic indication message:
Frame 47: 122 bytes on wire (976 bits), 122 bytes captured (976 bits) on interface -, id 0
Ethernet II, Src: ca:06:06:9e:00:08 (ca:06:06:9e:00:08), Dst: ca:07:06:ad:00:1c (ca:07:06:ad:00:1c)
Internet Protocol Version 4, Src: 99.255.10.2, Dst: 99.255.60.6
Generic Routing Encapsulation (NHRP)
Flags and Version: 0x0000
Protocol Type: NHRP (0x2001)
Next Hop Resolution Protocol (NHRP Traffic Indication)
NHRP Fixed Header
Address Family Number: IPv4 (0x0001)
Protocol Type (short form): IPv4 (0x0800)
Protocol Type (long form): 0000000000
Hop Count: 255
Packet Length: 84
NHRP Packet Checksum: 0xe9dc [correct]
[NHRP Packet Checksum Status: Good]
Extension Offset: 68
Version: 1 (NHRP - rfc2332)
NHRP Packet Type: NHRP Traffic Indication (8)
Source Address Type/Len: NSAP format/4
.0.. .... = Type: NSAP format (0)
..00 0100 = Length: 4
Source SubAddress Type/Len: NSAP format/0
.0.. .... = Type: NSAP format (0)
..00 0000 = Length: 0
NHRP Mandatory Part
Source Protocol Len: 4
Destination Protocol Len: 4
Source NBMA Address: 99.255.10.2
Source Protocol Address: 172.30.1.1
Destination Protocol Address: 172.30.1.7
Packet Causing Indication
Internet Protocol Version 4, Src: 172.30.1.7, Dst: 10.2.10.1
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 100
Identification: 0x000f (15)
000. .... = Flags: 0x0
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..0. .... = More fragments: Not set
...0 0000 0000 0000 = Fragment Offset: 0
Time to Live: 254
Protocol: ICMP (1)
Header Checksum: 0xfb61 [validation disabled]
[Header checksum status: Unverified]
Source Address: 172.30.1.7
Destination Address: 10.2.10.1
Internet Control Message Protocol
Type: 8 (Echo (ping) request)
Code: 0
Checksum: 0xb321 [unverified] [in ICMP error packet]
[Checksum Status: Unverified]
Identifier (BE): 3 (0x0003)
Identifier (LE): 768 (0x0300)
Sequence Number (BE): 0 (0x0000)
Sequence Number (LE): 0 (0x0000)
HiPerConTracer Trace Service
Magic Number: 0x80040000
Send TTL: 128
Round: 5
Checksum Tweak: 0x0000
Send Time Stamp: Jan 7, 2057 08:02:17.879552000 UTC
Forward Transit NHS Record Extension
1... .... .... .... = Compulsory Flag: True
..00 0000 0000 0100 = Extension Type: 0x0004
Extension length: 0
Reverse Transit NHS Record Extension
1... .... .... .... = Compulsory Flag: True
..00 0000 0000 0101 = Extension Type: 0x0005
Extension length: 0
Cisco NAT Address Extension
0... .... .... .... = Compulsory Flag: False
..00 0000 0000 1001 = Extension Type: 0x0009
Extension length: 0
End of Extension
1... .... .... .... = Compulsory Flag: True
..00 0000 0000 0000 = Extension Type: 0x0000
Extension length: 0
Routing Protocols
In DMVPN phase 3, if EIGRP is configured as the routing protocol over the overlay
network, EIGRP, by default, changes the next-hop to the router it is running on.
In phase 3, the configuration of next-hop-self feature is
usually ignored as the redirect on the NHS and shortcut on NHC will take care of
spoke to spoke communication; its presence or absence in NHS configuration has no
effect on the next-hop behaviour.
It is important to note that control-plane and IP multicast traffic still go through the hub regardless of DMVPN phase. This means that if an IGP such as OSPF is configured, the NHS should be configured as the Designated Router (DR) to distribute updates to the NHCs. With EIGRP, the next-hop-self feature should be disabled in order for the Phase 3 redirects to be successful. All neighborships of IGPs are with the NHS. NHCs cannot form IGP neighborships as they do not send multicast traffic to each other.
Summarization
In phase 3, when summarisation is implemented at the hub, this gets advertised to the spokes with the hub as the next-hop.
When summary routes are configured on routers the summary routes hide network
convergence. The summary route is installed on the NHS tunnel IP (through EIGRP)
and advertised to spokes. This reduces the unnecessary traffic entering the tunnel
as remote NHCs do not have to receive the EIGRP updates or queries on routes
internal to the NHS subnet. Also, convergence in the internal subnets of NHCs do
not get sent to other NHCs by the NHSs.
When an NHRP redirect occurs, NHRP will install a more specific route to the
remote NHC hence no NHO occurs.
Both entries in show dmvpn detail will be DT1 and not DT2 showing no next-hop-override occurs.
HUB(config)#router eigrp EIGRP_HUB
HUB(config-router)#address-family ipv4 unicast auto 1
HUB(config-router-af)#af-interface tunnel0
HUB(config-router-af-interface)#summary-address 10.7.0.0/16
The spokes will receive this summary route from the hub:
SPOKE2#show ip route eigrp
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
+ - replicated route, % - next hop override
Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 21 subnets, 3 masks
D 10.1.10.0/24 [90/76800640] via 172.30.1.1, 04:23:39, Tunnel0
D 10.1.11.0/24 [90/76800640] via 172.30.1.1, 04:23:39, Tunnel0
D 10.1.12.0/24 [90/76800640] via 172.30.1.1, 04:23:39, Tunnel0
D 10.7.0.0/16 [90/104960000] via 172.30.1.1, 00:00:14, Tunnel0
D 10.8.10.0/24 [90/104960000] via 172.30.1.8, 03:24:09, Tunnel0
D 10.8.11.0/24 [90/104960000] via 172.30.1.8, 03:24:09, Tunnel0
D 10.8.12.0/24 [90/104960000] via 172.30.1.8, 03:24:09, Tunnel0
If the spokes send or receive traffic from a host in a component network of the summary address, NHRP installs a more specific route in the routing table that is identified with the code ‘H’.
SPOKE2#ping 10.7.10.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.7.10.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/85/152 ms
SPOKE2#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
+ - replicated route, % - next hop override
Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 22 subnets, 3 masks
D 10.1.10.0/24 [90/76800640] via 172.30.1.1, 04:24:14, Tunnel0
D 10.1.11.0/24 [90/76800640] via 172.30.1.1, 04:24:14, Tunnel0
D 10.1.12.0/24 [90/76800640] via 172.30.1.1, 04:24:14, Tunnel0
C 10.2.10.0/24 is directly connected, Loopback10
L 10.2.10.1/32 is directly connected, Loopback10
C 10.2.11.0/24 is directly connected, Loopback11
L 10.2.11.1/32 is directly connected, Loopback11
C 10.2.12.0/24 is directly connected, Loopback12
L 10.2.12.1/32 is directly connected, Loopback12
C 10.2.13.0/24 is directly connected, Loopback13
L 10.2.13.1/32 is directly connected, Loopback13
D 10.7.0.0/16 [90/104960000] via 172.30.1.1, 00:00:49, Tunnel0
H 10.7.10.0/24 [250/1] via 172.30.1.7, 00:00:04, Tunnel0
D 10.8.10.0/24 [90/104960000] via 172.30.1.8, 03:24:44, Tunnel0
D 10.8.11.0/24 [90/104960000] via 172.30.1.8, 03:24:44, Tunnel0
D 10.8.12.0/24 [90/104960000] via 172.30.1.8, 03:24:44, Tunnel0
!output omitted...
SPOKE2#show ip route 10.7.10.0
Routing entry for 10.7.10.0/24
Known via "nhrp", distance 250, metric 1
Last update from 172.30.1.7 on Tunnel0, 00:03:17 ago
Routing Descriptor Blocks:
* 172.30.1.7, from 172.30.1.7, 00:03:17 ago, via Tunnel0
Route metric is 1, traffic share count is 1
The ip nhrp shortcut command on the spoke is required for
the installation of NHRP routes in the routing table. NHRP installed routes have
an administrative distance (AD) of 250.
This NHRP installed route is also reflected in the NHRP table:
SPOKE2#show ip nhrp
10.7.10.0/24 via 172.30.1.7
Tunnel0 created 00:00:21, expire 01:59:37
Type: dynamic, Flags: router used rib
NBMA address: 99.255.60.6
172.30.1.1/32 via 172.30.1.1
Tunnel0 created 04:35:26, never expire
Type: static, Flags: used
NBMA address: 99.255.10.2
In the NHRP table, entries have the flag "rib" indicating that a route has been generated and installed in the routing table.
In the output of the DMVPN mapping table, NHRP installed routes have the flag
DT1.
SPOKE2#show dmvpn detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface Tunnel0 is up/up, Addr. is 172.30.1.2, VRF ""
Tunnel Src./Dest. addr: 99.255.30.1/MGRE, Tunnel VRF ""
Protocol/Transport: "multi-GRE/IP", Protect ""
Interface State Control: Disabled
nhrp event-publisher : Disabled
IPv4 NHS:
172.30.1.1 RE NBMA Address: 99.255.10.2 priority = 0 cluster = 0
Type:Spoke, Total NBMA Peers (v4/v6): 3
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
1 99.255.60.6 172.30.1.7 UP 00:40:42 DT1 10.7.10.0/24
2 99.255.60.2 172.30.1.8 UP 00:03:29 DT2 10.8.10.0/24
99.255.60.2 172.30.1.8 UP 00:03:29 D 172.30.1.8/32
1 99.255.10.2 172.30.1.1 UP 05:15:48 S 172.30.1.1/32
Crypto Session Details:
--------------------------------------------------------------------------------
Pending DMVPN Sessions:
One key limitation of OSPF as the overlay protocol is that all the tunnel interfaces must be in the same area. Due to OSPF's strict summarization policies that do not permit summarization in an area, the spokes will have to receive all the routes in the same area as on the hub. Because of this, EIGRP and BGP have an edge of OSPF and IS-IS(link-state protocols) as the preferred overlay routing protocol.
Next-hop Override
When the shortcut feature has been applied to a specific route, if the prefixes are more specific, then the route learned through the overlay routing protocol will have the next-hop override flag(%) is set.
NHRP manipulates the routing table as necessary. An installed route is indicated
by the code ‘H’. A next hop override is indicated by the
code ‘%’. To display
explicit NHRP shortcuts, show ip route next-hop-override.
The shortcut has a symbol NHO and has a lower metric (255).
On the routing table, the route with the next-hop-override still displays its
original next-hop. However, to confirm that this next-hop is overridden, use the
command ip cef <prefix>.
SPOKE2#ping 10.8.10.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.8.10.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/39/48 ms
SPOKE2#
SPOKE2#show ip route eigrp
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
+ - replicated route, % - next hop override
Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 22 subnets, 3 masks
D 10.1.10.0/24 [90/76800640] via 172.30.1.1, 05:20:53, Tunnel0
D 10.1.11.0/24 [90/76800640] via 172.30.1.1, 05:20:53, Tunnel0
D 10.1.12.0/24 [90/76800640] via 172.30.1.1, 05:20:54, Tunnel0
D 10.7.0.0/16 [90/104960000] via 172.30.1.1, 00:57:29, Tunnel0
D % 10.8.10.0/24 [90/104960000] via 172.30.1.8, 04:21:24, Tunnel0
D 10.8.11.0/24 [90/104960000] via 172.30.1.8, 04:21:24, Tunnel0
D 10.8.12.0/24 [90/104960000] via 172.30.1.8, 04:21:24, Tunnel0
SPOKE2#
SPOKE2#show ip route 10.8.10.0
Routing entry for 10.8.10.0/24
Known via "eigrp 1", distance 90, metric 104960000, type internal
Redistributing via eigrp 1
Last update from 172.30.1.8 on Tunnel0, 04:21:27 ago
Routing Descriptor Blocks:
* 172.30.1.8, from 172.30.1.1, 04:21:27 ago, via Tunnel0
Route metric is 104960000, traffic share count is 1
Total delay is 105000 microseconds, minimum bandwidth is 100 Kbit
Reliability 255/255, minimum MTU 1400 bytes
Loading 2/255, Hops 2
SPOKE2#
SPOKE2#show dmvpn detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface Tunnel0 is up/up, Addr. is 172.30.1.2, VRF ""
Tunnel Src./Dest. addr: 99.255.30.1/MGRE, Tunnel VRF ""
Protocol/Transport: "multi-GRE/IP", Protect ""
Interface State Control: Disabled
nhrp event-publisher : Disabled
IPv4 NHS:
172.30.1.1 RE NBMA Address: 99.255.10.2 priority = 0 cluster = 0
Type:Spoke, Total NBMA Peers (v4/v6): 3
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
1 99.255.60.6 172.30.1.7 UP 00:56:58 DT1 10.7.10.0/24
2 99.255.60.2 172.30.1.8 UP 00:19:45 DT2 10.8.10.0/24
99.255.60.2 172.30.1.8 UP 00:19:45 D 172.30.1.8/32
1 99.255.10.2 172.30.1.1 UP 05:32:04 S 172.30.1.1/32
!Some output omitted
The next-hop override flag is also set in the NHRP table for this route:
SPOKE2#show ip nhrp detail
10.7.10.0/24 via 172.30.1.7
Tunnel0 created 01:12:43, expire 00:47:15
Type: dynamic, Flags: router used rib
NBMA address: 99.255.60.6
10.8.10.0/24 via 172.30.1.8
Tunnel0 created 00:35:30, expire 01:24:29
Type: dynamic, Flags: router used rib nho
NBMA address: 99.255.60.2
172.30.1.1/32 via 172.30.1.1
Tunnel0 created 05:47:49, never expire
Type: static, Flags: used
NBMA address: 99.255.10.2
172.30.1.8/32 via 172.30.1.8
Tunnel0 created 00:35:30, expire 01:24:29
Type: dynamic, Flags: router used
NBMA address: 99.255.60.2
Routing in DMVPN
DMVPN was developed by Cisco Systems with EIGRP in mind, as a routing protocol. To fully utilise the features of DMVPN, EIGRP is recommended as the routing protocol of the overlay network. However, other routing protocols such as OSPF, RIP, IS-IS can be used as routing protocols as well in DMVPN configurations. If using EIGRP for the internal networks, split horizon needs to be disabled on the tunnel interface. As EIGRP always maintain loop-free routes using the feasibility condition, disabling split-horizon does not introduce loops. However, it might be preferrable to disable it entirely.
HUB#configure terminal
HUB(config)#interface tunnel 10
HUB(config-if)#no ip split-horizon eigrp 100
HUB(config-if)#no ip next-hop-self eigrp 100
When configuring a routing protocol over the overlay network, never advertise the underlay network in routing protocol over the overlay network.
Verification of DMVPN
Viewing DMVPN Tunnel States
The tunnel interface experiences varying states based on the stage of tunnel creation or if the tunnel experienced problems during establishment or operation. Addition of encryption to the tunnel also affects the state of the DMVPN tunnels.
show dmvpn interface <interface> detail
The specified interface is the tunnel interface.
This command displays tunnel interface, role, state, peers and uptime. Tunnel states can be any of the following:
- INTF: Line protocol of DMVPN tunnel is down
- IKE: DMVPN tunnels configured with IPSec have not yet successfully established an IKE session.
- IPSec: An IKE session is established but an IPSec SA has not yet been established.
- NHRP: The DMVPN spoke router has not yet successfully registered.
- Up: Spoke has registered with hub and received ACK (positive registration reply). An up interface state must be maintained for traffic to flow.
SPOKE8#show dmvpn interface tunnel0 detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface Tunnel0 is up/up, Addr. is 172.30.1.8, VRF ""
Tunnel Src./Dest. addr: 99.255.60.2/MGRE, Tunnel VRF ""
Protocol/Transport: "multi-GRE/IP", Protect ""
Interface State Control: Disabled
nhrp event-publisher : Disabled
IPv4 NHS:
172.30.1.1 RE NBMA Address: 99.255.10.2 priority = 0 cluster = 0
Type:Spoke, Total NBMA Peers (v4/v6): 4
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
1 99.255.10.2 172.30.1.1 UP 00:21:37 S 172.30.1.1/32
1 99.255.30.1 172.30.1.2 UP 00:06:04 D 172.30.1.2/32
1 99.255.60.6 172.30.1.7 UP 00:21:20 D 172.30.1.7/32
1 99.255.60.2 172.30.1.8 UP 00:21:20 DLX 172.30.1.8/32
show dmvpn detail
Run from the hub displays the mapping between the tunnel address and NBMA address. This command displays local tunnel and NBMA ip addresses, tunnel health monitoring and VRF contexts IPSec crypto info.
SPOKE8#show dmvpn detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface Tunnel0 is up/up, Addr. is 172.30.1.8, VRF ""
Tunnel Src./Dest. addr: 99.255.60.2/MGRE, Tunnel VRF ""
Protocol/Transport: "multi-GRE/IP", Protect ""
Interface State Control: Disabled
nhrp event-publisher : Disabled
IPv4 NHS:
172.30.1.1 RE NBMA Address: 99.255.10.2 priority = 0 cluster = 0
Type:Spoke, Total NBMA Peers (v4/v6): 4
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
1 99.255.10.2 172.30.1.1 UP 00:25:40 S 172.30.1.1/32
1 99.255.30.1 172.30.1.2 UP 00:10:07 D 172.30.1.2/32
1 99.255.60.6 172.30.1.7 UP 00:25:22 D 172.30.1.7/32
1 99.255.60.2 172.30.1.8 UP 00:25:22 DLX 172.30.1.8/32
Crypto Session Details:
--------------------------------------------------------------------------------
Pending DMVPN Sessions:
Viewing the NHRP Cache
show ip nhrp [brief | detail]
The displays the local NHRP cache. The detail keyword lists routers that submitted
an NHRP request and its request ID. The NHRP cache displays NHRP mapping entries and NHRP message flags.
SPOKE8#show ip nhrp brief
Target Via NBMA Mode Intfc Claimed
172.30.1.1/32 172.30.1.1 99.255.10.2 static Tu0 < >
172.30.1.2/32 172.30.1.2 99.255.30.1 dynamic Tu0 < >
172.30.1.7/32 172.30.1.7 99.255.60.6 dynamic Tu0 < >
172.30.1.8/32 172.30.1.8 99.255.60.2 dynamic Tu0 < >
SPOKE8#
SPOKE8#show ip nhrp detail
172.30.1.1/32 via 172.30.1.1
Tunnel0 created 00:33:52, never expire
Type: static, Flags: used
NBMA address: 99.255.10.2
172.30.1.2/32 via 172.30.1.2
Tunnel0 created 00:18:18, expire 01:41:41
Type: dynamic, Flags: router used
NBMA address: 99.255.30.1
172.30.1.7/32 via 172.30.1.7
Tunnel0 created 00:33:34, expire 01:26:25
Type: dynamic, Flags: router implicit used
NBMA address: 99.255.60.6
172.30.1.8/32 via 172.30.1.8
Tunnel0 created 00:33:34, expire 01:26:25
Type: dynamic, Flags: router unique local
NBMA address: 99.255.60.2
(no-socket)
Requester: 172.30.1.7 Request ID: 7
SPOKE8#
The local NHRP cache contains:
- Network entry for hosts (/32 or 128) or subnets (/24, /25, /16) and tunnel to NBMA address.
- Interface number, duration of tunnel existence. Included is the amount of time to expiry of the tunnel for dynamic entries. Static entries do not expire.
- NHRP mapping types;
- Static: an entry created statically, on tunnel interface.
- Dynamic: an entry created from a spoke that register.
- Incomplete: a temporary entry placed locally while an NHRP resolution request is processing.
- Local: a typical entry is of a local network that was advertised for an NHRP resolution reply.
- (no socket): These mapping entries do not have an associated IPSec socket and encryption isn’t triggered yet.
- NBMA address: transport IP address where the entry was received.
NHRP message flags specify attributes of an NHRP cache entry or of the peer for which the entry was created.
- Used: this NHRP mapping was used to forward data packets within last 60 seconds.
- Implicit: Mapping entry was learned implicitly for example source mapping information gleaned from an NHRP resolution request was received by local router or forwarded through local router.
- Unique: This mapping entry must be unique and can’t be overwritten with a mapping entry that has the same tunnel IP address but different NBMA address.
- Router: mapping entry is from a remote router that provides access to a network or host behind the remote router.
- Rib: mapping entry has a corresponding routing entry in the routing table [H].
- Nho: mapping entry has a corresponding path overriding the next hop for a remote network as installed by another routing protocol.
- Nhop: Mapping entry for remote next hop address for remote tunnel interface and its associated NBMA address.
show ip nhrp traffic
The show ip nhrp traffic command is used to view the
statistics on NHRP packets sent and received.
HUB#show ip nhrp traffic
Tunnel0: Max-send limit:100Pkts/10Sec, Usage:0%
Sent: Total 16
8 Resolution Request 0 Resolution Reply 0 Registration Request
4 Registration Reply 0 Purge Request 0 Purge Reply
0 Error Indication 4 Traffic Indication 0 Redirect Suppress
Rcvd: Total 12
8 Resolution Request 0 Resolution Reply 4 Registration Request
0 Registration Reply 0 Purge Request 0 Purge Reply
0 Error Indication 0 Traffic Indication 0 Redirect Suppress
Overlay Routes
show ip route
Overlay routes have the tunnel interface as the exit interface.
Optimization of DMVPN Operation
Unique IP NHRP Registration
During registration, NHCs instruct NHS to keep their NBMA address unique. If a
loss of connectivity occurs between the NHC and NHS and a new NBMA address is
given to the NHC, say through DHCP, before the NHRP NHC cache timeout, the new
registration is blocked. This can be suppressed by using the command;
ip nhrp registration no-unique on the NHC tunnel interface.
Verify using the command show ip nhrp 10.100.100.8.
Recursive Routing Problems
Recursive routing happens when the transport/NBMA network is advertised into the same routing protocol that runs on the overlay network. One of the main preconditions for recursive routing is the routing protocol in the overlay network having a lower Administrative Distance (AD) than the routing protocol being used in the underlay network. An example of this can be where EIGRP is running on the overlay network and OSPF in the underlay network.
The two routers that have a DMVPN tunnel formed will share their routes using the EIGRP over the tunnel. EIGRP having an AD (90) lower than OSPF (110) will have its underlay route selected as the preferred route over the OSPF route by the remote router. This remote router will attempt to route underlay traffic through the tunnel. This will obviously fail. As it keeps attempting to route underlay traffic through the tunnel, the OSPF neighborship in the underlay network times out and gets terminated. As the router loses the tunnel, it creates a new OSPF neighborship with its underlay neighbor resulting in the reformation of the tunnel. Then EIGRP sends the same update of the underlay network across the tunnel. This process repeats continuously.
Solution
- Removing the transport network from the overlay routing protocol resolves recursive routing.
- Front-door VRF
Front-Door Virtual Route Forwarding (FVRF)
A DMVPN tunnel source or destination can be made to be separate from the DMVPN tunnel itself. The VRF associated with the transport network is known as front-foor VRF. Using a front-door VRF for every DMVPN tunnel prevents route recursion because the transport and overlay networks remain in separate routing tables. The transport network and interface can be associated with the transport VRF and the tunnel network and interface associated with the tunnel VRF.
Configuring FVRF
The VRF can be created using the older global configuration command
ip vrf <vrf_name> or the newer global configuration command
vrf definition <vrf_name> which supports IPv6 in
addition to support for IPv4 through address families.
Front-door VRF can be configured using the following sequence of steps:
-
Create the front-door VRF;
ip vrf <vrf_name>. If configuring IPv6 FVRFs, use the commandvrf definition <vrf_name>as it is the only command of the two that supports IPv6.SPOKE1(config)#vrf definition FVRF
SPOKE1(config-vrf)#address-family ipv4If the VRF is created using the command
vrf definition <vrf_name>, the appropriate address-family needs to be activated. In the above example, IPv4 has been activated. -
Associate the front-door FVRF to the interface;
vrf forwarding <vrf_name>.SPOKE1(config-if)#interface g0/0
SPOKE1(config-if)#vrf forwarding FVRF
SPOKE1(config-if)#ip address 99.255.20.2 255.255.255.252
-
Make the DMVPN VRF-aware:
tunnel vrf <fvrf_name>.SPOKE1(config)#interface tunnel 0
SPOKE1(config-if)#ip address 172.30.1.100 255.255.255.0
SPOKE1(config-if)#tunnel mode gre multipoint
SPOKE1(config-if)#tunnel source g0/0
SPOKE1(config-if)#tunnel vrf FVRF
Note: FVRF interfaces with static IP addressing require only a static default route in the FVRF context. Interfaces that receive addresses through DHCP do not require the configuration of a static route.
SPOKE1(config)#ip route vrf FVRF 0.0.0.0 0.0.0.0 99.255.20.1
Verification of FVRF
FVRF configuration and operation can be verified using the command
show dmvpn detail.
SPOKE1#show dmvpn detail
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface Tunnel0 is up/up, Addr. is 172.30.1.100, VRF ""
Tunnel Src./Dest. addr: 99.255.20.2/MGRE, Tunnel VRF "FVRF"
Protocol/Transport: "multi-GRE/IP", Protect ""
Interface State Control: Disabled
nhrp event-publisher : Disabled
IPv4 NHS:
172.30.1.1 RE priority = 0 cluster = 0
Type:Spoke, Total NBMA Peers (v4/v6): 1
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb Target Network
----- --------------- --------------- ----- -------- ----- -----------------
1 99.255.10.2 172.30.1.1 UP 00:25:24 S 172.30.1.1/32
!output omitted
DMVPN Failure Detection and High Availability
Hold time
By default, an NHRP mapping stays in the NHRP cache for 7200 seconds (2 hours)
(holdtime). It can be modified to the recommended value of 600 seconds using the
tunnel configuration command ip nhrp holdtime <1 – 65535>.
Multi-hub Topologies
Additional hubs can be added by additional mapping statements to the tunnel interface. A muti-hub topology increases redundancy and provides high availability.
NHRP Timeout
NHRP timeout period is one third of the holdtime period which is 2400 seconds.
This can be changed with the command ip nhrp registration timeout 240.
IPv6 DMVPN Configuration
IPv6 commands are similar to IPv4 commands. To create mGRE tunnels;
SPOKE8(config-if)#tunnel mode gre multipoint ipv6
A unique IPv6 link-local address should be assigned to the tunnel inteface when the tunnelling protocol is IPv6. IPv6 routing protocols use link-local addresses to discover each other and install in the RIB.
IPv6 DMVPN Commands
show ipv6 nhrp (brief|detail)show dmvpn [ipv6] [detail]show ipv6 nhrp trafficshow ipv6 nhrp nhs detail
Securing the DMVPN Tunnel with IPSec
Securing DMVPN tunnels is explained here.
