Pages

Wednesday 10 February 2021

CONFIGURING MPLS LAYER 3 VPNs In CISCO IOS

  1. Introduction
  2. L3VPN Components
  3. Configuration
  4. Troubleshooting

Introduction and Overview

MPLS Layer 3 VPNs (L3VPNs) are used by service providers (SPs) to provide layer 3 connectivity between an enterprise and its different remote offices/locations through the service provider's network. MPLS L3VPNs are defined by RFC 4364.

Service providers are able to provide different customers with layer 3 connectivity to their remote locations using the same underlaying infrastructure. This is possible regardless of whether the different customer sites use the same overlapping RFC 1918 IP addresses or not. Traffic from different customers is differentiated using VPNv4 addresses and routed to their respective destinations using the SP's core network.

The following topology will be used to configure MPLS Layer 3 VPNs.

The routers R1, R5, and R6 in the topology above are referred to as Provider Edge (PE) routers or edge Label Switch Routers (LSR). The MPLS network is usually referred to as the P network and the customer network, the C network. All LSRs in the P network have MPLS configured. PE routers have VRFs for each customer. Customer Edge (CE) routers have no knowledge of MPLS or VRFs. CE routers connect to Provider Edge (PE) routers.

At the PE routers, VRFs are created for each customer as one PE router may connect to many CE routers. Each customer’s routes will be separated from other customers in their own routing table.

MPLS L3VPN Components

MPLS L3VPNs consist of prefix exchange between CE and PE and P routers in the service provider's core network.

Service Provider Core

In the service provider core of an MPLS L3VPN, the P routers run MPLS over an interior gateway protocol(IGP) such as IS-IS, OSPF; the result is that packets are switched using MPLS along routing paths created by the underlaying IGP. Intermediate LSRs in the provider core network do not have the customer prefixes in their routing tables. All P routers (intermediate and edge LSRs) run an IGP and are configured for MPLS. LDP is usually configured as the label exchange protocol. Other labeling methods are supported such as MP-BGP, RSVP-TE, etc.

PE Routers

For purposes of providing MPLS L3VPN services, only PE routers run BGP and they form an iBGP peering through the VPNv4 address family.

On the PE, a VRF is created for each customer. This ensures that each customer has their prefixes in a separate routing table. In the VRF configuration, a route distinguisher (RD) is configured. This is used by MP-BGP on the PE to differentiate the customer's prefixes (including overlapping IP addresses) when switching customer traffic through the provider core network. The PE interface connecting to each customer's CE is placed in its corresponding VRF. Additionally, the route-target (RT) is configured on the PE devices to determine into which VRF routing table a prefix should be placed.

Route Distinguisher (RD)

The Route Distinguisher (RD) is a 64-bit (8-byte) value usually of the format ASN:nn or IP address:nn where ASN is the Autonomous system number and nn is any random number. RD is used by the PE to distinguish and uniquely identify the different VRFs; to differentiate the multiple prefixes from different customers. This allows for overlapping IPv4 addresses between customers. It is locally significant and does not need to match on the remote PE.

The RD is very critical to the successful operation of MPLS L3VPNs particularly when clients are using RFC 1918 prefixes in their enterprise networks.

VPNv4 Addresses

The RD is used to create VPNv4 addresses. This is a 96-bit value that consists of the RD (64-bit) and the IP address/network address (32-bit). It is usually in the format RD:IP address. An example can be 65000:10000:10.10.20.0 where the value 65000:10000 is the RD and 10.10.20.0 is the prefix being shared. VPNv4 addresses make addresses from the different customers unique. This results in the ability of the SP to route traffic for different customers regardless of whether they use overlapping address space or not. VPNv4 NLRI main attributes include:

  • 8-byte route distinguisher(RD)
  • IPv4 prefix and length
  • Next hop
  • MPLS VPN label

MPLS Layer 3 VPN Label Stack

L3VPN needs at least 2 labels to forward traffic across an MPLS L3VPN. The first label attached is the VPN label and the second is the transport/Label Distribution Protocol (LDP) label. On an MPLS packet, the transport label is the top most label. It is typically derived from LDP. It is sometimes referred to as the IGP label. The LDP label(transport label) is used for label switching from PE to PE through the MPLS domain as intermediate LSRs run only MPLS and not BGP. The transport label basically tells the SP MPLS core LSRs which PE traffic is destined to i.e., which PE device is the exit point out of the L3VPN to the CE. The VPN Label is derived from VPNv4 advertisements of PE and is used by the PE router to determine which CE traffic is destined to.

The VPN label is used by the egress LSR/PE router to determine customer specifics about the packet and what to do with it. The VPN label tells the PE router which CE traffic is destined to. It is derived from VPNv4 advertisements of PEs. Other applications such as MPLS TE, FRR, etc., require even more labels. VPN labels are exchanged between PE routers over the MP-BGP peering. Labeled packets are advertised along with BGP extended communities path attributes. MPLS generates labels for VPNv4 routes and adds [V] after the prefix to indicate that the label is a VPNv4 label. In the L3VPN label stack, the VPN label is usually at the bottom of the stack (nearest to the Layer 3 header) as indicated by the 1-value is the S-section of the MPLS header.

Route Target (RT)

A route-target (RT) is a 64-bit value similar in format to the RD i.e. ASN:nn or IP address:nn. It is used by MP-BGP to determine the VRF into which the different customer prefixes should be installed. RT is defined by RFC 4360 (BGP extended communities attribute). The routes defined in the VRF of one customer are tagged for export to the remote PE customer VRF using the command route-target export ASN:nn. To import these routes into the appropriate VRF at the remote PE router, the command route-target import ASN:nn is used. The new BGP extended community attribute "route-target" is used to control what prefixes enter and exit the VRF table.

  • export route target: what routes will go from VRF into BGP.
  • import route target: what routes will go from BGP into VRF.
The RT allows for granular control over what sites have what routes.

As an alternative to the use of RT, the route-maps "import map" and "export map" allow control over import and export on a per prefix basis.

RT controls the route’s VPN VRF membership(s). It is possible to have multiple RTs per VPNv4 route but one RD per VPNv4 route. VPNv4 speakers only accept VPNv4 routes with an RT matching a local VRF. Some exceptions include route reflectors. VPNv4 routes can have more than one RT. This allows for complex VPN topologies such as full mesh, hub and spoke, central services. The RT value used for export in the VRF on the ingress PE router should be the same RT value used in the RT-import of the appropriate VRF on the remote egress PE.

PE-CE

The CE forms an IGP peer relationship with the PE. Through this peering, it exchanges its prefixes with the PE. The PE isolates these prefixes received from the CE into the VRF created for this customer using the configured RT import values. For ingress traffic, the PE redistributes these customer prefixes from the IGP VRF table into MP-BGP VRF for that customer as defined by the export RT values. This configuration is implemented using the corresponding BGP VRF address-family. For egress traffic, the prefixes are redistributed from MP-BGP VRF into the IGP.

If the PE and CE routers have an eBGP peering, the prefixes are exchanged using BGP and no IGP (and redistribution) is required. MPLS is not configured on any client facing interfaces. Network traffic at the CE-PE interface uses normal IPv4 or IPv6 routing and all IPv4 protocols supported. Some IPv6 protocols supported.

Configuration

To create L3VPNs, the following steps should be taken.

  • Step 1: Ensure MPLS infrastructure is in place. i.e. IP routing, MPLS.
  • Step 2: Create the VRFs on the PE/edge LSR and interfaces connecting to each customer device should be configured to be in the appropriate VRF.
  • Step 3: Configure iBGP peering between the PE routers using VPNv4 address-family.
  • Step 4: Enable IGP peering and route exchange between the PE and CE devices.
  • Step 5: Redistribute routes between MP-BGP VRF address-family and PE-CE routing protocol (this is not needed if PE-CE routing uses eBGP).
  • Step 6: Test connectivity.

Step 1: Ensure MPLS Infrastructure is in Place

When configuring MP-BGP on an LSR, ensure that BGP peer address and the MPLS LDP ID match for ease of configuration. Guidance on how to configure an operational MPLS infrastructure can be found here.

Step 2: Create VRFs on PE Routers

Virtual Routing and Forwarding (VRFs) are used to create separate routing tables for each customer on a router. These routing tables are separated from the global routing table (RIB). The RT and RD are then configured under each VRF. There are two ways to create a VRF: using the command ip vrf <vrf-name> or vrf definition <vrf_name>.

  1. ip vrf <vrf-name> (older method): creates VRFs for IPv4 address family only.

    R1(config)#ip vrf CUST_X_HQ
    R1(config-vrf)#rd 300:100
    R1(config-vrf)#route-target export 300:100
    R1(config-vrf)#route-target import 300:110
    R1(config-vrf)#ip vrf CUST_Z_2
    R1(config-vrf)#rd 200:2
    R1(config-vrf)#route-target export 200:110
    R1(config-vrf)#route-target import 200:100

    Two VRFs CUST_X_HQ and CUST_Z_2 have been created.

  2. vrf definition <vrf_name>: VRFs can be created for the IPv4 or IPv6 address families.

    R6(config)#vrf definition CUST_Z_HQ
    R6(config-vrf)#rd 200:1
    R6(config-vrf)#address-family ipv4
    R6(config-vrf-af)#route-target export 200:100
    R6(config-vrf-af)#route-target import 200:110

    The VRF created is CUST_Z_HQ. VRF names do not need to match as they are locally significant.

vrf upgrade-cli

The global configuration command vrf upgrade-cli is an important command to use particularly when introducing VRFs in IPv6 in a network where IPv4 VRFs are operational. It has been previously mentioned that the vrf definition <vrf-name> command supports creation IPv6 VRFs. If the VRFs were created as below:

R1(config)#ip vrf CUST_X_HQ
R1(config-vrf)#rd 300:100
R1(config-vrf)#route-target export 300:100
R1(config-vrf)#route-target import 300:110
R1(config-vrf)#ip vrf CUST_Z_2
R1(config-vrf)#rd 200:2
R1(config-vrf)#route-target export 200:110
R1(config-vrf)#route-target import 200:100

The command vrf upgrade-cli multi-af-mode common-policies will apply the same RT values configured for address-family IPv4 to IPv6 configuration automatically. This makes applying the same configuration of IPv4 VRFs to IPv6 much easier. When configuring RT or other settings in new format VRF, the address family configuration takes precedence over any other similar configuration that may have been configured under the VRF sub-mode.

Verification

To verify the creation of VRFs, use the command show vrf.

R1(config)#vrf upgrade-cli multi-af-mode common-policies
You are about to upgrade to the multi-AF VRF syntax commands.
You will lose any IPv6 address configured on interfaces
belonging to upgraded VRFs.

Are you sure ? [yes]: yes
Number of VRFs upgraded: 2
R1(config)#
R1(config)#do show vrf detail
VRF CUST_X_HQ (VRF Id = 1); default RD 300:100; default VPNID <not set>
  No interfaces
Address family ipv4 (Table ID = 1 (0x1)):
  Export VPN route-target communities
    RT:300:100
  Import VPN route-target communities
    RT:300:110
  No import route-map
  No export route-map
  VRF label distribution protocol: not configured
  VRF label allocation mode: per-prefix
Address family ipv6 not active
    
VRF CUST_Z_2 (VRF Id = 2); default RD 200:2; default VPNID <not set>
  Interfaces:
    Gi2/0
Address family ipv4 (Table ID = 2 (0x2)):
  Export VPN route-target communities
    RT:200:110
  Import VPN route-target communities
    RT:200:100
  No import route-map
  No export route-map
  VRF label distribution protocol: not configured
  VRF label allocation mode: per-prefix
Address family ipv6 not active

R1(config)#

Step 3: Configure iBGP Peering between the PE routers using VPNv4 address-family

Configure an iBGP peering between the PE routers. This iBGP peering should be formed using the VPNv4 address family. iBGP neighborships occur over a maximum of 255 hops. To configure an iBGP neigborship on R1;

R1(config)#router bgp 100
R1(config-router)#bgp log-neighbor-changes
R1(config-router)#no bgp default ipv4-unicast
R1(config-router)#neighbor 30.255.6.6 remote-as 100
R1(config-router)#neighbor 30.255.6.6 update-source Loopback255
R1(config-router)#address-family vpnv4 unicast
R1(config-router-af)#neighbor 30.255.6.6 activate
R1(config-router-af)#exit-address-family

!On R6

R6(config)#router bgp 100
R6(config-router)#bgp log-neighbor-changes
R6(config-router)#no bgp default ipv4-unicast
R6(config-router)#neighbor 30.255.1.1 remote-as 100
R6(config-router)#neighbor 30.255.1.1 update-source Loopback255
R6(config-router)#address-family vpnv4
R6(config-router-af)#neighbor 30.255.1.1 activate
R6(config-router-af)#exit-address-family

The command neighbor 30.255.1.1 send-community extended is automatically added by MP-BGP as it uses extended communities to route VPNv4 traffic. This can be verified by the command show run | section bgp.

Verification

Once the iBGP peering has been completed, this neighborship can be confirmed using the command show bgp vpnv4 unicast all summary. The result of this command is indicated in the following snippet.

R1#show bgp vpnv4 unicast all summary
BGP router identifier 30.255.1.1, local AS number 100
BGP table version is 56, main routing table version 56
39 network entries using 6552 bytes of memory
39 path entries using 2496 bytes of memory
6/6 BGP path/bestpath attribute entries using 864 bytes of memory
5 BGP extended community entries using 1040 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 10952 total bytes of memory
BGP activity 47/8 prefixes, 47/8 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
30.255.5.5      4          100      87     105       56    0    0 01:28:23        5
30.255.6.6      4          100      67     103       56    0    0 01:28:04       12

This shows that the iBGP peering was created. A second iBGP peer(R5) was configured.

To confirm if extended BGP communities are being sent: debug bgp vpnv4 unicast updates

*Apr 14 05:57:04.319: BGP(4): 30.255.6.6 rcvd UPDATE w/ attr: nexthop 30.255.6.6, origin ?, localpref 100, metric 2, extended community RT:200:100 OSPF DOMAIN ID:0x0005:0x000003E80200 OSPF RT:0.0.0.1:2:0 OSPF ROUTER ID:10.255.67.2:0
*Apr 14 05:57:04.331: BGP(4): 30.255.6.6 rcvd 200:1:10.7.7.1/32, label 44
*Apr 14 05:57:04.351: BGP(4): Revise route installing 1 of 1 routes for 10.7.7.1/32 -> 30.255.6.6(CUST_Z_2) to CUST_Z_2 IP table

Step 4: IGP Peering and Route Exchange Between the PE And CE Devices

This can be implemented through configuration of an IGP between the PE-CE routers or BGP with an eBGP relationship. Some customers may prefer running BGP at the PE-CE. This routing protocol is to share the customer routes from the CE to the PE.

When configuring the routing protocol, the physical interface that the PE connects to the CE should be placed in the appropriate VRF that was configured for the customer to which that interface points. Between CE R1 and PE R2, the configured routing protocol is EIGRP with named EIGRP being configured on R2.

R1(config)#interface g3/0
R1(config-if)#vrf forwarding CUST_Z_2
% Interface GigabitEthernet3/0 IPv4 disabled and address(es) removed due to enabling VRF CUST_Z_2
R1(config-if)#ip address 10.255.19.1 255.255.255.252
R1(config)#router eigrp EIGRP_NAMED
R1(config-router)#address-family ipv4 unicast vrf CUST_Z_2 autonomous-system 10
R1(config-router-af)#network 10.255.19.0 0.0.0.3
R1(config-router-af)#topology base
R1(config-router-af)#redistribute bgp 100 metric 100000 100 200 1 1500

On R9:

R9(config-if)#router eigrp EIGRP_Z_S2
R9(config-router)#address-family ipv4 unicast autonomous-system 10
R9(config-router-af)#network 10.255.19.0 0.0.0.3
R9(config-router-af)#network 10.90.0.0 0.0.255.255

Note that VRFs are only configured on the PE routers. On CE routers, VRFs are not required for MPLS L3VPN functionality. If they are configured, then it could be for another purpose unrelated to MPLS L3VPN.

The IGPs on the running on the PE-CE interfaces of the different PE-CE pairings does not have to be the same. To illustrate this, the PE-CE IGP on R7 is OSPF.

Step 5: Redistribute Routes Between MP-BGP VRF Address-family and PE-CE Routing Protocol

Route redistribution between the IGP and MP-BGP occurs at the PE router. Customer routes learned from the remote PE router are stored in the MP-BGP VPNv4 table. These can then be redistributed into the IGP running on the PE-CE link. Also, routes learned from the connected customer site through the IGP can be redistributed into MP-BGP to be forwarded to the remote customer site. This is accomplished using the redistribution command in the appropriate address-family.

On R2:

R1(config)#router bgp 100
R1(config-router)#no bgp default ipv4-unicast
!REDISTRIBUTE CUST_X_HQ ROUTES INTO MP-BGP (LOCAL CUSTOMER SITE TO REMOTE SITES)
R1(config-router)#address-family ipv4 vrf CUST_X_HQ
R1(config-router-af)#redistribute eigrp 10
R1(config-router-af)#exit-address-family
R1(config-router)#exit
!REDISTRIBUTE MP-BGP ROUTES INTO EIGRP (REMOTE CUSTOMER SITES TO LOCAL SITE)
R1(config)#router eigrp EIGRP_Z_S2
R1(config-router)#address-family ipv4 vrf CUST_X_HQ autonomous-system 10
R1(config-router-af)#topology base
R1(config-router-af-topology)#redistribute bgp 100 100000 100 250 10 1500

On R5

R5(config)#router bgp 100
R5(config-router)#address-family ipv4 vrf CUST_X_2
R5(config-router-af)#redistribute eigrp 10
R5(config-router-af)#exit-address-family
R5(config-router)#exit
R5(config)#router eigrp EIGRP_Z_S2
R1(config-router)#address-family ipv4 vrf CUST_X_2 autonomous-system 10
R1(config-router-af)#topology base
R1(config-router-af-topology)#redistribute bgp 100 100000 100 250 10 1500
R5(config-router-af)#exit-address-family

Note that with eBGP, no redistribution of PE-CE routes is required. These routes will simply be shared through the eBGP relationship from MP-BGP.

Verification

To verify that the appropriate routes have been redistributed;

On the PE router:

R1#show ip route vrf CUST_Z_2

Routing Table: CUST_Z_2                                                    
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP  
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area      
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2      
       E1 - OSPF external type 1, E2 - OSPF external type 2                
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP    
       + - replicated route, % - next hop override                          
                                                                            
Gateway of last resort is not set                                          
                                                                            
      10.0.0.0/8 is variably subnetted, 18 subnets, 3 masks                
D        10.9.1.0/24                                                        
           [90/2570240] via 10.255.19.2, 01:04:15, GigabitEthernet3/0      
D        10.9.2.0/24                                                        
           [90/2570240] via 10.255.19.2, 01:04:15, GigabitEthernet3/0      
D        10.9.3.0/24                                                        
           [90/2570240] via 10.255.19.2, 01:04:15, GigabitEthernet3/0      
D        10.9.4.0/24                                                        
           [90/2570240] via 10.255.19.2, 01:04:15, GigabitEthernet3/0      
B        10.10.1.0/24 [200/2] via 30.255.6.6, 00:30:17                      
B        10.10.2.0/24 [200/2] via 30.255.6.6, 00:30:17                      
B        10.10.3.0/24 [200/2] via 30.255.6.6, 00:30:17                      
B        10.10.4.0/24 [200/2] via 30.255.6.6, 00:30:17                      
B        10.10.5.0/24 [200/2] via 30.255.6.6, 00:30:17                      
B        10.10.6.0/24 [200/2] via 30.255.6.6, 00:30:17                      
B        10.10.7.0/24 [200/2] via 30.255.6.6, 00:30:17                      
B        10.10.8.0/24 [200/2] via 30.255.6.6, 00:30:17                      
B        10.10.9.0/24 [200/2] via 30.255.6.6, 00:30:17                      
B        10.10.10.0/24 [200/2] via 30.255.6.6, 00:30:17                    
B        10.10.255.1/32 [200/2] via 30.255.6.6, 00:30:17                    
C        10.255.19.0/30 is directly connected, GigabitEthernet3/0          
L        10.255.19.1/32 is directly connected, GigabitEthernet3/0          
B        10.255.67.0/30 [200/0] via 30.255.6.6, 00:39:18                    
R1#
R1#show bgp vpnv4 unicast vrf CUST_Z_2
BGP table version is 38, local router ID is 30.255.1.1          
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete                  
RPKI validation codes: V valid, I invalid, N Not found          
                                                                
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 200:2 (default for vrf CUST_Z_2)            
*>i 10.7.1.0/24      30.255.6.6               2    100      0 ?
*>i 10.7.2.0/24      30.255.6.6               2    100      0 ?
*>i 10.7.3.0/24      30.255.6.6               2    100      0 ?
*>i 10.7.4.0/24      30.255.6.6               2    100      0 ?
*>i 10.7.5.0/24      30.255.6.6               2    100      0 ?
*>i 10.7.7.1/32      30.255.6.6               2    100      0 ?
*>  10.9.1.0/24      10.255.19.2        2570240         32768 ?
*>  10.9.2.0/24      10.255.19.2        2570240         32768 ?
*>  10.9.3.0/24      10.255.19.2        2570240         32768 ?
*>  10.9.4.0/24      10.255.19.2        2570240         32768 ?
*>i 10.10.255.1/32   30.255.6.6               2    100      0 ?
*>  10.255.19.0/30   0.0.0.0                  0         32768 ?
*>i 10.255.67.0/30   30.255.6.6               0    100      0 ?
R1#

On the CE router:

R9>show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override
                                                
Gateway of last resort is not set              
                                                
      10.0.0.0/8 is variably subnetted, 18 subnets, 3 masks
D EX     10.7.1.0/24 [170/51456] via 10.255.19.1, 00:26:27, GigabitEthernet0/0
D EX     10.7.2.0/24 [170/51456] via 10.255.19.1, 00:26:27, GigabitEthernet0/0
D EX     10.7.3.0/24 [170/51456] via 10.255.19.1, 00:26:27, GigabitEthernet0/0
D EX     10.7.4.0/24 [170/51456] via 10.255.19.1, 00:26:27, GigabitEthernet0/0
D EX     10.7.5.0/24 [170/51456] via 10.255.19.1, 00:26:27, GigabitEthernet0/0
D EX     10.7.7.1/32 [170/51456] via 10.255.19.1, 00:07:41, GigabitEthernet0/0
C        10.9.1.0/24 is directly connected, Loopback1
L        10.9.1.1/32 is directly connected, Loopback1
C        10.9.2.0/24 is directly connected, Loopback2
L        10.9.2.1/32 is directly connected, Loopback2
C        10.9.3.0/24 is directly connected, Loopback3
L        10.9.3.1/32 is directly connected, Loopback3
C        10.9.4.0/24 is directly connected, Loopback4
L        10.9.4.1/32 is directly connected, Loopback4
D EX     10.10.255.1/32                        
           [170/51456] via 10.255.19.1, 00:26:27, GigabitEthernet0/0
C        10.255.19.0/30 is directly connected, GigabitEthernet0/0
L        10.255.19.2/32 is directly connected, GigabitEthernet0/0
D EX     10.255.67.0/30                        
           [170/51456] via 10.255.19.1, 00:26:27, GigabitEthernet0/0
      99.0.0.0/32 is subnetted, 1 subnets      
C        99.255.9.9 is directly connected, Loopback255
R9>
R9>ping 10.7.7.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.7.7.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 84/100/128 ms
R9>

show bgp vpnv4 unicast all

Remote customer site routes are exchanged through L3VPN and are installed in the MP-BGP VRF table. This can be verified by the command; show bgp vpnv4 unicast all. This shows the routes in the MP-BGP table for all VRFs.

R1#show bgp vpnv4 unicast all                              
BGP table version is 40, local router ID is 30.255.1.1            
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete                    
RPKI validation codes: V valid, I invalid, N Not found            
                                                                  
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 200:1                                        
 *>i 10.10.1.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.2.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.3.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.4.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.5.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.6.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.7.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.8.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.9.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.10.0/24    30.255.6.6               2    100      0 ?  
 *>i 10.10.255.1/32   30.255.6.6               2    100      0 ?  
 *>i 10.255.67.0/30   30.255.6.6               0    100      0 ?  
Route Distinguisher: 200:2 (default for vrf CUST_Z_2)              
     Network          Next Hop            Metric LocPrf Weight Path
 *>  10.9.1.0/24      10.255.19.2        2570240         32768 ?  
 *>  10.9.2.0/24      10.255.19.2        2570240         32768 ?  
 *>  10.9.3.0/24      10.255.19.2        2570240         32768 ?  
 *>  10.9.4.0/24      10.255.19.2        2570240         32768 ?  
 *>i 10.10.1.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.2.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.3.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.4.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.5.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.6.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.7.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.8.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.9.0/24     30.255.6.6               2    100      0 ?  
 *>i 10.10.10.0/24    30.255.6.6               2    100      0 ?  
 *>i 10.10.255.1/32   30.255.6.6               2    100      0 ?  
 *>  10.255.19.0/30   0.0.0.0                  0         32768 ?  
 *>i 10.255.67.0/30   30.255.6.6               0    100      0 ?  
Route Distinguisher: 300:100 (default for vrf CUST_X_HQ)          
 *>i 10.80.1.0/24     30.255.5.5           10880    100      0 ?  
 *>i 10.80.2.0/24     30.255.5.5           10880    100      0 ?  
 *>i 10.80.3.0/24     30.255.5.5           10880    100      0 ?  
 *>i 10.80.4.0/24     30.255.5.5           10880    100      0 ?  
     Network          Next Hop            Metric LocPrf Weight Path
 *>i 10.255.58.0/30   30.255.5.5               0    100      0 ?  
Route Distinguisher: 300:110                                      
 *>i 10.80.1.0/24     30.255.5.5           10880    100      0 ?  
 *>i 10.80.2.0/24     30.255.5.5           10880    100      0 ?  
 *>i 10.80.3.0/24     30.255.5.5           10880    100      0 ?  
 *>i 10.80.4.0/24     30.255.5.5           10880    100      0 ?  
 *>i 10.255.58.0/30   30.255.5.5               0    100      0 ?  
R1#

Note that the routes learned through MP-BGP VPNv4, have the next-hop as the iBGP remote PE peer.

R1#show bgp vpnv4 unicast all 10.80.1.0
BGP routing table entry for 300:100:10.80.1.0/24, version 15
Paths: (1 available, best #1, table CUST_X_HQ)
  Not advertised to any peer
  Refresh Epoch 1
  Local, imported path from 300:110:10.80.1.0/24
    30.255.5.5 (metric 3) from 30.255.5.5 (30.255.5.5)
      Origin incomplete, metric 10880, localpref 100, valid, internal, best
      Extended Community: RT:300:110 Cost:pre-bestpath:128:10880
        0x8800:32768:0 0x8801:10:256 0x8802:65281:2560 0x8803:65281:1500
        0x8806:0:1493108744
      mpls labels in/out nolabel/5018
BGP routing table entry for 300:110:10.80.1.0/24, version 11
Paths: (1 available, best #1, no table)
  Not advertised to any peer
  Refresh Epoch 1
  Local      
    30.255.5.5 (metric 3) from 30.255.5.5 (30.255.5.5)
      Origin incomplete, metric 10880, localpref 100, valid, internal, best
      Extended Community: RT:300:110 Cost:pre-bestpath:128:10880
        0x8800:32768:0 0x8801:10:256 0x8802:65281:2560 0x8803:65281:1500
        0x8806:0:1493108744
      mpls labels in/out nolabel/5018
R1#          

From this output, the details of EIGRP composite metrics are encoded in the BGP extended community.

EIGRP

When redistributing prefixes into EIGRP, ensure that the seed metric is configured.

Verification: show ip eigrp vrf <vrf-name> topology. Some prefixes will have VPNv4 as the route source.

R1>show ip eigrp vrf CUST_Z_2 topology
EIGRP-IPv4 VR(EIGRP_PE) Topology Table for AS(10)/ID(10.255.19.1)
           Topology(base) TID(0) VRF(CUST_Z_2)
Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
       r - reply Status, s - sia Status
                  
P 10.10.8.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.9.1.0/24, 1 successors, FD is 328990720
        via 10.255.19.2 (328990720/327761920), GigabitEthernet3/0
P 10.10.5.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.9.2.0/24, 1 successors, FD is 328990720
        via 10.255.19.2 (328990720/327761920), GigabitEthernet3/0
P 10.255.19.0/30, 1 successors, FD is 1310720
        via Connected, GigabitEthernet3/0
P 10.9.3.0/24, 1 successors, FD is 328990720
        via 10.255.19.2 (328990720/327761920), GigabitEthernet3/0
P 10.255.67.0/30, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.9.4.0/24, 1 successors, FD is 328990720
        via 10.255.19.2 (328990720/327761920), GigabitEthernet3/0
P 10.10.4.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.10.9.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.10.6.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.10.7.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.10.255.1/32, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.10.2.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.10.10.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.10.1.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
P 10.10.3.0/24, 1 successors, FD is 72089600
        via Redistributed (72089600/0)
                  
R1>                
R1>show ip eigrp vrf CUST_Z_2 topology 10.10.1.0/24
EIGRP-IPv4 VR(EIGRP_PE) Topology Entry for AS(10)/ID(10.255.19.1)
           Topology(base) TID(0) VRF(CUST_Z_2)
EIGRP-IPv4(10): Topology base(0) entry for 10.10.1.0/24
  State is Passive, Query origin flag is 1, 1 Successor(s), FD is 72089600
  Descriptor Blocks:
  30.255.6.6, from Redistributed, Send flag is 0x0
      Composite metric is (72089600/0), route is External
      Vector metric:
        Minimum bandwidth is 100000 Kbit
        Total delay is 1000000000 picoseconds
        Reliability is 200/255
        Load is 1/255
        Minimum MTU is 1500
        Hop count is 0
        Originating router is 10.255.19.1
      External data:
        AS number of route is 100
        External protocol is BGP, external metric is 2
        Administrator tag is 0 (0x00000000)
R1>

OSPF

Enabling OSPF on an interface that is already part of a VRF makes the OSPF process VRF-aware.

External Routes as Inter-area Routes

The OSPF behaviour when redistributing BGP routes into OSPF is for the routes to be external routes E1 or E2.

The following output shows the OSPF database of a remote PE router:

R5#show ip ospf 1001 database
                                                        
            OSPF Router with ID (10.255.58.1) (Process ID 1001)
                                                        
                Router Link States (Area 1)              
                                                        
Link ID         ADV Router      Age         Seq#       Checksum Link count
10.255.58.1     10.255.58.1     1196        0x80000006 0x00BD57 1
88.255.8.8      88.255.8.8      38          0x80000006 0x00E82C 5
                                                        
                Net Link States (Area 1)                
                                                        
Link ID         ADV Router      Age         Seq#       Checksum
10.255.58.2     88.255.8.8      38          0x80000004 0x00617C
                                                        
                Type-5 AS External Link States          
                                                        
Link ID         ADV Router      Age         Seq#       Checksum Tag
10.10.1.1       10.255.58.1     28          0x80000001 0x00EEB2 3489661028
10.10.2.1       10.255.58.1     28          0x80000001 0x00E3BC 3489661028
10.10.3.1       10.255.58.1     28          0x80000001 0x00D8C6 3489661028
10.10.4.1       10.255.58.1     28          0x80000001 0x00CDD0 3489661028
10.10.5.1       10.255.58.1     28          0x80000001 0x00C2DA 3489661028
10.255.110.0    10.255.58.1     28          0x80000001 0x00A1A1 3489661028

On the CE router, these prefixes are E2 routes:

R8>show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override
                      
Gateway of last resort is not set
                      
      10.0.0.0/8 is variably subnetted, 16 subnets, 3 masks
O E2     10.10.1.1/32 [110/2] via 10.255.58.1, 00:02:39, GigabitEthernet0/0
O E2     10.10.2.1/32 [110/2] via 10.255.58.1, 00:02:39, GigabitEthernet0/0
O E2     10.10.3.1/32 [110/2] via 10.255.58.1, 00:02:39, GigabitEthernet0/0
O E2     10.10.4.1/32 [110/2] via 10.255.58.1, 00:02:39, GigabitEthernet0/0
O E2     10.10.5.1/32 [110/2] via 10.255.58.1, 00:02:39, GigabitEthernet0/0
.....
!!!!Additional output omitted
...

With VPNv4, OSPF on the PE LSRs can be configured to consider these routes as internal(inter-area). OSPF and BGP VPNv4 operate differently from normal redistribution. These routes are considered internal routes for the purpose of loop prevention and traffic engineering. Note that when redistributing prefixes into OSPF, excluding the subnets keyword results in classful prefixes being advertised.

The output of show ip ospf <process-ID> on the PE router displays the output "Domain ID type.. value ...". The value is inherited from the OSPF process ID.

R1#show ip ospf 1000
 Routing Process "ospf 1000" with ID 10.255.1.1
   Domain ID type 0x0005, value 0.0.3.232
 Start time: 00:00:36.340, Time elapsed: 01:48:49.108
 Supports only single TOS(TOS0) routes
 Supports opaque LSA
 Supports Link-local Signaling (LLS)
 Supports area transit capability
 Supports NSSA (compatible with RFC 3101)
 Connected to MPLS VPN Superbackbone, VRF CUST_X_HQ
 Event-log disabled
 It is an area border and autonomous system boundary router
 Redistributing External Routes from,
    bgp 100, includes subnets in redistribution
 Router is not originating router-LSAs with maximum metric
 Initial SPF schedule delay 5000 msecs
 Minimum hold time between two consecutive SPFs 10000 msecs
 Maximum wait time between two consecutive SPFs 10000 msecs
 Incremental-SPF disabled
 Minimum LSA interval 5 secs
 Minimum LSA arrival 1000 msecs
 LSA group pacing timer 240 secs
 Interface flood pacing timer 33 msecs
 Retransmission pacing timer 66 msecs
 Number of external LSA 0. Checksum Sum 0x000000
 Number of opaque AS LSA 0. Checksum Sum 0x000000
 Number of DCbitless external and opaque AS LSA 0
 Number of DoNotAge external and opaque AS LSA 0
 Number of areas in this router is 1. 1 normal 0 stub 0 nssa
 Number of areas transit capable is 0
 External flood list length 0
 IETF NSF helper support enabled
 Cisco NSF helper support enabled
 Reference bandwidth unit is 100 mbps
    Area 1
        Number of interfaces in this area is 1
        Area has no authentication
        SPF algorithm last executed 01:11:49.224 ago
        SPF algorithm executed 2 times
        Area ranges are
        Number of LSA 9. Checksum Sum 0x02ADF2
        Number of opaque link LSA 0. Checksum Sum 0x000000
        Number of DCbitless LSA 0
        Number of indication LSA 0
        Number of DoNotAge LSA 0
        Flood list length 0

On the remote PE R5:

R5(config-if)#do show ip ospf 1001
 Routing Process "ospf 1001" with ID 10.255.58.1
   Domain ID type 0x0005, value 0.0.3.233
 Start time: 00:15:57.372, Time elapsed: 00:13:48.868
 Supports only single TOS(TOS0) routes
 Supports opaque LSA  
 Supports Link-local Signaling (LLS)
 Supports area transit capability
 Supports NSSA (compatible with RFC 3101)
 Connected to MPLS VPN Superbackbone, VRF CUST_X_2
 Event-log disabled  
 It is an area border and autonomous system boundary router
 Redistributing External Routes from,
    bgp 100, includes subnets in redistribution
 Router is not originating router-LSAs with maximum metric
 Initial SPF schedule delay 5000 msecs
 Minimum hold time between two consecutive SPFs 10000 msecs
 Maximum wait time between two consecutive SPFs 10000 msecs
 Incremental-SPF disabled
 Minimum LSA interval 5 secs
 Minimum LSA arrival 1000 msecs
 LSA group pacing timer 240 secs
 Interface flood pacing timer 33 msecs
 Retransmission pacing timer 66 msecs
 Number of external LSA 6. Checksum Sum 0x04DD7F
 Number of opaque AS LSA 0. Checksum Sum 0x000000
 Number of DCbitless external and opaque AS LSA 0
 Number of DoNotAge external and opaque AS LSA 0
 Number of areas in this router is 1. 1 normal 0 stub 0 nssa
 Number of areas transit capable is 0
 External flood list length 0
 IETF NSF helper support enabled
 Cisco NSF helper support enabled
 Reference bandwidth unit is 100 mbps
    Area 1            
        Number of interfaces in this area is 1
        Area has no authentication
        SPF algorithm last executed 00:11:09.780 ago
        SPF algorithm executed 4 times
        Area ranges are
        Number of LSA 3. Checksum Sum 0x0218F6
        Number of opaque link LSA 0. Checksum Sum 0x000000
        Number of DCbitless LSA 0
        Number of indication LSA 0
        Number of DoNotAge LSA 0
        Flood list length 0

If the domain ID matches on the local and remote PE devices, prefixes that were originally external after redistribution will be considered as inter-area routes after redistribution from BGP to OSPF e.g., OSPF with process ID 1000 on both the local and remote PE devices; because the OSPF process ID is matching, then the routes can be encoded as inter-area routes.

The domain-ID can be configured to be the same on the local and remote PE by:

  • Configuring the same OSPF process ID on the local and remote PE.
  • A second option is to configure the same domain-id value on the local and remote PE.

The domain-ID value can be modified in OSPF using the OSPF command domain-id <ip-address>; the domain-ID value is in the form of an IPv4 address. If the domain-ID value is different in the OSPF process of the local and remote PE LSRs, then the routes redistributed from BGP to OSPF will be flagged as E2 external routes. The domain-ID is encoded(in HEX) in the BGP path attributes as an extended community value. The domain-ID value takes precedence over the OSPF process ID value; i.e., if the process IDs are different but the domain-IDs are the same, then the routes will be advertised as inter-area routes.

R5(config)#router ospf 1001 vrf CUST_X_2
R5(config-router)#domain-id 0.0.3.232
R5(config-router)#do show ip ospf 1001
 Routing Process "ospf 1001" with ID 10.255.58.1
   Domain ID type 0x0005, value 0.0.3.232
 Start time: 00:15:57.372, Time elapsed: 02:33:44.228
 Supports only single TOS(TOS0) routes
 Supports opaque LSA
 Supports Link-local Signaling (LLS)
 Supports area transit capability
 Supports NSSA (compatible with RFC 3101)
 Connected to MPLS VPN Superbackbone, VRF CUST_X_2
 Event-log disabled
 It is an area border and autonomous system boundary router
 Redistributing External Routes from,
    bgp 100, includes subnets in redistribution
 Router is not originating router-LSAs with maximum metric
 Initial SPF schedule delay 5000 msecs
 Minimum hold time between two consecutive SPFs 10000 msecs
 Maximum wait time between two consecutive SPFs 10000 msecs
 Incremental-SPF disabled
 Minimum LSA interval 5 secs
 Minimum LSA arrival 1000 msecs
 LSA group pacing timer 240 secs
 Interface flood pacing timer 33 msecs
 Retransmission pacing timer 66 msecs
 Number of external LSA 0. Checksum Sum 0x000000
 Number of opaque AS LSA 0. Checksum Sum 0x000000
 Number of DCbitless external and opaque AS LSA 0
 Number of DoNotAge external and opaque AS LSA 0
 Number of areas in this router is 1. 1 normal 0 stub 0 nssa
 Number of areas transit capable is 0
 External flood list length 0
 IETF NSF helper support enabled
 Cisco NSF helper support enabled
 Reference bandwidth unit is 100 mbps
    Area 1    
        Number of interfaces in this area is 1
        Area has no authentication
        SPF algorithm last executed 02:31:05.160 ago
        SPF algorithm executed 4 times
        Area ranges are
        Number of LSA 9. Checksum Sum 0x059C39
        Number of opaque link LSA 0. Checksum Sum 0x000000
        Number of DCbitless LSA 0
        Number of indication LSA 0
        Number of DoNotAge LSA 0
        Flood list length 0
              
R5(config-router)#end
R5>show ip ospf 1001 database
                                                          
            OSPF Router with ID (10.255.58.1) (Process ID 1001)
                                                          
                Router Link States (Area 1)              
                                                          
Link ID         ADV Router      Age         Seq#       Checksum Link count
10.255.58.1     10.255.58.1     109         0x80000008 0x00B959 1
88.255.8.8      88.255.8.8      67          0x80000007 0x00E62D 5
                                                          
                Net Link States (Area 1)                  
                                                          
Link ID         ADV Router      Age         Seq#       Checksum
10.255.58.2     88.255.8.8      67          0x80000005 0x005F7D
                                                          
                Summary Net Link States (Area 1)          
                                                          
Link ID         ADV Router      Age         Seq#       Checksum
10.10.1.1       10.255.58.1     789         0x80000001 0x000E51
10.10.2.1       10.255.58.1     789         0x80000001 0x00035B
10.10.3.1       10.255.58.1     789         0x80000001 0x00F765
10.10.4.1       10.255.58.1     789         0x80000001 0x00EC6F
10.10.5.1       10.255.58.1     789         0x80000001 0x00E179
10.255.110.0    10.255.58.1     789         0x80000001 0x00C040

The OSPF database on R5 shows that the routes are now considered inter-area.

On the remote site client CE, the routes are now considered inter-area:

R8>show ip route ospf

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override
            
Gateway of last resort is not set
            
      10.0.0.0/8 is variably subnetted, 16 subnets, 3 masks
O IA     10.10.1.1/32 [110/3] via 10.255.58.1, 00:10:22, GigabitEthernet0/0
O IA     10.10.2.1/32 [110/3] via 10.255.58.1, 00:10:22, GigabitEthernet0/0
O IA     10.10.3.1/32 [110/3] via 10.255.58.1, 00:10:22, GigabitEthernet0/0
O IA     10.10.4.1/32 [110/3] via 10.255.58.1, 00:10:22, GigabitEthernet0/0
O IA     10.10.5.1/32 [110/3] via 10.255.58.1, 00:10:22, GigabitEthernet0/0
O IA     10.255.110.0/30 [110/2] via 10.255.58.1, 00:10:22, GigabitEthernet0/0

show ip cef displays both the transport label and VPN label.

Superbackbone

On the PE router of the provider, in the OSPF output of the command show ip ospf <process-ID>, the message, Connected to the MPLS VPN superbackbone.

R5(config-router)#do show ip ospf 1001
 Routing Process "ospf 1001" with ID 10.255.58.1
   Domain ID type 0x0005, value 0.0.3.232
 Start time: 00:15:57.372, Time elapsed: 02:33:44.228
 Supports only single TOS(TOS0) routes
 Supports opaque LSA
 Supports Link-local Signaling (LLS)
 Supports area transit capability
 Supports NSSA (compatible with RFC 3101)
 Connected to MPLS VPN Superbackbone, VRF CUST_X_2
 Event-log disabled
 It is an area border and autonomous system boundary router
 Redistributing External Routes from,
    bgp 100, includes subnets in redistribution
 Router is not originating router-LSAs with maximum metric
 Initial SPF schedule delay 5000 msecs
 Minimum hold time between two consecutive SPFs 10000 msecs
 Maximum wait time between two consecutive SPFs 10000 msecs
 Incremental-SPF disabled
 Minimum LSA interval 5 secs
 Minimum LSA arrival 1000 msecs
 LSA group pacing timer 240 secs
 Interface flood pacing timer 33 msecs
 Retransmission pacing timer 66 msecs
 Number of external LSA 0. Checksum Sum 0x000000
 Number of opaque AS LSA 0. Checksum Sum 0x000000
 Number of DCbitless external and opaque AS LSA 0
 Number of DoNotAge external and opaque AS LSA 0
 Number of areas in this router is 1. 1 normal 0 stub 0 nssa
 Number of areas transit capable is 0
 External flood list length 0
 IETF NSF helper support enabled
 Cisco NSF helper support enabled
 Reference bandwidth unit is 100 mbps
    Area 1    
        Number of interfaces in this area is 1
        Area has no authentication
        SPF algorithm last executed 02:31:05.160 ago
        SPF algorithm executed 4 times
        Area ranges are
        Number of LSA 9. Checksum Sum 0x059C39
        Number of opaque link LSA 0. Checksum Sum 0x000000
        Number of DCbitless LSA 0
        Number of indication LSA 0
        Number of DoNotAge LSA 0
        Flood list length 0
              
R5(config-router)#end

The superbackbone is a hierarchy above area 0. With a superbackbone hierarchy in the OSPF domain, permitted hierarchies include:

  • Super-backbone (MPLS)→ Area 0 → Area 1
  • Super-backbone (MPLS)→ Area 1 → Area 1
    • Not permitted is:
  • Super-backbone (MPLS)→ Area 1 → Area 0. Here, a virtual link would be required to be configured to connect the Area 0 to the superbackbone as it would be considered a discontiguous area.

Traffic Enginneering: Primary and Backup Paths

Load-balancing across multiple paths can be accomplished by modifying the OSPF process ID or OSPF domain number. This results in some paths becoming external routes. OSPF always chooses internal routes over external routes.

OSPF Sham-links

OSPF sham-links are similar to virtual links going over the SP network. It does not have to be in area 0.

To configure OSPF sham-links:

  1. Advertise a new prefix (/32) of a loopback into the VRF and advertise this into BGP: Remember, the netmask must be /32 for this feature to work. This is done on both PE devices.
  2. Verify IP connectivity between the two /32 prefixes.
  3. Under the OSPF process, configure the sham-link area 1 sham-link 8.8.8.8 3.3.3.3. where 8.8.8.8 is the source (/32 of local device) and 3.3.3.3 is remote OSPF device. Do the same but in the reverse direction on the remote device.
  4. Verify with the command show ip ospf <process-id> neighbor. It shows neighborship over the sham-link. Routes that were previously learned through BGP are now learned thru OSPF (because of lower AD); verify this on PE router using command: show ip route.

Practical use case is that if you had a point-to-point VPN and you are migrating to MPLS L3VPN from legacy frame relay service, the backdoor link will be preferred from an OSPF point of view because intra-area is preferred to inter-area preferred to external routes.

Traffic engineering can be implemented based on the cost.

Load-balancing

If the SP network employs the use of BGP route reflector, and will only advertise what it considers as the best path of the available paths and not use the available paths simultaneously. For RR groups these updates based on the RD and performs path selection between the two of them. To enable the RR to use both paths, we need to configure different RDs so that the RR can advertise both paths to the remote PE. The RR will receive these as two separate and independent best paths and they can be advertised to the remote neighbor. The command maximum-paths <value> needs to be configured to enable multipaths.

BGP as PE-CE

If prefixes are to be shared using BGP as the protocol between at the PE-CE interconnection, on the PE device, the peering should be implemented in the VRF address-family. At the CE devices, the ASN for BGP should be the same. However, because BGP will detect the same ASN, it will reject the advertised route to prevent loop formation. There are two fixes for this:

  • On the CE, configure the neighbor with the command allow-as-in
  • Alternatively, on the PE, as-override to the CE iBGP peer on both ends of the LSP.

Loops can be introduced by the above configuration options. One solution is the BGP site-of-origin. Under BGP SOO, prefixes from one site are tagged with SOO code (say 1000:1) as they move to the PE. This gets advertised as a BGP extended community. The PE devices will filter out any other prefixes that have that SOO code set. This is configured on the PE device for the CE eBGP peer. This is configured under the VRF address-family. neighbor 155.1.67.7 soo 1000:1. SOO can be verified with command show bgp vpnv4 unicast all <prefix>. Lookout for SoO.

Verification

On the RR to confirm sharing of prefixes with PE: show bgp vpnv4 unicast all neighbors <neighbor-ip-address> advertised-routes.

Step 6: Test Connectivity

A ping test or traceroute test can be used to confirm the operation of the MPLS L3VPN.

traceroute

R7>traceroute 10.9.1.1
Type escape sequence to abort.
Tracing the route to 10.9.1.1
VRF info: (vrf in name/id, vrf out name/id)
  1 10.255.67.2 20 msec 16 msec 20 msec
  2 30.255.46.1 [MPLS: Labels 4003/1000 Exp 0] 124 msec 92 msec 104 msec
  3 30.255.24.1 [MPLS: Labels 2006/1000 Exp 0] 144 msec 52 msec 76 msec
  4 10.255.19.1 [MPLS: Label 1000 Exp 0] 28 msec 76 msec 56 msec
  5 10.255.19.2 112 msec * 104 msec

In the output of the traceroute command, two (2) MPLS labels are displayed; one label is the transport (MPLS) label and the other label is the VPN label. This confirms operation of MPLS L3VPN. In this case, the transport label is 4003 and VPN label 1000. As can be noticed, the transport label is continually swapped from 4003 to 2006 as the packet is switched from hop to hop while the VPN label is maintained with the packet from the ingress PE across the LSP to the egress PE where it finally gets popped and the packet is then redistributed into the IGP at the PE-CE link and routed using IP.

ping

R7>ping 10.9.2.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.9.2.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 44/65/100 ms

The ping command confirms the connectivity from customer site to remote customer site across the SP MPLS L3VPN.

To view the MPLS LFIB

R6>show mpls forwarding-table
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
Label      Label      or Tunnel Id     Switched      interface    
16         Pop Label  30.255.114.0/30  0             Gi0/0      30.255.46.1
17         4006       30.255.112.0/30  0             Gi0/0      30.255.46.1
           5024       30.255.112.0/30  0             Gi1/0      30.255.56.1
18         Pop Label  30.255.24.0/30   0             Gi0/0      30.255.46.1
19         4000       30.255.11.11/32  0             Gi0/0      30.255.46.1
           5022       30.255.11.11/32  0             Gi1/0      30.255.56.1
20         4012       30.255.2.2/32    0             Gi0/0      30.255.46.1
21         4023       30.2.10.1/32     0             Gi0/0      30.255.46.1
22         Pop Label  30.255.5.5/32    0             Gi1/0      30.255.56.1
23         4003       30.255.1.1/32    0             Gi0/0      30.255.46.1
24         Pop Label  30.255.115.0/30  0             Gi1/0      30.255.56.1
25         4007       30.255.113.0/30  0             Gi0/0      30.255.46.1
           5001       30.255.113.0/30  0             Gi1/0      30.255.56.1
26         4009       30.255.12.0/30   0             Gi0/0      30.255.46.1
27         4005       30.255.13.0/30   0             Gi0/0      30.255.46.1
           5003       30.255.13.0/30   0             Gi1/0      30.255.56.1
28         4004       30.255.35.0/30   0             Gi0/0      30.255.46.1
           5037       30.255.35.0/30   0             Gi1/0      30.255.56.1
29         4011       30.255.3.3/32    0             Gi0/0      30.255.46.1
           5004       30.255.3.3/32    0             Gi1/0      30.255.56.1
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
Label      Label      or Tunnel Id     Switched      interface    
30         4013       30.3.10.1/32     0             Gi0/0      30.255.46.1
           5005       30.3.10.1/32     0             Gi1/0      30.255.56.1
31         4014       30.3.9.1/32      0             Gi0/0      30.255.46.1
           5006       30.3.9.1/32      0             Gi1/0      30.255.56.1
32         4015       30.3.8.1/32      0             Gi0/0      30.255.46.1
           5007       30.3.8.1/32      0             Gi1/0      30.255.56.1
33         4016       30.3.7.1/32      0             Gi0/0      30.255.46.1
           5008       30.3.7.1/32      0             Gi1/0      30.255.56.1
34         4017       30.3.6.1/32      0             Gi0/0      30.255.46.1
           5009       30.3.6.1/32      0             Gi1/0      30.255.56.1
35         4018       30.3.5.1/32      0             Gi0/0      30.255.46.1
           5010       30.3.5.1/32      0             Gi1/0      30.255.56.1
36         4019       30.3.4.1/32      0             Gi0/0      30.255.46.1
           5011       30.3.4.1/32      0             Gi1/0      30.255.56.1
37         4020       30.3.3.1/32      0             Gi0/0      30.255.46.1
           5012       30.3.3.1/32      0             Gi1/0      30.255.56.1
38         4021       30.3.2.1/32      0             Gi0/0      30.255.46.1
           5013       30.3.2.1/32      0             Gi1/0      30.255.56.1
39         4022       30.3.1.1/32      0             Gi0/0      30.255.46.1
           5014       30.3.1.1/32      0             Gi1/0      30.255.56.1
40         No Label   10.7.3.0/24[V]   0             Gi2/0      10.255.67.1
41         Pop Label  30.255.4.4/32    0             Gi0/0      30.255.46.1
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
Label      Label      or Tunnel Id     Switched      interface    
42         No Label   30.4.10.1/32     0             Gi0/0      30.255.46.1
43         No Label   30.4.9.1/32      0             Gi0/0      30.255.46.1
44         No Label   30.4.8.1/32      0             Gi0/0      30.255.46.1
45         No Label   30.4.7.1/32      0             Gi0/0      30.255.46.1
46         No Label   30.4.6.1/32      0             Gi0/0      30.255.46.1
47         No Label   30.4.5.1/32      0             Gi0/0      30.255.46.1
48         No Label   30.4.4.1/32      0             Gi0/0      30.255.46.1
49         No Label   30.4.3.1/32      0             Gi0/0      30.255.46.1
50         No Label   10.10.255.1/32[V]   \                      
                                       0             Gi2/0      10.255.67.1
51         No Label   10.255.67.0/30[V]   \                      
                                       2372          aggregate/CUST_Z_HQ
52         4024       30.2.9.1/32      0             Gi0/0      30.255.46.1
53         4025       30.2.8.1/32      0             Gi0/0      30.255.46.1
54         4026       30.2.7.1/32      0             Gi0/0      30.255.46.1
55         4027       30.2.6.1/32      0             Gi0/0      30.255.46.1
56         4028       30.2.5.1/32      0             Gi0/0      30.255.46.1
57         4029       30.2.4.1/32      0             Gi0/0      30.255.46.1
58         4030       30.2.3.1/32      0             Gi0/0      30.255.46.1
59         4031       30.2.2.1/32      0             Gi0/0      30.255.46.1
60         4032       30.2.1.1/32      0             Gi0/0      30.255.46.1
61         No Label   10.7.1.0/24[V]   0             Gi2/0      10.255.67.1
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop
Label      Label      or Tunnel Id     Switched      interface    
62         No Label   10.7.4.0/24[V]   0             Gi2/0      10.255.67.1
63         No Label   10.7.2.0/24[V]   0             Gi2/0      10.255.67.1
64         No Label   10.7.5.0/24[V]   0             Gi2/0      10.255.67.1
65         No Label   30.4.2.1/32      0             Gi0/0      30.255.46.1
66         No Label   30.4.1.1/32      0             Gi0/0      30.255.46.1
67         Pop Label  30.4.40.0/24     0             Gi0/0      30.255.46.1
R6>

L3VPN routes have a [V] next to the learned route. To view the FIB for the different VRFs run the command show ip cef vrf <vrf_name>.

R6>show ip cef vrf CUST_Z_HQ
Prefix               Next Hop             Interface        
0.0.0.0/0            no route                              
0.0.0.0/8            drop                                  
0.0.0.0/32           receive                                
10.7.1.0/24          10.255.67.1          GigabitEthernet2/0
10.7.2.0/24          10.255.67.1          GigabitEthernet2/0
10.7.3.0/24          10.255.67.1          GigabitEthernet2/0
10.7.4.0/24          10.255.67.1          GigabitEthernet2/0
10.7.5.0/24          10.255.67.1          GigabitEthernet2/0
10.9.1.0/24          30.255.46.1          GigabitEthernet0/0
10.9.2.0/24          30.255.46.1          GigabitEthernet0/0
10.9.3.0/24          30.255.46.1          GigabitEthernet0/0
10.9.4.0/24          30.255.46.1          GigabitEthernet0/0
10.10.255.1/32       10.255.67.1          GigabitEthernet2/0
10.255.19.0/30       30.255.46.1          GigabitEthernet0/0
10.255.67.0/30       attached             GigabitEthernet2/0
10.255.67.0/32       receive              GigabitEthernet2/0
10.255.67.1/32       attached             GigabitEthernet2/0
10.255.67.2/32       receive              GigabitEthernet2/0
10.255.67.3/32       receive              GigabitEthernet2/0
127.0.0.0/8          drop                                  
224.0.0.0/4          drop                                  
224.0.0.0/24         receive                                
Prefix               Next Hop             Interface        
240.0.0.0/4          drop                                  
255.255.255.255/32   receive

Additionally, to view the MPLS L3VPN on a specific route in the VRF LIB

R6>show ip cef vrf CUST_Z_HQ 10.9.1.0
10.9.1.0/24
  nexthop 30.255.46.1 GigabitEthernet0/0 label 4003 1000
R6>        
R6>show ip cef vrf CUST_Z_HQ 10.7.1.0
10.7.1.0/24
  nexthop 10.255.67.1 GigabitEthernet2/0
R6>show ip cef vrf CUST_Z_HQ 10.7.1.0 detail
10.7.1.0/24, epoch 0
  local label info: other/61
  nexthop 10.255.67.1 GigabitEthernet2/0
R6>show ip cef vrf CUST_Z_HQ 10.9.1.0 detail
10.9.1.0/24, epoch 0, flags rib defined all labels
  recursive via 30.255.1.1 label 1000
    nexthop 30.255.46.1 GigabitEthernet0/0 label 4003
R6>

Troubleshooting

LDP Adjacency

  • Is LDP actually enabled: show mpls interfaces
  • Is LDP transport working: debug mpls ldp transport events
  • Is LDP session authenticated: LDP(like BGP) uses TCP auth (option 19)

LDP Advertisements

Advertise labels for all IGP learned routes.

  • Are labels actually bound:
    • show mpls ldp binding
    • show mpls forwarding-table
    • debug mpls ldp binding
    • debug mpls ldp transport events: able to view LDP messages being exchanged. If an LDP peer does not have a route to its neighbor's LDP ID, this debug command will display this appropriate message.
  • Is allocation being filtered?:
    • Advertise filter vs. allocate filter: labels can be advertised for prefixes that are in a specified access-list. It is important to advertise labels for all IGP learned routes.
    • Typically only /32 needs an allocation.

PE-CE Routing

  • Is loop prevention being violated?:
    • OSPF don-bit and domain-tag
    • BGP AS-Path and Site-of-Origin(SoO)
    • EIGRP Site-of-Origin
  • Is the VRF just hung?
    • clear ip route vrf *
    • Forces re-import/export

VPNv4 BGP

Is the route-target being filtered

  • Wrong import/export policy
  • Default route-target filter
  • VPNv4 Route reflection
debug bgp vpnv4 unicast updates to view debugging messages associated with VPNv4 failures. Critical error message is "DENIED due to: extended community not supported". show vrf detail.

Are VPNv4 routes being sent/received

  • show bgp vpnv4 unicast all
  • show bgp vpnv4 unicast all <prefix> : The in() and out MPLS labels of a prefix can be viewed.
  • show bgp vpnv4 unicast all neighbor advertised-routes
  • debug bgp vpnv4 unicast updates
  • clear bgp vpnv4 unicast * <in|out>

MPLS Data Plane

  • Is the VPNv4 peering to a /32? The loopback needs to be a /32 netmask for the VPN data plane to work.
    • PHP may happen one hop too soon if peering to the wrong address. This will result in a traffic blackhole.
    • Problems in route summarization.
  • Are LDP and IGP synced? LFIB can only install label for RIB.LRIB intersection

    • Posted by at
    Labels: , , , ,

    No comments:

    Post a Comment

    Subscribe to: Post Comments (Atom)