The first step in troubleshooting IP connectivity issues and CEF from a Layer 3 perspective is to consult the ARP table for identified IP devices that are experiencing connectivity issues. When CEF cannot locate a valid adjacency for a destination prefix, it punts the packets to the CPU for ARP resolution and, in turn, completion of the adjacency.
For example, if the ARP table already lists a particular host, punting it to the process level does not trigger an ARP request. If an entry is incomplete (no response from ARP request) or incorrect in the ARP table, it is also incomplete or incorrect in the CEF adjacency table. This section covers reviewing the ARP table.
Example 4-2 illustrates sample output from the show arp command used to display the contents of the ARP table.
Example 4-2 Displaying the ARP Table in Cisco IOS
Based on the show arp command output, investigate whether the ARP table information is correct. In this example, you see three entries. The 172.18.114.243 entry is incomplete, which means that the device with the address 172.18.114.243 did not respond to the ARP request or is simply powered off. To verify whether the other listings are correct, you might need to access the end device and verify the locally configured MAC address. Example 4-3 illustrates determining the IP address and MAC address of a Microsoft Windows XP laptop.
Example 4-3 Determining
the IP Address and MAC Address of a Microsoft Windows XP Laptop
In Example 4-3, the MAC (physical) address of the host is 00-09-6B-60-15-74. This differs from the ARP table of the local router. A protocol analyzer such as Ethereal is necessary to decode frames on Ethernet to investigate why the router has an incorrect entry. Example 4-4 shows an example of troubleshooting ARP entries with the debug arp command.
NOTE Because this is simply a mock setup for illustrative purposes, using a Cisco IOS debug is not an issue. Generally, in large networks, debugs such as debug arp are intrusive and should not be used in production environments.
Example 4-4 Troubleshooting Incorrect ARP Entries Using the debug arp Command
From the debug output, it is apparent that two devices are replying to the ARP request sent by the router: the test host and a rogue or misconfigured device. The router always populates the ARP table with the most recent ARP response. Because the rogue entry arrived second, the ARP table was incorrect. Another example of invalid ARP entries is with Frame Relay point-to-point or other types of point-to-point interfaces. Because the ARP entry is incorrect, the CEF adjacency table entry will also be incorrect.
NOTE When the ping process on a Cisco router or switch running Cisco IOS attempts to send an ICMP echo to a host for which an ARP entry does not exist, the router or switch initiates an ARP request. Because the first ICMP echo is dropped after a defined period awaiting the ARP response, the first ICMP echo fails to be sent. As a result, initiating an ICMP ping with default parameters to a device for which a current ARP entry does not exist generally results in a success rate of 4/5 (80 percent) because five ICMP echoes are sent by default in Cisco IOS.
Moreover, a common IP connectivity problem associated with CEF is where a more specific route, such as a host route (/32), is learned on a different interface than the connected route (for example, a host, 192.168.1.10, connected on interface Ethernet 0/0 of Cisco IOS router). According to the routing table, 192.168.1.0/24 is learned as a directly connected route. If the router learns a more specific route, such as 192.168.1.10/32, from another interface, the router forwards the packets to the more specific route. This is a common issue; you should rule it out early in troubleshooting.
Learn how to troubleshoot Cisco's Express Forwarding network switching technology in "Basic IP Connectivity and CEF Troubleshooting," Chapter 4 from the book Cisco Express Forwarding by Nakia Stringfield, Russ White and Stacia McKee.
Basic IP Connectivity and CEF Troubleshooting
Accurately describe the problem
Scope the network topology
Review the OSI model for troubleshooting
Verify the ARP table
Verify the IP routing table
Verify the CEF FIB table
Verify the adjacency table
Conduct hardware-specific troubleshooting
Reproduced from the book Cisco Express Forwarding. Copyright 2007, Cisco Systems, Inc. Reproduced by permission of Pearson Education, Inc., 800 East 96th Street, Indianapolis, IN 46240. Written permission from Pearson Education, Inc. is required for all other uses.
This was first published in July 2007