The following sections outline basic CEF troubleshooting using a stepwise approach based on Figure 4-3 and Example 4-5. These sections move to focusing on CEF.
The steps for verifying the CEF table are as follows:
|Step 1||Verify the CEF configuration.|
|Step 2||Confirm the IP CEF switching path, including using CEF accounting
counters to confirm the switching path.
|Step 3||Verify CEF switching details.|
Verifying the CEF Configuration
When verifying the CEF table (FIB), first verify whether CEF is indeed enabled globally and on an interface basis using the following commands:
show ip cef
show cef interface
Example 4-10 Verifying That IP CEF Is Enabled Globally and Per-Interface
If CEF is not enabled globally, the show ip cef command returns the message "%CEF not running," as shown in Example 4-11.
Example 4-11 Example of Router Not Running CEF
Although the previous step shows that CEF is enabled on the interfaces in respect to Figure 4-3, the show ip interface and the show cef interface commands confirm that CEF is enabled on a per-interface basis. Example 4-12 illustrates the show ip interface command.
Example 4-12 Verifying That CEF Is Enabled on a Per-Interface Basis Using the show ip interface Command
Confirming the IP CEF Switching Path
The next step in troubleshooting the CEF table is to verify the switching path that the router in question is using. The show interfaces stat command displays the switching path stats on Cisco IOS routers, as shown in Example 4-13. The processor row includes processswitched (software-switched) packets, while the router cache row includes both CEFswitched and fast-switched packets. Note that high-end routers and switches that support distributed CEF display an additional row referred to as Hardware, Parallel Express Forwarding (PXF), or Distributed.
Example 4-13 Displaying Interface Switching Statistics
From Example 4-13, the router is using route cache for switching. The command does not tell you how much is CEF switched or how much is fast switched because route cache includes both CEF-switched and fast-switched packets. Another way to determine whether the router is fast-switching or CEF-switching packets is to view the contents of the fastswitching table using the show ip cache command. If prefix entries exist in the fastswitching table, the traffic to the destination entries are being fast switched. Example 4-14 illustrates using the show ip cache command to verify CEF switching.
Example 4-14 Viewing the Fast-Switching (IP Cache) Table
Example 4-14 illustrates that the route cache counters from the show interface stats command were CEF switched by the router in our example. Example 4-14 views the output of the show interface stats command, checks the fast-switching table using the show ip cache command, and then disables CEF. After disabling CEF, the fast-switching table was rechecked and entries were observed.
In reference to Figure 4-3, where the host 10.1.1.100 could not ping 10.18.118.184, the troubleshooting step of ruling in or ruling out CEF is to disable CEF switching, globally or per interface, and reattempt to send the ICMP echoes. Of course, in a production environment, disabling CEF might not be an option; however, during a change-control window or isolated situation, this troubleshooting is useful. Always consult with a Cisco Technical Assistance Center (TAC) engineer before disabling CEF.
When disabling CEF on a per-interface basis for troubleshooting, pay close attention to the ingress and egress interfaces. Generally, when disabling CEF on a per-interface basis for troubleshooting, disable CEF on both the ingress and egress interface. In a mixed-mode environment, where the ingress and egress interface use different switching configurations, refer to Table 4-1 for the resulting switching method.
Table 4-1 Resulting Switching Method Based on Ingress and Egress Switching Configuration
|Ingress Interface||Egress Interface||Resulting Switching Method|
|Process||Fast Switching||Fast Switching|
Based on the information in Table 4-1, CEF switching occurs on the ingress. Therefore, use the no ip route-cache cef command on the ingress interface to disable CEF. In contrast, because Cisco IOS builds a fast-switching cache entry after switching a packet, packets ingress on a process-switched interface and egress through a fast-switched interface. Therefore, use the no ip route-cache command on the egress interface to disable fast switching. Alternatively, disabling CEF on a global basis is permissible on several platforms, mostly low-end platforms that use the no ip cef command. Check the release notes and configuration guide for your specific platform when attempting to disable CEF globally or per interface.
NOTE To reenable CEF on a per-interface basis, both fast switching and CEF have to be enabled. As a result, not only is the ip route-cache cef interface configuration command required, but the ip route-cache interface configuration command is also required for CEF to function on a per-interface basis.
Furthermore, newer mid- to high-end platforms do not support disabling CEF on a global or per-interface basis. When attempting to disable CEF on a platform such as the Catalyst 6500, Cisco 12000, CRS, and Cisco 7600 that do not support disabling CEF, the CLI returns an error. Example 4-15 illustrates an example of attempting to disable CEF on a platform that does not support disabling CEF.
Example 4-15 Attempting to Disable CEF on a Platform Such as the Catalyst 4500 That Does Not Support Disabling CEF
Using CEF Accounting Counters to Confirm the Switching Path
Cisco IOS supports CEF accounting, with specific limitations pertaining to hardwareswitched traffic. However, with software switching, CEF supports an accounting option for packet and byte counters.
When troubleshooting CEF, you can view the packet and byte counters on a per-prefix basis. To enable CEF accounting, use the ip cef accounting per-prefix global configuration command. Simply use the show ip cef prefix command to display the counters for a specific prefix. Example 4-16 illustrates the use of CEF accounting.
Example 4-16 CEF Accounting
NOTE When you enable network accounting for dCEF from global configuration mode, accounting information grouped by IP prefix is not sent to the route processor (RP) for viewing through the show ip cef command. However, the accounting information is collected by dCEF processes on the line card. In this situation, use the show cef linecard command to view dCEF statistics.
Verifying the CEF Switching Details
In most production environments, you usually cannot disable CEF, even on a per-interface basis, during normal production. This is because in many production networks, traffic rates exceed the software-switching capabilities of the router, and disabling CEF forces software switching. Therefore, disabling CEF to troubleshoot CEF is not always an option. The next step in these situations is to verify the CEF table information.
Example 4-17 illustrates the use of the show ip cef command to gather details about the CEF table on a per-entry basis. The values chosen for this example match those for the troubleshooting example.
Example 4-17 Gathering CEF Table Details
From the output shown in Example 4-17, the host 10.18.118.184 next hop is 172.18.114.1 and is a valid cached adjacency through Ethernet0/0. The first step is to verify this output against Figure 4-3 and the show ip route command from Example 4-5. The details are correct. The next step is to verify the next-hop CEF entry for 10.18.118.184, which is 172.18.114.1. Again, this information appears to be correct based on Figure 4-3. If this information was not correct, the next step is to either disable CEF and test connectivity or open a Cisco TAC case.
Another possible reason for packet loss with CEF is CEF drop adjacency. CEF drop adjacencies define hardware-switched drops for prefixes. CEF drop adjacencies allow dropping frames in hardware rather than punting every frame for software switching. This is an effective method in preventing denial of service (DoS) attacks and high CPU usage due the CPU processing excessive drops. If a CEF drop adjacency exists, it is generally because of one of the following reasons:
- Unsupported features.
- Packets destined to prefixes associated with punt adjacencies that exceed a predefined rate. Punting rate limiters exist to prevent DoS attacks and high CPU usage caused by excessive punts.
- Unresolved or missing FIB entry.
- Unsupported frame type.
Example 4-18 illustrates a sample output from the show cef drop command. In respect to Figure 4-3 and Example 4-5, attempting to relate the CEF drop counters to the ICMP packets can yield additional details about why the packets are being dropped. However, in a production environment, it is difficult to correlate the ICMP packet loss to show cef drop counters because production environments generally have multiple flows passing traffic simultaneously. Nonetheless, if the show cef drop counters remain 0 during the ping tests, you can rule out the notion that the ping failed because of CEF drop adjacencies. In later code, the show ip cef switching statistics command gives detailed information about why a drop occurs and replaces the show cef not command.
Example 4-18 show cef drop Command Example
Table 4-2 defines the fields associated with the show cef drop command.
Table 4-2 show cef drop Field Descriptions
|Slot||Refers to the slot for the respective ingress packet counts. For Cisco IOS routers that do not support dCEF this value is always RP.|
|Encap_fail||Indicates the number of packets dropped after exceeding the limit for packets punted to the processor because of missing adjacency information such as an unresolved ARP request. Note that CEF throttles packets punted to the process level at a rate of one packet per second to aid in susceptibility of DoS attacks.|
|Unresolved||Indicates the number of packets dropped because of an unresolved prefix in the FIB table.|
|Unsupported||Indicates the number of packets fast-dropped by CEF (drop adjacency) because of an unsupported feature.|
|No_route||Indicates the number of packets dropped because of a missing prefix in the FIB table.|
|No_adj||Indicates the number of packets dropped because of incomplete adjacency.|
|Chksum_Err||Indicates the number of IP version 4 (IPv4) packets received with a checksum error.|
In controlled environments, preferably in nonproduction environments, use debugs to troubleshoot CEF. If you can narrow your issue to a specific host or subnet and the router or switch under investigation is logging drops or receives, you can use debugs that are limited to specific destinations to troubleshoot the issue. Limiting debugs is done by limiting debug output to specific IP sources or destinations configured in an access list. Example 4-19 illustrates an example of configuring an access control list (ACL) to limit the output of a CEF debug to a specific destination.
Example 4-19 Controlling Debug Output Defined by an ACL
Based on the example in Figure 4-3 and Example 4-5, the debug output illustrates the router processing "receive" packets destined for its own interface. Receive packets include packets punted by CEF for software switching. Drop packets appear in the debug output in the same manner. As a result, this debug, used in controlled environments, can yield additional information helpful in troubleshooting.
Learn how to troubleshoot Cisco's Express Forwarding network switching technology in "Basic IP Connectivity and CEF Troubleshooting," Chapter 4 from the book Cisco Express Forwarding by Nakia Stringfield, Russ White and Stacia McKee.
Basic IP Connectivity and CEF Troubleshooting
Accurately describe the problem
Scope the network topology
Review the OSI model for troubleshooting
Verify the ARP table
Verify the IP routing table
Verify the CEF FIB table
Verify the adjacency table
Conduct hardware-specific troubleshooting
Reproduced from the book Cisco Express Forwarding. Copyright 2007, Cisco Systems, Inc. Reproduced by permission of Pearson Education, Inc., 800 East 96th Street, Indianapolis, IN 46240. Written permission from Pearson Education, Inc. is required for all other uses.