EN 
01.12.2025 Iva WELCOME IN MY WORLD

This website is originally written in the Czech language. Most content is machine (AI) translated into English. The translation may not be exact and may contain errors.

Tento článek si můžete zobrazit v originální české verzi. You can view this article in the original Czech version.
Cisco WLC C9800 - provoz, dohled a troubleshooting

Cisco WLC C9800 - operation, monitoring and troubleshooting

Edited 09.10.2023 16:00 | created | Petr Bouška - Samuraj |
The last part of the Cisco Catalyst 9800 Wireless Controller IOS XE based configuration description. It deserves to be much more extensive, but only briefly covers certain areas of client operation and monitoring. Small sections of troubleshooting and a few mentions of Windows-side information. At the end is a description of setting up Smart Licensing Using Policy.
displayed: 8 776x (2 717 CZ, 6 059 EN) | Comments [0]

Note: The description in the article is based on the Cisco Catalyst 9800-L Wireless Controller with Cisco IOS XE Cupertino version 17.9.3 (currently recommended version). To which are connected Cisco Catalyst 9164I AP access points.

When we implement a good WLAN deployment, the entire infrastructure can function relatively without intervention. However, at the beginning, we often deal with some fine-tuning and problems where some clients cannot connect, are being disconnected, etc. Radioactive Trace logs help us a lot. For example, when clients regularly disconnect and reconnect, we can determine from the log that we need to increase the session timeout.

When we implement a more secure WLAN setup and want to use modern protocols and algorithms, older clients won't connect. We must consider whether to allow an older security method (perhaps even create a separate WLAN) or not support old clients.

One solution is to use Open Security and Web Auth. Because the communication is not encrypted, almost every client can connect. And they can likely handle web authentication. However, there's the problem that the communication is not encrypted. Also, you need to log in every time you connect.

When troubleshooting issues on a mobile device and we want to browse logs (find them by MAC address), it's often necessary to look in the Wi-Fi network settings and change Use randomized MAC to Use device MAC. Otherwise, we can't search by the device's MAC address.

During operation, we can monitor information about clients and access points. Various statistics and utilization. We may respond with certain changes if necessary.

Monitoring and Operation

The Dashboard offers a range of interesting information for operational monitoring. We can click on everything and get to pages with detailed information, where a filter is usually applied.

WLC C9800 WebUI - Configuration - Tags & Profiles - WLANs - Filter

Best Practices

  • Administration - Best Practices

We can use Cisco recommendations regarding our configuration.

WLC File System

  • Administration - Management - File Manager

Simple access to WLC files via web. We can upload or download.

CLI

  • Administration - Command Line Interface

For quick use of a specific command, we don't need to connect to the CLI via SSH, but we can enter it in the GUI.

Connected Clients

  • Monitoring - Wireless - Clients

We see here a list of connected clients with many details. We can add a Filter (table header) and, for example, display clients on a specific AP or with a specific protocol. Clicking on the key icon next to the MAC address sets the address in Radioactive Trace.

On the Excluded Clients tab, we see excluded clients and can remove them (allow).

Clicking on a client displays a large amount of details about it.

  • 360 View - the main tab shows where and how the client is connected, its identification, statistics of used applications, etc.
  • Mobility History - is useful for tracking how the client traveled between APs and what type of roaming it used
  • General - Security Information - here we see what policies (protocols, encryption, etc.) the client used for connection
WLC C9800 WebUI - Monitoring - Wireless - Clients - Security Informati

Access Points

  • Monitoring - Wireless - AP Statistics

We see here a list of APs, their main configuration, and various statistics. We can switch to configuration (gear icon). In the AP detail (click on the row), we see, for example, slots for radios (Slot 0 - 2.4 GHz, Slot 1 - 5 GHz, Slot 2 - 6 GHz) and channel utilization. Next to the AP name is the AP Operational Configuration icon, where we see what tags and configurations are applied to the AP (i.e., list of SSIDs that the AP broadcasts).

WLC C9800 WebUI - Monitoring - Wireless - AP Statistics - 360 View
  • Monitoring - Wireless - Radio Statistics

We see here radio statistics for individual bands and APs. What channel the AP currently has set, what is the channel width, transmit power, and more. By clicking on an AP, we learn more details. For example, what neighboring APs it sees, how noisy the channels are, etc.

It might be useful to look at the transmit power. In an optimal design, all APs should transmit relatively strongly (1 is the highest level). Low power is typically only in dense deployments.

WLC C9800 WebUI - Monitoring - Wireless - Radio Statistics

WiFi Measurement Applications

There are many applications we can use to measure the quality and strength of wireless signals. A separate area is professional Site Survey tools. An example of a mobile Android application is Aruba Utilities. One of the applications for Windows is inSSIDer.

It's good to verify the signal strength in different places and test switching between APs (Roaming). For measurement, it's useful to turn on Advertise AP Name, and inSSIDer then displays AP names.

Troubleshooting

  • Troubleshooting

There are various tools for troubleshooting, such as

  • Logs - web access to Syslog, Webserver Log, License Log
  • Packet Capture - ability to capture communication

Radioactive Trace

  • Troubleshooting - Radioactive Trace

A very useful tool when troubleshooting issues with a specific client. We enter the client's MAC address and can download (generate) its logs for a specific period. We can thus look for problems when failing to connect to WLAN. Or we find events when the client unexpectedly disconnects. For example, if Session Timeout is applied, the log shows (only a small part of the events)

14:17:57 [client-orch-sm] [16032]: (note): MAC: e0dc.ffeb.2663  Client delete initiated. Reason: CO_CLIENT_DELETE_REASON_SESSION_TIMEOUT
14:17:57 [client-orch-state] [16032]: (note): MAC: e0dc.ffeb.2663  Client state transition: S_CO_DELETE_IN_PROGRESS -> S_CO_DELETED
14:17:59 [client-orch-state] [16032]: (note): MAC: e0dc.ffeb.2663  Client state transition: S_CO_INIT -> S_CO_ASSOCIATING

If we need more details, we can turn on Debug with the Start button (after setting the MAC address). Then the problematic event needs to occur, we stop the debug and generate/download extensive logs.

WLC C9800 WebUI - Troubleshooting - Radioactive Trace

CPU Core Number 7 Utilization at 100%

On the dashboard, I constantly have a warning about high CPU utilization. Cisco's response is that it's okay.

This process is used by the WLC to listen for packets in an effort to improve packet throughput and reduce packet latency and jitter, this increases the amount of CPU cycles that "ucode_pkt_PPE0" process consumes at baseline even without any traffic. This is the reason why is normal to see the CPU at a high level for that specific process on the core 7. So taking into consideration this process and the rest of the outputs we consider that the CPU behaves as expected and under normal operation standard.

WLC C9800 WebUI - Dashboard - CPU Usage

Solutions on the Windows OS side

Logs in Windows

When troubleshooting WiFi connection issues in Windows, it's useful to look at the logs.

  • Event Viewer - Applications and Services Logs - Microsoft - Windows - WLAN-AutoConfig

Windows roaming between APs and preferring 5 GHz band

When the signal strength of an AP decreases (for example during movement) to which a Windows notebook is connected, at some point the WiFi adapter starts searching for another AP candidate. It will connect to it if it has better parameters. If we find that we remain connected to the original AP for a long time (which already has a weak signal), we can try to adjust the signal strength threshold. This is the Roaming Aggressiveness setting on the adapter.

In Windows, we can set this in the properties of the wireless network adapter on the Advanced tab. The Roaming Aggressiveness item, default value is Medium.

Windows Network Adapter Properties - Roaming Aggressiveness

Another setting we can adjust in the network adapter parameters is Preferred Band. In many situations, it may be better to use the 5 GHz band.

Windows supported security methods (settings)

Using a command line, we can display information about the network adapter. What IEEE 802.11 standards (radio types), bands, authentication, and encryption (security standards) it supports.

C:\>netsh wlan show drivers

Interface name: Wi-Fi

    Driver                    : Intel(R) Wi-Fi 6E AX211 160MHz
    Vendor                    : Intel Corporation
    Provider                  : Intel
    Date                      : 18.06.2023
    Version                   : 22.240.0.6
    INF file                  : oem214.inf
    Type                      : Native Wi-Fi Driver
    Radio types supported     : 802.11b 802.11g 802.11n 802.11a 802.11ac 802.11ax
    FIPS 140-2 mode supported : Yes
    802.11w Management Frame Protection supported : Yes
    Hosted network supported  : No
    Authentication and cipher supported in infrastructure mode:
                                Open             None
                                Open             WEP-40bit
                                Open             WEP-104bit
                                Open             WEP
                                WPA-Enterprise   TKIP
                                WPA-Enterprise   CCMP
                                WPA-Personal     TKIP
                                WPA-Personal     CCMP
                                WPA2-Enterprise  TKIP
                                WPA2-Enterprise  CCMP
                                WPA2-Personal    TKIP
                                WPA2-Personal    CCMP
                                Open             Vendor defined
                                WPA3-Personal    CCMP
                                Vendor defined   Vendor defined
                                WPA3-Enterprise 192 Bits GCMP-256
                                OWE              CCMP
                                WPA3-Enterprise  CCMP
                                WPA3-Enterprise  TKIP
    Number of supported bands : 3
                                2.4 GHz [ 0 MHz - 0 MHz]
                                5 GHz   [ 0 MHz - 0 MHz]
                                6 GHz   [ 0 MHz - 0 MHz]
    IHV service present       : Yes
    IHV adapter OUI           : [00 00 00], type: [00]
    IHV extensibility DLL path: C:\Windows\System32\DriverStore\FileRepository\netwtw6e.inf_amd64_fa3402905034e59a\IntelIHVRouter12.dll
    IHV UI extensibility ClSID: {00000000-0000-0000-0000-000000000000}
    IHV diagnostics CLSID     : {00000000-0000-0000-0000-000000000000}
    Wireless Display Supported: Yes (Graphics Driver: Yes, Wi-Fi Driver: Yes)

Smart Licensing Using Policy

Licensing C9800 uses Smart Software Licensing just like IOS XE switches and other devices or applications. There is an older Legacy Smart Licensing, which was used from IOS XE version 16.10.1, and a newer recommended Smart Licensing Using Policy, which came with IOS XE version 17.3.2.

We manage licenses in the Smart Software Licensing / Manager portal and need a Smart Account. Devices connect directly or indirectly to the Cisco Smart Software Manager (CSSM) cloud. Here we will look at the simplest option called Direct connect to CSSM.

For connecting to CSSM, a specific communication method (Transport) is used:

  • Smart Transport - recommended modern communication method, address https://smartreceiver.cisco.com/licservice/license
  • Call-home Transport - older method used for Smart Licensing, it can (but is not recommended) be used for Smart Licensing Using Policy, address https://tools.cisco.com/its/service/oddce/services/DDCEService

License settings and connection to CSSM can also be done in the GUI. But if we encounter problems, we must use the CLI.

  • Licensing - Service Settings

Configuration using CLI is not complicated. In practice, I encountered a problem where I could not connect to CSSM via the GUI. It also failed in the CLI, and an error could be found:

Trust Establishment:
  Attempts: Total=2, Success=0, Fail=2 Ongoing Failure: Overall=2 Communication=2
  Last Response: NO REPLY on Oct 02 10:45:16 2023 CET

Oct  2 10:46:48: %SESSION_MGR-5-FAIL: Chassis 1 R0/0: wncd: Authorization failed or unapplied for client (xxxx.xxxx.xxxx)
 on Interface capwap_900000f8 AuditSessionID 6F64A8C00001B353EF8F9CEF. Failure reason: Authc fail. Authc failure reason: No
  Response from Client.

I had to use two commands that we never used when configuring C9300 and C9500 switches. Setting the client certificate (self-signed certificate TP-self-signed-xxxxxxxxxx) and the source L3 interface (VLANs for WMI).

Smart Licensing Using Policy configuration is described in the link at the beginning of the chapter, including other variants. Here is just a brief overview. Certain conditions must be met, such as DNS resolution and HTTPS communication to the Cisco address smartreceiver.cisco.com.

conf t
 ip domain lookup
 ip http client source-interface <interface>
 ip http client secure-trustpoint <TP>
 license smart transport smart
 license smart url default
end
write

In the CSSM portal, we generate a registration Token, which we use in the next command to establish trust between the WLC (PI - Product Instance) and CSSM. A pair of encryption keys is created, and a Trust Code is installed.

license smart trust idtoken <token> all force

After establishing the trust relationship, we can manually trigger synchronization (it usually happens automatically after a few minutes):

license smart sync all

Some commands to display information and statuses:

show license status
show license all
show license tech support | s Trus

The last command shows data about various operations (such as Trust Establishment, Trust Acknowledgement, and Trust Sync) and the last response, in case of an error, its reason (Failure Reason).

Author:

Related articles:

Cisco WLC

The Wireless LAN Controller is a Cisco device for central management of WiFi networks (access points). The articles describe the function and configuration of Cisco wireless networks. From Access Points (i.e. access points), to WLC, Wireless Control System (WCS) and Cisco Prime Infrastructure.

If you want write something about this article use comments.

Comments

There are no comments yet.

Add comment

Insert tag: strong em link

Help:
  • maximum length of comment is 2000 characters
  • HTML tags are not allowed (they will be removed), you can use only the special tags listed above the input field
  • new line (ENTER) ends paragraph and start new one
  • when you respond to a comment, put the original comment number in squar brackets at the beginning of the paragraph (line)