Logging

Logging can be used for fault identification, security auditing and network forensics. Logging aims to answer the three questions: what, when, who? Log messages are generated by the various services and protocols running on a network device. By default, output from system messages and debug commands is sent to a logging process. The logging process controls the distribution of logging messages to various destinations, such as the logging buffer, terminal lines, syslog server. Messages are also sent to the console. These are then viewed using syslog, local logging, debugs, conditional debugs. Logs are also used for forensics and compliance. It is important to note that the time must be consistent. This can be done by configuring NTP to provide accurate datetime values. Troubleshooting can be difficult if you cannot correlate timestamps across devices.

Logging Messages

Syslog messages can be output to the console, local buffer or remote syslog server. The messages follow this format:

%FACILITY-SUBFACILITY-SEVERITY-MNEMONIC: message text

Where:

• FACILITY-SUBFACILITY: protocol, module or process that generated the message.
• SEVERITY: This is a level from 0 - 7 that specifies how important the message is. When you enable logging for a specific level, it also enables all levels above it. For example, if you enable level 4(warnings), it also will enable levels 0 - 3.

  Level Number	Severity	Description	Syslog Definition
  0	Emergencies	System is unusable	LOG_EMERG
  1	Alerts	Action must be taken immediately	LOG_ALERT
  2	Critical	Critical conditions	LOG_CRIT
  3	Errors	Error conditions	LOG_ERR
  4	Warnings	Warning conditions	LOG_WARNING
  5	Notifications	Normal but significant conditions	LOG_NOTICE
  6	Informational	informational messages	LOG_INFO
  7	Debugging	debug-level messages	LOG_DEBUG

• MNEMONIC: a code that identifies the action reported
• Message text: A plain text description of the event

Local Logging

By default, all logging is displayed on the console. This can be turned off with the no logging command. By default, monitor and buffer logging display messages with severity levels of debugging and lower. For SSH/Telnet, the logging session settings can be configured using the command terminal monitor

Logging Console

To enable logging to console, use the command logging console. By default, the logging severity level for console is debugging. To control the logging options, use the command logging console <level>.

Logging Buffer

For local logging, Cisco IOS can save syslog messages to an internal buffer i.e., all the logs are saved on the device itself. You can display these messages by using the show logging command.

These messages are lost when the device is rebooted. The default buffer size is 8192 bytes. When the buffer is full, the older logs get overwritten by the newer ones. The size of the buffer can be increased using the command: logging buffered <size>.

Logging Monitor

To enable logging to remote logins via SSH or telnet, use the command logging monitor <level>. The default logging severity is debugging. However, by default, the log messages do not appear in the login session of of SSH or telnet. To view the log messages in the login session of a vty line, issue the command terminal monitor.

Logging messages can be disabled from a vty login session using the command terminal no monitor.

logging synchronous

The command logging synchronous [level <severity-level>|all] | limit <0-2147483647>. Enables synchronous logging of messages. It can be configured to prevent logging messages from interrupting the typing at a login session. The optional values:

• severity-level: indicates that messages with a severity level equal to or higher are printed asynchronously. The default is 2.
• limit: specifies the number of buffers to be queued for the terminal after which new messages are dropped. The default is 20.

Logging Flash

Logs can be stored as files on flash memory. This is accomplished using the command logging file flash:<file-name> &max-file-size> <min-file-size > <severity-level> where

• The maximum file size is in the range 4096 - 2147483647 bytes.
• The default size is 4096.
• The minimum file-size is in the range 1024 - 2147483647 bytes.
• The severity-level is in the range 0 - 7.

The command logging file flash:log_message.txt 40960 4096 3 stores log messages in a file in a flash memory.

Debug

Debugging is a powerful tool for troubleshooting complex network issues. It displays detailed information about all supported protocols and features of the network device. Use the debug command with caution, on production networks, as it is a resource intensive tool and may impact operations. Debugging can disrupt operations under high-CPU load conditions as debugging is processed by the CPU.

Best practices when using debugging include the following:

• show process cpu sorted: helps understand current CPU utilization. If the router is already loaded, debugging may impact the router.
• Consider output volume and time before enabling debug. If output is voluminous, this may impact router performance.
• Excessive console logging can cause the network device to hang as its CPU prioritizes this output. Use vty terminal monitor or logging buffered instead.
• Disable debug commands after troubleshooting using no debug all or undebug all.

Debugging messages can be enabled using the command debug <command>. The debugging messages can then be sent to a logging service, in this case the console using the command logging console.

The following is an example of the debugging of OSPF: debug ip ospf adj.

Debug using IP Access Lists

It allows debugging information that matches a configured IP access control list.

access-list 100 permit ip any 192.168.1.1 0.0.0.0
debug ip packet 100

Any debugs that do not match the access control list are not logged.

Conditional Debugs

Generates detailed information about a specific object such as matching a specific interface, username, MAC address, VLAN, etc. Limit the number of packets punted to the CPU. Conditional debugs are configured using the command debug condition <interface>. Conditional debugs can be stacked as such that the debugging messages that are displayed match all conditions in the stack.

debug condition gigabitethernet1/0
debug condition ip 1092.168.1.1
debug ip ospf hello

The above configuration will generate debug messages if they match all three commands.

To log debug messages for packets matching an ACL and specific interface: debug interface <interface-id>. Verify with show debug.

With conditional debugs, you can attach an access-list to filter debug traffic to permitted traffic in the acces-list.

Verification

show debug

After disabling the debug, ensure that the debug condition is disabled as well.

show debug

Enabled debugging can be viewed using this command.

Syslog

Logs generated by a device can be sent to a logging buffer whose size can be modified, or an off-box syslog collector. Syslog listens for log messages on UDP port 514. Syslog is the standard for computer message logging. It allows the separation of software which generates messages from the systems that store it and the software that analyzes and reports it. It can be used for the security auditing and system management, debugging messages and analysis. The syslog is supported by a wide range of devices, including routers and receivers over multiple platforms. The syslog is used to intergrate the log data from many various types of systems into the central repository.

Rate-limit: used to limit the rate of messages per second.

The syslogs can be forwarded to a syslog server using the following process

1. Specify the syslog server: logging host <ip-address>.
2. Specify the type of messages: the message type to be logged is specified using the command logging trap <severity>.

Filtering Logs

Logging Discriminator

A logging discriminator can be used to filter generated logs based on the text in the log. Log discriminators can be used to filter out specific logs that are repeatedly generated consuming the device log buffer. Some logs have a high severity level but are not so consequential such as up-down messages. It can be applied to the console, monitor, buffer or syslog server.

When creating a log discriminator, be as specific as possible and test the discriminator extensively before deploying it in a production environment. This will minimize the risk of filtering important messages.

To create a syslog message discriminator, use the command logging discriminator <discriminator-name> <facility> <mnemonics> <message-body> <includes> <severity> rate-limit> where:

• facility: (Optional) Message subfilter for the facility pattern in an event message.
• mnemonics: (Optional) Message subfilter for the mnemonic pattern in an event message.
• msg-body: (Optional) Message subfilter for the msg-body pattern in an event message.
• drops: Drops messages that match the pattern, including the specified regular expression.
• includes: Delivers messages that match the pattern, including the specified regular expression string.
• string: (Optional) Expression used for message filtering.
• severity: (Optional) Message subfilter by severity level or group.
• sev-num: (Optional) Integer that identifies the severity level or multiple levels. Multiple levels must be separated with a comma (,)/
• rate-limit: (Optional) Specifies a number of messages to be processed within a unit of time.
• msglimit: (Optional) Integer in the range of 1 to 10000 that identifies the number of messages not to be exceeded.

To configure a logging discriminator:

1. Configure the logging discriminator name and filtering options:
   logging discriminator msg-body drops Configured from console by console

2. Application:
   • To apply the logging discriminator to the console, use the command logging console discriminator <discriminator-name>
   • To apply the logging discriminator to the monitor, use the command logging monitor discriminator <discriminator>.

Verification

Configured logging settings can be viewed using the command show logging.

Timestamps

Timestamps are controlled by the service timestamps command. This command by itself enables the adding of timestamps to debug messages and, by default, sets the timestamps to uptime.

Having logs and debug messages timestamped is extremely important when troubleshooting particularly across different devices. To enable timestamps on logging and debug messages:

• service timestamps debug datetime msec: Adds timestamps to debug messages including milliseconds. The datetime option includes the date and the time.
• service timestamps log datetime msec: Adds the timestamps to logging messages. Options include:
  • datetime: This option alone enables the date (day of month and month) and time(hours, minutes and seconds). Additional options include:
    • localtime: use the local timezone for timestamps.
    • msec: include milliseconds in the timestamp.
    • show-timezone: add timezone information to the timestamp.
    • year: include the current year in the timestamp.
  • uptime: does not include additional options as does datetime.

Sequence Numbers

If there is more than one log message with the same time stamp, you can display messages with sequence numbers to view these messages. By default, sequence numbers in log messages are not displayed. Sequence numbers can be added to log messages using the command service sequence-numbers. When enabled, they are prepended to the log message appearing at the start of a log message.