Logging can be used for fault identification, security auditing and network forensics. Logging aims to answer the three questions: what, when, who? Log messages are generated by the various services and protocols running on a network device. By default, output from system messages and debug commands is sent to a logging process. The logging process controls the distribution of logging messages to various destinations, such as the logging buffer, terminal lines, syslog server. Messages are also sent to the console. These are then viewed using syslog, local logging, debugs, conditional debugs. Logs are also used for forensics and compliance. It is important to note that the time must be consistent. This can be done by configuring NTP to provide accurate datetime values. Troubleshooting can be difficult if you cannot correlate timestamps across devices.
Logging Messages
Syslog messages can be output to the console, local buffer or remote syslog server. The messages follow this format:
%FACILITY-SUBFACILITY-SEVERITY-MNEMONIC: message text
Where:
- FACILITY-SUBFACILITY: protocol, module or process that generated the message.
- SEVERITY: This is a level from 0 - 7 that specifies how important
the message is. When you enable logging for a specific level, it also enables
all levels above it. For example, if you enable level 4(warnings), it also will
enable levels 0 - 3.
Level Number Severity Description Syslog Definition 0 Emergencies System is unusable LOG_EMERG 1 Alerts Action must be taken immediately LOG_ALERT 2 Critical Critical conditions LOG_CRIT 3 Errors Error conditions LOG_ERR 4 Warnings Warning conditions LOG_WARNING 5 Notifications Normal but significant conditions LOG_NOTICE 6 Informational informational messages LOG_INFO 7 Debugging debug-level messages LOG_DEBUG - MNEMONIC: a code that identifies the action reported
- Message text: A plain text description of the event
Local Logging
By default, all logging is displayed on the console. This can be turned off with
the no logging
command.
By default, monitor and buffer logging display messages with severity
levels of debugging and lower.
For SSH/Telnet, the logging session settings can be configured using the command
terminal monitor
Logging Console
To enable logging to console, use the command
logging console
. By default, the logging severity level for console is
debugging. To control the logging options, use the command logging console <level>
.
Logging Buffer
For local logging, Cisco IOS can save syslog messages to an internal buffer i.e.,
all the logs are saved on the device itself. You
can display these messages by using the show logging
command.
These messages are lost when the device is rebooted. The default buffer size is
8192 bytes. When the buffer is full, the older logs get overwritten by the newer ones.
The size of the buffer can be increased using the command:
logging buffered <size>
.
Logging Monitor
To enable logging to remote logins via SSH or telnet, use the command
logging monitor <level>
. The default logging severity is debugging.
However, by default, the log messages do not appear in the login session of of SSH
or telnet. To view the log messages in the login session of a vty line, issue
the command terminal monitor
.
Logging messages can be disabled from a vty login session using the command
terminal no monitor
.
logging synchronous
The command logging synchronous [level <severity-level>|all] |
limit <0-2147483647>
.
Enables synchronous logging of messages. It can be configured to prevent logging
messages from interrupting the typing at a login session.
The optional values:
- severity-level: indicates that messages with a severity level equal to or higher are printed asynchronously. The default is 2.
- limit: specifies the number of buffers to be queued for the terminal after which new messages are dropped. The default is 20.
Logging Flash
Logs can be stored as files on flash memory. This is accomplished using the
command
logging file flash:<file-name> &max-file-size> <min-file-size
> <severity-level>
where
- The maximum file size is in the range 4096 - 2147483647 bytes.
- The default size is 4096.
- The minimum file-size is in the range 1024 - 2147483647 bytes.
- The severity-level is in the range 0 - 7.
logging file flash:log_message.txt 40960 4096 3
stores log messages in a file in a flash memory.
Debug
Debugging is a powerful tool for troubleshooting complex network issues.
It displays detailed information about all supported protocols and features of
the network device. Use the debug
command with caution, on
production networks, as it is a resource intensive tool and may impact operations.
Debugging can disrupt operations under high-CPU load conditions as debugging
is processed by the CPU.
Best practices when using debugging include the following:
show process cpu sorted
: helps understand current CPU utilization. If the router is already loaded, debugging may impact the router.- Consider output volume and time before enabling debug. If output is voluminous, this may impact router performance.
- Excessive console logging can cause the network device to hang as its
CPU prioritizes this output. Use vty
terminal monitor
or logging buffered instead. - Disable debug commands after troubleshooting using
no debug all
orundebug all
.
Debugging messages can be enabled using the command debug <command>
.
The debugging messages can then be sent to a logging service, in this case the
console using the command logging console
.
The following is an example of the debugging of OSPF: debug ip ospf
adj
.
Debug using IP Access Lists
It allows debugging information that matches a configured IP access control list.
access-list 100 permit ip any 192.168.1.1 0.0.0.0
debug ip packet 100
Any debugs that do not match the access control list are not logged.
Conditional Debugs
Generates detailed information about a specific object such as matching a specific
interface, username, MAC address, VLAN, etc.
Limit the number of packets punted to the CPU. Conditional debugs are configured using the command
debug condition <interface>
. Conditional debugs can be
stacked as such that the debugging messages that are displayed match all conditions
in the stack.
debug condition gigabitethernet1/0
debug condition ip 1092.168.1.1
debug ip ospf hello
The above configuration will generate debug messages if they match all three commands.
To log debug messages for packets matching an ACL and specific interface:
debug interface <interface-id>
. Verify with
show debug
.
With conditional debugs, you can attach an access-list to filter debug traffic to permitted traffic in the acces-list.
Verification
show debug
After disabling the debug, ensure that the debug condition is disabled as well.
show debug
Enabled debugging can be viewed using this command.
Syslog
Logs generated by a device can be sent to a logging buffer whose size can be modified, or an off-box syslog collector. Syslog listens for log messages on UDP port 514. Syslog is the standard for computer message logging. It allows the separation of software which generates messages from the systems that store it and the software that analyzes and reports it. It can be used for the security auditing and system management, debugging messages and analysis. The syslog is supported by a wide range of devices, including routers and receivers over multiple platforms. The syslog is used to intergrate the log data from many various types of systems into the central repository.
Rate-limit: used to limit the rate of messages per second.
The syslogs can be forwarded to a syslog server using the following process
- Specify the syslog server:
logging host <ip-address>
. - Specify the type of messages: the message type to be logged is specified
using the command
logging trap <severity>
.
Filtering Logs
Logging Discriminator
A logging discriminator can be used to filter generated logs based on the text in the log. Log discriminators can be used to filter out specific logs that are repeatedly generated consuming the device log buffer. Some logs have a high severity level but are not so consequential such as up-down messages. It can be applied to the console, monitor, buffer or syslog server.
When creating a log discriminator, be as specific as possible and test the discriminator extensively before deploying it in a production environment. This will minimize the risk of filtering important messages.
To create a syslog message discriminator, use the command logging
discriminator <discriminator-name> <facility> <mnemonics>
<message-body> <includes> <severity> rate-limit>
where:
- facility: (Optional) Message subfilter for the facility pattern in an event message.
- mnemonics: (Optional) Message subfilter for the mnemonic pattern in an event message.
- msg-body: (Optional) Message subfilter for the msg-body pattern in an event message.
- drops: Drops messages that match the pattern, including the specified regular expression.
- includes: Delivers messages that match the pattern, including the specified regular expression string.
- string: (Optional) Expression used for message filtering.
- severity: (Optional) Message subfilter by severity level or group.
- sev-num: (Optional) Integer that identifies the severity level or multiple levels. Multiple levels must be separated with a comma (,)/
- rate-limit: (Optional) Specifies a number of messages to be processed within a unit of time.
- msglimit: (Optional) Integer in the range of 1 to 10000 that identifies the number of messages not to be exceeded.
To configure a logging discriminator:
- Configure the logging discriminator name and filtering options:
logging discriminator msg-body drops Configured from console by console
- Application:
- To apply the logging discriminator to the console, use the command
logging console discriminator <discriminator-name>
- To apply the logging discriminator to the monitor, use the command
logging monitor discriminator <discriminator>
.
- To apply the logging discriminator to the console, use the command
Verification
Configured logging settings can be viewed using the command show logging
.
Timestamps
Timestamps are controlled by the service timestamps
command.
This command by itself enables the adding of timestamps to debug messages and,
by default, sets the timestamps to uptime.
Having logs and debug messages timestamped is extremely important when troubleshooting particularly across different devices. To enable timestamps on logging and debug messages:
service timestamps debug datetime msec
: Adds timestamps to debug messages including milliseconds. The datetime option includes the date and the time.service timestamps log datetime msec
: Adds the timestamps to logging messages. Options include:- datetime: This option alone enables the date (day of month and
month) and time(hours, minutes and seconds). Additional options include:
- localtime: use the local timezone for timestamps.
- msec: include milliseconds in the timestamp.
- show-timezone: add timezone information to the timestamp.
- year: include the current year in the timestamp.
- datetime: This option alone enables the date (day of month and
month) and time(hours, minutes and seconds). Additional options include:
- uptime: does not include additional options as does datetime.
Sequence Numbers
If there is more than one log message with the same time stamp, you can display
messages with sequence numbers to view these messages. By default, sequence numbers
in log messages are not displayed.
Sequence numbers can be added to log messages using the command service
sequence-numbers
. When enabled, they are prepended to the log message
appearing at the start of a log message.