Introduction to rsyslog system log service

rsyslog

RSYSLOG is the rocket-fast system for log processing.
  • rsyslog is a system management service for CentOS 6 and later versions. It provides high performance, excellent security and modular design.
  • Although rsyslog was originally a regular syslogd, it has developed into a Swiss army knife-style logging tool that can accept input from various sources, convert it, and output it to different destinations.
  • When limited processing is applied, RSYSLOG can deliver more than one million messages per second to local destinations. Even in remote destinations and finer processing, performance is often considered "amazing".

rsyslog features

  • Multithreading
  • UDP, TCP, SSL, TLS, RELP
  • MySQL, PGSQL, Oracle realize log storage
  • Powerful filter, can filter any part of the log information
  • Custom output format
  • Suitable for enterprise-level relay chain
rsyslog

rsyslog is the system's own service

  • Rsyslog has been inherited when the system is installed
[[email protected] ~]# rpm -qi rsyslog
Name        : rsyslog
Version     : 8.1911.0
Release     : 6.el8
Architecture: x86_64
Install Date: Mon 31 May 2021 06:55:55 PM CST
Group       : System Environment/Daemons
Size        : 2428362
License     : (GPLv3+ and ASL 2.0)
Signature   : RSA/SHA256, Tue 21 Jul 2020 09:42:03 AM CST, Key ID 05b555b38483c65d
Source RPM  : rsyslog-8.1911.0-6.el8.src.rpm
Build Date  : Tue 21 Jul 2020 09:33:16 AM CST
Build Host  : x86-02.mbox.centos.org
Relocations : (not relocatable)
Packager    : CentOS Buildsys <[email protected]>
Vendor      : CentOS
URL         : http://www.rsyslog.com/
Summary     : Enhanced system logging and kernel message trapping daemon
Description :
Rsyslog is an enhanced, multi-threaded syslog daemon. It supports MySQL,
syslog/TCP, RFC 3195, permitted sender lists, filtering on any message part,
and fine grain output format control. It is compatible with stock sysklogd
and can be used as a drop-in replacement. Rsyslog is simple to set up, with
advanced features suitable for enterprise-class, encryption-protected syslog
relay chains.
  • Package: rsyslog
  • Main program: /usr/sbin/rsyslogd
  • CentOS 6:/etc/rc.d/init.d/rsyslog {start|stop|restart|status}
  • CentOS 7,8:/usr/lib/systemd/system/rsyslog.service
  • Configuration files: /etc/rsyslog.conf, /etc/rsyslog.d/*.conf
  • Library file: /lib64/rsyslog/*.so

rsyslog configuration file

  • /etc/rsyslog.conf
cat /etc/rsyslog.conf | sed -n '/^[^#]/p'
module(load="imuxsock" 	  # provides support for local system logging (e.g. via logger command)
       SysSock.Use="off") # Turn off message reception via local log socket; 
			  # local messages are retrieved through imjournal now.
module(load="imjournal" 	    # provides access to the systemd journal
       StateFile="imjournal.state") # File to store the position in the journal
input(type="imudp" port="514")
input(type="imtcp" port="514")
global(workDirectory="/var/lib/rsyslog")
module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")
include(file="/etc/rsyslog.d/*.conf" mode="optional")
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
authpriv.*                                              /var/log/secure
mail.*                                                  -/var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 :omusrmsg:*
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot.log

Configuration file content:

It consists of three parts

  • MODULES: Related module configuration
  • GLOBAL DIRECTIVES: global configuration
  • RULES: Rules configuration related to logging

Module

#### MODULES ####

module(load="imuxsock" 	  # provides support for local system logging (e.g. via logger command)
       SysSock.Use="off") # Turn off message reception via local log socket; 
			  # local messages are retrieved through imjournal now.
module(load="imjournal" 	    # provides access to the systemd journal
       StateFile="imjournal.state") # File to store the position in the journal
#module(load="imklog") # reads kernel messages (the same are read from journald)
#module(load"immark") # provides --MARK-- message capability

# Provides UDP syslog reception
# for parameters see http://www.rsyslog.com/doc/imudp.html
#module(load="imudp") # needs to be done just once
input(type="imudp" port="514")

# Provides TCP syslog reception
# for parameters see http://www.rsyslog.com/doc/imtcp.html
#module(load="imtcp") # needs to be done just once
input(type="imtcp" port="514")
  • Decide which modules to load, load those that are needed, and not load those that are not needed
rpm -ql rsyslog | grep imux 
/usr/lib64/rsyslog/imuxsock.so

Global Settings

Working path, configuration file path, module format

#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
global(workDirectory="/var/lib/rsyslog")

# Use default timestamp format
module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")

# Include all config files in /etc/rsyslog.d/
include(file="/etc/rsyslog.d/*.conf" mode="optional")

rule

  • Rules are the core of the log
  • It specifies what kind of log to put
#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log
  • Log file with the most content: var/log/messages
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

Including any type of info and above, except mail, authpriv, cron, which are all written in /var/log/messages

The horizontal line in front of the folder-indicates the asynchronous mechanism, not to write to the disk immediately, but to write to the buffer after a while, improve performance, single security has hidden dangers

Description of configuration format

Configure Priority priority level format

*: 表示所有级别
none:没有级别,即不记录
PRIORITY:指定级别(含)以上的所有级别
=PRIORITY:仅记录指定级别的日志信息

Configure target log format

文件路径:通常在/var/log/,文件路径前的-表示异步写入
用户:将日志事件通知给指定的用户,* 表示登录的所有用户
日志服务器:@host,把日志送往至指定的远程UDP日志服务器 @@host 将日志发送到远程TCP日志服务器
管道: | COMMAND,转发给其它命令处理

Display format of the log file

  • There are many log files, such as: /var/log/messages, cron, secure, etc.,
  • The basic format is similar. The format is as follows:
事件产生的日期时间 主机 进程(pid):事件内容
  • View system security log
[[email protected] ~]# tail /var/log/secure 
May 31 18:32:13 C8-192 sshd[30815]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.0.0.88
Jun  1 17:12:42 C8-192 sshd[821]: Server listening on 0.0.0.0 port 22.
Jun  1 17:12:42 C8-192 sshd[821]: Server listening on :: port 22.
Jun  1 17:12:42 C8-192 polkitd[799]: Loading rules from directory /etc/polkit-1/rules.d
Jun  1 17:12:42 C8-192 polkitd[799]: Loading rules from directory /usr/share/polkit-1/rules.d
Jun  1 17:12:42 C8-192 polkitd[799]: Finished loading, compiling and executing 2 rules
Jun  1 17:12:42 C8-192 polkitd[799]: Acquired the name org.freedesktop.PolicyKit1 on the system bus
Jun  1 17:26:36 C8-192 sshd[1704]: Accepted publickey for root from 10.0.0.88 port 49324 ssh2: RSA SHA256:SkkJUczJ2TjwOv/dIQbqe5s9mQlhDLk+YXeNiOK2Fs0
Jun  1 17:26:36 C8-192 systemd[1707]: pam_unix(systemd-user:session): session opened for user root by (uid=0)
Jun  1 17:26:36 C8-192 sshd[1704]: pam_unix(sshd:session): session opened for user root by (uid=0)

Log configuration example

Establish custom logging for ssh service

  • The default sshd service log is written into /var/log/messages and the corresponding level system log
  • We can record the sshd service to a custom directory by modifying the configuration file

Modify the configuration file of the sshd service

  • Find the sshd configuration file and modify the relevant content of the log
sed -ri.bak '/^SyslogFacility/a SyslogFacility Local2' /etc/ssh/sshd_config

Modify the configuration file of rsyslog

  • Add custom local2 logging location
echo -e "#sshd.log\nLocal2.* /var/log/sshd.log" >> /etc/rsyslog.conf

Restart the service to take effect

service sshd reload && systemctl restart rsyslog

Write log to test

[[email protected] ~]# cat /var/log/sshd.log
Jun  1 23:59:40 C8-192 root[2734]: i am sshd.log
Jun  2 00:03:56 C8-192 sshd[2994]: Server listening on 0.0.0.0 port 22.
Jun  2 00:03:56 C8-192 sshd[2994]: Server listening on :: port 22.
Jun  2 00:04:05 C8-192 sshd[2994]: Received signal 15; terminating.
Jun  2 00:09:43 C8-192 sshd[3132]: Accepted publickey for root from 10.0.0.88 port 49360 ssh2: RSA SHA256:SkkJUczJ2TjwOv/dIQbqe5s9mQlhDLk+YXeNiOK2Fs0
Jun  2 00:09:45 C8-192 sshd[3135]: Received disconnect from 10.0.0.88 port 49360:11: disconnected by user
Jun  2 00:09:45 C8-192 sshd[3135]: Disconnected from user root 10.0.0.88 port 49360
Jun  2 00:09:46 C8-192 sshd[3159]: Accepted publickey for root from 10.0.0.88 port 49362 ssh2: RSA SHA256:SkkJUczJ2TjwOv/dIQbqe5s9mQlhDLk+YXeNiOK2Fs0
Jun  2 00:09:48 C8-192 sshd[3162]: Received disconnect from 10.0.0.88 port 49362:11: disconnected by user
Jun  2 00:09:48 C8-192 sshd[3162]: Disconnected from user root 10.0.0.88 port 49362