[Efficient coding] Can't remember the log query command? It's okay, this article will help you remember

Hello, I’m the code farmer Fei brother , I’ve always wanted to fly, and I’m running that old man for the time being.

Areas of expertise: Java high-concurrency programming, MySQL database The
focus of this article: record various commands for viewing logs in daily work.
It is full of dry goods, it is recommended to collect it, and you need to check it often when you need it. Friends who have questions and needs, welcome to leave a message~~~.

Article Directory


There are two log files, normal.log file and normal-05-29-2021-1.log.gz

Find the log file

Sometimes we only know the name of the log file, but do not know the address of the log file. In this case, how to find the log file? There are two commands that can be used here: one is the path command; the other is the locate command. The difference between these two commands is that the path command searches for a specific directory; the locate command searches a database (/var/lib/locatedb), which contains local information.
In this example, search: the location of the normal.log file:

Use the path command

The basic syntax of the path command is: find < path > < expression > < cmd >
where: path: the directory to be searched and all its subdirectories, the default is the current directory
expression: the characteristics of the file to be searched.
cmd: Perform specific processing on search results.

find / -name normal.log -type f

This means the command is in the root directory / down, find the named normal.logfile, f denotes find a file. The search results are:

Insert picture description here

Use locate command

The use format of locate command islocate [选项] [名称]

locate -r normal.log$

Find all files ending in normal.log. The search results are as follows:

Insert picture description here

Basic log viewing commands:

head command

The head command can be used to view the content of the beginning of the file, and another commonly used parameter -n is used to display the number of lines, the default is 10, which displays the content of 10 lines.

1. Command format:

head [参数] [文件]

2. Command parameters, the meaning of various operations:

parametermeaning
-qHide file name
-vShow file name
-c<tree>Display the number of bytes
-n<number of lines>Display the number of lines

3. Examples

  1. The command to display the first 10 lines of the normal.log file is:
head normal.log
  1. The command to query the first 20 lines of the normal.log file is:
head -n 20  normal.log
  1. Query all logs except the last 20 lines of the normal.log log file;
head -n -20  normal.log

tail command

The tail command can be used to view the contents of a file. There is a commonly used parameter -f that is often used to view the log file that is changing. tail -f filenameThe last content in the filename file will be displayed on the screen and refreshed continuously. As long as the filename is updated, you can see the latest file content.

1. Command format

tail [参数] [文件]  

2. Command parameters

parameterDescription
-fLoop reading
-qDo not display processing information
-vDisplay detailed processing information
-c<number>Number of bytes displayed
-n<number of lines>Display the last n lines of the file

3. Examples

  1. To display the last 10 lines of the normal.log file:
tail normal.log
  1. The command to query the last 20 lines of the log at the end of the log
tail  -n  20  normal.log 
  1. Command to query all logs after 20 lines
tail -n +20 normal.log  

cat command

The cat (English: concatenate) command is used to concatenate files and print them to the standard output device.

1. Command format

cat [-AbeEnstTuv] [--help] [--version] fileName

2. Command parameters

parameterDescription
-nAll output lines are numbered starting from 1.
-bSimilar to -n, except that blank lines are not numbered.
-sWhen it encounters a blank line with more than two consecutive lines, it is replaced with a blank line.
-vUse ^ and M- symbols, except for LFD and TAB.

3. Examples

  1. Add the line number to the document content of normal.log and enter it in the document test.txt:
cat -n normal.log > test.txt

grep command

Before introducing the query command, let's first introduce the grep command. This command is an important command. This command is used to find data containing keywords, including finding qualified strings from files.

  1. Command grep [option] pattern [文件名]Format: .
  2. Command parameters, the meaning of various operations:

Command parameters

parametereffect
-?At the same time show the matching line up and down? Line, such as: grep -2 pattern filename displays the upper and lower lines of the matching line at the same time
-b—Byte-offset print the fast number in front of the matching line
-c—Count only print the number of matched lines, not the matched content
-f—File=File Extract the template from the file. The empty file contains 0 templates, so nothing matches.
-h—No-filename When searching multiple files, do not display the matching file name prefix
-i—Ignore-case ignore case differences
-q-Quiet Cancel the display, only return the exit status, 0 means that the matching line was found
-l—File-with-matches print a list of files matching the template
-n--Line-number print the line number before the matched line
-s--Silent does not display error messages about non-existent or unreadable files
-v—Revert-match reverse search, only display the unmatched lines
-w--Word-regexp If it is quoted by <and >, search the expression as a word.
-V--Version Display software version information

Regular expression:

Rule expressionmeaning
^The beginning of the anchor line, such as:'^test' matches all lines beginning with test
$The end of the anchor line, such as:'test$' matches all lines ending with grep
.Match a non-newline character, such as:'gr.p' matches gr followed by an arbitrary character, and then p.
*Match zero or more previous characters such as: '*test'match all lines with one or more spaces followed by grep.

grep usage examples

Example 1: Find the process

Find all processes whose name contains dcs
Command: ps -ef|grep dcs
Output:

[[email protected] dcs-api]# ps -ef|grep dcs
root      1527     1  0 5月27 ?       00:14:22 java -jar ai-dcs.jar
root     17456 13312  0 10:58 pts/2    00:00:00 grep --color=auto dcs

The first one is the process found; the second result is the grep process itself, not the real process.

Example 2: Find the number of specified processes

Command: ps -ef|grep -c dsc
output:

[[email protected] dcs-api]# ps -ef|grep -c dcs
3

Scenario 1: View logs according to line number ---- filter logs near keywords

After introducing the use of various commands, let's take a look at it. Usually we get very few logs with grep, and we need to check the logs near it.

  1. Get the line number of the key log:
 cat -n normal.log |grep '推送接口'

Output:

[[email protected] dcs-]# cat -n normal.log |grep '推送接口'
   290	2021.06.02 at 06:00:00 698 CST INFO [http-nio-8087-exec-3]-[traceId:] ReportController pushReport() @59 - 推送接口接收到的参数是=[]

The line number of the key log is 290 line
2. Query the log of the first 10 lines and the last 10 lines of the keyword:

cat -n normal.log |tail -n +289|head -n 20

Among them, it tail -n +289means to query all logs after line 289. head -n 20It means to check the first 20 records in the previous query results.

Scenario 2: How to query by date?

Usually we need to find logs for a specified period of time.

sed -n '/2021.06.02 at 06:00:00/,/2021.06.02 at 06:05:40/p' normal.log

What needs special explanation is: the above two dates must be the log printed out in the log, otherwise it is invalid.

  1. Regarding the printing of the date, you can first pass grep '2021.06.02 at 06:05:40' normal.logto ensure that the log can be obtained in step 4. This is a very useful command to query the log based on the time period.
  2. If there are a lot of logs we are looking for and it is inconvenient to print them on the screen, there are two ways:
    (1) Use the more and less commands, such as:, cat -n normal.log |grep '推送接口'|moreso that the pages are printed, and the pages are turned by clicking the space bar.
    (2) Use >xxx.txtand save it in a file, then you can pull down this file for analysis, such as:cat -n normal.log |grep '推送接口' >test.txt

Scenario 3: How to view the compressed log file?

In an actual production environment, log files will be compressed by time or by size. So how to query the compressed log file? The following is: normal-05-29-2021-1.log.gz log file, we can
view it through the following command:

zcat -n normal-05-29-2021-1.log.gz |grep '推送'

The output is:

[[email protected] dcs-api]# zcat -n normal-05-29-2021-1.log.gz |grep '推送'
2021.05.29 at 06:00:00 696 CST INFO [http-nio-8087-exec-1]-[traceId:] ReportController pushReport() @59 - 推送接口接收到的参数是=[]

Scenario 4: Display the size of the log file

Under normal circumstances, we llquery related logs directly through commands

Insert picture description here

However, the downside of this command is that the displayed file size is displayed in bytes, which is not very friendly. Through the command: ll -h

Insert picture description here


In order to help more Xiaobai advance Java engineers from zero, I got a set of "Java Engineers Learning and Growth Knowledge Graph" from the CSDN official side, the size is 870mm x 560mm, and there is a desk size after unfolding , It can also be folded into the size of a book. Interested friends can learn about it, and all friends are welcome to learn about it.

Insert picture description here

to sum up

This article believes that the various commands used in daily work to query logs are more than enough to meet the daily work needs.