(An interview trick every day) Redis persistence mechanism (AOF and RDB) and how to achieve the best persistence in the production process must be asked

First of all, I wish the children a happy Children’s Day

1. RDB mechanism

RDB actually stores data in the form of snapshots. Presumably everyone should know snapshots. Similar to the storage of a snapshot of a photo or a certain period of time, it is conceivable that the problem of data loss cannot be avoided. Basic The production environment on the above will not be recommended separately

1.1 RDB advantages

  1. The RDB file is compact and fully backed up, which is very suitable for backup and disaster recovery.
  2. When generating RDB files, the redis main process will fork() a child process to handle all saving work, and the main process does not need to perform any disk IO operations.
  3. RDB is faster in restoring large data sets than AOF.

1.2 RDB disadvantages

Roughly the same as the description, some data will be lost (data after the snapshot will be lost)

2. AOF mechanism

AOF is different, each received command will be appended to the file through (write, append).

2.1 Advantages of AOF

  1. Compared with RDB, the advantages of AOF can better ensure the integrity of the data. Insert operation is performed every 1s and the write performance is high.
  2. When the AOF log file is too large, there will be a rewrite operation but it will not affect the client's read and write
  3. The commands of the AOF log file are recorded in a very readable manner. This feature is very suitable for emergency recovery of catastrophic accidental deletion. For example, if someone accidentally emptied all the data with the fluxhall command, as long as the background rewrite has not occurred at this time, then you can copy the AOF file immediately, delete the last fluxhall command, and then put the AOF file back. Through the recovery mechanism, automatically recover all data

2.2 Disadvantages of AOF

  1. For the same data, AOF will be larger than RDB file
  2. After AOF is turned on, the supported write QPS will be lower than the write QPS supported by RDB, because AOF is generally configured to fsync log files once per second (of course, once fsync per second, the performance is still very high)

How to achieve the optimal persistence of the production process?

Most interviewees will say that it is one of AOF or RDB, but it is obviously not optimal.
Instead, hybrid persistence should be used for persistence operations (released in Redis 4.X version)
and when mixed persistence is turned on After

  1. When AOF is rewritten, it is no longer simply converting the memory data into RESP commands and writing the AOF file, but the RDB snapshot processing of the memory before the rewriting moment
  2. And the RDB snapshot content and the incremental AOF command to modify the memory data exist together, and they are written into the new AOF file. The new file is not called appendonly.aof at the beginning. It will be renamed after the new AOF file is rewritten. Overwrite the original AOF file and complete the replacement of the new and old AOF files.
  3. Therefore, when Redis restarts, you can load the RDB content first, and then replay the incremental AOF log, which can completely replace the previous AOF full file replay, so the restart efficiency is greatly improved.

Hybrid persistent file format

Insert picture description here