Redis expiration strategy, elimination mechanism

1. What is the use of Redis to set an expiration time for cached data?

The memory is limited. If all the data in the cache is kept all the time, it is easy to Out Of Memory.

Redis comes with the function of setting the expiration time for cached data:

192.168.1.1>exp key 60 #数据在60s后过期
(integer)1
192.168.1.1>setex key 60 value #数据在60s后过期(setex:set+expire)
OK
192.168.1.1>ttl key #查看数据还有多久过期
(integer)56

Note: In Redis, except for the string type which has its own unique command setex to set the expiration time, other methods need to rely on the expire command to set the expiration time.
In addition, the persist command can remove the expiration time of a key.

What else does the expiration time do?

In many cases, it is common for businesses to only need certain data to exist in a certain period of time. For example, our SMS verification code is only valid for one day.

If you use a traditional database for processing, you usually judge the expiration by yourself, which is troublesome and poor performance.

2. How does Redis judge whether the data is out of date?

There is a hash table called expiration dictionary in Redis to store data expiration time.
The key of the expired dictionary points to a key in the Redis database, and the value of the expired dictionary is an integer of type long. This integer holds the expiration time of the database key pointed to by the key (UNIX timestamp with millisecond precision).

Insert picture description here


The expired dictionary is stored in the structure of redisDb:

typedef struct redosDb{
	...
	
	dict *dict;//数据库键空间,保存这数据库中所有的键值对
	dict *expries;//过期字典,保存这键的过期时间
}redisDb

3. Do you understand the deletion strategy of expired data?

  1. Lazy deletion: The data will be checked for expiration only when the key is taken out, which is the most CPU-friendly, but may cause too many expired keys to not be deleted
  2. Periodic deletion: extract a batch of keys at regular intervals to delete expired keys. And the bottom layer of Redis will reduce the impact of the delete operation on the CPU by limiting the duration and frequency of the delete operation.

Periodic deletion is more friendly to memory, and lazy deletion is more friendly to CPU.
Both have their own merits, so Redis uses regular deletion + lazy/lazy deletion.

However, just setting the expiration time for the key is still problematic, because there may still be cases where many expired keys are missed by regular deletion and lazy deletion, which causes a large number of keys to accumulate in memory and then Out Of Memory.
How to solve this problem?
It depends on Redis's memory elimination strategy.

4. Do you understand the Redis memory elimination mechanism?

Related question: There are 2000w data in MySQL, and only 20W data in Redis. How to ensure that the data in Redis are all hot data?

Redis provides 6 data elimination strategies:

  1. Valtile-lru (least recently used): Select the least recently used data from the data set with the expiration time set to eliminate it.
  2. volatile-ttl: select the data to be expired from the data set that has set expiration time
  3. Volatile-random: arbitrarily select data to eliminate from the data set that has set expiration time
  4. allkeys-lru (least recently used): When the memory is not enough to accommodate the newly written data, in the key space, remove the least recently used key
  5. allkeys-random: arbitrarily select data from the data set to eliminate
  6. no-eviction: Prohibit eviction of data, that is, when the memory is not enough to hold the new data, the new write operation will report an error.

After 4.0, two types have been added:
7. volatile-lfu (least frequently used): select the least frequently used data from the data set with an expiration time set to eliminate
8. allkeys-lfu (least frequently user): when the memory is not enough to accommodate When new data is written, in the key space, remove the least frequently used key