Deploy Redis: Configure memory strategy

table of Contents

Key expiration delete method

Configure memory policy

Memory elimination strategy

As a lightweight memory-based data structure service, Redis stores all data in memory. Although RDB or AOF persistence writes data to disk, Redis uses memory space in the actual operating environment. It is always larger than the space it persists to disk, and the memory cost is also relatively expensive, so we should make some configuration of its memory strategy.

Key expiration delete method

In fact, in terms of saving Redis memory, we have summarized a method in the previous log, which is to set the expiration time of the key, and bind an expiration time to the key through the EXPIRE command, so that we do not need to actively call the DEL command to delete the key. After the key is passed, it will be automatically recycled by Redis . There are three recycling methods for expired keys: regular deletion, passive deletion and periodic deletion.

Timed deletion, that is, strictly enforced according to the key expiration time we set, the key will be cleared immediately after expiration. The advantage of using this method is that it can maximize memory savings, because useless keys can be recycled immediately. However, the disadvantage of timed deletion is that if a large number of keys expire in a certain period of time, Redis will take a lot of time to delete them, resulting in low server efficiency during that period.

Passive deletion. In this way, when the key we set expires, Redis will not delete it immediately, but will continue to store it in memory. The next time our client accesses this expired key, Redis will It is deleted from the memory, that is, when the key is used, we will check whether it has expired. The effect of this method is almost the opposite of passive deletion. Since the deletion of the key is not processed immediately, it will not cause too much pressure on the server. However, when a key expires and has not been used for a long time, it will remain inactive. In the memory, with the accumulation of such expired keys that have not been accessed, it will still cause a certain amount of memory waste, and even eventually lead to memory leaks.

The third type is to delete regularly, every certain time (for example, 10 seconds, you can modify it in the configuration file and find the " hz " parameter in redis.conf ) Redis will randomly select 20 keys with an expiration time from the expires dictionary scan, if found to have expired keys will immediately be deleted when selecting the 20 keys More than 25% of the key is deleted, then randomly select 20 keys repeat the above operation. The advantage of this method is that it is a balanced solution of the above two methods, which will not occupy the CPU for a long time , and it can also clear out the expired keys regularly to prevent them from being missed in the memory. Of course, there are still shortcomings, this is how to control the frequency of periodic deletion? If the setting is too frequent, the CPU occupancy is still quite high. If the setting interval is too long, then there may still be more and more expired keys.

At this time, we can change our angle and turn to configuring Redis 's memory strategy.

Configure memory policy

Although we can set the key timeout period to allow Redis to recycle these no longer used keys to save memory usage, the above strategy will inevitably cause the memory to be full, or the stored data is very large, which leads to direct memory. When the memory is full, we need to configure the memory elimination strategy, that is, how does Redis remove some data when the memory is full.

First, we use the command INFO MEMORY to view some current Redis memory usage information:

You can look at the three highlighted parts. The first used_memory indicates that a total of 827,504 bytes of memory are allocated in Redis. The second maxmemory indicates the maximum memory space limit of Redis. Note that the value here is 0. This means that there is no maximum memory space limit, and Redis can use all the available space on the physical host. The third maxmemory_policy is the key point. It represents Redis's elimination strategy when the memory space is full, that is, delete some of the keys to release the memory. The memory elimination strategy in the above figure is noeviction, which means not to eliminate the keys, and do it again when the memory is full When a new key is created, an error will be returned directly.

Memory elimination strategy

For Redis's memory elimination strategy, we can use the command config set maxmemory-policy to set. Before configuration, let's take a look at some of the memory elimination strategies provided by Redis:

  1. noeviction: The key is not eliminated. When the memory space is full, writing will return an error.
  2. allkeys-lru: The least recently used method. When the memory space is full, the least recently used key in the memory will be eliminated, that is, the key that has not been accessed for the longest time.
  3. Volatile-lru: Similar to the least recently used, but when the memory space is full, the least recently used key is eliminated from the keys with an expiration time.
  4. allkeys-lfu: The least commonly used method recently. When the memory is full, the keys that have been accessed the least frequently will be eliminated.
  5. Volatile-lru: Similar to the above, when the memory is full, the key with the least number of access times is eliminated from the keys with the expiration time set.
  6. allkeys-random: When the memory is full, the keys in the memory are randomly eliminated. It can be any key, no matter if you use it frequently or infrequently.
  7. Volatile-random: similarly, after the memory is used up, any key is eliminated from the keys with an expiration time.
  8. volatile-ttl: Among the keys with expiration time set, the keys that are about to expire are eliminated.

As shown in the figure above, we can set maxmemory-policy to allkeys-lru, which means that when the memory is full, use the least recently used method to eliminate random keys.