- One, master-slave replication
- Environment construction (one area N structure)
- Second, the sentinel mode
- Environment setup (a sentinel)
- 3. Cache penetration, cache breakdown, cache avalanche
- 1. Cache penetration
- 2. Cache breakdown
- 3. Cache avalanche
One, master-slave replication
One master server master, multiple slave servers slave. Main functions: data redundancy, failure recovery, load balancing, and high-availability clusters.
The master can write, but the slave node cannot write and can only read. The information in the master will be saved by the slave node. The data of the slave node is still retained when the master node is down, and the identity of the slave node remains unchanged. After the master node recovers, the slave node Can continue to connect
View the basic information of master-slave replication
127.0.0.1:6379> info replication # Replication role:master # 当前角色master connected_slaves:0 # 从机0个 master_failover_state:no-failover master_replid:f298b23a9266dd0860de5be3d776ada013e40e86 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
Environment construction (one area N structure)
1. Start three Redis containers first
[email protected]:~# docker run -p 6379:6379 -d --name="redis00" bc8d70f9ef6c 2e76a9ca22c00e027d55d971b0d5bc265d16ad33f7891733b87fcbfcda592eef [email protected]:~# docker run -p 6389:6389 -d --name="redis01" bc8d70f9ef6c 51610b97d4cbdf745d1514800f3a00f5ffc7ea1d4feb23a4e53781bfb50777d0 [email protected]:~# docker run -p 6399:6399 -d --name="redis02" bc8d70f9ef6c 1fcda7dd77cba51aa036591fe1a17dfb672f168e7b4d06ca15bbee17414329af [email protected]:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1fcda7dd77cb bc8d70f9ef6c "docker-entrypoint.s…" 3 minutes ago Up 3 minutes 6379/tcp, 0.0.0.0:6399->6399/tcp redis02 51610b97d4cb bc8d70f9ef6c "docker-entrypoint.s…" 4 minutes ago Up 3 minutes 6379/tcp, 0.0.0.0:6389->6389/tcp redis01 2e76a9ca22c0 bc8d70f9ef6c "docker-entrypoint.s…" 7 seconds ago Up 5 seconds 0.0.0.0:6379->6379/tcp redis00
2. Copy the official redis.conf file from the host to the three containers
[email protected]:~# docker cp ./redis.conf 2e76a9ca22c0:/usr/local/bin/ [email protected]:~# docker cp ./redis.conf 51610b97d4cb:/usr/local/bin/ [email protected]:~# docker cp ./redis.conf 1fcda7dd77cb:/usr/local/bin/
3. Enter the container to modify the configuration file, taking the redis00 machine as an example:
[email protected]:~# docker exec -it 2e76a9ca22c0 /bin/bash [email protected]:/data/# cd /usr/local/bin [email protected]:/usr/local/bin# ls docker-entrypoint.sh redis-check-aof redis-sentinel gosu redis-check-rdb redis-server redis-benchmark redis-cli redis.conf # 为了好辨识各个机器，配置文件名为：redis+机器编号+端口号.conf # 所以三个机器各名为：redis0079.conf、redis0189.conf、redis0299.conf [email protected]:/usr/local/bin# mv redis.conf redis0079.conf
In order to edit files, view ip information and ping the test network in the container, you also need to install tools. All three machines execute this command
[email protected]:/usr/local/bin# apt update && apt install -y vim && apt install -y iproute2 && apt install -y iputils-ping
The configuration file redis0079.conf of the machine redis00 is modified as follows
port 6379 ==改为==> port 6379 daemonize no ==改为==> daemonize yes pidfile /var/run/redis_6379.pid ==改为==> pidfile /var/run/redis_6379.pid logfile "" ==改为==> logfile "6379.log" dbfilename dump.rdb ==改为==> dbfilename dump6379.rdb
The configuration file redis0189.conf of the machine redis01 is modified as follows
port 6389 ==改为==> port 6389 daemonize no ==改为==> daemonize yes pidfile /var/run/redis_6389.pid ==改为==> pidfile /var/run/redis_6389.pid logfile "" ==改为==> logfile "6389.log" dbfilename dump.rdb ==改为==> dbfilename dump6389.rdb
The configuration file redis0299.conf of the machine redis02 is modified as follows
port 6399 ==改为==> port 6399 daemonize no ==改为==> daemonize yes pidfile /var/run/redis_6399.pid ==改为==> pidfile /var/run/redis_6399.pid logfile "" ==改为==> logfile "6399.log" dbfilename dump.rdb ==改为==> dbfilename dump6399.rdb
4. Start redis
[email protected]:/usr/local/bin# redis-server /usr/local/bin/redis0079.conf
[email protected]:/usr/local/bin# redis-server /usr/local/bin/redis0189.conf
[email protected]:/usr/local/bin# redis-server /usr/local/bin/redis0299.conf
View process information on the host
At this time, each redis service is the master node
5. Configure the slave node
The master node does not need to be configured, and the slave node configures the master node address
Get the ip address of the master node first
Then use the **
slaveofhost port ** command to set the address of the master node
The above configuration using commands is only temporary, and the modification of the configuration file is permanent
In the redis0189.conf file of redis01 and the redis0299.conf file of redis02
# replicaof <masterip> <masterport>改为replicaof 172.18.0.4 6379
Then restart the redis service of redis01 and redis02, you can
If you want to cancel the slave identity, just comment out the configuration file mode, and you can use the command in the command mode
slaveof no one
Second, the sentinel mode
In master-slave replication, once the master node goes down, other nodes will not change, so the entire cluster cannot provide external services. The sentinel mode is to increase N sentinels, monitor the hosts of the entire cluster, and elect a master node. Once the master node goes down, another master node is newly elected.
The redis-sentinel tool in the redis installation directory acts as a sentinel. Generally, there are multiple sentinels, because if one sentinel is down, the cluster will also be down.
Environment setup (a sentinel)
Still use the environment of master-slave replication, one master node, two slave nodes, and then continue the following configuration
1. Modify the redis0079.conf, redis0189.conf, redis0299.conf files, and the modified content is the same
protected-mode yes# 改为，关闭安全模式，允许无密码访问protected-mode no
bind 127.0.0.1 -::1# 改为，监听所有地址bind * -::*
2. Add a configuration file on any node.
sentinel.confAnd then configure
# 哨兵 监视 监视名 地址 端口 票数sentinel monitor sentinel01 172.18.0.4 6379 1
3. Run the sentry tool
[email protected]:/usr/local/bin# redis-sentinel sentinel.conf
Shut down the master, a few seconds later, one of the slave nodes will be elected as the master node
Even if the host that just went down is restored, it can only be used as a slave node
3. Cache penetration, cache breakdown, cache avalanche
1. Cache penetration
We generally use the cache because the database query is slow and consumes more resources, which means that the data is stored in the memory, but because there is too much data or other reasons, not all the data in the database is put in the memory, so we will Set up a strategy, if you can't find the data in the cache, look for it in the database. Now there is a hacker who forges a large number of requests, and the requested data is not in the cache, so the background programs will all look in the database, the cache loses its meaning, and the database will be under a lot of pressure.
The request passes through the cache and reaches the database. This is cache breakdown.
There are many types, two of which are most commonly used:
- Because of forged requests or due to code bugs, requests generally have the same key multiple times and the value is null, and the same key must be queried in the database every time, but the partial check is meaningless because it always returns null. So we can set the requested key to null and store it in the cache
- Use Bloom filter (commonly used)
Principle of Bloom filter :
First, there is a bitmap, we can imagine it as an array, it has a length, a subscript, and the stored value can only be 0 or 1.
Then use different hash functions to calculate the key of our data, which can be 3 hash functions
Then calculate the remainder of the obtained hash value to the length of the bitmap, and the result of the remainder is the subscript of the bitmap
Then, after the remainder of the three results, the value of the corresponding bitmap subscript is set to 1, which means that the key exists
Then as the key increases, some positions in the bitmap may overlap, but it doesn’t matter
When the last request comes, calculate the hash value of the requested key, find the corresponding bit subscript, determine whether the key exists, and access the cache if it exists
If the bitmap subscript value corresponding to the result of a requested key calculation is 1, then the key may or may not exist, because there is a probability that different keys have the same hash value, and 3 hash values are still calculated
If the bitmap subscript value corresponding to the result of a requested key calculation is 0, then the key must not exist
Therefore, the Bloom filter has a certain probability of misjudgment, but we can accept it.
2. Cache breakdown
When we store data, we will take into account that some keys may not have a large amount of access, and they will waste space if they are kept, and all keys have an expiration time. Even if they expire, they can be stored in the cache after the request is processed again. in. If a key suddenly becomes hot data and tens of millions of requests are aimed at this key, and the key just reaches the expiration time, then tens of millions of requests will access the database at the same time because they cannot get the results in the cache. After the database crashes, the keys in the cache gradually become invalid over time, but the database cannot get the value, then the keys gradually become invalid, and finally the service ends.
At the moment when a key expires, a large number of requests hit the database, causing the database to go down, and then the same applies to each key, which is an avalanche.
There is no good solution:
- Lock the key resource in the program so that only one thread or process can access it at a time. Of course, queues can also be used.
3. Cache avalanche
Basically similar to cache breakdown, but cache breakdown is for the instant when a key fails, and cache avalanche is for the moment when multiple keys fail at the same time
- Fuse mechanism