[Redis reading notes] 3. Redis cluster

One, master-slave replication

Redis's master-slave replication steps are divided into: synchronization and command propagation.

  • Synchronization (PSYNC) : Synchronize the data of the master server to the slave server.
  • Command propagation: The newly generated commands during the synchronization process are synchronized to the slave server again.
Insert picture description here
  1. The client sends the SLAVEOF command to the slave server, requesting master-slave replication.
  2. The slave server sends a PSYNC command to the master server to request synchronization.

If it is the first synchronization:

  1. The main server executes the BGSAVE command, generates an RDB file in the background, and records all write operations in a buffer.
  2. After the BGSAVE command is executed, the master server sends the RDB file to the slave server, and loads the RDB file after receiving the file from the server.
  3. The master server sends the write command in the buffer to the slave server, and the slave server executes the write command.
  4. When the master server receives the write command again, it will synchronize to the slave server to ensure consistent status.

If it is not the first synchronization:

  1. Both the master and slave servers maintain a replication offset. By comparing the replication offsets of the master and slave servers, you can know whether they are consistent.
  2. The master-slave server has a first-in, first-out copy backlog buffer. When a write command is written into the buffer of the master server, it is also sent to the buffer of the slave server.
  3. When executing the PSYNC command, if the offset of the slave server is in the buffer of the main service, partial synchronization is performed; if it is not in the buffer of the main service, full synchronization is performed.
  4. The master-slave server program will generate a new running ID every time it starts. During the first synchronization, the master server will give the running ID to the slave server, and the slave server will save it. When the connection is disconnected and reconnected, it will compare the saved running ID with the current one. Whether the running ID of the main server is consistent.
  5. If they are consistent, partial synchronization can be performed. If they are inconsistent, it means that the main server has been restarted during the disconnection, and then a complete synchronization can be performed.

Two, high availability

Redis uses Sentinel to solve high availability problems.

  1. Sentinel to monitor multiple master servers and his slave servers.
  2. If the master server goes offline, one of the following slave servers will be automatically set to go online.
  3. Then execute the copy command to synchronize all the slave servers with the new master server.
  4. Continue to monitor the original master server, if it is resurrected, set it as a slave service area.
Insert picture description here

Three, cluster data structure

Insert picture description here

Four, slot assignment

The Redis cluster saves key-value pairs in the database through sharding. The entire database of the cluster is divided into 16384 slots, and each key in the database belongs to one of the 16384 slots. Each node can handle 0 or more slots of data.

If 16384 slots are processed by nodes, the cluster is online, otherwise, it is offline. If there is a slot's data and no node is processing, then the cluster is considered offline.

The clusterNode structure of each node records which write slots those nodes handle:

struct clusterNode{
    unsigned char slots[16384/8];
    int numslots´╝Ť
}

numslots records the number of slots that cluterNode is responsible for processing. The slots array is a binary bit array that records the information of each slot. 1 represents the node processing this slot, and 0 represents the node not processing this slot. Divide by 8 because a char has 8 bits.

Each node uses clusterState to record the state of the cluster, and the slots array in the clusterState structure records the assignment information of all slots in the cluster.

typedef struct clusterState{
    clusterNode *slots[16384];
}