Problems encountered by large concurrent servers and simple response framework ideas

Article Directory

Server design goals

  • High Performance (HP);
  • High availability (High Availability, HA);
  • Scalability;

Distributed design issues

  • Load balancing;
  • Distributed storage
  • Distributed Computing;

Server performance bottlenecks

Data copyCause high IO usage, use cache to alleviate
Environment switchProcess/thread context switching will take up more machine cycles, so it is not good to create threads blindly, that is, create threads rationally:
①Single-core server: using finite state machine is the most efficient;
②Multi-core server: multi-threading can give full play to efficiency And performance;
Memory allocationUse memory pool as a unified memory resource management
Lock contentionHow to reduce waiting and empty consumption caused by locks, and how to avoid deadlocks and lock competitions

Large concurrent server design and iterative update ideas

Large concurrent server design

Large-scale website architecture evolution process, iterative update ideas

  • Step1 : Web (eg HTTP) Server and Application (eg DataBase) Server are separated.
    If there is only one server, the front and back ends are deployed on the same server, and everything is damaged. Separate to ensure high availability (HA);
  • Step2 :
    ① Web resource separation:
    static resources (html, js, css, img and other disk files), static requests are responded to by the front-end server (HTTP Server, eg Apache/Nginx/...);
    dynamic resources (jsp, php, etc.) Execute files or business operations), dynamic requests are responded to by the back-end server (Application Server, eg JBoss/Tomcat/...);
    ② Cache processing (cacheable location):
    (1) Client (Browser) caches, reducing the need for websites Access;
    (2) Front Page Cache (Front Page Cache), which reduces requests to Web application servers;
    (3) Page Fragment Cache (Edge Side Includes, ESI), which can also cache non-dynamic content in dynamic pages to reduce The requested workload;
    (4) Local data caching to perform database data, reducing Web Server queries to the database;
  • Step3 : Web Server cluster + read-write separation:
    (1) Add a Web server at the front end and add a Load Balance mechanism (eg Task Server);
    (2) If the front end is under pressure and the back end is under pressure, the back-end server (eg DB ) Perform read-write separation (replication mechanism, master-slave database, while the master database replicates and updates the synchronized slave database), and at the same time, the access to the back-end server needs to add a DAL layer;
  • Step4 : CDN (Content Distribution Net), distributed cache, sub-database and sub-table:
    (1) Content distribution enables access requests from different regions to be accepted nearby, improves user experience, and supports operators to record in different modes;
    (2) ) Distributed caching allows data on all cache servers to be shared, improving cache query efficiency. Currently popular distributed caching schemes include: memcached, membase, redis, etc. Basically, the current NoSQL schemes can be used for distributed caching Solution;
    (3) With the development of the business, the database tables will become very large. At this time, the DB Server is divided into databases and tables. Generally, horizontal partitioning is adopted to improve the database query speed;
  • Step5 : Multiple data centers + distributed storage and computing:
    NoSQL has higher concurrency and is much cheaper than SQL. For data that does not require high consistency, NoSQL can be used for storage; SQL maintains better data consistency. Focus on transactional operations, but will also encounter problems such as locks, large table joins and other issues that affect DB performance;
    distributed storage must have a dedicated file system to ensure rapid indexing of files and full and effective use of storage ( Dedicated small file system, large file index system, etc.), there will also be a dedicated file system inside the commercial database, and there will also be a corresponding distributed computing framework compatible with the distributed file system, as follows:

Through the above steps, there are the following large-scale website architecture diagrams:

For the customer service server in the above figure, it may not be flat, but hierarchical. There is a cluster for various dedicated services to provide corresponding services; there will be a NoSQL database in the data center to respond to some query services to reduce the pressure of SQL DB.

List of Load Balance methods

Load balancing locationmethod
Front-end load balancingDNS load balancing :
In the DNS server, the same name can be configured for multiple different addresses, and different clients can access the same name to obtain different addresses;

reverse proxy : use a proxy server to send requests to internal servers, Let the proxy server evenly forward the request to one of the multiple internal Web servers, so as to achieve the purpose of load balancing. (The standard proxy method is that customers use a proxy to access multiple external Web servers, and this proxy method is that multiple customers use a proxy to access internal Web servers, so it is called reverse proxy mode);

NAT-based load balancing NAT (Network Address Translation) protocol, which enables internal LAN IP to access external data (application layer protocol);

LVS : a load balancing open source software;

F5 hardware load balancing : expensive;
Application server load balancingA task server is added to record the load of each client server :
(1) The application server receives tasks passively : the task server distributes tasks to the application server with lower load according to the load of each application server;
(2) The application server takes the initiative Receiving tasks: The application server feels that it can carry more load, so it proactively applies to the task server. But it is not very friendly for dedicated client servers;
(3) Generally there are redundant backup task servers with backup for failover, which communicate through heartbeat to ensure HA;
Database load balancingReplication mechanism : master-slave expansion (scalability), separation of reading and writing;

sub-database sub-table :
vertical partition (sub-database) : database and paging according to business table type;
horizontal partition (sub-table) : cut the entire table horizontally, Each library table has various services and is a common mechanism in display
Vertical partition (sub-library)
Horizontal partition (sub-table)

The existence of sub-databases and sub-tables needs to be searched and indexed with DAL, and hash indexes are commonly used, which puts forward new requirements for the access design of the library tables.