Build and develop scaffolding from scratch to ensure the idempotence of services and prevent repeated requests

Article Directory

What is idempotence?

The result of multiple executions is the same as the result of one execution. For example, query operations are naturally idempotent.

Reason for repeated request

Let's take the following order as an example. There are generally several reasons for repeated orders:

  • The user's hands are shaking faster, which leads to repeated orders.
  • Network jitter causes failure or timeout retransmission, such as nginx, Fegin, RPC framework, etc.

solution

Solution 1: The front-end synchronization blocking button is grayed out

The front-end synchronization blocking button is grayed out. After the user clicks the "Publish" button, the user cannot continue to click the "Publish button" before the network request is returned or timed out. The button can be grayed out or circled on the interface.

Advantages : very low implementation cost

Disadvantages :

It can prevent the user's misoperation by shaking hands.

Can not prevent remote call retry, and malicious replay.

Solution 2: Work with front and back ends, and pre-generate order numbers

The order number can be generated in advance (the order number is generated when entering the order page ), and then the unique constraint of the order number in the database can be used to avoid repeated write orders.

The sequence diagram is as follows :

Details as follow:

Timing of order number generation

It is when entering the order page, not when submitting the order.

Order number generation rules

  • Small-scale systems can be generated using MySQL Sequence or Redis. Large-scale systems can also use methods similar to the snowflake algorithm to generate GUIDs in a distributed manner.
  • It’s better to include some category, time and other information in the order number to facilitate business processing. It cannot be a purely self-increasing ID. Otherwise, it is easy for others to calculate your approximate sales volume based on the order number. Therefore, the production algorithm of the order number is guaranteed. Under the premise of repetition, many business rules are usually added in it.

Whether the order number is the primary key

Method 1 : Use the order number as the primary key

If the order number is not incremented, it may cause frequent page splits, resulting in performance degradation when the concurrency is high, so it is necessary to ensure that the order number is globally incremented.

Method 2 : Have an auto-incremented primary key and order number column and set a unique index

Because the order number is not the primary key, the query based on the order number will have one more return to the table, and if the order number is not incremented, the secondary order number index will also have a page split.

Can the order number be generated by the front end?

No, the order number must be generated on the back-end. The back-end generation can ensure that it is globally unique and can be used for security authentication. Order numbers that are not issued by the back-end will not be processed.

When submitting an order, one is to check the database with the order number first, and let the business code verify whether it exists, and the other is to directly use the primary key unique constraint of the database table to throw an exception. Which of the two processing methods has better performance?

Choose the latter. When the database is checked and it is confirmed that it does not exist before inserting, the data may have changed, and the order exists, but an exception is still to be thrown. The check is of little significance.

Scheme 3: General scheme, lock mode

Use locks to control repeated requests within a period of time. Note: The granularity of locks is user + business .

The request process is as follows :

  • 1. When requesting an interface, acquire a lock
    . The granularity of the lock: the same operation logic of the same user. The
    lock name rule: business name + user ID
  • 2. Set an expiration time of 10 seconds for the lock to prevent business logic execution errors, and the user has been locked
  • 3. If it is locked, return to "Processing, please do not submit repeatedly"
  • 4. Not locked, execute normal logic, after the logic ends, delete the lock

achieve

The realization of scheme three is as follows:

Custom annotations to limit repeated submissions

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
public @interface RepeatSubmitLimit {
    /**
     * 业务key,例如下单业务 order
     */
    String businessKey();

    /**
     * 业务参数,用于做更细粒度锁,例如锁到具体 订单id #orderId
     */
    String businessParam() default "";

    /**
     * 是否用户隔离,默认启用
     */
    boolean userLimit() default true;

    /**
     * 锁时间 默认10s
     */
    int time() default 10;
}

Customized aspect interception filtering processing

@Component
@Aspect
@Slf4j
public class LimitSubmitAspect {
    LFUCache<Object, Object> LFUCACHE = CacheUtil.newLFUCache(100, 60 * 1000);

    @Pointcut("@annotation(RepeatSubmitLimit)")
    private void pointcut() {
    }

    @Around("pointcut()")
    public Object handleSubmit(ProceedingJoinPoint joinPoint) throws Throwable {


        Method method = ((MethodSignature) joinPoint.getSignature()).getMethod();
        //获取注解信息
        RepeatSubmitLimit repeatSubmitLimit = method.getAnnotation(RepeatSubmitLimit.class);
        int limitTime = repeatSubmitLimit.time();
        String key = getLockKey(joinPoint, repeatSubmitLimit);
        Object result = LFUCACHE.get(key, false);
        if (result != null) {
            throw new BusinessException("请勿重复访问!");
        }
        LFUCACHE.put(key, StpUtil.getLoginId(), limitTime * 1000);
        try {
            Object proceed = joinPoint.proceed();
            return proceed;
        } catch (Throwable e) {
            log.error("Exception in {}.{}() with cause = \'{}\' and exception = \'{}\'", joinPoint.getSignature().getDeclaringTypeName(),
                    joinPoint.getSignature().getName(), e.getCause() != null ? e.getCause() : "NULL", e.getMessage(), e);
            throw e;
        } finally {
            LFUCACHE.remove(key);
        }
    }

    private static final ParameterNameDiscoverer NAME_DISCOVERER = new DefaultParameterNameDiscoverer();

    private static final ExpressionParser PARSER = new SpelExpressionParser();

    private String getLockKey(ProceedingJoinPoint joinPoint, RepeatSubmitLimit repeatSubmitLimit) {
        String businessKey = repeatSubmitLimit.businessKey();
        boolean userLimit = repeatSubmitLimit.userLimit();
        String businessParam = repeatSubmitLimit.businessParam();
        if (userLimit) {
            businessKey = businessKey + ":" + StpUtil.getLoginId();
        }

        if (StrUtil.isNotBlank(businessParam)) {
            Method method = ((MethodSignature) joinPoint.getSignature()).getMethod();
            EvaluationContext context = new MethodBasedEvaluationContext(null, method, joinPoint.getArgs(), NAME_DISCOVERER);
            String key = PARSER.parseExpression(businessParam).getValue(context, String.class);
            businessKey = businessKey + ":" + key;
        }
        return businessKey;
    }
}

Usage example

    @RepeatSubmitLimit(businessKey = "tokenInfo", businessParam = "#name")
    @GetMapping("/api/v1/tokenInfo")
    public Response tokenInfo(String name) {
    }

Example request : http://localhost:8080/api/v1/tokenInfo?name=123

The lock granularity is:taokeninfo:1:123

Anti-weight effect :

{
	code: "500",
	msg: "请勿重复访问!"
}

reference:

  • Back-end storage practice course