Spring Cloud Gateway does not support grayscale publishing, so implement one yourself! !

Hello, everyone, I am Misty Jam, a third-rate programmer from a third-rate company in a third-rate city. This is our 157th original article. If you like it, please remember to give me a like and forward.
The series of tutorials will be converted to a paid position when the traffic is reduced, but not when the traffic is high, so hurry up and learn~

Preface

This article originated from a question raised by fans: how to solve the chaos of multi-environment unified registry service instances?

How to understand?

Suppose that the AccountService of the development environment has been registered in Nacos, and now Xiao Zhang needs to modify it and upgrade it. After starting AccountService locally, he also registered to Nacos, but when debugging, he often jumps directly to the development environment after requesting through the gateway. If that happens, Xiao Zhang will not be able to debug at ease.

image-20210608205745475

In fact, this problem ultimately comes down to how to implement gray-scale publishing based on Spring Cloud Gateway, and let request traffic reach a specific instance through specified rules.

In the SpringCloud 2020 version, it is officially recommended to use Spring Cloud LoadBalancer to replace the original Ribbon load balancer. So this article is directly based on Spring Cloud LoadBalancer.

tips: what is gray release
Grayscale release (also known as canary release) refers to a release method that can smoothly transition between black and white. A/B testing can be performed on it, that is, some users continue to use product feature A, and some users start to use product feature B. If users have no objections to B, then gradually expand the scope and migrate all users to B. Come. Gray release can ensure the stability of the overall system, and problems can be found and adjusted at the initial gray level to ensure its impact.

Goals

The goal is very clear. Xiao Zhang hopes that the requests made during debugging can reach his local development environment directly to facilitate debugging.

Realization ideas

To achieve this goal, we need to solve two key issues:

How to distinguish between different instances

It is necessary to give Xiao Zhang a special identifier for the AccountService service instance started locally to distinguish it from the development environment.

Here we can use the metadata of the registry to distinguish, which can be spring.cloud.nacos.discovery.metadata.version = devspecified by configuration, or directly add metadata information to the nacos service list.

image-20210608114019908

Implement custom load balancing rules, through custom rules so that the load balancer can find the service instances we need

Xiao Zhang needs to add a label version=devto the request header when requesting the service . After obtaining the request header information, the custom load balancer goes to the service instance to find the service instance configured with mtadata.version=dev.

Spring Cloud LoadBalancer (SCL)

SCL load balancing strategy

There is such a description in the official Spring Cloud LoadBalancer document :

Spring Cloud provides its own client-side load-balancer abstraction and implementation. For the load-balancing mechanism, ReactiveLoadBalancerinterface has been added and a Round-Robin-based and Random implementations have been provided for it. In order to get instances to select from reactive ServiceInstanceListSupplieris used. Currently we support a service-discovery-based implementation of ServiceInstanceListSupplierthat retrieves available instances from Service Discovery using a Discovery Client available in the classpath.

Combining with other content in the document, extract several key information:

Spring Cloud LoadBalancer provides two load balancing algorithms: Round-Robin-based and Random , Round-Robin-based is used by default

image-20210608124754654

ServiceInstanceListSupplierThe service instance that meets the requirements can be selected through implementation

By LoadBalancerClientselection policy notes, specify the service level load balancing strategy and examples

Tip: If you need to explore the implementation principle of SCL, you can GatewayReactiveLoadBalancerClientAutoConfigurationstart with it.

Custom grayscale release

In combination with the above, we have two ways to achieve grayscale using Spring Cloud LoadBalancer:

  1. Simple and rude, directly implement a new load balancing strategy, and then LoadBalancerClientspecify the service instance to use this strategy through annotations.
  2. Customize the service instance filtering logic, and filter out the service instances that meet the requirements when returning to the front-end instance. Of course, you also need to LoadBalancerClientspecify the service instance through annotations to use this selector.

Code

Imprint

The version used by the SpringCloud project is the graduate version recommended by SpringCloud alibaba

<spring-boot.version>2.4.2</spring-boot.version>
<alibaba-cloud.version>2021.1</alibaba-cloud.version>
<springcloud.version>2020.0.0</springcloud.version>

Custom load balancing strategy

First, let's look at the first implementation method, which is achieved through a custom load balancing strategy.

  1. SCL is introduced in the gateway module, and the Ribbon load balancer that comes with the nacos registry needs to be removed at the same time.
<dependency>
    <groupId>com.alibaba.cloud</groupId>
    <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
    <exclusions>
        <exclusion>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
        </exclusion>
    </exclusions>
</dependency>

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-loadbalancer</artifactId>
</dependency>
  1. Custom load balancing strategy VersionGrayLoadBalancer
/**
 * Description:
 * 自定义灰度
 * 通过给请求头添加Version 与 Service Instance 元数据属性进行对比
 * @author Jam
 * @date 2021/6/1 17:26
 */
@Log4j2
public class VersionGrayLoadBalancer implements ReactorServiceInstanceLoadBalancer {

    private final ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider;
    private final String serviceId;

    private final AtomicInteger position;

    public VersionGrayLoadBalancer(ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider, String serviceId) {
        this(serviceInstanceListSupplierProvider,serviceId,new Random().nextInt(1000));
    }

    public VersionGrayLoadBalancer(ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider,
                                   String serviceId, int seedPosition) {
        this.serviceId = serviceId;
        this.serviceInstanceListSupplierProvider = serviceInstanceListSupplierProvider;
        this.position = new AtomicInteger(seedPosition);
    }

    @Override
    public Mono<Response<ServiceInstance>> choose(Request request) {

        ServiceInstanceListSupplier supplier = this.serviceInstanceListSupplierProvider.getIfAvailable(NoopServiceInstanceListSupplier::new);

        return supplier.get(request).next()
                .map(serviceInstances -> processInstanceResponse(serviceInstances,request));

    }


    private Response<ServiceInstance> processInstanceResponse(List<ServiceInstance> instances, Request request) {
        if (instances.isEmpty()) {
            log.warn("No servers available for service: " + this.serviceId);
            return new EmptyResponse();
        } else {
            DefaultRequestContext requestContext = (DefaultRequestContext) request.getContext();
            RequestData clientRequest = (RequestData) requestContext.getClientRequest();
            HttpHeaders headers = clientRequest.getHeaders();

            // get Request Header
            String reqVersion = headers.getFirst("version");

            if(StringUtils.isEmpty(reqVersion)){
                return processRibbonInstanceResponse(instances);
            }

            log.info("request header version : {}",reqVersion );
			// filter service instances
            List<ServiceInstance> serviceInstances = instances.stream()
                    .filter(instance -> reqVersion.equals(instance.getMetadata().get("version")))
                    .collect(Collectors.toList());

            if(serviceInstances.size() > 0){
                return processRibbonInstanceResponse(serviceInstances);
            }else{
                return processRibbonInstanceResponse(instances);
            }
        }
    }

    /**
     * 负载均衡器
     * 参考 org.springframework.cloud.loadbalancer.core.RoundRobinLoadBalancer#getInstanceResponse
     * @author javadaily
     */
    private Response<ServiceInstance> processRibbonInstanceResponse(List<ServiceInstance> instances) {
        int pos = Math.abs(this.position.incrementAndGet());
        ServiceInstance instance = instances.get(pos % instances.size());
        return new DefaultResponse(instance);
    }

}

Get the version attribute in the request header, and then match it according to the version attribute in the service instance metadata. Refer to the Round-Robin-based implementation method for qualified instances.

  1. Write configuration classes VersionLoadBalancerConfigurationto replace the default load balancing algorithm
/**
 * Description:
 * 自定义负载均衡器配置实现类
 * @author javadaily
 * @date 2021/6/3 16:02
 */
public class VersionLoadBalancerConfiguration {
    @Bean
    ReactorLoadBalancer<ServiceInstance> versionGrayLoadBalancer(Environment environment,
                                                                 LoadBalancerClientFactory loadBalancerClientFactory) {
        String name = environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new VersionGrayLoadBalancer(
                loadBalancerClientFactory.getLazyProvider(name, ServiceInstanceListSupplier.class), name);
    }

}

The VersionLoadBalancerConfiguration configuration class cannot be annotated with @Configuration.

  1. Use annotations in the gateway startup class to @LoadBalancerClientspecify which services use custom load balancing algorithms

Pass @LoadBalancerClient(value = "auth-service", configuration = VersionLoadBalancerConfiguration.class), enable custom load balancing algorithm for auth-service;

Or by @LoadBalancerClients(defaultConfiguration = VersionLoadBalancerConfiguration.class)enabling custom load balancing algorithms for all services.

Custom service instance filtering logic

Next, let's look at the second implementation method, by implementing ServiceInstanceListSupplier to customize the service filtering logic, we can directly inherit DelegatingServiceInstanceListSupplier to achieve.

  1. Introduce Spring Cloud LoadBalancer in the gateway module (same as above)
  2. Custom service instance filtering logicVersionServiceInstanceListSupplier
/**
 * 自定义服务实例筛选逻辑
 * @author javadaily
 * 参考:org.springframework.cloud.loadbalancer.core.ZonePreferenceServiceInstanceListSupplier
 */
@Log4j2
public class VersionServiceInstanceListSupplier extends DelegatingServiceInstanceListSupplier {


    public VersionServiceInstanceListSupplier(ServiceInstanceListSupplier delegate) {
        super(delegate);
    }


    @Override
    public Flux<List<ServiceInstance>> get() {
        return delegate.get();
    }

    @Override
    public Flux<List<ServiceInstance>> get(Request request) {
        return delegate.get(request).map(instances -> filteredByVersion(instances,getVersion(request.getContext())));
    }


    /**
     * filter instance by requestVersion
     * @author javadaily
     */
    private List<ServiceInstance> filteredByVersion(List<ServiceInstance> instances, String requestVersion) {
        log.info("request version is {}",requestVersion);
        if(StringUtils.isEmpty(requestVersion)){
            return instances;
        }

        List<ServiceInstance> filteredInstances = instances.stream()
                .filter(instance -> requestVersion.equalsIgnoreCase(instance.getMetadata().getOrDefault("version","")))
                .collect(Collectors.toList());

        if (filteredInstances.size() > 0) {
            return filteredInstances;
        }

        return instances;
    }

    private String getVersion(Object requestContext) {
        if (requestContext == null) {
            return null;
        }
        String version = null;
        if (requestContext instanceof RequestDataContext) {
            version = getVersionFromHeader((RequestDataContext) requestContext);
        }
        return version;
    }

    /**
     * get version from header
     * @author javadaily
     */
    private String getVersionFromHeader(RequestDataContext context) {
        if (context.getClientRequest() != null) {
            HttpHeaders headers = context.getClientRequest().getHeaders();
            if (headers != null) {
                //could extract to the properties
                return headers.getFirst("version");
            }
        }
        return null;
    }
}

The implementation principle is the same as the custom load balancing strategy, matching the service instances that meet the requirements according to the version.

  1. Write configuration classes VersionServiceInstanceListSupplierConfigurationto replace the default service instance filtering logic
public class VersionServiceInstanceListSupplierConfiguration {


    @Bean
    ServiceInstanceListSupplier serviceInstanceListSupplier(ConfigurableApplicationContext context) {
        ServiceInstanceListSupplier delegate = ServiceInstanceListSupplier.builder()
                .withDiscoveryClient()
                .withCaching()
                .build(context);
        return new VersionServiceInstanceListSupplier(delegate);
    }
}
  1. Use the annotation @LoadBalancerClient in the gateway startup class to specify which services use a custom load balancing algorithm

Pass @LoadBalancerClient(value = "auth-service", configuration = VersionServiceInstanceListSupplierConfiguration.class), enable custom load balancing algorithm for auth-service;

Or by @LoadBalancerClients(defaultConfiguration = VersionServiceInstanceListSupplierConfiguration.class)enabling custom load balancing algorithms for all services.

test

  1. Start multiple AccountService instances, and configure metadata version = dev for the instance on port 58302
image-20210608113938479
image-20210608114019908
  1. Specify the request header when postman calls the interface
image-20210608114109694
  1. Observe the two implementation logics through debug mode, and observe whether the results meet expectations.

summary

In this article, we have implemented a simple gray-scale publishing function based on SCL by extending the load balancing algorithm and modifying the service instance screening logic. You can refer to this to implement the extended SCL load balancing algorithm or customize your own service screening logic.