Set up a Tomcat source code reading environment, run and debug without bugs. Promise me, learn Tomcat with me, don't give up!

Article Directory

I. Introduction

(The preface has a few verbose sentences, you can skip it directly, see the source code to run and build)

Tomcat, A familiar and unfamiliar running webtool. Usually, to build a new webproject is to copy and change it according to the existing project. If you encounter a problem, you can use Baidu to adjust a parameter and change the path to solve it.

But what is the function of this parameter in the configuration file? Why should the file be placed in this path? Know nothing! Sometimes Baidu can't solve it for a long time. It takes only a few days. The efficiency is very low. If you understand the source code, it's good. So I made up my mind to thoroughly understand the Tomcatsource code.

Found a copy, write well: start with the beginning of the overall structure introduced Tomcatby connectors and containers composed, server, , service, Host, Contextvarious containers nested directly to the whole ignorant, read a long book, what can not remember, Retired and gave up halfway. Intermittently, for half a year and a year, I have been wandering in the first few chapters, stranded how many times, and when I picked it up and put it down, my self-confidence was almost exhausted. . .

You can't just read the book, you have to look at the source code. After downloading the source code, it's finally a bit on the road. The Tomcatversion in the book may be different from the version of the source code you downloaded. There will be some differences in the description. Some classes have been deleted, and some methods have been refactored. However, it is undeniable that the books written by predecessors play a very good guide. It works, everything is difficult at the beginning, and a good start is half the battle.

After reading the book and the source code for a period of time, I found that it was still not enough. The description in the book was more general. Looking at the method call stack compared to the source code, it really depends on guessing. If it matches the theory, it means that the guess is reasonable. No, you must run the source code, debug, and learn, how can you rely on guessing!

Two, run Tomcat source code

1. Download the source code

TomcatSource code of each version: https://archive.apache.org/dist/tomcat/

TomcatInstallation package: https://tomcat.apache.org/

Why download the source code and the installation package again?

The source code webappsis not compiled with the need to replace the installation package, and Tomcatuse the ant+build.xmldependency management, this approach is relatively old, are now used maven, gradlethe so manually replaced maven, but some packages mavencan not find the warehouse, It can be obtained from the Tomcatinstallation package libdirectory.

What I download here are two versions, Tomcat8.5.9and the Tomcatlatest version (10.0.6). Later, I will explain the source code operation and build process of the two versions respectively.

2, Tomcat8.5.9 source code operation

Download it apache-tomcat-8.5.9-src, open it with IDEA and import it: File --> Project Structure --> Modules --> + --> import module.

Complete source code directory

(1) New pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>Tomcat8.5.9</artifactId>
    <name>Tomcat8.5.9</name>
    <version>8.5.9</version>
 
    <build>
        <finalName>Tomcat8.5.9</finalName>
        <sourceDirectory>java</sourceDirectory>
        <resources>
            <resource>
                <directory>java</directory>
            </resource>
        </resources>
        <testResources>
           <testResource>
                <directory>test</directory>
           </testResource>
        </testResources>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3</version>
                <configuration>
                    <encoding>UTF-8</encoding>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
 
    <dependencies>
        <!--test会用到-->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.easymock</groupId>
            <artifactId>easymock</artifactId>
            <version>3.4</version>
        </dependency>

        <!--org.apache.catalina.ant用到-->
        <dependency>
            <groupId>ant</groupId>
            <artifactId>ant</artifactId>
            <version>1.7.0</version>
        </dependency>

        <!--org.apache.naming.factory.webservices.ServiceRefFactory用到-->
        <dependency>
            <groupId>wsdl4j</groupId>
            <artifactId>wsdl4j</artifactId>
            <version>1.6.2</version>
        </dependency>

        <!--org.apache.naming.factory.webservices用到-->
        <dependency>
            <groupId>javax.xml</groupId>
            <artifactId>jaxrpc</artifactId>
            <version>1.1</version>
        </dependency>

        <!--org.apache.jasper.compiler.JDTCompiler用到-->
        <dependency>
            <groupId>org.eclipse.jdt.core.compiler</groupId>
            <artifactId>ecj</artifactId>
            <version>4.5.1</version>
        </dependency>
    </dependencies>
</project>

(2) Replace webapps and add lib package

Because the webappssample project WEB-INF/classes/under the source package is .javanot compiled and there is no .class file, you can webappsdelete the whole and copy the installation package webappsto the source package.

Create a new lib directory in the jasper.jarsource package and copy it from the lib directory in the installation package to the lib directory of the source package. The purpose of this is twofold:

When Tomcat is running, it will scan the lib directory by default. Although there will be no error if there is no lib directory, the console will have a warning log indicating that the lib directory does not exist.

Missing lib package warning

There are a lot of jar packages under the installation package lib, and there is no need to copy them, and copy them on demand jasper.jar. Although jasper.jarthe code and source code in the org.apache.jasperpackage are completely heavy, there are very important configuration information in this jar package. The defined javax.servlet.ServletContainerInitializerimplementation class is org.apache.jasper.servlet.JasperInitializer, TomcatAt startup, Contextthe lifecycle listener ContextConfig.configureStartwill scan the configuration information in the jar in the lib directory and do some class initialization operations. This is the active instantiation JasperInitializer, otherwise the access jspwill report an error. (Baidu's tutorials are the same, all are written in the source code and JasperInitializerinitialized with one . It is not recommended to do this, and you must find the reason.)

Solve the problem that JasperInitializer cannot be initialized

(3) Run Bootstrap#main

org.apache.catalina.startup.Bootstrap#mainIt is Tomcatthe source of one-click start and stop. mainBefore running this method, you need to specify the location of the configuration, and Tomcatyou must reference server.xmland wait for the configuration to run.

Need to understand two concepts first:

  • catalina.homeIt is the Tomcatinstallation directory and also a public directory, such as bin and lib are shared by all webs.
  • catalina.baseIt is the webproject deployment directory and the working directory, such as conf, logs, webapps, etc. The web can be private.

TomcatRead some configuration, the default is in the two directories, such as conf, webapps, liband so on. I VM optionsset up here catalina.homeand catalina.baseboth are the source directory and the specified Logconfiguration.

-Dcatalina.home=C:/study/tomcat/apache-tomcat-8.5.9-src
-Dcatalina.base=C:/study/tomcat/apache-tomcat-8.5.9-src
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=C:/study/tomcat/apache-tomcat-8.5.9-src/conf/logging.properties
Edit Configurations...

(4) Run and visit

After completing the above three steps, you can run:

It's running, yeah

If testthere is an error in the directory, you can delete the location where the error is reported (it is better not to delete the entire testdirectory, you can testwrite the test class in the directory yourself debug). After normal operation, visit http://127.0.0.1:8080/ in the browser, click on the hyperlinks in the page, such as Documentation, Configurationetc., you can visit normally.

ROOT Home
/docs/config

(5) -config specifies the configuration path to run

webappsWeb project is under deployment automatically, but actual production is unlikely to allow all web projects are deployed webapps, the less likely to start a Tomcatrun all the web, resulting in abnormal run once a web Tomcat hung up, other web Will also be affected. So generally by -configspecifying the configuration, one web runs one Tomcatto ensure process isolation.

# Program arguments, example1.conf是定制的server.xml配置
-config C:/study/tomcat/conf/example1.conf start
-config start

example1.conf:

<?xml version='1.0' encoding='utf-8'?>
<Server port="8112" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">

    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-" maxThreads="20" minSpareThreads="20" />
	
    <Connector executor="tomcatThreadPool"  port="8012" protocol="org.apache.coyote.http11.Http11NioProtocol"
               connectionTimeout="60000"
		acceptCount= "10000" 
               redirectPort="8443"  URIEncoding="UTF-8"  maxPostSize="-1" maxHttpHeaderSize ="102400"/>
    <Engine name="Catalina" defaultHost="demo1">
	
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>
      <Host name="demo1" >
        <!-- Context 指定web项目路径 -->
        <Context path="/" reloadable="true" docBase="C:/study/tomcat/web/example1"  workDir="C:/study/tomcat/web/example1/WEB-INF/work/" >
	  <Resources>
      </Resources>
        </Context>
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="C:/study/tomcat/logs/"
               prefix="example1." suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>

3. Tomcat10.0.6 source code operation

It apache-tomcat-8.5.9-srcis similar to the build process, with a slight difference that some dependencies need to be adjusted. (If Tomcat10.0.6you directly copy the lib of the entire installation package to the source code package, you won’t get an error, so you can copy it all)

pom.xmlTwo more dependent than version 8.5.9 jakartaee-migrationand biz.aQute.bndlib:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>Tomcat10.0.6</artifactId>
    <name>Tomcat10.0.6</name>
    <version>10.0.6</version>
 
    <build>
        <finalName>Tomcat10.0.6</finalName>
        <sourceDirectory>java</sourceDirectory>
        <resources>
            <resource>
                <directory>java</directory>
            </resource>
        </resources>
        <testResources>
           <testResource>
                <directory>test</directory>
           </testResource>
        </testResources>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3</version>
                <configuration>
                    <encoding>UTF-8</encoding>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
 
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.easymock</groupId>
            <artifactId>easymock</artifactId>
            <version>3.4</version>
        </dependency>
        <dependency>
            <groupId>ant</groupId>
            <artifactId>ant</artifactId>
            <version>1.7.0</version>
        </dependency>
        <dependency>
            <groupId>wsdl4j</groupId>
            <artifactId>wsdl4j</artifactId>
            <version>1.6.2</version>
        </dependency>
        <dependency>
            <groupId>javax.xml</groupId>
            <artifactId>jaxrpc</artifactId>
            <version>1.1</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.apache.tomcat/jakartaee-migration -->
        <!-- 10.0.6 新加 -->
        <dependency>
            <groupId>org.apache.tomcat</groupId>
            <artifactId>jakartaee-migration</artifactId>
            <version>1.0.0</version>
        </dependency>

        <!-- 10.0.6 新加 -->
        <dependency>
            <groupId>biz.aQute.bnd</groupId>
            <artifactId>biz.aQute.bndlib</artifactId>
            <version>5.3.0</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>
</project>

The dependency is removed ecj, because the latest version in the maven repository of this dependency is not enough Tomcat10.0.6, it needs to be Tomcat10.0.6copied from the installation package lib to the lib of ecj-4.18.jarthe source package, and the dependency needs to be manually specified:

ecj-4.18

After the dependency is adjusted, other operations are the apache-tomcat-8.5.9-srcsame. However, after running 10.0.6, there are some garbled characters in the console output, not Chinese garbled characters, I tried to debug the code, but it didn't solve it. (Students who have solved the garbled can inform us)

Tomcat10 is also running, but there are garbled characters.  .

Third, build a source code reading environment

To read and learn the Tomcat source code, three things are needed:

  1. Reading book introduction, here are two books about Tomcat "Analysis of Tomcat Kernel Design" Wang Jian and "Analysis of Tomcat Architecture" Liu Guangrui.
  2. Set up the Tomcat source code running environment and debug.
  3. Read the latest source code of Tomcat and compare the source code with multiple versions. Tomcat Githubis open sourced in China, and the git clonelatest code is available locally. One advantage of reading the latest source code is that you can see the commit record of each modification.

The following is the source code reading environment I built, you can git clonelearn together:

https://gitee.com/stefanpy/tomcat-source-code-learning

Fourth, summary: how to read the source code

It is essential to build a source code reading environment. You must have books in your hands, source code in your eyes, and debug in your heart.

The book has a guiding and summarizing function. There are so many and complicated Tomcat source codes, so I don't know where to start reading. You can follow the book's table of contents and study by chapter, from the whole to the details, from the outside to the inside, from simple to complex, and quickly establish a Tomcat infrastructure network:

  1. Tomcat is an HTTP server and Servletcontainer. It has two core components: a connector and a container. The linker implements the HTTP function, and the container implements the loading Servletfunction.
  2. One Servercan contain multiple Service, one Servicecontains multiple Conectorand one Engine, one Enginecontains multiple Host, one Hostcontains multiple Context, and one Contextcontains multiple Wrapper. Are these container names and hierarchical relationships a bit confusing? A Context is a familiar web service, which Wrappercan be understood as the right Servletpackaging.
Tomcat overall architecture-from the network
  1. The response to the request is from Connector -> Container -> Connector . The connector is responsible for external communication, receiving the request to do some encapsulation, and then handing it over to the container for processing, and then returning to the connector for response after the container is processed.
Connectors and containers-from the network
  1. Connectors, containers, where do you start to learn? Connectors involve network programming, HTTP protocol, etc.; in the container, there are class loading, various design patterns, the responsibility chain, and observer (event monitoring) patterns are the most used, and they are easy to understand and absorb. If you are not familiar with network programming (NIO, net, HTTP), you can learn from the container first. If you are familiar with the RPC framework Netty, the connector is simple.
  2. Study the Tomcat source code with tasks and questions. Divide the arduous project of learning the entire Tomcat source code into multiple small tasks, and study and study with questions. For example, you can start with the server.xmlconfiguration you are familiar with daily , and understand the configuration inside, why is this configuration, and why is this path like this; the question is how does Tomcat do hot reloading? How to deploy and load a web project? How to process a request? How is the Tomcat life cycle realized and how to start and stop with one click? Why does Tomcat need to customize the class loader and how to break parental delegation? and many more. With tasks and questions, and timely positive feedback, can we insist on gnawing down the hard bone of Tomcat.

I have been learning the Tomcat source code for a while, and I left it halfway for many times. I fumbled and summed up a set of learning methods that suit me. Really, everything is difficult at the beginning. A good beginning is half the success. A good method is twice the result with half the effort. I will continue to share my experience and process of learning the Tomcat source code later, I hope it will be useful to you.