docker - Docker persistence, data sharing, compose multi-container deployment

One, understand docker persistence

Insert picture description here

1. Docker persistence solution

  • A volum based on the local file system. You can use the -v parameter to use the host's directory as the container's data volume when executing docker create or docker run. This part of the function is based on the volume management of the local file system
  • Plugin-based volume, support third-party storage solutions, such as NAS, aws
Insert picture description here

  • Type of Volume:
  • Collect the managed data volume, which is automatically created by the docker background
  • Bind mounted volume, the specific mounting location is specified by the user

2. Data volume for data persistence

Insert picture description here
  • Reference address: https://github.com/docker-library/mysql/tree/master/5.7#'Officially provided MySQL Dockerfile'
  • Data volume persistent data will not be deleted with the deletion of the container, which solves the problem of data security
  • The name of the data volume generated by default is very long and inconvenient to use. Use the -v parameter to specify the name.
    Example:-v <volume名称>:<挂载路径>
# 运行一个允许空密码登入的mysql
[[email protected] ~]# docker run -d --name mysql1 -e MYSQL_ALLOW_EMPTY_PASSWORD mysql

[[email protected] ~]# docker ps -a
CONTAINER ID   IMAGE     COMMAND                  CREATED          STATUS                      PORTS     NAMES
36a16497b015   mysql     "docker-entrypoint.s…"   14 seconds ago   Exited (1) 12 seconds ago             mysql1
# 没有启动成功,查看mysql1日志,发现还有密码问题导致无法启动
[[email protected] ~]# docker logs mysql1
2021-05-29 16:55:46+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.25-1debian10 started.
2021-05-29 16:55:46+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2021-05-29 16:55:46+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.25-1debian10 started.
2021-05-29 16:55:46+00:00 [ERROR] [Entrypoint]: Database is uninitialized and password option is not specified
    You need to specify one of the following:
    - MYSQL_ROOT_PASSWORD
    - MYSQL_ALLOW_EMPTY_PASSWORD
    - MYSQL_RANDOM_ROOT_PASSWORD
# 删掉之前的mysql
[[email protected] ~]# docker rm mysql1

# 虽然容器被删除了,但是volume依旧在,可以执行 docker volume rm 删除
[[email protected] ~]# docker volume ls
DRIVER    VOLUME NAME
local     dc21763599f6430ea2a8decee10495c51510fc562f108ec22496cc07c4f91023
[[email protected] ~]# docker volume  rm dc21763599f6430ea2a8decee10495c51510fc562f108ec22496cc07c4f91023
dc21763599f6430ea2a8decee10495c51510fc562f108ec22496cc07c4f91023
# 设置允许空密码
[[email protected] ~]# docker run -d --name mysql1 -e MYSQL_ALLOW_EMPTY_PASSWORD=true mysql

# 容器up了
[[email protected] ~]# docker ps -a
CONTAINER ID   IMAGE     COMMAND                  CREATED         STATUS         PORTS                 NAMES
604a6a4136f0   mysql     "docker-entrypoint.s…"   4 seconds ago   Up 3 seconds   3306/tcp, 33060/tcp   mysql1

# 查看本地volume
[[email protected] ~]# docker volume ls
DRIVER    VOLUME NAME
local     cffb1dc331784981ef76baa7929a38b50a4407c6c6bc160a1b60e48f6263e4a8

# 查看这个volume的详细信息
[[email protected] ~]# docker volume inspect  cffb1dc331784981ef76baa7929a38b50a4407c6c6bc160a1b60e48f6263e4a8
[
    {
        "CreatedAt": "2021-05-30T01:03:53+08:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/cffb1dc331784981ef76baa7929a38b50a4407c6c6bc160a1b60e48f6263e4a8/_data",
        "Name": "cffb1dc331784981ef76baa7929a38b50a4407c6c6bc160a1b60e48f6263e4a8",
        "Options": null,
        "Scope": "local"
    }
]

# 利用-v 指定volume的名称和路径
[[email protected] ~]# docker run -d --name mysql2 -v mysql-data:/var/lib/mysql -e MYSQL_ALLOW_EMPTY_PASSWORD=true mysql

# 生成了一个mysql-data的volume
[[email protected] ~]# docker volume  ls
DRIVER    VOLUME NAME
local     cffb1dc331784981ef76baa7929a38b50a4407c6c6bc160a1b60e48f6263e4a8
local     mysql-data

# 查看mysql-data信息,可以看到挂载路径
[[email protected] ~]# docker volume inspect  mysql-data 
[
    {
        "CreatedAt": "2021-05-30T01:13:50+08:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/mysql-data/_data",
        "Name": "mysql-data",
        "Options": null,
        "Scope": "local"
    }
]

Verify that the volume is valid

# 进入mysql2查看数据库
[[email protected] ~]# docker exec -it mysql2 bash -c mysql
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
4 rows in set (0.00 sec)

# 创建docker库
mysql> create database docker;
Query OK, 1 row affected (0.01 sec)

# 强制删除所有的mysql容器
[[email protected] ~]# docker rm -f $(docker ps -aq)

# 查看容器,已经被删除了
[[email protected] ~]#  docker ps -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

# 查看volume,mysql-data的数据卷还在
[[email protected] ~]# docker volume ls
DRIVER    VOLUME NAME
local     mysql-data

# 使用-v指定volume名称和路径(注意,一定要和上面的一致,否则会重新创建一个volume),重新创建一个mysql容器,
[[email protected] ~]# docker run -d --name mysql1 -v mysql-data:/var/lib/mysql -e MYSQL_ALLOW_EMPTY_PASSWORD=true mysql

# 进入数据库查看,之前创建docker库还在
[[email protected] ~]# docker exec -it mysql1 bash -c mysql
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| docker             |
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.01 sec)

3. Bind mounting of data persistence

  • The difference between bind mount and data volume:
  • data volume: Need to define the created volume in the Dockerfile
  • bind mount: only need to specify the local path and the path in the container when the container is running -v
    -v <指定本地路径>:<容器内路径>

Make a container for nginx to access Baidu

# 用Dockerfile生成镜像
[[email protected] nginx]# cat Dockerfile 
FROM nginx:latest
WORKDIR /usr/share/nginx/html
COPY index.html index.html

# 利用百度首页生成index.html文件
[[email protected] nginx]# curl www.baidu.com > index.html

# 制作镜像
[[email protected] nginx]# docker build -t test/baidu-nginx .

# 运行容器
[[email protected] nginx]# docker run -d -p 80:80 --name nginx-web test/baidu-nginx

[[email protected] nginx]# docker ps -a
CONTAINER ID   IMAGE              COMMAND                  CREATED         STATUS         PORTS                               NAMES
7f70c92f40c3   test/baidu-nginx   "/docker-entrypoint.…"   9 seconds ago   Up 8 seconds   0.0.0.0:80->80/tcp, :::80->80/tcp   nginx-web

Access test

Insert picture description here


Use bind volume

[[email protected] nginx]# ls
Dockerfile  index.html
# 将当前目录映射到容器的/usr/share/nginx/html目录下
[[email protected] nginx]# docker run -d -p 80:80 -v $(pwd):/usr/share/nginx/html --name nginx-web test/baidu-nginx

# 在当前目录下生成一个test.html文件
[[email protected] nginx]# vim test.html 
<head>
<meta charset="utf-8">
<title>YY</title>
<h1>精美壁纸</h1><br>
<img src="http://pic1.cxtuku.com/00/03/34/b3846967c5ac.jpg" />
</head>

To access the test, you can access the newly added test.html file

Insert picture description here

4. Practical demonstration of bind mounting

# 下载代码
[[email protected] ~]# git clone https://github.com/LTP7534/skeleton.git

[[email protected] ~]# cd skeleton

[[email protected] skeleton]# docker build -t test/flask-skeleton .

[[email protected] skeleton]# [[email protected] skeleton]# docker run -d -p 80:5000 -v $(pwd):/skeleton --name flask-skeleton test/flask-skeleton:latest

[[email protected] skeleton]# docker ps -a
CONTAINER ID   IMAGE                  COMMAND            CREATED         STATUS         PORTS                                   NAMES
30b1f87f4cbc   test/skeleton:latest   "scripts/dev.sh"   6 seconds ago   Up 5 seconds   0.0.0.0:80->5000/tcp, :::80->5000/tcp   flask-skeleton

Insert picture description here

Two, understand docker compose

1. Deploy wordprocess (exercise)

  • Reference address: http://hub.docker.com/_/wordpress
  • Deploy wordpress applications through multiple containers

1.1 Create a mysql database container

[[email protected] skeleton]# docker run -d --name mysql -v mysql-data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=123456 -e MYSQL_DATABASE=wordpress mysql

[[email protected] skeleton]# docker ps -a
CONTAINER ID   IMAGE     COMMAND                  CREATED         STATUS         PORTS                 NAMES
7fc6b75c1989   mysql     "docker-entrypoint.s…"   4 seconds ago   Up 3 seconds   3306/tcp, 33060/tcp   mysql
[[email protected] skeleton]# 

1.2 Deploy wordpress container

[[email protected] skeleton]# docker run -d --name wordpress -e WORDPRESS_DB_HOST=mysql:3306 -e WORDPRESS_DB_USER=root -e WORDPRESS_DB_PASSWORD=123456 --link mysql -p 8080:80 wordpress


[[email protected] skeleton]# docker ps -a
CONTAINER ID   IMAGE       COMMAND                  CREATED         STATUS         PORTS                                   NAMES
275a8f58ff01   wordpress   "docker-entrypoint.s…"   9 seconds ago   Up 7 seconds   0.0.0.0:8080->80/tcp, :::8080->80/tcp   distracted_zhukovsky
7fc6b75c1989   mysql       "docker-entrypoint.s…"   4 minutes ago   Up 4 minutes   3306/tcp, 33060/tcp                     mysql

1.3 Browser access

Insert picture description here


Insert picture description here


Insert picture description here


Insert picture description here

2. Docker compose deployment application

Reference address: https://docs.docker.com/compose/compose-file/compose-file-v3/

  • Docker compose solves the problem of multi-container deployment management
  • docker compose is a tool
  • This tool defines a multi-container docker application through a yml file
  • With one command, you can create or manage multiple containers according to the definition of the yml file.
Insert picture description here


The three major concepts of docker compose

Services :

A Services represents a container, this container can be created from the image created by Dockerfile or the image pulled from dockerhub

Service start is similar to docker run, we can specify network and volume for service

E.g:

# 示例1:
services:
 db:
   image: postgres:9.4     # 从dockerhub上拉取的镜像
   volumes:
     - "db-data":/var/lib/postgres/data
   networks:
     - back-tier 

Is equivalent to:

docker run -d --networ kback-tier -v db-data:/var/lib/postgres/data postgres:9.4

# 示例2:
services:
 worker:
   build: ./worker     # 利用Dockerfile创建的镜像
   link:
     - db
     - redis
   networks:
     - back-tier 

Networks :

  • E.g
Insert picture description here

Volumes:

  • E.g:
Insert picture description here

2.1 Install docker compose

[[email protected] ~]# sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

[[email protected] ~]# chmod +x /usr/local/bin/docker-compose

[[email protected] ~]# docker-compose --version
docker-compose version 1.29.2, build 5becea4c


2.2 Deploy wordpress with docker compose

  • docker-compose -f <yml文件> up---- Start to orchestrate the container and run the container directly
[[email protected] nginx]# vim docker-compose.yml
version: '3'

services:
  wordpress:
    image: wordpress
    ports:
      - 8080:80
    depends_on:
      - mysql
    environment:
      WORDPRESS_DB_HOST: mysql
      WORDPRESS_DB_PASSWORD: 123456
      WORDPRESS_DB_USER: root
    networks:
      - my-bridge

  mysql:
    image: mysql:5.7
    environment:
      MYSQL_ROOT_PASSWORD: 123456
      MYSQL_DATABASE: wordpress
    volumes:
      - mysql-data:/var/lib/mysql
    networks:
      - my-bridge

volumes:
  mysql-data:

networks:
  my-bridge:
    driver: bridge
# 查看yml文件
[[email protected] nginx]# ls
docker-compose.yml

# 开始编排,-f 默认的文件名就是docker-compose.yml,因此这里可以省略指定文件名
# -d参数:后台运行(加上这个参数后,debug日志就不会打印出)
# 一般用于调试时会直接up,不加-d,可以实时查看log
[[email protected] nginx]# docker-compose up&


# 打印出 docker-compose 启动的容器
[[email protected] nginx]# docker-compose ps
      Name                     Command               State                  Ports                
-------------------------------------------------------------------------------------------------
nginx_mysql_1       docker-entrypoint.sh mysqld      Up      3306/tcp, 33060/tcp                 
nginx_wordpress_1   docker-entrypoint.sh apach ...   Up      0.0.0.0:8080->80/tcp,:::8080->80/tcp

Access test

Insert picture description here
# docker-compose stop 可以停止docker-compose启动的容器(不会删除)
[[email protected] nginx]# docker-compose stop

# 查看容器状态
[[email protected] nginx]# docker-compose ps
      Name                     Command               State    Ports
-------------------------------------------------------------------
nginx_mysql_1       docker-entrypoint.sh mysqld      Exit 0        
nginx_wordpress_1   docker-entrypoint.sh apach ...   Exit 0      

# docker-compose start 会将停止的容器再开启
[[email protected] nginx]# docker-compose start 
Starting mysql     ... done
Starting wordpress ... done


# 又会处于运行状态
[[email protected] nginx]# docker-compose ps
      Name                     Command               State                  Ports                
-------------------------------------------------------------------------------------------------
nginx_mysql_1       docker-entrypoint.sh mysqld      Up      3306/tcp, 33060/tcp                 
nginx_wordpress_1   docker-entrypoint.sh apach ...   Up      0.0.0.0:8080->80/tcp,:::8080->80/tcp


# docker-compose down 会将相关的容器,镜像,数据卷等删除
[[email protected] nginx]# docker-compose down
Stopping nginx_wordpress_1 ... done
Stopping nginx_mysql_1     ... done
Removing nginx_wordpress_1 ... done
Removing nginx_mysql_1     ... done
Removing network nginx_my-bridge

# 编排的容器以及被删除
[[email protected] nginx]# docker-compose ps
Name   Command   State   Ports
------------------------------

# -d参数:后台运行(加上这个参数后,debug日志就不会打印出)
[[email protected] nginx]# docker-compose up -d
Creating network "nginx_my-bridge" with driver "bridge"
Creating nginx_mysql_1 ... done
Creating nginx_wordpress_1 ... done


# 查看docker-compose生成的镜像
[[email protected] nginx]# docker-compose images
    Container       Repository    Tag       Image Id       Size  
-----------------------------------------------------------------
nginx_mysql_1       mysql        5.7      2c9028880e58   447 MB  
nginx_wordpress_1   wordpress    latest   0adda6ed742f   550.5 MB

# 进入容器docker-compose exec <yml中定义的Services> bash
[[email protected] nginx]# docker-compose exec mysql bash
[email protected]:/# mysql -u root -p123456
mysql> 

# 生成的网络名称格式 : "当前目录名称_yml中定义的networks名称" 如这个nginx_my-bridge
[[email protected] nginx]# docker network ls
NETWORK ID     NAME              DRIVER    SCOPE
5748917a989f   bridge            bridge    local
ef2dd6544dee   host              host      local
6b5d1976ff28   nginx_my-bridge   bridge    local    
0051745c4534   none              null      local
[[email protected] nginx]# docker-compose down
Stopping nginx_wordpress_1 ... done
Stopping nginx_mysql_1     ... done
Removing nginx_wordpress_1 ... done
Removing nginx_mysql_1     ... done
Removing network nginx_my-bridge

3. docker-compose horizontal expansion and load balancing

  • For example, we have a web service that gets data from the total redis. When there is only one web container, it can be directly connected and accessed, but if there are multiple identical webs, they cannot be accessed directly, and the ports will conflict. This is the case. Load balancing

3.1 Practical example

First look at the web application script, it will get data from redis, and then web display

3.1.1 First perform the orchestration deployment of a single web container

[[email protected] flask-redis]# vim app.py 
from flask import Flask
from redis import Redis
import os
import socket

app = Flask(__name__)
redis = Redis(host=os.environ.get('REDIS_HOST', '127.0.0.1'), port=6379)


@app.route('/')
def hello():
    redis.incr('hits')
    return 'Hello Container World! I have been seen %s times and my hostname is %s.\n' % (redis.get('hits'),socket.gethostname())


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000, debug=True)

View Dockerfile

[[email protected] flask-redis]# vim Dockerfile 
FROM python:2.7
LABEL maintaner="Peng Xiao [email protected]"
COPY . /app
WORKDIR /app
RUN pip install flask redis
EXPOSE 5000
CMD [ "python", "app.py" ]

View docker-compose

[[email protected] flask-redis]# vim docker-compose.yml 
version: "3"

services:

  redis:
    image: redis

  web:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 8080:5000
    environment:
      REDIS_HOST: redis

Arrange to map port 5000 in the container to 8080

[[email protected] flask-redis]# docker-compose up -d

[[email protected] flask-redis]# docker-compose ps
       Name                      Command               State                    Ports                  
-------------------------------------------------------------------------------------------------------
flask-redis_redis_1   docker-entrypoint.sh redis ...   Up      6379/tcp                                
flask-redis_web_1     python app.py                    Up      0.0.0.0:8080->5000/tcp,:::8080->5000/tcp

Access test

Insert picture description here

3.1.2 Multi-web container deployment

  • However, when there are multiple web containers, the above yml file cannot be used. There will be port conflicts, and haproxy is needed.
  • Expand container commanddocker-compose up --scale <service名称>=<扩容数目> -d
  • This method is more scalable and more flexible
Insert picture description here
# 先移除之前的
[[email protected] flask-redis]# docker-compose down
Removing flask-redis_redis_1 ... done
Removing flask-redis_web_1   ... done
Removing network flask-redis_default

View related files

[[email protected] flask-redis]# cat app.py 
from flask import Flask
from redis import Redis
import os
import socket

app = Flask(__name__)
redis = Redis(host=os.environ.get('REDIS_HOST', '127.0.0.1'), port=6379)

@app.route('/')
def hello():
    redis.incr('hits')
    return 'Hello Container World! I have been seen %s times and my hostname is %s.\n' % (redis.get('hits'),socket.gethostname())

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=80, debug=True)


[[email protected] flask-redis]# cat Dockerfile 
FROM python:2.7
LABEL maintaner="Peng Xiao [email protected]"
COPY . /app
WORKDIR /app
RUN pip install flask redis
EXPOSE 80
CMD [ "python", "app.py" ]


[[email protected] flask-redis]# cat docker-compose.yml 
version: "3"
services:

  redis:
    image: redis

  web:
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      REDIS_HOST: redis

  lb:
    image: dockercloud/haproxy
    links:
      - web
    ports:
      - 8080:80
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock 

Single container operation

# 运行容器
[[email protected] flask-redis]# docker-compose up -d

# 访问web
[[email protected] flask-redis]# curl 192.168.10.70:8080
Hello Container World! I have been seen 1 times and my hostname is b04337110d23.

# 查看运行容器
[[email protected] flask-redis]# docker-compose ps -a 
       Name                      Command               State                            Ports                         
----------------------------------------------------------------------------------------------------------------------
flask-redis_lb_1      /sbin/tini -- dockercloud- ...   Up      1936/tcp, 443/tcp, 0.0.0.0:8080->80/tcp,:::8080->80/tcp
flask-redis_redis_1   docker-entrypoint.sh redis ...   Up      6379/tcp                                               
flask-redis_web_1     python app.py                    Up      5000/tcp   

Multi-web container operation (can be directly expanded on the original basis)

# 这是web容器数量为3
[[email protected] flask-redis]# docker-compose up --scale web=3 -d
flask-redis_redis_1 is up-to-date
Creating flask-redis_web_2 ... done
Creating flask-redis_web_3 ... done
flask-redis_lb_1 is up-to-date

# 查看容器,多出了web_2 和 web_3
[[email protected] flask-redis]# docker-compose ps -a 
       Name                      Command               State                            Ports                         
----------------------------------------------------------------------------------------------------------------------
flask-redis_lb_1      /sbin/tini -- dockercloud- ...   Up      1936/tcp, 443/tcp, 0.0.0.0:8080->80/tcp,:::8080->80/tcp
flask-redis_redis_1   docker-entrypoint.sh redis ...   Up      6379/tcp                                               
flask-redis_web_1     python app.py                    Up      5000/tcp                                               
flask-redis_web_2     python app.py                    Up      5000/tcp                                               
flask-redis_web_3     python app.py                    Up      5000/tcp  

# 访问测试,正常
[[email protected] flask-redis]# curl 192.168.10.70:8080
Hello Container World! I have been seen 2 times and my hostname is b04337110d23.
 

4. Use docker-compose to deploy more complex applications

  • voting app: for voting
  • results app: used to view voting results
Insert picture description here

# 下载代码
[[email protected] voting]# git clone https://github.com/LTP7534/voting.git

# 查看docker-compose.yml
[[email protected] voting]# cat docker-compose.yml 
version: "3"

services:
  voting-app:
    build: ./voting-app/.
    volumes:
     - ./voting-app:/app
    ports:
    # 容器的80端口映射到本地的5000端口
      - "5000:80"
    links:
      - redis
    networks:
      - front-tier
      - back-tier

  result-app:
    build: ./result-app/.
    volumes:
      - ./result-app:/app
    ports:
      - "5001:80"
    links:
      - db
    networks:
      - front-tier
      - back-tier

  worker:
    build: ./worker
    links:
      - db
      - redis
    networks:
      - back-tier

  redis:
    image: redis
    ports: ["6379"]
    networks:
      - back-tier

  db:
    image: postgres:9.4
    volumes:
      - "db-data:/var/lib/postgresql/data"
    networks:
      - back-tier

volumes:
  db-data:
  
networks:
  # front-tier 和 back-tier 都没有指明driver,默认时bridge
  front-tier:
  back-tier: