docker创建keepalived镜像实现高可用

必要条件：

  • 1、在服务器中安装 centos 镜像
  • 2、进入centos_01容器安装keepalived（或者keepalived+nginx），若有前端则加上nginx

环境：
前提条件 -> 已在 linux 中安装好docker环境。

1、在服务器中安装 centos 镜像

下载镜像：

1

docker  pull  centos:7.7.1908


查看镜像：

1

docker images


安装第一个centos容器：centos_01

#f1cb7c7d58b7  就是上图中的镜像ID

sudo docker run -it --name centos_01 f1cb7c7d58b7

#然后退出容器

exit;

2、进入centos_01容器安装keepalived+nginx

进入容器：

#efbad978d17f 是容器ID

docker exec -it efbad978d17f /bin/bash

安装keepalived 和安装nginx

#安装依赖环境

yum install -y gcc openssl-devel popt-devel

#安装keepalived

yum install -y keepalived

#或者本地安装指定版本

wget https://www.keepalived.org/software/keepalived-1.3.4.tar.gz

#使用yum安装nginx需要包括Nginx的库，安装Nginx的库

rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm

# 使用下面命令安装nginx

yum install -y nginx

#安装网络包(需要使用ifconfig和ping命令)

yum install -y net-tool

修改/etc/keepalived/keepalived.conf文件:

#进入到 容器 centos_01 的根目录

cd /

#进入 keepalived 安装目录

cd /etc/keepalived

# 打开配置文件

vi keepalived.conf

修改/etc/keepalived/keepalived.conf文件        

! Configuration File for keepalived

global_defs {

    router_id LVS_DEVEL

}

vrrp_script chk_nginx {

    script "/etc/keepalived/nginx_check.sh" #检测nginx进程是否在线

    # script "</dev/tcp/127.0.0.1/8080" 检测服务端口是否在线

     interval 2 #检查脚本的频率,单位（秒）
     weight -30 #端口检查失败,优先级减少30,默认减少2[注意:两台主机配置的weight相同时候必须保证weight的值大于priority差值]

}

vrrp_instance VI_1 {

    state MASTER 

    interface eth0 

    virtual_router_id 51

    priority 101

    advert_int 2

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        172.17.0.210 

    }

    track_script {

       chk_nginx

    }

}

创建 /etc/keepalived/check_nginx.sh 脚本文件

A=`ps -ef | grep nginx | grep -v grep | wc -l`

if [ $A -eq 0 ];then

    nginx

    sleep 2

    if [ `ps -ef | grep nginx | grep -v grep | wc -l` -eq 0 ];then

        #killall keepalived

        ps -ef|grep keepalived|grep -v grep|awk '{print $2}'|xargs kill -9

    fi

  

fi

再对check_nginx.sh赋于执行权限：

chmod +x check_nginx.sh

注：keepalived是通过检测keepalived进程是否存在判断服务器是否宕机，如果keepalived进程在但是nginx进程不在了那么keepalived是不会做主备切换，所以我们需要写个脚本来监控nginx进程是否存在，如果nginx不存在就将keepalived进程杀掉。
在主nginx上需要编写nginx进程检测脚本（check_nginx.sh），判断nginx进程是否存在，如果nginx不存在就将keepalived进程杀掉，并将vip漂移到备份机器上。

设置 keepalived 开机启动

1

2

#设置开机自动启动

systemctl enable keepalived.service

启动keepalived服务：

1

2

# 启动

systemctl start keepalived.service

安装所有需要的依赖和环境后，将容器新增的内容重新提交到镜像文件，制作成新的镜像以供使用

1

2

3

4

#查看容器ID

docker ps

#把这个容器打包为一个镜像，5d112 就是容器 centos_01 的ID

docker commit 5d112 centos_keepalived_nginx:v1

执行完成之后，我们用 docker images 命令看一下

1

docker images

将镜像centos_keepalived_nginx:v1保存为keepalived.tar.gz

docker save -o keepalived.tar.gz centos_keepalived_nginx:v1    


这个镜像就是我们前面打包后的镜像。将keepalived.tar.gz放入另一台含有docker的服务器/home/docker/docker_test/images目录下，例如：192.168.2.2

在192.168.2.2服务器里操作：

cd home/docker/docker_test/images

从 eepalived.tar.gz 导入镜像

docker load -i keepalived.tar.gz

启动含有(keepalived+nginx)的容器。

1

docker run --privileged=true -tid --name  keepalived_master centos_keepalived_nginx:v1 /usr/sbin/init

进入keepalived_master容器：

1

docker exec -it keepalived_master bash

进入/usr/share/nginx/html，修改index.html文件

修改标题为:

Welcome to nginx Master!

启动keepalived_salve容器：

1

2

3

4

5

#启动一个容器

docker run --privileged=true -tid --name  keepalived_slave centos_keepalived_nginx:v1 /usr/sbin/init

#进入容器

docker exec -it keepalived_slave bash

修改keepalived_salve容器中nginx index.html文件

1

vim /usr/share/nginx/html/index.html

修改标题为:

Welcome to nginx Slave!

修改keepalived_salve容器中keepalived.conf文件 (master容器中，保持和镜像中设置一样即可，不需要更改)

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

! Configuration File for keepalived

global_defs {

notification_email {

762357658@qq.com

}

notification_email_from itsection@example.com

smtp_server mail.example.com

smtp_connect_timeout 30

router_id LVS_DEVEL

}

vrrp_script chk_nginx {

script "/etc/keepalived/nginx_check.sh"

interval 2

weight -5

fall 3

rise 2

}

vrrp_instance VI_1 {

state BACKUP

interface eth0

virtual_router_id 2

priority 100

advert_int 2

authentication {

auth_type PASS

auth_pass 1111

}

virtual_ipaddress {

172.17.0.210

}

track_script {

chk_nginx

}

}

其实，由配置中可以看出，主要是state和priority两个参数的调整，其中master节点的priority值一定要比backup大才行！
原理说明：
1、 通过vrrp协议广播，每个keepalived vrrp都去争取master
2、 以virtual_router_id为组队标识。 同为一个vip服务的keepalived的virtual_router_id要保持相同
3、 以priority 为权值，同一个virtual_router_id下那个priority大那个就是master,其它为backup

改完之后，重新加载

1

2

systemctl daemon-reload

systemctl restart keepalived.service

验证

查看两个容器中keepalived服务状态，两台服务状态不一样

1

systemctl status keepalived.service

双机主备切换验证


 关闭主机


关闭主机后，备机自动出现服务ip
当主机再次启动后，服务ip切换至主机

关闭keepalived 服务


关闭keepalived 服务后，服务ip自动切换至备机
当主机keepalived服务再次启动后，服务ip自动切换至主机

关闭nginx服务


停止nginx服务后，服务ip自动切换至备机
当nginx服务重新启动后，服务ip自动切换至主机
 

关闭检测的端口8080服务


停止服务后，服务ip自动切换至备机
当服务重新启动后，服务ip自动切换至主机

以上是在192.168.2.2服务器进行命令操作，也可用docker-compse.yml操作

version: '3.9'
services:
  keepalivebase:
    image: 'centos:8'
    container_name: 'keepalivebase'
    network_mode: "host"
    restart: always
    volumes:
      - "/home/docker/volumes/xxx/_data/keepalived_data/keepalived.conf:/etc/keepalived/keepalived.conf"
      - "/etc/localtime:/etc/localtime"
      - "/etc/timezone:/etc/timezone"
    privileged: true

  portainer:
        image: portainer/portainer:latest
        container_name: portainer
        restart: always
        environment:
            TZ: Asia/Shanghai
            LANG: en_US.UTF-8
        ports:
            - "9000:9000"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
            - "/etc/localtime:/etc/localtime"
            - "/etc/timezone:/etc/timezone"



version: '3.9'
services:  
    keepalive01:
        image: 'centos8_keepalived_nginx:v1'
        container_name: 'keepalive01'
        privileged: true
        entrypoint: "/sbin/init"
        environment:
            - TZ=Asia/Shanghai
        volumes:
            - ./keepalived_master.conf:/etc/keepalived/keepalived.conf
            - ./backup.sh:/etc/keepalived/backup.sh
            - ./fault.sh:/etc/keepalived/fault.sh
            - ./master.sh:/etc/keepalived/master.sh
            - ./index-master.html:/usr/share/nginx/html/index.html
        command: /bin/bash -c "chmod +x /etc/keepalived/*.sh && chmod -x /etc/keepalived/keepalived.conf"
        ports:
            - "80:80"
        networks:
            keepalive-ha:
                ipv4_address: '172.29.0.11'
    keepalive02:
        image: 'centos8_keepalived_nginx:v1'
        container_name: 'keepalive02'
        privileged: true
        environment:
            - TZ=Asia/Shanghai
        entrypoint: "/sbin/init"
        volumes:
            - ./keepalived_buckup.conf:/etc/keepalived/keepalived.conf
            - ./backup.sh:/etc/keepalived/backup.sh
            - ./fault.sh:/etc/keepalived/fault.sh
            - ./master.sh:/etc/keepalived/master.sh
            - ./index-slave.html:/usr/share/nginx/html/index.html
        command: /bin/bash -c "chmod +x /etc/keepalived/*.sh && chmod -x /etc/keepalived/keepalived.conf"
        ports:
            - "81:80"
        networks:
            keepalive-ha:
                ipv4_address: '172.29.0.12'
    cul_test:
        image: 'centos:8'
        container_name: 'cul_test'
        privileged: true
        environment:
            - TZ=Asia/Shanghai
        entrypoint: "/sbin/init"
        stdin_open: true
        tty: true
        networks:
            keepalive-ha:
                ipv4_address: '172.29.0.13'
    portainer:
        image: portainer/portainer:latest
        container_name: portainer
        restart: always
        environment:
            TZ: Asia/Shanghai
            LANG: en_US.UTF-8
        ports:
            - "9000:9000"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
        networks:
            keepalive-ha:
                ipv4_address: '172.29.0.50'

networks:
    keepalive-ha:
        name: keepalive-ha
        driver: bridge
        # driver: overlay
        ipam:
            config:
                - subnet: '172.29.0.0/16'

docker load -i keepalived.tar.gz

docker-compose  -f /home/xxx/xxx/docker-compose.yml up -d 

    keepalived的主备状态与state值设置无关；
    主备机由priority值和vrrp_script中的weight值之和决定，大的为主；
    主备比较权值=priority值+weight值*标志位，当vrrp_script检测脚本为true时标志位为1，反之为0；
    为保证正常的主备切换，weight值应大于主备priority值之差。