跳至主要內容

WEB集群部署

fastbee大约 6 分钟

WEB集群部署

服务器资源分配
服务器角色服务器 A服务器 B
Nginx 负载均衡❌（单点部署于 B）✅（主负载均衡）
Spring Boot 应用✅（Fastbee，含 Ignite 节点）✅（Fastbee，含 Ignite 节点）
数据库✅（MySQL + Tdengine）主从模式待实现....
缓存✅（Redis 单点，带持久化）主从架构待实现...
1. 环境准备（两台服务器均执行）

  • 安装 Docker 和 Docker Compose

    # 安装Docker
curl -fsSL https://get.docker.com | sh
systemctl start docker && systemctl enable docker

# 安装Docker Compose
curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
2. 服务器 A 部署：数据库、Redis、Fastbee服务

  • ​ 假设两台服务器A和B，公网IP：123.0.0.1 和 124.0.0.2 内网: 10.1.1.10 和 20.2.2.20

  • ​ 打包后端jar包 fastbee-admin.jar

  • ​ 打包前端 dist 包

  • 配置docker-compose文件

    按照文档提示进行修改

    version: '3.8'

services:
redis:
 image: redis:7.0.0
 container_name: redis
 ports:
   - 6379:6379
 privileged: true
 network_mode: host
 volumes:
   - /var/data/redis:/usr/local/etc/redis
   - /var/data/redis/data:/data
 command: [ '--requirepass', 'admin123', '--appendonly', 'yes' ] #密码是admin123,自行修改

mysql:
 image: mysql:5.7
 container_name: mysql
 environment:
   MYSQL_ROOT_PASSWORD: admin123  #密码是admin123,自行修改
   MYSQL_DATABASE: fastbee  #数据库名，自行更改
 ports:
   - 3306:3306
 privileged: true
 network_mode: host
 volumes:
   - /var/data/mysql/data:/var/lib/mysql
   - /var/data/mysql/initdb:/docker-entrypoint-initdb.d
 command:
   [
     'mysqld',
     '--character-set-server=utf8',
     '--collation-server=utf8_unicode_ci',
     '--default-time-zone=+8:00',
     '--lower-case-table-names=1',
     '--bind-address=0.0.0.0'
   ]

# 假设两台服务器A和B，公网IP：123.0.0.1 和 124.0.0.2   内网:  10.1.1.10 和 20.2.2.20
java:
 image: openjdk:8-jre
 container_name: java
 ports:
   - 8080:8080
   - 47500-47509:47500-47509
   - 47101:47101
   - 11211:11211
   - 10800:10800
   - 48500:48500
   - 48880:48880
 environment:
   WORKER_HOST: 123.0.0.1                           # 服务器A公网IP 或内网IP
   IGNITE_LOCAL_IP: 10.1.1.10                       # 服务器A内网IP
   IGNITE_CLUSTER_NODES: 10.1.1.10,20.2.2.20        # 服务器A和B内网IP
 network_mode: host  
 volumes:
   - /var/data/java/fastbee-admin.jar:/server.jar
   - /var/data/java/license:/license
   - /var/data/java/libtaos.so:/usr/lib/libtaos.so
   - /var/data/java/uploadPath:/uploadPath
   - /var/data/java/logs:/logs
   - /etc/localtime:/etc/localtime
 depends_on:
   - redis
   - mysql
   - tdengine
 entrypoint:
   - sh
   - -c
   - |
     exec java \
       -Xmx4g -Xms4g \          #4核8G配置,根据情况调优
       -DIGNITE_QUIET=false \
       -DIGNITE_NO_DISCO_ORDER=true \
       -Djava.net.preferIPv4Stack=true \
       -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m \
       -XX:+UseG1GC -XX:G1HeapRegionSize=32m \
       -XX:ParallelGCThreads=4 -XX:ConcGCThreads=2 \
       -Dfile.encoding=UTF-8 \
       -Dignite.heap.offheap.size=4g \    #4核8G配置,根据情况调优
       -jar /server.jar

tdengine:
 image: tdengine/tdengine:3.3.2.0
 hostname: tdengine
 container_name: tdengine
 ports:
   - 6030:6030
   - 6041:6041
   - 6043-6049:6043-6049
   - 6043-6049:6043-6049/udp
 network_mode: host
 volumes:
   - /var/data/tdengine/log:/var/log/taos
   - /var/data/tdengine/data:/var/lib/taos
   - /var/data/tdengine/conf:/etc/taos
   - /etc/localtime:/etc/localtime
3. 服务器 B 部署：Nginx 负载均衡、Spring Boot 应用

nginx.conf负载均衡配置

user nginx;
worker_processes auto;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 1024;
    use epoll;
    multi_accept on;
}

http {
    include mime.types;
    default_type application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;

    keepalive_timeout 65;
    keepalive_requests 1000;
    client_max_body_size 100m;

    gzip on;
    gzip_min_length 1k;
    gzip_buffers 16 64k;
    gzip_http_version 1.1;
    gzip_comp_level 5;
    gzip_types text/plain application/javascript application/x-javascript text/javascript text/css application/xml;
    gzip_vary on;
    gzip_proxied expired no-cache no-store private auth;
    gzip_disable "MSIE [1-6]\.";

    # 服务器A和B的内网IP  自行更改
    upstream backend {
        server 10.1.1.10:8080 max_fails=1 fail_timeout=10s;  # 服务A内网IP,自行修改
        server 20.2.2.20:8080 max_fails=1 fail_timeout=10s;  # 服务B内网IP,自行修改
        keepalive 60;
        least_conn;
    }

    server {
        listen 80;
        server_name localhost;
        charset utf-8;

        # SSL 默认访问端口号为443
        listen 443 ssl;
        server_name  localhost;
        charset utf-8;

        # 证书文件的路径
        ssl_certificate /usr/share/nginx/ssl/fastbee.crt;
        # 私钥文件的路径
        ssl_certificate_key /usr/share/nginx/ssl/fastbee.key;
        ssl_session_timeout 10m;
        # 请按照以下协议配置
        ssl_protocols TLSv1.2 TLSv1.3;
        # 请按照以下套件配置，配置加密套件，写法遵循openssl 标准
        ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
        ssl_prefer_server_ciphers on;

        #................其他配置

        location /prod-api/ {
            proxy_pass http://backend/;
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header REMOTE-HOST $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Connection "";
            proxy_redirect off;

            proxy_connect_timeout 300s;
            proxy_send_timeout 300s;
            proxy_read_timeout 300s;
            send_timeout 300s;
        }
    }
}
  • docker-compose配置修改
version: '3.8'

services:
  nginx:
    image: nginx:stable
    container_name: nginx
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/data/nginx/vue:/usr/share/nginx/html
      - /var/data/nginx/view:/usr/share/nginx/view
      - /var/data/nginx/h5:/usr/share/nginx/h5
      # - /var/data/nginx/ssl:/usr/share/nginx/ssl    #SSL证书有则放开
      - /var/data/nginx/logs:/var/log/nginx
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
    network_mode: host

  # 假设两台服务器A和B，公网IP：123.0.0.1 和 124.0.0.2   内网:  10.1.1.10 和 20.2.2.20
  java:
    image: openjdk:8-jre
    container_name: java
    ports:
      - 8080:8080
      - 47500-47509:47500-47509
      - 47101:47101
      - 11211:11211
      - 10800:10800
      - 48500:48500
      - 48880:48880
    environment:
      WORKER_HOST: 124.0.0.2                          # 服务器A公网IP 或内网IP
      IGNITE_LOCAL_IP: 20.2.2.20                      # 服务器A内网IP
      IGNITE_CLUSTER_NODES: 10.1.1.10,20.2.2.20       # 服务器A和B内网IP
    volumes:
      - /var/data/java/fastbee-admin.jar:/server.jar
      - /var/data/java/license:/license
      - /var/data/java/libtaos.so:/usr/lib/libtaos.so
      - /var/data/java/uploadPath:/uploadPath
      - /var/data/java/logs:/logs
      - /etc/localtime:/etc/localtime
    network_mode: host
    entrypoint:
      - sh
      - -c
      - |
        exec java \
          -Xmx4g -Xms4g \
          -DIGNITE_QUIET=false \
          -DIGNITE_NO_DISCO_ORDER=true \
          -Djava.net.preferIPv4Stack=true \
          -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m \
          -XX:+UseG1GC -XX:G1HeapRegionSize=32m \
          -XX:ParallelGCThreads=4 -XX:ConcGCThreads=2 \
          -Dfile.encoding=UTF-8 \
          -Dignite.heap.offheap.size=4g \
          -jar /server.jar
4. 服务目录

按照上面修改好后

  • 将项目中 docker目录下面，ServerA中 data复制至 服务器A /var 目录下
  • 将项目中 docker目录下面，ServerB中 data复制至 服务器B /var 目录下

执行目录授权

cd /var/data

setenforce 0

chmod 777 -R /var/data
5.部署

在服务器A /var/data 执行

docker-compose up -d

# 查看部署情况
#[root@VM-xxx data]# docker ps
CONTAINER ID   IMAGE                       COMMAND                    CREATED          STATUS          PORTS     NAMES
429020770a4d   openjdk:8-jre               "sh -c 'exec java \\\n…"   16 minutes ago   Up 16 minutes             java
eebb0c7c40e9   tdengine/tdengine:3.3.2.0   "/tini -- /usr/bin/e…"     16 minutes ago   Up 16 minutes             tdengine
501697f9bfa7   redis:7.0.0                 "docker-entrypoint.s…"     16 minutes ago   Up 16 minutes             redis
0b31966146f5   mysql:5.7                   "docker-entrypoint.s…"     16 minutes ago   Up 16 minutes             mysql

在服务器B /var/data 执行

docker-compose up -d

# 查看部署情况
#[root@VM-xxx data]# docker ps
CONTAINER ID   IMAGE           COMMAND                    CREATED              STATUS              PORTS     NAMES
9e7622669ec5   openjdk:8-jre   "sh -c 'exec java \\\n…"   About a minute ago   Up About a minute             java
4ed9ca408461   nginx:stable    "/docker-entrypoint.…"     About a minute ago   Up About a minute             nginx
6.kafka负载均衡
1.分区与副本机制

数据分片：每个topic被划分到多个分区，部分在不同Broker上，生产者发送消息，通过分区策略将消息均匀分配到各分区，实现写入负载均衡

副本备份: 每个分区有多个副本，Leader处理读写请求， Follower同步数据，Leader故障，Follower替换

2. 消费者组

同一消费者组内的消费者通过分配策略（Round Robin，Range），分摊分区消费任务，例如通过轮训分配，确保同一消费者组中分区数据平衡

3.实施(重点)

kafka的分区无法在代码实现，需要借助外部工具，例如之前提到的 kafka-console-ui

下面举例实现数据上报的消费负载均衡：

目前数据上报主题 fmq_property_post 消费实例是 10个，假设现在集群部署两个web服务节点，该主题的消费实例总数是 20，那么要达到数据上报消费负载均衡需要满足 分区数 >= 所有节点主题消费实例总数

更改分区数

images
images
./images/kafka-banlc
./images/kafka-banlc

验证kafka负载均衡

生产者发送数据，查看 消费者组 - 消费情况 -每个分区的消费是否有并相对均衡

11
11
7.验证Ignite集群

方式1：查看java容器 sys-info.log

Cluster [hosts=2, CPUs=8, servers=2, clients=0, topVer=4, minorTopVer=1] hosts =2 表示集群节点为2

Metrics for local node (to disable set 'metricsLogFrequency' to 0)
    ^-- Node [id=592e782c, name=fastbee-server, uptime=03:37:01.004]
    ^-- Cluster [hosts=2, CPUs=8, servers=2, clients=0, topVer=4, minorTopVer=1]
    ^-- Network [addrs=[10.0.24.6], localHost=10.0.24.6, discoPort=47501, commPort=47101]
    ^-- CPU [CPUs=4, curLoad=0.83%, avgLoad=1.04%, GC=0%]
    ^-- Heap [used=612MB, free=85.05%, comm=4096MB]
    ^-- Outbound messages queue [size=0]
    ^-- Public thread pool [active=0, idle=0, qSize=0]
    ^-- System thread pool [active=0, idle=4, qSize=0]
    ^-- Striped thread pool [active=0, idle=8, qSize=0]
  • 方式2：可视化界面

​ 待开发....

8.后端负载均衡验证

  • 方式1：在 Spring Boot 中添接口打印请求来源 IP：

    定位ToolController，后端全路径为 iot/tool/test

@GetMapping("/test")
public String test(HttpServletRequest request) {
    return "接收到请求，来源 IP: " + request.getRemoteAddr();
}
  • 方式2：停止某节点 JAVA容器服务
docker stop java

前端访问无出现 正常，即验证成功

上次编辑于:
贡献者: kerwincui
蜂信物联 | 曲靖蜂信科技有限公司
Copyright © 2025 fastbee