Kubernetes 部署 Nacos 2.4.0 集群：StatefulSet + NFS + MySQL 持久化

背景

业务需要高可用的服务注册发现中心，要求：

• Nacos 2.4.0 集群部署（3 节点）
• 数据持久化（MySQL 5.7）
• 开启鉴权（默认无密码风险）
• APISIX 网关暴露

环境信息

1. NFS 动态存储

复用现有 APISIX 的 nfs-client-provisioner：

# 查看 StorageClass
kubectl get storageclass

# 回显
NAME          PROVISIONER   RECLAIMPOLICY   VOLUMEBINDINGMODE
nfs-storage   nfs-client    Delete          Immediate

如未部署，参考以下步骤：

# 下载部署文件
wget https://gitee.com/mrchis/kubernetes/releases/download/v4.0.18/nfs-subdir-external-provisioner-nfs-subdir-external-provisioner-4.0.18.zip

# 修改配置
sed -i 's/namespace:.*/namespace: nacos/g' rbac.yaml deployment.yaml
sed -i 's#registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2#registry.cn-beijing.aliyuncs.com/xngczl/nfs-subdir-external-provisione:v4.0.0#g' deployment.yaml
sed -i 's#10.3.243.101#192.168.1.3#g' deployment.yaml
sed -i 's#/ifs/kubernetes#/data/k8s/nfs/data#g' deployment.yaml

kubectl create -f rbac.yaml
kubectl apply -f deployment.yaml

2. MySQL 部署（StatefulSet）

apiVersion: v1
kind: Namespace
metadata:
  name: nacos
---
apiVersion: v1
kind: Secret
metadata:
  name: mysql-password
  namespace: nacos
data:
  mysql_root_password: cm9vdA==  # base64(root)
---
apiVersion: v1
kind: Service
metadata:
  name: nacos-mysql-svc
  namespace: nacos
spec:
  ports:
    - port: 3306
      targetPort: 3306
      nodePort: 30306
  selector:
    app: mysql
  type: NodePort
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos-mysql
  namespace: nacos
spec:
  serviceName: "nacos-mysql-svc"
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - name: mysql
          image: registry.cn-shanghai.aliyuncs.com/mrchi/mysql:5.7.44
          ports:
            - containerPort: 3306
          env:
            - name: MYSQL_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-password
                  key: mysql_root_password
          args:
            - --character-set-server=utf8mb4
            - --collation-server=utf8mb4_unicode_ci
            - --lower_case_table_names=1
            - --default-time_zone=+8:00
          volumeMounts:
            - name: mysql-data
              mountPath: /var/lib/mysql
  volumeClaimTemplates:
    - metadata:
        name: mysql-data
      spec:
        storageClassName: "nfs-storage"
        accessModes: ["ReadWriteMany"]
        resources:
          requests:
            storage: 10Gi

初始化 Nacos 数据库

wget https://gitee.com/mrchis/kubernetes/releases/download/v2.4.0/v2.4.0-nacos-mysql-init.sql

# 连接 MySQL（NodePort 30306）
mysql -u root -p -h 192.168.1.3 -P 30306

mysql> CREATE DATABASE nacos_config;
mysql> USE nacos_config;
mysql> SOURCE /root/v2.4.0-nacos-mysql-init.sql;

3. Nacos 集群部署

ConfigMap 配置

apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-config
  namespace: nacos
data:
  jvm-xmn: "428m"
  jvm-xms: "656m"
  jvm-xmx: "856m"
  mode: "cluster"
  mysql-database-num: "1"
  mysql-service-db-name: "nacos_config"
  mysql-service-db-param: "characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true&useSSL=false"
  mysql-service-host: "nacos-mysql-0.nacos-mysql-svc.nacos.svc.cluster.local"
  mysql-service-port: "3306"
  mysql-service-user: "root"
  mysql-service-pd: "root"
  nacos-servers: "nacos-0.nacos-headless.nacos.svc.cluster.local:8848 nacos-1.nacos-headless.nacos.svc.cluster.local:8848"
  spring-datasource-platform: "mysql"
  nacos-auth-enable: "true"
  nacos-auth-token: "SecretKeyM1Z2WDc4dnVyZkQ3NmZMZjZ3RHRwZnJjNFROdkJOemEK"
  nacos-core-auth-server-identity-key: "test"
  nacos-core-auth-server-identity-value: "test"

StatefulSet 部署

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
  namespace: nacos
spec:
  selector:
    matchLabels:
      app: nacos
  serviceName: "nacos-headless"
  replicas: 2
  template:
    metadata:
      labels:
        app: nacos
    spec:
      containers:
        - name: nacos
          image: registry.cn-shanghai.aliyuncs.com/mrchi/nacos-server:v2.4.0
          ports:
            - name: server
              containerPort: 8848
            - name: clien-rpc
              containerPort: 9848
            - name: old-ratf-rpc
              containerPort: 7848
            - name: ratf-rpc
              containerPort: 9849
          env:
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: JVM_XMN
              valueFrom:
                configMapKeyRef:
                  key: jvm-xmn
                  name: nacos-config
            - name: JVM_XMS
              valueFrom:
                configMapKeyRef:
                  key: jvm-xms
                  name: nacos-config
            - name: JVM_XMX
              valueFrom:
                configMapKeyRef:
                  key: jvm-xmx
                  name: nacos-config
            - name: MODE
              valueFrom:
                configMapKeyRef:
                  key: mode
                  name: nacos-config
            - name: MYSQL_DATABASE_NUM
              valueFrom:
                configMapKeyRef:
                  key: mysql-database-num
                  name: nacos-config
            - name: MYSQL_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  key: mysql-service-db-name
                  name: nacos-config
            - name: MYSQL_SERVICE_DB_PARAM
              valueFrom:
                configMapKeyRef:
                  key: mysql-service-db-param
                  name: nacos-config
            - name: MYSQL_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  key: mysql-service-host
                  name: nacos-config
            - name: MYSQL_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  key: mysql-service-pd
                  name: nacos-config
            - name: MYSQL_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  key: mysql-service-port
                  name: nacos-config
            - name: MYSQL_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  key: mysql-service-user
                  name: nacos-config
            - name: NACOS_SERVERS
              valueFrom:
                configMapKeyRef:
                  key: nacos-servers
                  name: nacos-config
            - name: MEMBER_LIST
              valueFrom:
                configMapKeyRef:
                  key: nacos-servers
                  name: nacos-config
            - name: SPRING_DATASOURCE_PLATFORM
              valueFrom:
                configMapKeyRef:
                  key: spring-datasource-platform
                  name: nacos-config
            - name: NACOS_AUTH_ENABLE
              valueFrom:
                configMapKeyRef:
                  key: nacos-auth-enable
                  name: nacos-config
            - name: NACOS_CORE_AUTH_SERVER_IDENTITY_KEY
              valueFrom:
                configMapKeyRef:
                  key: nacos-core-auth-server-identity-key
                  name: nacos-config
            - name: NACOS_CORE_AUTH_SERVER_IDENTITY_VALUE
              valueFrom:
                configMapKeyRef:
                  key: nacos-core-auth-server-identity-value
                  name: nacos-config
            - name: NACOS_AUTH_TOKEN
              valueFrom:
                configMapKeyRef:
                  key: nacos-auth-token
                  name: nacos-config
          volumeMounts:
            - name: datadir
              mountPath: /home/nacos/data
            - name: logdir
              mountPath: /home/nacos/logs
  volumeClaimTemplates:
    - metadata:
        name: datadir
      spec:
        storageClassName: "nfs-storage"
        accessModes: ["ReadWriteMany"]
        resources:
          requests:
            storage: 5Gi
    - metadata:
        name: logdir
      spec:
        storageClassName: "nfs-storage"
        accessModes: ["ReadWriteMany"]
        resources:
          requests:
            storage: 5Gi

Service 配置

# 无头服务（集群通信）
apiVersion: v1
kind: Service
metadata:
  name: nacos-headless
  namespace: nacos
spec:
  ports:
    - port: 8848
      name: server
    - port: 9848
      name: clien-rpc
    - port: 7848
      name: old-ratf-rpc
    - port: 9849
      name: ratf-rpc
  clusterIP: None
  selector:
    app: nacos
---
# 外部访问服务
apiVersion: v1
kind: Service
metadata:
  name: nacos-external
  namespace: nacos
spec:
  ports:
    - port: 8848
      targetPort: 8848
      nodePort: 31848
  type: NodePort
  selector:
    app: nacos

4. 验证与运维

查看资源状态

kubectl get pod,svc,pvc -n nacos

查看初始化日志

kubectl logs -f nacos-1 -n nacos

修改默认密码

# 进入 MySQL 容器
mysql -u root -proot
USE nacos_config;

# 查看当前用户
SELECT username, password FROM users;

# 生成 bcrypt 密码（Python）
python3 -c 'import bcrypt; print(bcrypt.hashpw(b"New!@Passwd132", bcrypt.gensalt(10)).decode())'

# 更新密码
UPDATE users SET password = '$2b$10$xxx' WHERE username = 'nacos';

# 重启生效
kubectl rollout restart statefulset nacos -n nacos

APISIX 网关集成

# 通过 APISIX 暴露 Nacos 服务
# 域名: nacosnc-payx.kun.global

# 查询集群状态
curl http://nacosnc-payx.kun.global/nacos/v1/console/server/state

# 查看节点列表
curl http://nacosnc-payx.kun.global/nacos/v2/core/cluster/node/list

关键配置说明

复盘

问题根因：2.4.0 版本移除默认密码，首次启动需初始化

解决：通过 MySQL 直接修改 users 表密码字段

改进措施：

• 文档化密码初始化流程
• ConfigMap 集中管理配置
• NFS 存储监控告警

本文首发于 wr.mrchi.cn，转载请注明出处。