Skip to content

helm 部署

本文介绍如何在 Kubernetes 上通过 helm 部署 HENGSHI SENSE,以及部署后如何进行版本升级和数据备份。

部署前准备

helm 版本下载文件
v3.xhelm-v3.13.3-linux-amd64.tar.gz

提示

将 helm 二进制包下载到部署机器,拷贝到/usr/bin目录下

执行 helm version 输出以下字样说明安装完成

version.BuildInfo Version:"v3.13.3"

  1. 添加 helm 源

    shell
    helm repo add hengshi-charts https://hs-chart-repo.s3.cn-north-1.amazonaws.com.cn/charts/
  2. 更新 helm 源

    shell
    helm repo update
    helm search repo hengshi-charts # -l 列出所有版本
  3. 导入离线镜像

    shell
    wget https://download.hengshi.com/releases/hengshi-sense-xxx.tar.gz
    docker load -i hengshi-sense-xxx.tar.gz

提示

k8s/helm 环境需要将镜像推送到集群所使用的镜像仓库,如 registry、harbor、阿里云镜像仓库、腾讯云镜像仓库

集群部署

1.自定义配置文件

提示

默认hengshi最小实例数为0即不启动, 需要等待engine初始化。请注意执行命令后的NOTEST提示

示例配置 example-config.yaml

shell
hsImage:
hengshi:
  address: "registry.hengshi.org/hengshi-sense:6.0-xxxxx"
gpdb:
  address: "registry.hengshi.org/gpdb:6.25.4.0"
hsContainer:
gpdb:
  segment_replicas: 3
  gpdb_segments:
    - segment-0
    - segment-1
    - segment-2
# 如需更改存储卷大小,可以开启下列参数
hsVolume:
storage_class_name: "longhorn" # 替换成你正在使用的StorageClass
# 公有云磁盘可能会有最小限制,例如阿里云最小为20G
#  gpdb:
#    master_size: "50Gi"
#    segment_size: "50Gi"
#  metadb:
#    hengshi_db_size: "50Gi"
#  minio:
#    hengshi_minio_size: "50Gi"
#  redis:
#    hengshi_redis_size: "10Gi"

2.安装 Charts

shell
# 获取当前 chart 最新版本 -l 参数查看所有版本
helm search repo hengshi-charts
NAME                        	CHART VERSION	APP VERSION	DESCRIPTION
hengshi-charts/hengshi-sense	1.2.2        	4.2        	HENGSHI SENSE
shell
helm -n [namespace] install -f example-config.yaml hengshi-sense-6.0 hengshi-charts/hengshi-sense --version [chart_version] # chart_version 为 helm search repo hengshi-charts 列出的可用版本

# Output
NOTES:
Initialize the engine by following these commands:
  kubectl -n test exec -it master-0 -- /entrypoint.sh -m initsystem
  kubectl -n test exec -it master-0 -- /entrypoint.sh -m startsystem

Run hengshi by following these commands.
  kubectl -n test scale deployment hengshi-sense --replicas=1


Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get svc hengshi --namespace test -o jsonpath="{.spec.ports[0].nodePort}")
  export NODE_IP=$(kubectl get nodes --namespace test -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

3.修改 configmap (如需修改帐号密码等自定义配置,请执行此项,否则忽略)

shell
# 可修改的 configmap 名称: hengshi-sense greenplum metadb
kubectl edit configmap [configmap_name] -n [namespace]

配置内容参考

4.初始化 engine(已替换 hengshi 内置引擎 此步可跳过无需初始化)

shell
kubectl -n [namespace] exec -it master-0 -- /entrypoint.sh -m initsystem
kubectl -n [namespace] exec -it master-0 -- /entrypoint.sh -m startsystem

单机部署

1.自定义配置文件

提示

默认hengshi最小实例数为0即不启动, 需要等待engine初始化。请注意执行命令后的NOTEST提示

示例配置 example-config.yaml

shell
hsImage:
  hengshi:
    address: "registry.hengshi.org/hengshi-sense:6.0-xxxxxx"
  gpdb:
    address: "registry.hengshi.org/gpdb:6.25.4.0"
# 如果你需要更改存储卷大小,可以开启下列参数
hsVolume:
  storage_class_name: "longhorn" # 替换成你正在使用的StorageClass
# 公有云磁盘可能会有最小限制,例如阿里云最小为20G
#  gpdb:
#    master_size: "50Gi"
#    segment_size: "50Gi"
#  metadb:
#    hengshi_db_size: "50Gi"
#  minio:
#    hengshi_minio_size: "50Gi"
#  redis:
#    hengshi_redis_size: "10Gi"

2.安装 Charts

shell
# 获取当前 chart 最新版本 -l 参数查看所有版本
helm search repo hengshi-charts
NAME                        	CHART VERSION	APP VERSION	DESCRIPTION
hengshi-charts/hengshi-sense	1.2.2        	4.2        	HENGSHI SENSE
shell
# 使用自定义配置安装一个:"HengshiSense" 的指定版本应用
helm -n [namespace] install -f example-config.yaml hengshi-sense-6.0 hengshi-charts/hengshi-sense --version [chart_version] # chart_version 为 helm search repo hengshi-charts 列出的可用版本

# Output
NOTES:
Initialize the engine by following these commands:
  kubectl -n test exec -it master-0 -- /entrypoint.sh -m initsystem
  kubectl -n test exec -it master-0 -- /entrypoint.sh -m startsystem

Run hengshi by following these commands.
  kubectl -n test scale deployment hengshi-sense --replicas=1


Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get svc hengshi --namespace test -o jsonpath="{.spec.ports[0].nodePort}")
  export NODE_IP=$(kubectl get nodes --namespace test -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

3.修改 configmap (如需修改帐号密码等自定义配置,请执行此项,否则忽略)

shell
# 可修改的 configmap 名称: hengshi-sense greenplum metadb
kubectl edit configmap [configmap_name] -n [namespace]

配置内容参考

4.初始化 engine

shell
kubectl -n [namespace] exec -it master-0 -- /entrypoint.sh -m initsystem
kubectl -n [namespace] exec -it master-0 -- /entrypoint.sh -m startsystem

稍等几分钟查看 hengshi-sense 服务启动完成,即可通过 k8s 任意节点 NODE_IP:PORT 访问, 如需自行配置 Nginx 转发可查看当前 Service8080端口暴露的 NodePort 端口转发即可。

shell
kubectl -n [namespace] get svc
NAME            TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                                                         AGE
hengshi-sense   NodePort    10.0.0.146   <none>        8080:31204/TCP,54320:30524/TCP,15432:31062/TCP,5005:32465/TCP   286d

替换或关闭衡石组件

参考以下示例配置关闭或开启。

shell
# 以下任一组件置为 false, 安装后则不会启动
metadb_enabled: true
gpdb_enabled: true
doris_enabled: false
redis_enabled: true
minio_enabled: true
flink_enabled: false

# 示例:使用 AnalyticDB MySQL 替换 hengshi 内置 greenplum 引擎
# 1. gpdb_enabled 参数在自定义配置文件中将其置为 false
# 2. 参考 安装与启动 > 系统配置 > 引擎高级配置 中的示例配置添加到 "配置项(configmap)"
HS_ENGINE_TYPE: "mysql"
IS_ENGINE_EMBEDDED: false
SYSTEM_ENGINE_URL: "jdbc:mysql://192.168.211.4:3306/testdb?user=root&password=Test123@"
INTERNAL_ENGINE_DATASET_PATH: "enginedb"
INTERNAL_ENGINE_TMP_PATH: "enginetmp"
INTERNAL_ENGINE_OUTPUT_PATH: "enginecommon"

# 去掉以下默认的引擎变量
HS_ENGINE_TYPE: greenplum
HS_ENGINE_HOST: "master-0.gp-hs"
HS_ENGINE_PORT: "15432"
HS_ENGINE_DB: hengshi
HS_ENGINE_USR: hengshi
HS_ENGINE_PWD: hengshi202020



# 示例:使用 RDS PostgreSQL 替换 hengshi 内置 metadb
# 1. metadb_enabled 参数在自定义配置文件中将其置为 false
# 2. 参考 安装与启动 > 系统配置 > 替代衡石 metadb 中的示例配置添加到 "配置项(configmap)", 并在云上数据库创建相应的帐号和权限
HS_PG_HOST=xxx
HS_PG_PORT=5432
HS_PG_DB="hengshi"
HS_PG_USR=hengshi
HS_PG_PWD="hengshi"
INTERNAL_STORAGE_DB_NAME="hengshi_sense_internal_storage"
HS_SYSLOG_HOST=xxxx
HS_SYSLOG_PORT=5432
HS_SYSLOG_DB=syslog
HS_SYSLOG_USR=syslog
HS_SYSLOG_PWD=syslog

自定义配置文件完整示例

参考下方完整示例。

shell
# Module enable/disable
metadb_enabled: true
metadb_slave_enabled: false
gpdb_enabled: true
doris_enabled: false
redis_enabled: true
minio_enabled: true
flink_enabled: false

# Configmap
hsConfigmap:
hengshi: hengshi-sense
gpdb: greenplum
metadb: metadb
doris: doris-config
hs_engine_type: "greenplum"

hs_pg_if_external: false
hs_pg_host: "metadb-0.hengshi-hs"
hs_pg_port: "54320"
hs_pg_db: "hengshi"
hs_pg_usr: "hengshi"
hs_pg_pwd: "hengshi202020"

hs_engine_if_external: false
hs_engine_host: "master-0.gp-hs"
hs_engine_port: "15432"
hs_engine_db: "hengshi"
hs_engine_usr: "hengshi"
hs_engine_pwd: "hengshi202020"
hs_engine_query_pwd: "query202020"
hs_engine_etl_pwd: "etl202020"

hs_syslog_host: "metadb-0.hengshi-hs"
hs_syslog_db: "syslog"
hs_syslog_usr: "syslog"
hs_syslog_pwd: "syslog202020"

hs_minio_if_external: false
hs_minio_host: "http://minio-0.hengshi-hs"
hs_minio_usr: "hengshi"
hs_minio_pwd: "hengshi202020"
hs_minio_port: "9000"
hs_minio_region_code: "none"
hs_minio_bucket: "none"

hs_redis_if_external: false
hs_redis_host: "redis-0.hengshi-hs"
hs_redis_db: "0"
hs_redis_port: "6379"
hs_redis_pwd: "foobared"

# Service
hsService:
service_type:
  nodeport: NodePort
  clusterip: ClusterIP
  none: None
hengshi:
  name: "hengshi"
  sense_port: 8080
  apm_port: 5005
gpdb:
  name: "greenplum"
  greenplum_port: 15432
metadb:
  name: "metadb"
  metadb_port: 54320
metadbslave:
  name: "metadbslave"
  metadb_slave_port: 54321
minio:
  name: "minio"
  minio_port: 9001
redis:
  name: "redis"
  redis_port: 6379
flink:
  name: "flink"
  flink_rest_port: 8082
  flink_job_rpc_port: 6123
doris:
  name: "doris"

# Container
hsContainer:
hengshi:
  name: "hengshi-sense"
  replicas: 0
  start_probe_per: 10
  start_probe_success: 1
  start_probe_failure: 45
  read_probe_per: 10
  read_probe_success: 1
  read_probe_failure: 3
  live_probe_per: 10
  live_probe_success: 1
  live_probe_failure: 6
apmserver:
  name: "apmserver"
  start_probe_per: 5
  start_probe_success: 1
  start_probe_failure: 12
  read_probe_per: 5
  read_probe_success: 1
  read_probe_failure: 6
  live_probe_per: 5
  live_probe_success: 1
  live_probe_failure: 6
gpdb:
  master_name: "master"
  segment_name: "segment"
  master_replicas: 1
  segment_replicas: 1
  dns_policy: "ClusterFirst"
  gpdb_master: master-0
  gpdb_segments:
    - segment-0
metadb:
  name: "metadb"
  replicas: 1
metadbslave:
  name: "metadbslave"
  replicas: 1
minio:
  name: "minio"
  replicas: 1
redis:
  name: "redis"
  replicas: 1
flink:
  name: "flink"
  replicas: 1
doris:
  doris_be_replicas: 3
  doris_fe_replicas: 3
  doris_fe_hosts:
    - doris-fe-0
    - doris-fe-1
    - doris-fe-2
  doris_be_hosts:
    - doris-be-0
    - doris-be-1
    - doris-be-2

# Volume
hsVolume:
# 公有云磁盘可能会有最小限制,例如阿里云最小为20G
storage_class_name: "longhorn"
access_mode:
  once: ReadWriteOnce
  many: ReadWriteMany
hengshi:
  enable_pvc: false
  enable_mounts: true
  enable_volume: true
  access_mode:
    once: ReadWriteOnce
    many: ReadWriteMany
  exp_api_size: "5G"
  hengshi_geo_size: "5G"
gpdb:
  enable_mounts: true
  enable_volume: true
  master_size: "50Gi"
  segment_size: "50Gi"
metadb:
  enable_mounts: true
  enable_volume: true
  enable_empty_volume: false
  hengshi_db_size: "50Gi"
metadbslave:
  enable_mounts: true
  enable_volume: true
  hengshi_db_slave_size: "50Gi"
minio:
  enable_mounts: true
  enable_volume: true
  hengshi_minio_size: "50Gi"
redis:
  enable_mounts: true
  enable_volume: true
  hengshi_redis_size: "10Gi"
flink:
  enable_mounts: true
  enable_volume: true
  hengshi_flink_size: "10Gi"

# Resources
hsResources:
hengshi:
  requests:
    memory: "4Gi"
    cpu: "2"
  limits:
    memory: "16Gi"
    cpu: "4"
apmserver:
  requests:
    memory: "256Mi"
    cpu: "0.2"
  limits:
    memory: "2Gi"
    cpu: "1"
gpdb:
  requests:
    memory: "2Gi"
    cpu: "2"
  limits:
    memory: "2Gi"
    cpu: "2"
metadb:
  requests:
    memory: "1Gi"
    cpu: "1"
  limits:
    memory: "1Gi"
    cpu: "1"
minio:
  requests:
    memory: "1Gi"
    cpu: "1"
  limits:
    memory: "1Gi"
    cpu: "1"
redis:
  requests:
    memory: "1Gi"
    cpu: "1"
  limits:
    memory: "1Gi"
    cpu: "1"

# IMAGE
hsImage:
enable_secret: false
secret_name: secret-name
policy: "IfNotPresent"
hengshi:
  address: "registry.hengshi.org/hengshi-sense:develop"
gpdb:
  address: "registry.hengshi.org/gpdb:6.25.4.0"

# SCHEDULE
hsSchedule:
enable_node_selector: false
hengshi:
  node_selector_kv: "diskType: ssd"
gpdb:
  node_selector_kv: "diskType: ssd"
metadb:
  node_selector_kv: "diskType: ssd"
metadbslave:
  node_selector_kv: "diskType: ssd"
minio:
  node_selector_kv: "diskType: ssd"
redis:
  node_selector_kv: "diskType: ssd"
flink:
  node_selector_kv: "diskType: ssd"
doris:
  node_selector_kv: "diskType: ssd"

# Ingress
ingress:
enabled: false
className: "nginx"
annotations:
  ingress.kubernetes.io/force-ssl-redirect: "false"
  nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
  nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
  nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
  nginx.ingress.kubernetes.io/proxy-body-size: "500M"
hosts:
  - host: helm-cluster.hengshi.org
    paths:
      - path: /
        pathType: Prefix
tls: []
#  - secretName: ttt-hengshi-org-tls
#    hosts:
#      - ttt.hengshi.org
# affinity e.g:
#hs_affinity:
#  podAntiAffinity:
#    requiredDuringSchedulingIgnoredDuringExecution:
#      - labelSelector:
#          matchExpressions:
#            - key: "hengshi"
#              operator: In
#              values:
#              - "true"
#        topologyKey: "kubernetes.io/hostname"
#  nodeAffinity:
#    requiredDuringSchedulingIgnoredDuringExecution:
#      nodeSelectorTerms:
#      - matchExpressions:
#        - key: bigdata
#          operator: NotIn
#          values:
#          - "true"

升级/备份

  1. 容器部署升级

  2. k8s 环境备份与恢复

衡石分析平台使用手册