Pre-Deployment Preparation

1. Obtain the k8s deployment configuration files based on the version.

Installation Version	Deployment Files	Component Dependencies
3.x	k8s-yaml	metadb, engine, hengshi zookeeper
4.0.x	k8s-yaml	metadb, engine, hengshi, minio, zookeeper
4.1.x	k8s-yaml	metadb, engine, hengshi, minio, redis, flink, zookeeper
4.2.x	k8s-yaml	metadb, engine, hengshi, minio, redis, flink, zookeeper
4.3.x	k8s-yaml	metadb, engine, hengshi, minio, redis, flink, zookeeper
4.4.x	k8s-yaml	metadb, engine, hengshi, minio, redis, flink, zookeeper
4.5.0 ~ 4.5.6	k8s-yaml	metadb, engine, hengshi, minio, redis, flink
4.5.7 ~ 4.5.x(x>7)	k8s-yaml	metadb, engine, hengshi, minio, redis, flink, apm-server
5.0.x	k8s-yaml	metadb, engine, hengshi, minio, redis, flink, apm-server
5.1.x	k8s-yaml	metadb, engine, hengshi, minio, redis, flink (removed, no need to run), apm-server

2. Import offline images and modify the image address.

wget https://download.hengshi.com/releases/hengshi-sense-xxx.tar.gz
docker load -i hengshi-sense-xxx.tar.gz

Tip

Except for the gpdb image address, all other components should be replaced with the imported offline image tag, e.g., image: hengshi-sense:5.0-20231103-dp-427c5f image: gpdb:x.x.x For k8s/helm environments, push the images to the image registry used by the cluster, such as registry, harbor, Alibaba Cloud Image Registry, Tencent Cloud Image Registry.

3. Replace the $(POD_NAMESPACE) variable in gpdb.yaml with the current namespace, for example, hengshi.

sed -i 's/$(POD_NAMESPACE)/hengshi/'

4. Modify PVC

• Change storageClassName: xxx to the storageclass of the current cluster.
• Modify storage: xxx to the storage size for each service.
• Modify doris.yaml for the doris engine.

metadb.yaml
gpdb.yaml
redis.yaml
minio.yaml

5. Specify the namespace, for example, hengshi.

kubectl create namespace hengshi

engine

Deploy engine

If you need to modify the password for gpdb, you need to make changes in two places:

• gpdb.yaml

  GREENPLUM_PWD: hengshi202020
  GREENPLUM_QUERY_PWD: query202020
  GREENPLUM_ETL_PWD: etl202020

• configmap.yaml

  HS_ENGINE_PWD: hengshi202020
  ENGINE_QUERY_PASSWORD: query202020
  ENGINE_ETL_PASSWORD: etl202020

Initialize and start the engine.

kubectl -n hengshi apply -f gpdb.yaml
kubectl -n hengshi exec -it master-0 -- /entrypoint.sh -m initsystem
kubectl -n hengshi exec -it master-0 -- /entrypoint.sh -m startsystem

Tip

doris engine yaml: doris.yaml does not require initsystem and startsystem operations

Deploy Additional Components

Refer to the following deployment list YAML files.

kubectl -n hengshi apply -f configmap.yaml
kubectl -n hengshi apply -f service.yaml
kubectl -n hengshi apply -f metadb.yaml
kubectl -n hengshi apply -f minio.yaml
kubectl -n hengshi apply -f redis.yaml
kubectl -n hengshi apply -f hengshi.yaml
kubectl -n hengshi apply -f ingress.yaml

Tip

configmap.yaml HENGSHI configuration file service.yaml Internal cluster communication and externally exposed service file ingress.yaml Decide whether to deploy ingress based on needs

Expose HENGSHI Service

HENGSHI provides example configurations for external access, and you can choose one of them as needed.

nodePort

Expose the HENGSHI service via nodePort (By default, if ingress is not configured, the nodePort exposed by the service can be used to provide external services).

For example, the nodePort mapping port corresponding to 8080 in the following example.

apiVersion: v1
kind: Service
metadata:
  name: hengshi
spec:
  selector:
    hsapp: hengshi-sense
    hsrole: hengshi
  ports:
  - protocol: TCP
    name: "8080"
    port: 8080
    targetPort: 8080
  - protocol: TCP
    name: "5005"
    port: 5005
    targetPort: 5005
  type: NodePort

ingress

Expose the HENGSHI SENSE service externally via ingress (optional).

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hengshi-sense
  namespace: hengshi-sense
  annotations:
    ingress.kubernetes.io/force-ssl-redirect: "false"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "90"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "90"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "90"
spec:
  ingressClassName: nginx
  rules:
    - host: xxxx.hengshi.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: hengshi-sense
                port:
                  number: 8080

Tip

ingressClassName: Please modify the ingressClass of the current cluster host: Public domain name

Basic Operations

Stop Database Services Safely

Refer to the following commands to stop metadb and engine.

kubectl -n hengshi exec -it metadb-0 -- /docker-entrypoint.sh stop metadb single

kubectl -n hengshi exec -it master-0 -- /entrypoint.sh -m stopsystem

Restart the engine

Refer to the following command to restart the engine.

kubectl -n hengshi exec -it master-0 -- /entrypoint.sh gpstop -r

Clean Logs

During the runtime, HENGSHI SENSE generates runtime logs, which need to be cleaned periodically to free up storage space. Below is an example command to clean the rolling logs of the internal database.

kubectl -n hengshi exec -it master-0 -- /bin/bash
crontab -e # Write the following scheduled statement into it and save and exit
0 0 * * * /opt/hengshi/bin/clean_engine.sh -t -r -c -g -p
*/5 * * * * /opt/hengshi/bin/clean_engine.sh -l

Scale Up Engine

1. Modify StatefulSet/segment

kubectl -n hengshi edit StatefulSet/segment

• Fill in the SEGMENTS field with the appnames of all segments after scaling up (e.g., from 2 to 4)
• Set the replicas of StatefulSet/segment to the total number of segments after scaling up

apiVersion: v1
kind: ConfigMap
metadata:
  name: greenplum
data:
  MASTER: "master-0"
  SEGMENTS: |  #List of 4 segments
    segment-0
    segment-1
    segment-2
    segment-3
...
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: segment
spec:
  replicas: 4 #For example, 4 segments after scaling up

• Then run kubectl -n hengshi apply -f gpdb.yaml
• Wait until the status of all newly added and existing segment pods becomes running

2. Write new_host_file (list of new segments, e.g., originally 2 segments (0,1), now scaled up to 4 segments (0,1,2,3))

kubectl -n hengshi exec -it master-0 /bin/bash
cd /opt/hsdata/ && mkdir expand && cd expand
cat <<EOF > new_host_file
segment-2
segment-3
EOF

3. Perform the scaling operation

kubectl -n hengshi exec -it master-0 /bin/bash
cd /opt/hsdata/expand
psql postgres -c "create database expand"
gpexpand -f new_host_file  -D expand
  >y
  >0 #This will generate the gpexpand_inputfile_yyyymmdd_xxxxxx file
gpexpand -i gpexpand_inputfile_yyyymmdd_xxxxxx -D expand

If the scaling fails, you can refer to the following commands to roll back the engine.

kubectl -n hengshi exec -it master-0 /bin/bash
cd /opt/hsdata/expand
gpstart -aR
gpexpand -r -D expand

Engine Data Migration

1. Export Data from the Old Engine

 # dump db data
kubectl exec -it $old-gp /bin/bash
source $HS_HOME/engine-cluster
pg_dumpall > /opt/hsdata/engine.back.sql
exit

2. Copy Data to the New Machine

 # cp db data
kubectl cp $old-gp:/opt/hsdata/engine.back.sql engine.back.sql
kubectl cp engine.back.sql $master-0:/opt/hsdata/engine.back.sql

3. Import Data to the New Environment

 # load db data
kubectl exec -it $master-0 /bin/bash
source $HS_HOME/engine-cluster
psql postgres < /opt/hsdata/engine.back.sql
rm /opt/hsdata/engine.back.sql

Deploy Standalone Version (POC)

1. Modify Configuration Files for Standalone Configuration

Ensure that configmap.yaml, hengshi.yaml, and other configuration files are in the same directory as config_to_single.sh before execution.

./config_to_single.sh

2. Deploy Engine

Refer to Engine Deployment

3. Deploy Other Components

Refer to the following deployment YAML files.

kubectl -n hengshi apply -f configmap.yaml
kubectl -n hengshi apply -f service.yaml
kubectl -n hengshi apply -f metadb.yaml
kubectl -n hengshi apply -f minio.yaml
kubectl -n hengshi apply -f redis.yaml
kubectl -n hengshi apply -f hengshi.yaml