Engine New Node Environment Preparation

HENGSHI SENSE comes with built-in Greenplum/Doris engine support for multi-node deployment to build cluster solutions. This document mainly introduces the environment preparation work for adding engine computation nodes.

• Variable Description
  • The variable node-1 used in this document represents the hostname of an already deployed engine node.
  • The variable node-2 used in this document represents the hostname of the new node.

New Node Environment Preparation Work

1. Node Port Opening Instructions
   The new node requires all default (tcp/udp) ports to be mutually open with the already deployed engine nodes.

2. Add execution user, with the username hengshi as an example.

   grep hengshi /etc/passwd > /dev/null || sudo useradd -m hengshi
   sudo mkdir -p /opt/hengshi && sudo chown hengshi:hengshi /opt/hengshi

3. Configure the server-side sshd to listen on a non-22 port.
   For machines involved in the installation, including the local machine and those configured in the HS_ENGINE_SEGMENTS variable, if there are non-ssh 22 ports, you need to configure the actual port for each host in the deployment user's ~/.ssh/config.
   The local machine needs to configure the ports for localhost and the domain name returned by the hostname command. For example: if the hostname of the new machine is Node-A and HS_ENGINE_SEGMENTS=(Node-A), the corresponding listening ports are all 122.
   The .ssh/config configuration file should include the following settings:

   Host localhost
     Port 122
   Host Node-A
     Port 122

4. Configure passwordless SSH login on the new machine.

5. Create the installation path, with /opt/hengshi as the example installation path.

   sudo mkdir -p /opt/hengshi && sudo chown hengshi:hengshi /opt/hengshi

6. Install HENGSHI SENSE.

   sudo su - hengshi             # Switch to the product runtime user
   cd ~/pkgs/hengshi-sense-[version]           # Navigate to the extracted target directory
   ./hs_install -p /opt/hengshi    # Execute the installation

7. Initialize the OS under sudo privileges.

   sudo su - hengshi             # Switch to the product runtime user
   cd /opt/hengshi                 # Navigate to the installation target directory
   bin/hengshi-sense-bin init-os all  # Initialize the OS

8. When the query engine type is Greenplum, check the Greenplum versions in the old and new environments to ensure consistency.
   In the HENGSHI installation directory under lib, the gpdb symbolic link points to the currently used Greenplum version. Check whether the gpdb symbolic link in the old machine and the new expansion machine points to the same version. If not, copy the Greenplum from the old machine to the new machine and modify the gpdb symbolic link on the new machine to point to the Greenplum version of the old machine.
   For example:
   On the old machine node1:

   hengshi@node1$ cd /opt/hengshi
   hengshi@node1$ ls lib/gpdb -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 lib/gpdb -> gpdb-6.2.1.1-centos7

   On the new machine node2:

   hengshi@node2$ cd /opt/hengshi
   hengshi@node2$ ls lib/gpdb -l
   lrwxrwxrwx 1 hengshi hengshi 21 Oct 11 23:03 lib/gpdb -> gpdb-6.18.1.1-centos7

   At this point, the two gpdb versions are inconsistent: gpdb-6.2.1.1-centos7 != gpdb-6.18.1.1-centos7. Since the installation package used by the new machine is not the same as the one used in the old machine environment, you need to copy the gpdb from the old machine to the new machine and modify the gpdb symbolic link on the new machine.

   $ ssh node1
   hengshi@node1$ cd /opt/hengshi/lib
   hengshi@node1$ rsync -avzP gpdb-6.2.1.1-centos7 node2:/opt/hengshi/lib/
   hengshi@node1$ ssh node2 "cd /opt/hengshi/lib; rm -f gpdb; ln -s gpdb-6.2.1.1-centos7 gpdb"

9. When the query engine type is Doris, check the Doris versions in the old and new environments to ensure consistency.
   In the HENGSHI installation directory under lib, the doris-be and doris-fe symbolic links point to the currently used Doris version. Check whether the doris-be and doris-fe symbolic links in the old machine and the new expansion machine point to the same version. If not, copy the doris-be and doris-fe from the old machine to the new machine and modify the doris-be and doris-fe symbolic links on the new machine to point to the Doris version of the old machine.
   For example:
   On the old machine node1:

   hengshi@node1$ cd /opt/hengshi
   hengshi@node1$ ls lib/doris-be -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 doris-be -> doris-be-1.2.1.2-linux-64
   hengshi@node1$ ls lib/doris-fe -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 doris-fe -> doris-fe-1.2.1.2-linux-64

   On the new machine node2:

   hengshi@node2$ cd /opt/hengshi
   hengshi@node2$ ls lib/doris-be -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 doris-be -> doris-be-1.2.2.2-linux-64
   hengshi@node2$ ls lib/doris-fe -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 doris-fe -> doris-fe-1.2.2.2-linux-64

   At this point, the two Doris versions are inconsistent: doris-be-1.2.1.2-linux-64 != doris-be-1.2.2.2-linux-64 and doris-fe-1.2.1.2-linux-64 != doris-fe-1.2.2.2-linux-64. Since the installation package used by the new machine is not the same as the one used in the old machine environment, you need to copy the doris-fe and doris-be from the old machine to the new machine and modify the doris-fe and doris-be symbolic links on the new machine.

   $ ssh node1
   hengshi@node1$ cd /opt/hengshi/lib
   hengshi@node1$ rsync -avzP doris-fe-1.2.1.2-linux-64 node2:/opt/hengshi/lib/
   hengshi@node1$ rsync -avzP doris-be-1.2.1.2-linux-64 node2:/opt/hengshi/lib/
   hengshi@node1$ ssh node2 "cd /opt/hengshi/lib; rm -f doris-fe doris-be; ln -s doris-fe-1.2.1.2-linux-64 doris-fe;ln -s doris-be-1.2.1.2-linux-64 doris-be"