Engine New Node Environment Preparation

HENGSHI SENSE comes with built-in Greenplum/Doris engine support for multi-node deployment and cluster building solutions. This article mainly introduces the environment preparation work for adding engine computing nodes.

• Variable Explanation
  • The node-1 variable used in this document represents the hostname of the deployed engine node.
  • The node-2 variable used in this document represents the hostname of the new node.

Preparation for New Node Environment

1. Node Port Opening Instructions New nodes require all (tcp/udp) ports to be open by default with the deployed engine nodes.

2. Add an execution user, with the username "hengshi" in the example.

   grep hengshi /etc/passwd > /dev/null || sudo useradd -m hengshi
   sudo mkdir -p /opt/hengshi && sudo chown hengshi:hengshi /opt/hengshi

3. Configure the server's sshd to listen on a non-22 port. The machines involved in the installation include the local machine and the machines configured in the HS_ENGINE_SEGMENTS variable. If any of these machines use a port other than ssh 22, you need to configure the actual port for each host in the deployment user's ~/.ssh/config. The local machine needs to configure the port for both localhost and the domain name returned by the hostname command. For example: if the local machine's hostname is configured as localhost and HS_ENGINE_SEGMENTS=(Node-A), the corresponding listening port should be 122. The .ssh/config file should include the following configuration:

   Host localhost
     Port 122
   Host Node-A
     Port 122

4. Configure passwordless ssh login on the new machine.

5. Create the installation path, with the installation path being /opt/hengshi in the example.

   sudo mkdir -p /opt/hengshi && sudo chown hengshi:hengshi /opt/hengshi

6. Install HENGSHI SENSE.

   sudo su - hengshi             #Switch to the product running user
   cd ~/pkgs/hengshi-sense-[version]           #Switch to the extraction target directory
   ./hs_install -p /opt/hengshi    #Execute the installation

7. Initialize the OS with sudo privileges.

   sudo su - hengshi             #Switch to the product running user
   cd /opt/hengshi                 #Enter the installation target directory
   bin/hengshi-sense-bin init-os all  #Initialize the OS

8. When the query engine type is greenplum, check the greenplum versions in the new and old environments, which need to be consistent. The HENGSHI installation directory's lib directory contains soft links to the currently used greenplum version. Check if the gpdb soft links on the old machine and the newly expanded machine point to the same version. If not, copy the greenplum from the old machine to the new machine and modify the new machine's gpdb soft link to point to the old machine's greenplum version. For example: On the old machine node1:

   hengshi@node1$ cd /opt/hengshi
   hengshi@node1$ ls lib/gpdb -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 lib/gpdb -> gpdb-6.2.1.1-centos7

   On the new machine node2:

   hengshi@node2$ cd /opt/hengshi
   hengshi@node2$ ls lib/gpdb -l
   lrwxrwxrwx 1 hengshi hengshi 21 Oct 11 23:03 lib/gpdb -> gpdb-6.18.1.1-centos7

   In this case, the two gpdb versions are inconsistent gpdb-6.2.1.1-centos7 != gpdb-6.18.1.1-centos7. Since the installation package used on the new machine is not the same as the one used on the old machine, you need to copy the old machine's gpdb to the new machine and modify the new machine's gpdb soft link.

   $ ssh node1
   hengshi@node1$ cd /opt/hengshi/lib
   hengshi@node1$ rsync -avzP gpdb-6.2.1.1-centos7 node2:/opt/hengshi/lib/
   hengshi@node1$ ssh node2 "cd /opt/hengshi/lib; rm -f gpdb; ln -s gpdb-6.2.1.1-centos7 gpdb"

9. When the query engine type is doris, check the doris versions in the new and old environments, which need to be consistent. The HENGSHI installation directory's lib directory contains soft links to the currently used Doris version. Check if the doris-be and doris-fe soft links on the old machine and the newly expanded machine point to the same version. If not, copy the Doris from the old machine to the new machine and modify the new machine's doris-be and doris-fe soft links to point to the old machine's Doris version. For example: On the old machine node1:

   hengshi@node1$ cd /opt/hengshi
   hengshi@node1$ ls lib/doris-be -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 doris-be -> doris-be-1.2.1.2-linux-64
   hengshi@node1$ ls lib/doris-fe -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 doris-fe -> doris-fe-1.2.1.2-linux-64

   On the new machine node2:

   hengshi@node2$ cd /opt/hengshi
   hengshi@node2$ ls lib/doris-be -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 doris-be -> doris-be-1.2.2.2-linux-64
   hengshi@node2$ ls lib/doris-fe -l
   lrwxrwxrwx 1 hengshi hengshi 20 Oct 11 23:03 doris-fe -> doris-fe-1.2.2.2-linux-64

   In this case, the two doris versions are inconsistent doris-be-1.2.1.2-linux-64 != doris-be-1.2.2.2-linux-64 && doris-fe-1.2.1.2-linux-64 != doris-fe-1.2.2.2-linux-64. Since the installation package used on the new machine is not the same as the one used on the old machine, you need to copy the old machine's doris-fe and doris-be to the new machine and modify the new machine's doris-fe and doris-be soft links.

   $ ssh node1
   hengshi@node1$ cd /opt/hengshi/lib
   hengshi@node1$ rsync -avzP doris-fe-1.2.1.2-linux-64 node2:/opt/hengshi/lib/
   hengshi@node1$ rsync -avzP doris-be-1.2.1.2-linux-64 node2:/opt/hengshi/lib/
   hengshi@node1$ ssh node2 "cd /opt/hengshi/lib; rm -f doris-fe doris-be; ln -s doris-fe-1.2.1.2-linux-64 doris-fe;ln -s doris-be-1.2.1.2-linux-64 doris-be"