Standalone Installation and Startup

This article describes the process of installing HENGSHI SENSE in a standalone environment.

Before installation, please confirm the network environment. If it is an isolated environment and cannot connect to the internet, please first follow the instructions in Offline Environment Installation Dependencies to install the dependencies, and then continue with the instructions in this article. If the network environment can connect to the internet, please proceed with the installation according to the instructions in this article.

Preparation

1. Please refer to the Installation Environment document to prepare the installation environment.

2. Configure users and installation directories. Before proceeding, please check if you have sudo or root privileges. If not, obtain the appropriate privileges first, and then proceed with the following steps. Do not install using root. It is recommended to create a user with sudo privileges to install and run HENGSHI SENSE, and configure the installation path. In this example, the user hengshi is used, with the installation path set to /opt/hengshi. The following example demonstrates how to create the hengshi user and set the installation directory and permissions.

   grep hengshi /etc/passwd > /dev/null || sudo useradd -m hengshi         # Create hengshi user
   sudo mkdir -p /opt/hengshi && sudo chown hengshi:hengshi /opt/hengshi  # Set installation directory and permissions

3. The server sshd listens on a non-22 port. The following two situations require configuring the actual port for each host in the deployment user's ~/.ssh/config. Otherwise, proceed directly with the installation.

   1. If the SSH port of the machine deploying hengshi is not the default 22 port.
   2. If the machines deploying the engine segments are not on the same machine as the hengshi service and the SSH port is not the default 22 port.

   For example: If the local machine is configured with the hostname localhost, and HS_ENGINE_SEGMENTS=(Node-A), with the corresponding listening ports all set to 122. The .ssh/config file should include the following configuration:

   Host localhost
     Port 122
   Host Node-A
     Port 122

Installing HENGSHI SENSE

Follow these steps to complete the installation.

1. Switch to the user who will perform the installation. In this example, the username is hengshi.

sudo su - hengshi

2. Enter the target directory after decompressing the installation package.

cd ~/pkgs/hengshi-sense-[version]

3. Execute the installation command.

./hs_install -p /opt/hengshi

Tip

If you replace the HENGSHI metadb during service usage, ensure that the value of the export HS_PG_IF_EXTERNAL=false variable is true. You can change this in the configuration file conf/hengshi-sense-env.sh. This way, the installation and upgrade process will not require stopping the service.

4. The following message is displayed upon successful installation.

SUCCESSED! installed to xxxxx

System Configuration

Before starting the service, please read the Configuration File to set up the relevant configurations. If the built-in engine type requires Doris, please read the Doris Engine Configuration.

Start Service

Please follow these steps to start the service.

1. Switch to the user who starts the service, the example user name is hengshi.

sudo su - hengshi

2. Enter the installation directory.

cd /opt/hengshi

3. Initialize the OS. Please confirm that the user has sudo privileges before execution.

bin/hengshi-sense-bin init-os all

Tip

In an offline environment, after installing the dependencies, execute bin/hengshi-sense-bin init-os all-offline to skip dependency package installation.

4. Initialize HENGSHI SENSE.

bin/hengshi-sense-bin init all

5. Start the HENGSHI SENSE service.

bin/hengshi-sense-bin start all

6. When the service starts successfully, the following message is displayed, and the module status is IS ACTIVE.

 IS ACTIVE! : [metadb] @127.0.0.1:54320/hengshi
 IS ACTIVE! : [engine] @127.0.0.1:15432/hengshi
 IS ACTIVE! : [minio] minio server /opt/hengshi/minio-data
 IS ACTIVE! : [redis] redis-server .*:6379
 IS ACTIVE! : [flink] /opt/hengshi/lib/flink
 IS ACTIVE! : [syslog] syslog-ng --no-caps --module-path=/opt/hengshi/lib/syslog-ng/lib/syslog-ng -f /opt/hengshi/conf/syslog-ng.conf -p /opt/hengshi/logs/syslog-ng.pid -R /opt/hengshi/logs/syslog-ng.persist -c /opt/hengshi/logs/syslog-ng.ctl
 IS ACTIVE! : [hengshi] jar /opt/hengshi/lib/bootstrap-4.4-SNAPSHOT.jar
 IS ACTIVE! : [watchdog] /opt/hengshi/bin/hengshi-watchdog.sh

7. Access the service address via a browser, default address: http://localhost:8080 to use the HENGSHI SENSE service. If unable to access, please check if the service port HS_HENGSHI_PORT in the configuration file conf/hengshi-sense-env.sh is open to the public.

8. For nginx configuration, please refer to: Installation and Startup FAQ section 1.1.3 on how to serve through ports 80 and 443.

Operations After Starting the Service

When the HENGSHI SENSE service is running, it is necessary to regularly back up data to prevent data loss and promptly clean up unnecessary logs to free up storage space.

1. Scheduled Data Backup

   It is recommended to back up the database metadb daily, which can be backed up to local devices or remote devices. Scheduled backups are recommended to be performed during off-peak business hours, such as early morning, to avoid affecting user service usage. The following example is the execution command to back up data to a remote device at midnight every day. For detailed parameter explanations, please refer to Data Backup.

   0 0 * * * /opt/hengshi/bin/dbbackup.sh -m metadb -l /BACKUP/PATH -h $REMOTE_IP -r /BACKUP/PATH

2. Scheduled Log Cleanup

   During operation, HENGSHI SENSE generates runtime logs, which need to be cleaned up regularly to free up storage space. The following example is the command to clean up rolling logs of the internal database daily.

   0 0 * * * /opt/hengshi/bin/clean_engine.sh -t -r -c -g -p
   */5 * * * * /opt/hengshi/bin/clean_engine.sh -l

Tip

In a public network environment, do not expose the entire HENGSHI SENSE server port in non-essential scenarios. It is recommended to provide services externally through nginx + domain name. Avoid situations where attacks may occur due to component issues. In special cases, you can use the web service port (default is 8080) through the IP + port access form.

Stop Service

Execute the following command to stop the service.

sudo su - hengshi
cd /opt/hengshi
bin/hengshi-sense-bin stop all

After the service is successfully stopped, the following message is prompted, and the module status is "NOT ACTIVE".

 NOT ACTIVE! : [metadb] @127.0.0.1:54320/hengshi
 NOT ACTIVE! : [engine] @127.0.0.1:15432/hengshi
 NOT ACTIVE! : [minio] minio server /opt/hengshi/minio-data
 NOT ACTIVE! : [redis] redis-server .*:6379
 NOT ACTIVE! : [flink] /opt/hengshi/lib/flink
 NOT ACTIVE! : [syslog] syslog-ng --no-caps --module-path=/opt/hengshi/lib/syslog-ng/lib/syslog-ng -f /opt/hengshi/conf/syslog-ng.conf -p /opt/hengshi/logs/syslog-ng.pid -R /opt/hengshi/logs/syslog-ng.persist -c /opt/hengshi/logs/syslog-ng.ctl
 NOT ACTIVE! : [hengshi] jar /opt/hengshi/lib/bootstrap-4.4-SNAPSHOT.jar
 NOT ACTIVE! : [watchdog] /opt/hengshi/bin/hengshi-watchdog.sh

Check Service Status

During the service runtime, you can view the program's running status by executing the following command.

sudo su - hengshi
cd /opt/hengshi
bin/hengshi-sense-bin status all

In the prompt information during program runtime, focus on the status information of the following modules. IS ACTIVE indicates that the corresponding module is running, and NOT ACTIVE indicates that the corresponding module has stopped service.

 NOT ACTIVE! : [metadb] @127.0.0.1:54320/hengshi
 NOT ACTIVE! : [engine] @127.0.0.1:15432/hengshi
 NOT ACTIVE! : [minio] minio server /opt/hengshi/minio-data
 NOT ACTIVE! : [redis] redis-server .*:6379
 NOT ACTIVE! : [flink] /opt/hengshi/lib/flink
 NOT ACTIVE! : [syslog] syslog-ng --no-caps --module-path=/opt/hengshi/lib/syslog-ng/lib/syslog-ng -f /opt/hengshi/conf/syslog-ng.conf -p /opt/hengshi/logs/syslog-ng.pid -R /opt/hengshi/logs/syslog-ng.persist -c /opt/hengshi/logs/syslog-ng.ctl
 NOT ACTIVE! : [hengshi] jar /opt/hengshi/lib/bootstrap-4.4-SNAPSHOT.jar
 NOT ACTIVE! : [watchdog] /opt/hengshi/bin/hengshi-watchdog.sh