Standalone Installation and Startup

This document describes the process of installing HENGSHI SENSE in a standalone environment.

Before installation, please confirm the network environment. If it is an isolated environment and cannot connect to the internet, please first follow the guidance in Installing Dependencies in Offline Environment to install the dependency packages, and then proceed with the instructions in this document. If the network environment can connect to the internet, you can directly follow the instructions in this document for installation.

Preparation Work

1. Please refer to the Installation Environment document to prepare the machine configuration.

2. Configure User and Installation Directory
   The installation program must be executed as a non-root user, and passwordless sudo permission is temporarily required during installation and initialization. After this, sudo permission can be revoked for startup and operation.
   Do not use the root user for installation. It is recommended to create a user with sudo permissions to install and run HENGSHI SENSE.
   In this document, we use the hengshi user as an example, with the installation path set to /opt/hengshi. Below is an example of how to create the hengshi user and set up the installation directory and permissions.

   sudo -n -l
   # If the output is "sudo: a password is required," it indicates a lack of passwordless sudo permission. Grant the permission and verify again.
   grep hengshi /etc/passwd > /dev/null || sudo useradd -m hengshi
   # Create the hengshi user
   sudo mkdir -p /opt/hengshi && sudo chown hengshi:hengshi /opt/hengshi
   # Set up the installation directory and permissions

3. SSHD on the server listens on a non-22 port (if SSHD is set to 22, this step can be skipped)
   In the following two scenarios, you need to configure the actual port for each host in the deployment user's ~/.ssh/config:

   1. If the SSH port of the machine deploying hengshi is not the default 22 port.
   2. If the machines hosting the engine's segments are different from the machine hosting the hengshi service, and their SSH ports are not the default 22 port.

   For example: The local machine is configured with the hostname as localhost, and HS_ENGINE_SEGMENTS=(Node-A), with the corresponding listening ports set to 122.
   The .ssh/config file on each machine should include the following configuration:

   Host localhost
     Port 122
   Host Node-A
     Port 122

Install HENGSHI SENSE

Follow the steps below to complete the installation.

1. Switch to the user executing the installation. In the example, the username is hengshi.

sudo su - hengshi

2. Navigate to the target directory where the installation package has been extracted.

cd ~/pkgs/hengshi-sense-[version]

3. Execute the installation command.

./hs_install -p /opt/hengshi

Tip

If the HENGSHI metadb is replaced during service usage, ensure the value of the variable export HS_PG_IF_EXTERNAL=false is set to true. This can be modified in the configuration file conf/hengshi-sense-env.sh. Doing so will prevent the service from being stopped during the upgrade process.

4. Upon successful installation, the following message will be displayed.

SUCCESSED! installed to xxxxx

Configure the System

Before starting the service, please read Configuration File to set up the relevant configurations.
If the built-in engine type requires Doris, please read Doris Engine Configuration.

Start the Service

Follow the steps below to start the service.

1. Switch to the user for starting the service. In the example, the username is hengshi.

sudo su - hengshi

2. Navigate to the installation directory.

cd /opt/hengshi

3. Initialize the OS. Before executing, ensure the user has sudo permissions.

bin/hengshi-sense-bin init-os all

Note

For offline environments, use bin/hengshi-sense-bin init-os all-offline to skip online dependency updates.

4. Initialize HENGSHI SENSE.

bin/hengshi-sense-bin init all

5. Start the HENGSHI SENSE service.

bin/hengshi-sense-bin start all

6. When the service starts successfully, the following information will be displayed, and the module status will be IS ACTIVE.

 IS ACTIVE! : [metadb] @127.0.0.1:5301/hengshi
 IS ACTIVE! : [engine] @localhost:5302/hengshi
 IS ACTIVE! : [minio] minio server /opt/hengshi/minio-data
 IS ACTIVE! : [redis] 127.0.0.1@5307
 IS ACTIVE! : [syslog] syslog-ng --no-caps --module-path=/opt/hengshi/lib/syslog-ng/lib/syslog-ng -f /opt/hengshi/conf/syslog-ng.conf -p /opt/hengshi/logs/syslog-ng.pid -R /opt/hengshi/logs/syslog-ng.persist -c /opt/hengshi/logs/syslog-ng.ctl
 IS ACTIVE! : [apmserver] /opt/hengshi/lib/apm-server/apm-server -c /opt/hengshi/conf/apm-server.yaml
 IS ACTIVE! : [hengshi] jar /opt/hengshi/lib/bootstrap-6.0.0.jar
 IS ACTIVE! : [monit] /opt/hengshi/lib/monit/bin/monit -c /opt/hengshi/conf/monitrc

7. Access the service through a browser. The default address is: http://localhost:8080. Use the HENGSHI SENSE service. If the service is not accessible, check whether the service port HS_HENGSHI_PORT is open to external access in the configuration file conf/hengshi-sense-env.sh.

8. To configure nginx, refer to: Installation and Startup FAQ section 1.1.3 on how to enable services on ports 80 and 443.

Operations After Starting the Service

When the HENGSHI SENSE service is running, it is necessary to regularly back up data to prevent data loss and clean up unused logs to free up storage space.

1. Regular Data Backup

   It is recommended to back up the database metadb daily. Backups can be stored on local devices or remote devices. Regular backups should be scheduled during non-peak business hours, such as midnight, to avoid affecting user service usage. The following example demonstrates a command to back up data to a remote device daily at midnight. For detailed parameter explanations, please refer to Data Backup.

   0 0 * * * /opt/hengshi/bin/dbbackup.sh -m metadb -l /BACKUP/PATH -h $REMOTE_IP -r /BACKUP/PATH

2. Regular Log Cleanup

   During operation, HENGSHI SENSE generates runtime logs, which need to be cleaned up regularly to free up storage space. The following example demonstrates commands to clean up rolling logs of the internal database daily.

   0 0 * * * /opt/hengshi/bin/clean_engine.sh -t -r -c -g -p
   */5 * * * * /opt/hengshi/bin/clean_engine.sh -l

Tip

Do not expose the HENGSHI service port directly to the public network. It is recommended to provide services externally through nginx + domain name to avoid potential attacks caused by component vulnerabilities.

Stop Service

Stop the service by executing the following commands:

sudo su - hengshi
cd /opt/hengshi
bin/hengshi-sense-bin stop all

After the service is successfully stopped, the following information will be displayed, and the module status will be "NOT ACTIVE":

 NOT ACTIVE! : [metadb] @127.0.0.1:5501/hengshi
 NOT ACTIVE! : [engine] @localhost:5502/hengshi
 NOT ACTIVE! : [minio] minio server /opt/hengshi/minio-data
 NOT ACTIVE! : [redis] 127.0.0.1@5507
 NOT ACTIVE! : [syslog] syslog-ng --no-caps --module-path=/opt/hengshi/lib/syslog-ng/lib/syslog-ng -f /opt/hengshi/conf/syslog-ng.conf -p /opt/hengshi/logs/syslog-ng.pid -R /opt/hengshi/logs/syslog-ng.persist -c /opt/hengshi/logs/syslog-ng.ctl
 NOT ACTIVE! : [apmserver] /opt/hengshi/lib/apm-server/apm-server -c /opt/hengshi/conf/apm-server.yaml
 NOT ACTIVE! : [hengshi] jar /opt/hengshi/lib/bootstrap-6.0-SNAPSHOT.jar
 NOT ACTIVE! : [monit] /opt/hengshi/lib/monit/bin/monit -c /opt/hengshi/conf/monitrc

Check Service Status

During service operation, you can check the program's running status by executing the following commands.

sudo su - hengshi
cd /opt/hengshi
bin/hengshi-sense-bin health all

The first column represents the module status information:

• Healthy highlighted indicates the corresponding module is running
• Stopped highlighted indicates the corresponding module has stopped
• Starting highlighted indicates the corresponding module is starting

[Stopped Starting **Healthy**]: [metadb] hengshi@127.0.0.1:5301/hengshi
[Stopped Starting **Healthy**]: [engine] hengshi@localhost:5302/hengshi
[Stopped Starting **Healthy**]: [minio] minio server /opt/hengshi/minio-data
[Stopped Starting **Healthy**]: [redis] redis-server .*:5307
[Stopped Starting **Healthy**]: [syslog] syslog-ng --no-caps --module-path=/opt/hengshi/lib/syslog-ng/lib/syslog-ng -f /opt/hengshi/conf/syslog-ng.conf -p /opt/hengshi/logs/syslog-ng.pid -R /opt/hengshi/logs/syslog-ng.persist -c /opt/hengshi/logs/syslog-ng.ctl
[Stopped Starting **Healthy**]: [apmserver] /opt/hengshi/lib/apm-server/apm-server -c /opt/hengshi/conf/apm-server.yaml
[Stopped Starting **Healthy**]: [hengshi] jar /opt/hengshi/lib/bootstrap-5.4.5.jar
[Stopped Starting **Healthy**]: [monit] /opt/hengshi/lib/monit/bin/monit -c /opt/hengshi/conf/monitrc