Configuration File

The configuration file stores the configuration parameters for HENGSHI SENSE, allowing you to modify the configuration file to change the service mode of HENGSHI SENSE.

Configuration File Description

The configuration files are stored in the conf directory, including hengshi-sense-env.sh and engine-segment-hosts. The engine-segment-hosts file is used to store configuration information for the acceleration engine's segment table, while hengshi-sense-env.sh contains all configuration items except for the acceleration engine segment host list.

Tip

If conf/hengshi-sense-env.sh does not exist, you can generate this file by copying the conf/hengshi-sense-env.sh.sample template. Changes to the configuration files require a restart of the HENGSHI service to take effect.

Acceleration engine related configuration is only required when installing the acceleration engine.

Engine Management Module master The engine management module master is an independent machine, and its IP or hostname can be configured through the HS_ENGINE_HOST in the conf/hengshi-sense-env.sh configuration file.
Engine Data Query Module segment The engine data query module segment can be configured on multiple machines, and their IPs or hostnames are configured through the conf/engine-segment-hosts configuration file, with each line being the IP or hostname of a segment. By default, the system will generate a configuration containing only localhost.

Data Directory Configuration Recommendations

Please refer to the following recommendations for data directory configuration.

At initial installation, the program and system data require at least 3G of space.
If you upload many files and they occupy a large amount of space, it is recommended to configure the HS_HENGSHI_DATA directory to a larger disk. System data, uploaded data, and acceleration engine data are all stored in this directory.
If you need the acceleration engine to accelerate large amounts of data, it is recommended to configure the HS_ENGINE_HOME directory to a larger disk. After configuration, the engine acceleration data will be stored independently from HS_HENGSHI_DATA in this directory.

Common Configuration Variables Explanation

All configuration items need to be written into the conf/hengshi-sense-env.sh file.

Parameter	Purpose
JAVA_HOME	The environment directory for the runtime Java.
HS_HENGSHI_PORT	HENGSHI WEB service port, default is 8080. BI prohibits root startup, configure proxy forwarding for ports below 1024, such as nginx, etc.
HS_HENGSHI_DATA	The root directory where HENGSHI stores all data, default is under the installation directory. When there is a large partition dedicated to data storage, this variable can be configured.
HS_PG_HOST	The internal database host of the program. This variable can be modified to configure when not using HENGSHI metadb.
HS_PG_PORT	The listening port of the internal database of the program, default is 54320. This variable can be used to change the port when it conflicts with existing service ports.
HS_PG_DB	The database name of the internal database of the program.
HS_PG_USR	The username of the internal database of the program.
HS_PG_PWD	The password of the internal database of the program.
HS_ENGINE_HOME	The path of the engine data files.
HS_ENGINE_HOST	The engine host. After configuration, the master module of the engine will be deployed on the corresponding machine, default is localhost.
HS_ENGINE_SEGMENTS	Engine worker nodes, array, default is (localhost). For multiple nodes, the example is (Node-A Node-B Node-C).
HS_ENGINE_PORT	The engine port.

The maximum memory for HENGSHI startup JVM defaults to 6G. If you need to modify it, use HENGSHI_JAVA_OPTS. For example, to change it to a minimum of 1G and a maximum of 4G, configure export HENGSHI_JAVA_OPTS="-Xms1g -Xmx4g" in conf/hengshi-sense-env.sh.

Encrypting Passwords in Configuration Files

If there is a requirement to store only encrypted passwords in the conf/hengshi-sense-env.sh configuration file, you can proceed as follows:

Generate the encrypted password The following command will generate the encrypted text for the password 'abc123abc123abc123abc123abc123abc123abc123abc123':

cd /opt/hengshi;
bin/gete.sh "abc123abc123abc123abc123abc123abc123abc123abc123"
# Program output
U2FsdGVkX1/Ol74LahZWp6HlMspWmBHL5c4/s20Aw7o/z89q9pUITWfE/w2RCAu9
eyjEPtdGLOfXg3OGVcr9CJRPwgmpQWmWHvZ4j1aBmH8=

Replace the variable in the configuration file For example, replace the configuration for "HS_PG_RO_PWD":

HS_PG_RO_PWD="abc123abc123abc123abc123abc123abc123abc123abc123"
# Replace with
E_HS_PG_RO_PWD="U2FsdGVkX1/Ol74LahZWp6HlMspWmBHL5c4/s20Aw7o/z89q9pUITWfE/w2RCAu9
eyjEPtdGLOfXg3OGVcr9CJRPwgmpQWmWHvZ4j1aBmH8="

Note: If both plaintext and encrypted configurations exist simultaneously, the encrypted configuration takes precedence. For example, if both HS_PG_RO_PWD and E_HS_PG_RO_PWD are configured, "E_HS_PG_RO_PWD" will be the one that takes effect.

Variable mapping for encrypted configurations

Plaintext Variable	Encrypted Variable
HS_PG_PWD	E_HS_PG_PWD
HS_PG_RO_PWD	E_HS_PG_RO_PWD
HS_ENGINE_PWD	E_HS_ENGINE_PWD
HS_SYSLOG_PWD	E_HS_SYSLOG_PWD
HS_ACCESSLOG_PWD	E_HS_ACCESSLOG_PWD
QUARTZ_PWD	E_QUARTZ_PWD
GREENPLUM_QUERY_PWD	E_GREENPLUM_QUERY_PWD
GREENPLUM_ETL_PWD	E_GREENPLUM_ETL_PWD
MINIO_ROOT_PASSWORD	E_MINIO_ROOT_PASSWORD
REDIS_PASSWORD	E_REDIS_PASSWORD

Custom JDBC Configuration

If the version of the data source to be connected is listed in the "Special Data Source Version JDBC Driver Table," you need to download the corresponding version driver, unzip the driver zip, and copy the unzipped content to the plugins directory under the installation directory. The changes will take effect after restarting the HENGSHI service.

Special Data Source Version JDBC Driver Table:

Server Version	Download Link
hive-1.1	https://download.hengshi.com/3rd/hive-1.1-jdbc-minimal.zip

Note

After placing the JDBC driver into the plugins directory, you can only connect to the corresponding version of the service. Connectivity to other versions is not guaranteed. For example, if you download the hive-1.1 driver and place it in the plugins directory, you can only guarantee connectivity to hive-1.1, and connectivity to other hive versions is not guaranteed.

The above method modifies the default JDBC driver for the entire service. If you need to use more than two versions of JDBC drivers simultaneously, you can use the following method:

Taking hive1.1.1, hive1.1.0, and hive1.0.0 as examples, the system needs to use the default hive JDBC driver and these three versions of the hive JDBC driver simultaneously. Unzip the corresponding version dependency packages and place the jar files in a corresponding directory on the HENGSHI server, for example, /opt/hengshi/jdbc-jar/hive1.1.1, /opt/hengshi/jdbc-jar/hive1.1.0, /opt/hengshi/jdbc-jar/hive1.0.0.
Add the configuration file /opt/hengshi/jdbc-jar/jdbc_conf.json with the following content:

json

[
  {
    // Same as the options.type field of the corresponding data connection
    "name":"hive",
    "versions":[
      {
        // Specific version number
        "version":"1.1.1",
        // Path to the jar file
        "jdbcJarPath":"/opt/hengshi/jdbc-jar/hive1.1.1",
        // Full name of the driver's main class, can be omitted if it is the same as the default version.
        "driverName":"org.apache.hive.jdbc.HiveDriver",
        "configs":{
          // hive1.1 does not support autoCommit setting, here it is false.
          // All additional HENGSHI-specific configurations need to start with _hs_ so that they can be removed before building the connection pool.
          "_hs_set_auto_commit":false
        }
      },
      // Multiple versions are placed under different elements of the versions array
      {
        "version":"1.1.0",
        "jdbcJarPath":"/opt/hengshi/jdbc-jar/hive1.1.0",
        "driverName":"org.apache.hive.jdbc.HiveDriver",
        "configs":{
          "_hs_set_auto_commit":false
        }
      },
      {
        "version":"1.0.0",
        "jdbcJarPath":"/opt/hengshi/jdbc-jar/hive1.0.0",
        "driverName":"org.apache.hive.jdbc.HiveDriver",
        "configs":{
          "_hs_set_auto_commit":false
        }
      }
    ]
  }
  // Multiple different connections are placed under different elements of the outermost array
  // , {...}
]

Add the configuration item export HENGSHI_JDBC_CONFIG=/opt/hengshi/jdbc-jar/jdbc_conf.json in the configuration file conf/hengshi-sense-env.sh and restart the HENGSHI service.
When creating a new data connection, select the corresponding version. default refers to the version within HENGSHI, and others are custom driver versions.

Note

Cluster deployment: The above operations need to be performed on all [HENGSHI] nodes.
Docker container deployment: For adding configuration items, modify the docker-compose.yaml configuration file, add HENGSHI_JDBC_CONFIG: /opt/hengshi/jdbc-jar/jdbc_conf.json under hengshi>environment, and then restart the HENGSHI service.
k8s cluster deployment: Adjust the jar package and configuration file location, change the /opt/hengshi/jdbc-jar directory to /opt/hsdata/jdbc-jar. For adding configuration items, modify the configmap.yaml configuration file, add HENGSHI_JDBC_CONFIG: /opt/hengshi/jdbc-jar/jdbc_conf.json, and then restart the HENGSHI service.

Flink Web Interface Open Access Configuration

shell

# The default configuration is export FLINK_REST_BIND_ADDRESS=127.0.0.1
# Add the following configuration in conf/hengshi-sense-env.sh
export FLINK_REST_BIND_ADDRESS=0.0.0.0 # IP address 0.0.0.0 or the server's local IP

Note

The Flink Web interface can be accessed without a password. It is not recommended to directly open the public Flink port, as this may increase the risk of the server being maliciously attacked. If you need to use this feature, it is recommended to access it after configuring a reverse proxy tool such as nginx, avoiding direct access via the port.

Secondary Access Path Configuration

The requirement for secondary path access relies on reverse proxy tools such as Nginx, and different versions have different configuration methods.

Configuration Method One

This configuration method is applicable to version 5.0.2 and above.

Nginx Reference Configuration

During the configuration process, take the /bi secondary path as an example.

prefix style

text

   location /bi/ {             #<====[1/3] Path address prefix
        proxy_set_header Accept-Encoding "";  #<====[2/3] Do not miss this, it is required when compression is enabled
        proxy_set_header X-HS-Sub-Path "/bi/"; #<====[3/3] /bi/ path address prefix, consistent with the location declaration
        proxy_hide_header Access-Control-Allow-Origin;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://hengshi-server/;
   }

longest matching prefix style

text

   location ^~ /bi/ {             #<====[1/3] Path address prefix
        proxy_set_header Accept-Encoding "";  #<====[2/3] Do not miss this, it is required when compression is enabled
        proxy_set_header X-HS-Sub-Path "/bi/"; #<====[3/3] /bi/ path address prefix, consistent with the location declaration
        proxy_hide_header Access-Control-Allow-Origin;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://hengshi-server/;
   }

regex style

text

   location ~ ^/bi {             #<====[1/3] Path address prefix
        proxy_set_header Accept-Encoding "";  #<====[2/3] Do not miss this, it is required when compression is enabled
        proxy_set_header X-HS-Sub-Path "/bi/"; #<====[3/3] /bi/ path address prefix, consistent with the location declaration
        proxy_hide_header Access-Control-Allow-Origin;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        rewrite ^/bi/(.*) /$1 break;  #<==== Rewrite URI
        proxy_pass http://hengshi-server;  #<==== Path ending without URI, here there is no trailing '/'
   }

Ingress Reference Configuration

The configuration process uses the /bi secondary path as an example.

yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header Accept-Encoding "";     #<====[2/3] Do not miss this, it is required when compression is enabled
      proxy_set_header X-HS-Sub-Path "/bi/";   #<====[3/3] /bi/ path prefix, consistent with the location declaration
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Proto $scheme;
spec:
  ingressClassName: nginx
  rules:
    - host: example.hengshi.com
      http:
        paths:
          - path: /bi/     #<====[1/3] Path prefix
            pathType: Prefix

Configuration Method Two

This configuration method applies to versions 5.0.1 and 5.0.

Nginx Reference Configuration

During the configuration process, take the /bi secondary path as an example.

text

   upstream hengshi-server {
         server 127.0.0.1:8080;
      }

prefix style

   location /bi/ {             #<====[1/3] Path address prefix, no regex matching
        proxy_set_header Accept-Encoding "";  #<====[2/3] Do not miss this, required when compression is enabled
        proxy_hide_header Access-Control-Allow-Origin;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://hengshi-server/;
        sub_filter_once on;
        sub_filter '<base href="/">' '<base href="/bi/" />';  #<====[3/3] /bi/ path address prefix, consistent with the location declaration
    }

regex style

    location ^/bi/(.*) {             #<====[1/4] Path address rule
        proxy_set_header Accept-Encoding "";  #<====[2/4] Do not miss this, required when compression is enabled
        proxy_hide_header Access-Control-Allow-Origin;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://hengshi-server/$1;  #<====[3/4] Pass the part after the path in the uri '$1' to the upstream
        sub_filter_once on;
        sub_filter '<base href="/">' '<base href="/bi/" />';  #<====[4/4] /bi/ path address prefix, consistent with the location declaration
    }

exact match (This type can only match one api address and cannot match other apis, used in special scenarios)

    location = /bi/api/auth/login-info {       #<====[1/4] Exact match path
        proxy_set_header Accept-Encoding "";  #<====[2/4] Do not miss this, required when compression is enabled
        proxy_hide_header Access-Control-Allow-Origin;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://hengshi-server/api/auth/login-info;  #<====[3/4] Write the full path of the upstream
        sub_filter_once on;
        sub_filter '<base href="/">' '<base href="/bi/" />';  #<====[4/4] /bi/ path address prefix, consistent with the location declaration
    }

Ingress Reference Configuration

The configuration process uses the /bi secondary path as an example.

yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hengshi-sense
  namespace: hengshi
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header Accept-Encoding "";
      proxy_hide_header Access-Control-Allow-Origin;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Proto $scheme;
      sub_filter_once on;
      sub_filter '<base href="/">' '<base href="/bi/" />';
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
    - host: example.hengshi.com
      http:
        paths:
          - path: /bi(/|$)(.*)
            pathType: Prefix
            backend:
              service:
                name: hengshi-sense
                port:
                  number: 8080

Tip

The CONTEXT_PATH parameter is no longer effective in this version.

AI Assistant

Connect Data Source

Database

NoSQL/NewSQL

SQL on Hadoop

Cloud

Searching

Multi Dimensional Database

SaaS API

Create Dataset

Dataset Management

parameter

Dashboard

Control Settings

Chart Controls

Indicator Class

map

Table

Advanced Chart Calculations

Display Widgets

Functional Controls

Filter

HENGSHI SENSE Embedded Data Analysis

Data Permission Related Practices

Configuration File