Container Deployment

This article introduces how to deploy HENGSHI SENSE on a container, as well as how to perform version upgrades and data backups after deployment.

Pre-Deployment Preparation

Before deploying on a single machine, please complete the following preparations.

Check the Docker environment. Requirements:
- Docker version >= 17.09
- Install docker-compose.
Obtain and import the offline image.

shell

wget https://download.hengshi.com/releases/hengshi-sense-xxx.tar.gz
docker load -i hengshi-sense-xxx.tar.gz

Obtain the docker-compose deployment file according to the version.

Installation Version	Deployment File	Component Dependencies
3.x	docker-compose	metadb, engine, hengshi
4.0.x	docker-compose	metadb, engine, hengshi, minio
4.1.x	docker-compose	metadb, engine, hengshi, minio, redis, flink
4.2.x	docker-compose	metadb, engine, hengshi, minio, redis, flink
4.3.x	docker-compose	metadb, engine, hengshi, minio, redis, flink
4.4.x	docker-compose	metadb, engine, hengshi, minio, redis, flink
4.5.0 ~ 4.5.6	docker-compose	metadb, engine, hengshi, minio, redis, flink
4.5.7 ~ 4.5.x(x>7)	docker-compose	metadb, engine, hengshi, minio, redis, flink, apm-server
5.0.x	docker-compose	metadb, engine, hengshi, minio, redis, flink, apm-server
5.1.x	docker-compose	metadb, engine, hengshi, minio, redis, ~~flink (removed, no need to run)~~, apm-server
5.2.x	docker-compose	metadb, engine, hengshi, minio, redis, apm-server
5.3.x	docker-compose	metadb, engine, hengshi, minio, redis, apm-server

Standalone Deployment

Please follow the instructions below to deploy the HENGSHI service on a standalone machine.

Unzip the deployment manifest file

shell

# Unzip the downloaded docker-compose file
unzip -q docker-compose-x.x.zip
# Enter the configuration file directory, e.g., if you downloaded version 5.1, it would be cd docker-compose-5.1
cd docker-compose-x.x

Modify the configuration

For detailed variables, refer to .env

shell

# Rename the default configuration to .env and modify the variables in .env,
# HS_IMAGE_TAG must be changed to the tag of the imported offline image
cp hsenv .env

Tip

When configuring .env, ensure that the ports are not occupied. You can execute the netstat -anpt command to check.

If the built-in engine type is required to be Doris, adjust the HS_ENGINE_TYPE parameter.

Uncomment the "networks" section in the docker-compose file to avoid Doris failing to start due to IP changes.

Initialize the metadb and engine

shell

# Make sure to initialize the metadb and engine before installation, otherwise HENGSHI will not start
docker-compose run --rm metadb init metadb # Initialize metadb
docker-compose run --rm engine init engine # Initialize engine

If the operation is successful, you will see the following prompts.

shell

docker-compose run --rm metadb init metadb
# The following message in the logs indicates that metadb initialization is successful
INIT SUCCESS! : [metadb] /opt/hsdata/pg_data

docker-compose run --rm engine init engine
# The following message in the logs indicates that engine initialization is successful
INIT SUCCESS! : [engine] /opt/hsdata/engine-cluster
after config docker engine standalone

Start the service

shell

docker-compose up -d  # Start in detach mode

If the operation is successful, you will see the following prompts.

shell

docker-compose up -d
[+] Running 8/8
 ✔ Container hengshi-flink      Started   1.1s
 ✔ Container hengshi-metadb     Started   1.1s
 ✔ Container hengshi-engine     Started   1.0s
 ✔ Container hengshi-apmserver  Started   0.8s
 ✔ Container hengshi-minio      Started   1.0s
 ✔ Container hengshi-monit      Started   1.0s
 ✔ Container hengshi-redis      Started   1.2s
 ✔ Container hengshi-sense      Started   1.9s

Stop the service and delete the containers

shell

docker-compose down # Destroy the containers

If the operation is successful, you will see the following prompts.

shell

docker-compose down
[+] Running 9/8
 ✔ Container hengshi-monit      Removed    0.0s
 ✔ Container hengshi-apmserver  Removed    0.0s
 ✔ Container hengshi-engine     Removed   10.4s
 ✔ Container hengshi-sense      Removed    0.0s
 ✔ Container hengshi-flink      Removed   10.3s
 ✔ Container hengshi-metadb     Removed   10.4s
 ✔ Container hengshi-minio      Removed   10.4s
 ✔ Container hengshi-redis      Removed   10.2s
 ✔ Network hengshi_default      Removed    0.1s

Pause the service

shell

docker-compose stop  # Stop the container service

If the operation is successful, you will see the following prompts.

shell

docker-compose stop
[+] Running 8/8
 ✔ Container hengshi-monit      Stopped   0.0s
 ✔ Container hengshi-sense      Stopped   0.0s
 ✔ Container hengshi-apmserver  Stopped   0.0s
 ✔ Container hengshi-engine     Stopped  10.6s
 ✔ Container hengshi-minio      Stopped  10.5s
 ✔ Container hengshi-flink      Stopped  10.4s
 ✔ Container hengshi-metadb     Stopped  10.4s
 ✔ Container hengshi-redis      Stopped  10.5s

Start the paused service

shell

docker-compose start # Start the paused containers

If the operation is successful, you will see the following prompts.

shell

docker-compose start
[+] Running 8/8
 ✔ Container hengshi-flink      Started  1.2s
 ✔ Container hengshi-metadb     Started  0.9s
 ✔ Container hengshi-apmserver  Started  1.0s
 ✔ Container hengshi-redis      Started  1.0s
 ✔ Container hengshi-engine     Started  0.9s
 ✔ Container hengshi-monit      Started  1.2s
 ✔ Container hengshi-minio      Started  0.8s
 ✔ Container hengshi-sense      Started  0.7s

Example: .env File

shell

# HENGSHI SENSE image tag version
HS_IMAGE_TAG=$TAG

# Optional, configure the root path of the volumn, defaults to the directory of docker-compose.yaml. Can be configured. Please plan the data disk, it is recommended to configure the largest data disk.
HSHOME=.

# HENGSHI web service port
HS_HENGSHI_EXPOSE_PORT=8080

# HENGSHI data-gateway service port
DATA_GATEWAY_PUBLIC_EXPOSE_PORT=8079

# Optional, the port exposed by the container for metadb, default is 54320
HS_PG_EXPOSE_PORT=54320

# Optional, engine type, options are greenplum|doris
HS_ENGINE_TYPE=greenplum

# Optional, the port exposed by the engine when the engine type is greenplum
HS_ENGINE_EXPOSE_PORT=15432

# Optional, the port exposed by the engine when the engine type is doris
#HS_ENGINE_EXPOSE_PORT=9030

# Remote debug port
HS_JDWP_EXPOSE_PORT=5005

# Optional, MINIO service exposed port
HS_MINIO_EXPOSE_PORT=9891

# Optional, MINIO console exposed port
HS_MINIO_CONSOLE_EXPOSE_PORT=9892

# Optional, redis service exposed port
HS_REDIS_EXPOSE_PORT=6379

# Optional, flink service exposed port
HS_FLINK_EXPOSE_PORT=8082

# Optional, apmserver service exposed port
HS_APMSERVER_EXPOSE_PORT=8200

# MINIO connection address | login user | login key
MINIO_SERVER_HOST=hengshi-minio
MINIO_ROOT_USER=hengshi
MINIO_ROOT_PASSWORD=hengshi202020