Doris Engine Expansion

Doris Engine expansion refers to increasing the number of Doris BE nodes. During the expansion process, the current system operation and ongoing tasks are unaffected, and the performance of the current system is not compromised. Data balancing will be automatically performed. Depending on the size of the existing data in the cluster, the cluster will return to a balanced load state within a few hours to one day.

Adding Machines for Expansion

Preparations on New Machines Before Expansion

Before horizontal expansion, the following preparations need to be made on the new machines.

Please refer to Engine New Node Environment Preparation to complete the new node preparation work.

Expansion Operation

1. Start the BE service on the expansion node

$ ssh node2
hengshi@node1$ cd /opt/hengshi
hengshi@node1$ bin/hengshi-sense-bin start doris-be single

2. Check the startup status of the BE service on the expansion node

$ ssh node2
hengshi@node1$ cd /opt/hengshi
hengshi@node1$ bin/hengshi-sense-bin status doris-be single
# The following message indicates that the BE service has started successfully
[hengshi-sense-bin:48][ INFO ]: be status is started

3. Add the BE service on the expansion node to the Doris cluster

$ ssh node1
hengshi@node1$ bin/doris-client.sh
# Enter the mysql console, node2-host is the IP address of the scaling node,
mysql> ALTER SYSTEM ADD BACKEND 'node2-host:9050';
mysql> show backends;
# The result Alive field type is true, indicating that the node addition is successful. If the result is false, you may need to wait a few minutes and then execute show backends again.
+-----------+-----------------+--------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| BackendId | Cluster         | IP           | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime       | LastHeartbeat       | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag                      | ErrMsg | Version                  | Status                                                                                                                        | HeartbeatFailureCounter | NodeRole |
+-----------+-----------------+--------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| 10003     | default_cluster | node2-host   | 9050          | 9060   | 8040     | 8060     | 2023-01-11 14:26:57 | 2023-06-02 10:35:03 | true  | false                | false                 | 680       | 33.526 MB        | 416.611 GB    | 433.784 GB    | 3.96 %  | 3.96 %         | 0.000              | {"location" : "default"} |        | doris-1.2.1-rc01-075c88e | {"lastSuccessReportTabletsTime":"2023-06-02 10:34:05","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0                       | mix      |
+-----------+-----------------+--------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
# If the result still does not become true after repeated checks, please contact technical support.

4. Modify the Hengshi configuration file to add scaling node configuration information

$ ssh node1
hengshi@node1$ cd /opt/hengshi/conf
hengshi@node1$ vi hengshi-sense-env.sh
# Modify the DORIS_BE_HOSTS variable
export DORIS_BE_HOSTS=( `Deployed BE node host` `node2-host` )

Tip

Data balancing will be performed automatically. Depending on the size of the existing data in the cluster, the cluster will return to a balanced load state within a few hours to one day.