Doris Enable HA

HENGSHI Doris supports the deployment of multiple FE and BE nodes. Users can deploy a load balancing layer on top of multiple FEs to achieve high availability of Doris, and multiple BE nodes can ensure data security.

Cluster Installation and Deployment

The default cluster deployment method is to use multiple FE deployments, no additional configuration is required.

Single Node Installation and Deployment

To start the FE HA deployment on a single node, the idea of enabling HA is to achieve HA service by adding new nodes to build a Doris Cluster.

Tip

The total number of nodes in the Doris Cluster needs to be an odd number. The original deployment machine in this example is node1, with new machines being node2 and node3. Taking node2 as an example to illustrate the process of adding a new node, the operation for node3 is the same as that for node2, and so on.

Preparation for Adding Node Machine

Please refer to Engine New Node Environment Preparation to complete the preparation for the new node.

Add New Node to Deployed Doris Cluster

1. Start the new FE and BE node services

$ ssh node2
hengshi@node1$ cd /opt/hengshi
hengshi@node1$ bin/hengshi-sense-bin start doris-fe single
hengshi@node1$ bin/hengshi-sense-bin start doris-be single

2. Check the startup status of the newly added FE and BE services

$ ssh node2
hengshi@node1$ cd /opt/hengshi
hengshi@node1$ bin/hengshi-sense-bin status doris-be single
# The following message indicates that the BE service has started successfully
[hengshi-sense-bin:48][ INFO ]: doris-fe status is started
hengshi@node1$ bin/hengshi-sense-bin status doris-be single
# The following message indicates that the BE service has started successfully
[hengshi-sense-bin:48][ INFO ]: be status is started

3. Add the newly added FE and BE services to the Doris cluster

$ ssh node1
hengshi@node1$ bin/doris-client.sh
# Enter the mysql console, node2-host is the IP address of the scaling node,
mysql> ALTER SYSTEM ADD FOLLOWER 'node2-host:9010';
mysql> show frontends;
# The result Alive field type is true, indicating that the node addition is successful. If the result is false, you may need to wait a few minutes and then execute show frontends; again.
+---------------------------------+--------------+-------------+----------+-----------+---------+----------+----------+-----------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| Name                            | IP           | EditLogPort | HttpPort | QueryPort | RpcPort | Role     | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat       | IsHelper | ErrMsg | Version                  | CurrentConnected |
+---------------------------------+--------------+-------------+----------+-----------+---------+----------+----------+-----------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| 10.10.13.125_9010_1673418419332 | node2-host   | 9010        | 8030     | 9030      | 9020    | FOLLOWER | false     | 633068215 | true | true  | 1703188           | 2023-06-02 12:08:20 | true     |        | doris-1.2.1-rc01-075c88e | Yes              |
+---------------------------------+--------------+-------------+----------+-----------+---------+----------+----------+-----------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
# If the result still does not become true after repeated checks, please contact technical support.
mysql> ALTER SYSTEM ADD BACKEND 'node2-host:9050';
mysql> show backends;
# The result Alive field type is true, indicating that the node addition is successful. If the result is false, you may need to wait a few minutes and then execute show backends; again.
+-----------+-----------------+--------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| BackendId | Cluster         | IP           | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime       | LastHeartbeat       | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag                      | ErrMsg | Version                  | Status                                                                                                                        | HeartbeatFailureCounter | NodeRole |
+-----------+-----------------+--------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| 10003     | default_cluster | node2-host   | 9050          | 9060   | 8040     | 8060     | 2023-01-11 14:26:57 | 2023-06-02 10:35:03 | true  | false                | false                 | 680       | 33.526 MB        | 416.611 GB    | 433.784 GB    | 3.96 %  | 3.96 %         | 0.000              | {"location" : "default"} |        | doris-1.2.1-rc01-075c88e | {"lastSuccessReportTabletsTime":"2023-06-02 10:34:05","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0                       | mix      |
+-----------+-----------------+--------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
# If the result still does not become true after repeated checks, please contact technical support.

Tip

All subsequent operations are performed after the new nodes are added to the deployed Doris cluster.

Modify HENGSHI SENSE Configuration File, Add New Node Configuration Information

$ ssh node1
hengshi@node1$ cd /opt/hengshi/conf
hengshi@node1$ vi hengshi-sense-env.sh
# Modify the DORIS_FE_HOSTS variable
export DORIS_FE_HOSTS=( `node1-host` `node2-host` `node3-host` )
# Modify the DORIS_BE_HOSTS variable
export DORIS_BE_HOSTS=( `node1-host` `node2-host` `node3-host` )