Replace HENGSHI Minio

During the use of the HENGSHI SENSE service, you can replace the built-in HENGSHI Minio based on business needs. This article describes the process of using AWS S3 and Aliyun OSS to replace HENGSHI Minio.

Preparation Work

Before configuration, please complete the following preparations on AWS S3 or Alibaba Cloud OSS.

1. Create access_key and secret_access_key

   AWS operation reference: Managing Access Keys for IAM Users

   Alibaba Cloud operation reference: Access Control/Create AccessKey

2. Configure access_key and secret_access_key permissions to access the Bucket used by HENGSHI SENSE

AWS Policy Reference

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowListAllAndLocation",
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws-cn:s3:::*"
        },
        {
            "Sid": "FullAccessToSpecificBucket",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws-cn:s3:::[Your-Bucket-Name]",
                "arn:aws-cn:s3:::[Your-Bucket-Name]/*"
            ]
        }
    ]
}

Alibaba Cloud Policy Reference

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "oss:*",
      "Resource": [
        "acs:oss:oss-*:*:[Your-Bucket-Name]",
        "acs:oss:oss-*:*:[Your-Bucket-Name]/*"
      ]
    }
  ]
}

refer:

Amazon S3

Aliyun OSS

3. Endpoint (Access Domain)

   AWS access domain reference: Endpoints and ARNs (China)

   AWS access domain reference: Endpoints and ARNs (Global)

   Alibaba Cloud access domain reference: OSS Access Domain and Data Center

Modify MINIO-related Configuration

Single-node Deployment: Modify the following configuration items in the file ${HS_HOME}/conf/hengshi-sense-env.sh:

export MINIO_ROOT_USER=[access_key]
export MINIO_ROOT_PASSWORD=[secret_access_key]
export MINIO_SERVER_HOST=[endpoint]
export MINIO_SERVER_PORT=[endpoint_port] # Use 80 for HTTP protocol and 443 for HTTPS protocol
export OBJECT_REGION_CODE=[region_code]
export OBJECT_ROOT_BUCKET_NAME=[bucket_name]

export HS_MINIO_IF_EXTERNAL=true # Declare the use of external object storage to replace HENGSHI MINIO

Docker Deployment: Modify the following configuration items in the .env file located in the same directory as docker-compose.yml:

MINIO_ROOT_USER=[access_key]
MINIO_ROOT_PASSWORD=[secret_access_key]
MINIO_SERVER_HOST=[endpoint]
MINIO_SERVER_PORT=[endpoint_port] # Use 80 for HTTP protocol and 443 for HTTPS protocol
OBJECT_REGION_CODE=[region_code]
OBJECT_ROOT_BUCKET_NAME=[bucket_name]

HS_MINIO_IF_EXTERNAL=true # Declare the use of external object storage to replace HENGSHI MINIO

Cluster Deployment: Modify the following configuration items in ${HS_HOME}/ansible/vars.yml:

#------- hengshi_config will overwrite the 'deprecated' vars above ----------
hengshi_config:
- export HS_HENGSHI_PORT=8081
- export HS_PG_PORT=54320
....
- export MINIO_ROOT_USER=[access_key]
- export MINIO_ROOT_PASSWORD=[secret_access_key]
- export MINIO_SERVER_HOST=[endpoint]
- export MINIO_SERVER_PORT=[endpoint_port] # Use 80 for HTTP protocol and 443 for HTTPS protocol
- export OBJECT_REGION_CODE=[region_code]
- export OBJECT_ROOT_BUCKET_NAME=[bucket_name]
- export HS_MINIO_IF_EXTERNAL=true # Declare the use of external object storage to replace HENGSHI MINIO

K8s Deployment: Modify the following configuration items in the configmap under the hengshi namespace for hengshi-sense:

data:
  ENABLE_REFERENCE_DATASET: 'true'
  EVEREST_HISTORY_LOG_RESERVED_DAYS: '1'
  ....
  MINIO_ROOT_USER: "[access_key]"
  MINIO_ROOT_PASSWORD: "[secret_access_key]"
  MINIO_SERVER_HOST: "[endpoint]"
  MINIO_SERVER_PORT: "[endpoint_port]" # Use 80 for HTTP protocol and 443 for HTTPS protocol
  OBJECT_REGION_CODE: "[region_code]"
  OBJECT_ROOT_BUCKET_NAME: "[bucket_name]"
  HS_MINIO_IF_EXTERNAL: true # Declare the use of external object storage to replace HENGSHI MINIO

Note: After modifying the parameters, you need to restart HENGSHI SENSE for the changes to take effect.