Instructional Guide for OpenSearch Auto Scaling service¶

OpenSearch Auto Scaling dynamically adjusts the number of instances in your OpenSearch cluster based on CPU usage. This ensures optimal performance and cost efficiency by scaling out during high-demand periods and scaling in when demand decreases.

This guide provides step-by-step instructions for deploying and destroying the OpenSearch Auto Scaling service using the search-hq CLI commands. Follow the steps in each section carefully to ensure successful deployment and destruction of the service.

Deployment: Step-by-step instructions to deploy the OpenSearch Auto Scaling service.
Destruction: Instruction to destroy the OpenSearch Auto Scaling service.

Prerequisites¶

Launch an Instance from Search HQ Launch Pack¶

The Search HQ Launch Pack is an AMI available on AWS Marketplace. Before proceeding with the steps below, you need to launch an instance using this AMI. Follow the Setup Instructions to set up the instance, enabling you to use the search-hq CLI commands.

Deployment¶

Follow these steps to deploy the OpenSearch Auto Scaling service.

Step 1: Configure Your YAML File¶

Configure the settings in the conf/project.yaml file properly. Specifically, ensure the settings in the auto_scaling section are uncomment and correct.

For detailed information, refer to Instructional Guide for Project Configuration.

##  You can configure the project name.
project_name: your-project-name  # Replace 'your-project-name' with the name of your project.


##  You can configure the AWS region.
aws:
  region_name: ap-northeast-1  # AWS region where resources will be deployed. Modify if needed (e.g., 'ap-northeast-1').


##  You need to use the existing VPC.
vpc:
  vpc_id: your-vpc-id  # Replace 'your-vpc-id' with the ID of your existing VPC.


##  You need to use the existing OpenSearch.
opensearch:
  domain: your-opensearch-domain  # Replace 'your-opensearch-domain' with the domain VPC endpoint for OpenSearch.
  domain_name: your-opensearch-domain-name  # Replace 'your-opensearch-domain-name' with the specific domain name.

  ##  If the fine-grained access control in your OpenSearch is enabled,
  ##  provide the necessary information in the 'auth' section as required.
  ##  By default, this setting is disabled. To enable it, uncomment the 'auth' section.
  # auth:
  #   type: aws  ##  The authentication type. Options: `aws` for AWS IAM authentication or `basic` for account and password. If `basic` is selected, you need to provide master_user_secret_arn.
  #   master_user_secret_arn: auth-arn  # The ARN of the SSM Parameter Store or AWS Secrets Manager where the master user’s account and password information is stored.

  ##  You can build OpenSearch Auto Scaling mechanism by configuring the auto_scaling section.
  ##  By default, this setting is disabled. To enable it, uncomment the 'auto_scaling' section.
  auto_scaling:
    min_instance_count: 2  # Minimum number of instances for auto scaling.
    max_instance_count: 12  # Maximum number of instances for auto scaling.
    scale_down_cpu_util_threshold: 30  # CPU utilization threshold to scale down instances.
    scale_up_cpu_util_threshold: 80  # CPU utilization threshold to scale up instances.

Step 2: Deploy OpenSearch Auto Scaling service¶

Execute the following command to deploy the OpenSearch Auto Scaling service:

search-hq opensearch-auto-scaling deploy

This command will use the configuration from your conf/project.yaml file to set up the auto scaling for your OpenSearch domain.

Destruction¶

Step 1: Destroy the OpenSearch Auto Scaling service¶

To destroy the OpenSearch Auto Scaling service, execute the following command:

search-hq opensearch-auto-scaling destroy --destroy_infra=False

This command will disable and remove the OpenSearch Auto Scaling service.

Note

If you are using only the auto scaling service, set destroy_infra to True to completely remove the service. However, if you are using other Search HQ services simultaneously, set destroy_infra to False to prevent shared resources from being deleted.

Step 2: Destroy the associated resources (Optional)¶

By default, the destroy command will retain the Amazon S3 bucket and AWS Systems Manager Parameter Store associated with your search-hq deployment. You can choose to manually delete these resources if needed.

To remove associated resources, follow these additional steps:

Remove Objects from Amazon S3 Bucket: To delete the entire Amazon S3 bucket associated with your search-hq deployment, use the following command. This will remove the entire bucket and all objects within it.
```
python -m operation.cli.s3 delete_bucket
```
If you only need to delete a single object from the Amazon S3 bucket, use:
```
python -m operation.cli.s3 delete_object --key <S3_OBJECT_KEY>
```
Replace <S3_OBJECT_KEY> with the key of the object you want to delete from the Amazon S3 bucket.
Remove Parameters from AWS Systems Manager Parameter Store: To delete all parameters associated with your search-hq deployment from AWS Systems Manager Parameter Store, use the following command. This will remove all parameters.
```
python -m operation.cli.ssm delete_ssm_parameters
```
If you only need to delete a single parameter, use:
```
python -m operation.cli.ssm delete_ssm_parameter --parameter_name <PARAMETER_NAME>
```
Replace <PARAMETER_NAME> with the name of the parameter you want to delete from the AWS Systems Manager Parameter Store.

By following the steps outlined in this guide, you can deploy and manage the OpenSearch Auto Scaling service efficiently. Ensure your YAML configurations are correct, and use the provided search-hq commands to manage the service as needed.