Autoscaling K8S Cluster
There are different levels and ways to auto-scale an application when running in a Kubernetes cluster.
- When autoscaling the pods running in the Kubernetes cluster it is needed to run the Horizontal Pod AutoScaler
- To ensure that the Kubernetes cluster has enough nodes to schedule the pods to scale, it is recommended to use the Kubernetes Cluster AutoScaler.
Pre-requisites
- Kubernetes Cluster already deployed
- Kubernetes Metrics Server installed - Installing the Kubernetes Metrics Server - Amazon EKS
- Kubectl client installed
Limitations
- Manual scale get disabled by the HPA
Autoscaling Architecture on K8S Cluster
In grey the components needed to be added to have a full autoscaling architecture with AWS as example.
Steps to install Horizontal Pod Autoscaler for Core
- Deploy HPA with kubectl
kubectl autoscale deployment md-core --cpu-percent=95 --min=1 --max=5Steps to install Cluster Autoscaler (Manual Configuration)
Permissions
Cluster Autoscaler requires the ability to examine and modify EC2 Auto Scaling Groups. We recommend using IAM roles for Service Accounts to associate the Service Account that the Cluster Autoscaler Deployment runs as with an IAM role that is able to perform these functions
For that it is needed to attach the following policy __example_scripts/iam_policy_cluster_autoscaler.json to the service account with name cluster-autoscaler
One ASG Setup
Cluster Autoscaler can be configured manually if you wish by passing the --nodes argument at startup. The format of the argument is --nodes=<min>:<max>:<asg-name>, where <min> is the minimum number of nodes, <max> is the maximum number of nodes, and <asg-name> is the Auto Scaling Group name.
- Both
<min>and<max>must be within the range of the minimum and maximum instance counts already specified by the Auto Scaling group. - When manual configuration is used, the Auto Scaling groups must use EC2 instance types that provide equal CPU and memory capacity.
It is needed to replace the following variables in example_scripts/cluster-autoscaler-one-asg-aws.yaml before applying the changes <MIN_NODES>, <MAX_NODES> and <K8S_ASG_NAME>
kubectl apply -f example_scripts/cluster-autoscaler-one-asg-aws.yamlScript provided by OPSWAT with all Steps
OPSWAT provides an script that automatically replace all the variables from the files needed to create HPA and Cluster Autoscaler
cd example_scriptsinstall_hpa_and_cluster_autoscaler_aws.sh --min <min_nodes> --max <max_nodes> --region <asg_region> --cluster_name <cluster_name> --asg_name <k8s_asg_name> --account_id <aws_account_id>"