Replication & High Availability on K8S

When installing MetaDefender Storage Security in kubernetes as this technology provide the system with its self-healing feature, it might not be strictily necessary to set it up with at least 2 replicas for each of the pods in case of wanting to have High Availabitlity, but it is always a best practice to have 2 replicas of each pod running in different worker nodes. For those components that could not be running within the cluster, we will provide different solutions that are supported by our application.

However, there are a few components that may incure in high load in some situations, so for those components it is recommended to have more than 1 replica. These components are the following.

MDSS services

• Web Client -> to provide high availability to outside requests
• Api Gateway -> to provide high availability to outside requests
• Scanning Service -> to provide high availability when having to handle big number of files to scan

3rd Party components

• Database -> Deploy an external service for MongoDB with HA or MongoDB operator runnning on K8S cluster
• RabbitMQ -> Use external service for RabbitMQ with HA
• Redis Cache -> Use external service for Redis cache with HA

HA solutions for MongoDB

In production environments it's supported to use MongoDB Atlas running as a ReplicaSet Cluster with MDSS being compatible up to Mongo 6.0. For CSPs provided service it has been tested and is supported by the following:

• Amazon Document DB 5.0

If a highly available database is required inside the k8s cluster, then there are publicly available solutions that can deploy a MongoDB Replica Set. For a k8s cluster, MongoDB provides an operator for deploying a Replica Set:

https://www.mongodb.com/docs/kubernetes-operator/master/tutorial/deploy-replica-set/

HA Solution for Redis Cache

OPSWAT have tested the following services to provided HA to a redis service.

• AWS ElasticCache for Redis

HA Solution for RabbitMQ

OPSWAT have tested the following services to provided high availability to a RabbitMQ service.

• Amazon MQ for RabbitMQ

HA deployment for MDSS

MDSS containers can be scaled independently depending on the availability and performance requirements. For example, just the webclientand apigateway pods can be replicated to provide high availability to outside requests and for the web interface.

mdss_components:

  webclient:

    replicas: 3

  apigateway:

    replicas: 3

The mongomigrationsservice is used to keep the database up to date at startup and provides no benefits if replicated .

Create environment with HA components

For deploying all the components that provide a high availability, OPSWAT have prepared a terraform module to deploy all the 3rd parties applications (Redis, RabbitMQ & Document DB).

There are 2 ways of using that terraform module

• Applying changes directly using the terraform project
• Using the metadefenderk8s.sh to deploy mdss with it.

Automatic replicas using horizontal pod autoscaling (HPA)

The HPA can be enabled on all MDSS services from the values:

hpa:

  enabled: true

  minReplicas: 2

  maxReplicas: 3

  cpuTargetUtilization: 85

In the example above the number of replicas are adjusted within the specified limits depending on the measured CPU usage. The HPA is applied for all MDSS services, for a more granular autoscaling it's recommended to manually create a HPA separately from the helm deployment just for the desired services.