AWS Deployments

Introduction

This Deployment Guide provides step-by-step instructions for deploying MetaDefender Core on Amazon Web Services infrastructure.

Organizations interested in protecting their solutions deployed in AWS can leverage MetaDefender to analyze and sanitize files residing, or transitioning, their AWS deployment. MetaDefender can scan and either sanitize or check for known vulnerabilities, depending on the type of traffic it's seeing. The ideal use case would be an organization that allows files to be uploaded to AWS through an external facing web portal. Analyzing files before they are made accessible to the end-users is critical to ensure that no malicious content is allowed and distributed through the web application.

Advanced attacks are concealing the malicious payload and are relying on productivity files (documents, pdfs, images) as a distribution mechanism. Productivity files allow active content to be leveraged, but these features are frequently exploited to execute the malicious behavior.

This guide is for IT infrastructure architects, administrators and DevOps professionals who are seeking to prevent potential malicious traffic being allowed in their AWS Cloud deployment. Threat Prevention is ensured for both productivity files that might be uploaded and for known vulnerabilities that can be identified for all running services/applications deployed in AWS. The vulnerability scanning is checking known vulnerabilities for unpatched OS and running applications.

MetaDefender is provided as:

• AMI through the AWS Marketplace (MetaDefender products)
• Packaged installer available for download through the OPSWAT Portal.
• Container Image available in the OPSWAT Docker Hub Repository

There are currently no restrictions on AWS regions in which you can deploy MetaDefender. The only restriction is that MetaDefender is not available AWS GovCloud at this time.

OPSWAT Contact Information

Sales:sales-inquiry@opswat.com

Support:https://www.opswat.com/support

Other:https://www.opswat.com/contact

For installing our solution, please review the guideline listed below:

• Single EC2
• (Link Removed)
• EKS Cluster

Database Encryption

MetaDefender Core supports both native OS encryption or EBS encrypted volumes. The database is secured via credentials and can be configured to encrypt everything at rest. In general, OPSWAT recommends the use of a 3rd party database layer, e.g. Amazon Aurora or RDS, which should be encrypted. It is best practice to not share the database layer on the same hosts/VM/EC2 instance.

Sensitive Data Storage

Network traffic processed by MetaDefender Core is SSL encrypted during transmission. The product does not collect or store any customer data outside of analyzing the contents of files submitted for scanning.

During the scanning process, files are stored unencrypted on disk to allow the scan process to complete. Once the scan completes, the file is deleted unless configured to place the file in quarantine. If you wish to encrypt these quarantined files, please use native OS encryption or EBS encrypted volumes.

EC2 Key Management

The EC2 key pair is only used to SSH or RDP to the EC2 instance. This is a very uncommon scenario, used primarily for debugging. You can create a new key pair, or use an existing pair.

Costs

MetaDefender is made available as an annual subscription. The only mandatory service that needs to be purchase is MetaDefender Core. MetaDefender Core is a paid service and is currently licensed on the BYOL (Bring Your Own License) model. Contact our sales team via our Contact form, available here: https://www.opswat.com/contact

Considering that are over 60 different options to license MetaDefender, it is highly coupled to the use case and the advanced features that you are considering deploying. Note that the more functionality you are adding to MetaDefender, the more need will be of CPU from the EC2 instance.

Our recommendation would be a minimum 8 vCPU for our lower tiers.

We recommend General Purpose (m family) and Compute Optimized (c family) instance types, preferably that are using NVMe-based SSDs for high throughput expectations:

• m5.xlarge
• m5.2xlarge
• m5.4xlarge
• c5d.2xlarge
• c5d.4xlarge
• c5d.8xlarge

We recommend to go with Reserved Instances, considering that you are committing to an annual subscription for MetaDefender.

For pricing per instance, please refer to the official AWS pricelist: https://aws.amazon.com/ec2/pricing/

Sizing

MetaDefender needs an EC2 instance with minimum 8 vCPU, in order to have an optimal response rate to submitted files for analysis. However, depending on the use case and expected throughput (analysis SLA), higher tier instances are recommended.

The system requirements (hardware and supported operating system) are defined here: Recommended System Configuration

Operation Guidance for EC2 deployments

Troubleshooting

For troubleshooting MetaDefender Core with Kubernetes deployment , please refer to the following guides:

1. Troubleshooting K8S (installation, logging, cluster management tools)

For troubleshooting MetaDefender Core with EC2 deployment , please refer to the following guides:

1. Troubleshooting MetaDefender Core (installation, logging, node management)
2. Scripted license management (Licensing status and automation)
3. Performance Scaling (for large deployments)
4. Engine and Database health (view Health Check section below)

Health Check

It is important to set a health check of the MetaDefender instance. There are multiple factors that can result in failure of the system, some of them are AWS related (AZ fault, hardware fault) or application fault. Depending on the use case, each customer defines differently what application failed is.

For generic hardware or AZ fault, we recommend always a distributed environment as defined in (Link Removed). By deploying MetaDefender in different Availability Zones with a load balancer in front of them, you will always be sure that no hardware fault will result in service interruption.

1. If you have an ELB in front of MetaDefender instances, configure it to do the health checks using the engines status.

   • How to setup ELB to do health checks: https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-healthchecks.html
   • Use Engines status as API endpoint for health check: GET Engine Status

2. In case you are not using an ELB or you would like to have a more advanced health check, our recommendation would be to setup a Lambda function to check on each MetaDefender instance. In case of the health check monitored via Lambda, you will have the flexibility to actually adapt based on the REST API response provided by MetaDefender. One of the following should apply:

   • If all engines are healthy, the instance is considered healthy

   • If some of the engines are not up to date or failing, based on the internal policy you might invalidate this MetaDefender instance.

     • Recovery of MetaDefender will be detailed in the next section

   • If the REST API response is an error, the instance should be considered down

3. An even more advanced check can be considered actually submitting a file to be analyzed using a Lambda function. Note that depending on the file size and complexity, workflow configurations and the number of files in the queue, it might result in timing out the Lambda execution.

   • Submit always the same file to MetaDefender through the REST API
   • Compare the response with a baseline

Please see this page for more information on Regular Maintenance of the MetaDefender Core services and components. For database maintenance, please refer to Database Defragmentation and Optimization.

To check the status of your MetaDefender Core license, please review the Check Your License__Details section on License Activation page.

Backup and Recovery

MetaDefender service in general does not store any persistent data that needs to be recovered or backed-up. Regardless if it's a hardware or service failure, MetaDefender will not recover the files submitted in the queue and those files will need to be resubmitted for analysis. Using OPSWAT Central Management (OCM) or a central database (such as AWS Aurora/RDS), there is no backup required per service. All configurations will automatically be shared with the new instances when they are brought up (this is the recommended model to prevent data loss). In order to have a sustainable service, not affected by any hardware or software failures, the recommendations are to:

• Store the files in an S3 bucket before they are submitted to MetaDefender

  • In order to avoid any AZ or Region failure, would be recommended to use S3's Cross-Region Replication

• Define a pool of jobs for your webapp, which are in a pending state while MetaDefender analyzes the files

  • Best option would be to use the SQS service and your application to send the files from the SQS to MetaDefender

    • Every time the files are being uploaded to a temporary S3 bucket, there is a new job added to the SQS

  • If MetaDefender fails, the job is stilled defined in SQS

  • If AZ or even the region fails, if the S3 bucket is synced in multiple regions, the SQS queue should be easily be able to be reconstructed for the remaining files in the bucket

    • On initializing stage, check the files in the S3 bucket and if there is any file left, add them to the SQS

• Based on the result from MetaDefender, the file should be moved to the final location or it's sanitized copy should. Either way, the original file should be deleted from the temporary S3 bucket

• When the job is being removed from SQS, remove also the file from the S3 bucket

In case of failure of MetaDefender service, follow the instructions defined in the Troubleshooting section: Troubleshooting MetaDefender Core

Routine Maintenance

Always update the MetaDefender deployment with the new versions published either on:

• portal.opswat.com > Products section
• AWS Marketplace

Release Notes are available here.

MetaDefender provides a direct update mechanism for the licensed analysis engines (all the engines listed in Inventory > Technology). The application logic will require a product update, however the licensed engines are automatically updated (in online deployments). Both the engine and the signature updates for Anti-malware engines will be automatically downloaded and deployed on a daily basis. We recommend to configure the product to check for updates at least every 4h (for more details see Engines Update Configuration).

In general MetaDefender instance has Internet connection. If an offline deployment is considered, make sure you are using either the Central Management or the Update Downloader and that you are uploading the engines' signatures updates minimum once a day.

If you are deploying MetaDefender in your own AWS environment, the AWS service limits will not impact the product. A potential issue that can be caused by AWS service is using a lower-tier EC2 type than is recommended. Please see the Costs section above for details on which EC2 instance types we recommend for MetaDefender. Network bandwidth and EBS/General SSD IOPS can become a limitation in high-throughput environments. The solution to this is to simply use higher tier machines such as ones with 20+Gbps network bandwidth and moving to a provisioned IOPS (io2) disk(s).

MetaDefender supports the ability to rotate system credential and cryptographic keys to ensure the security of your deployment. Information for performing this rotation can be found on the Database Management page. MetaDefender users can also change user credentials via our POST - Change Password API.

Emergency Maintenance

In case of any availability failures of an AWS Service, recommended actions would be to pass the load to another MetaDefender instances, preferably deployed in a different AZ than the one affected. Any analysis in progress submitted to MetaDefender will be considered lost. However, is highly recommended to build a resilient system that won't rely on MetaDefender to recover the pending jobs, but to be managed by a queueing mechanism (e.g. AWS SQS). See Backup and Recovery suggested deployment scenarios.

Support

To receive support, please visit our Support Portal. From there after you log in click on "Cases" on menu at the top of the page and then click "New" on the right hand side to create a new case.

Support policies, costs, levels and SLA's are described on our website, at the Support section: https://www.opswat.com/support. Please view our support tiers in the table below:

Note: Please visithttps://www.opswat.com/supportfor the most updated information on support.

Accessibility

Reference Materials

All MetaDefender documentation is available on docs.opswat.com/mdcore

Localization

MetaDefender products and documentation are available exclusively in English for now.