OPSWAT Metascan is an advanced threat detection and prevention technology that runs multiple anti-malware engines concurrently to maximize the likelihood of catching known malware. While a single antivirus engine can detect 40%-80% of malware, Metascan enables cybersecurity specialists to scan files with over 30 market-leading anti-malware engines on-premises (Windows and Linux are supported) and in the cloud (MetaDefender Cloud) to achieve detection rates greater than 99% (See our report). Our solution not only increases detection rates, decreases outbreak detection times, but also provides resiliency for single vendor anti-malware solutions. Metascan is one of the critical software modules within OPSWAT MetaDefender Core and is continually enhanced by evaluating and adding the most effective antivirus (AV) providers into our engine supplier list. We are always on the lookout for new security partners to add to our multiscanning solution to better protect our customers from ever-more sophisticated cybercrimes. In this blog, we'll cover the technical evaluation processes for AVs before being added to Metascan.
Evaluation processes go through four distinct phases: package requirement validation, third-party components check, quick integration, and automation test.
The first phase of the evaluation is SDK (software development kit) package requirement validation. Based on our experience in integrating with more than 30 leading anti-malware engines, we came up with a standard and simple set of requirements:
- To facilitate the integration, the SDK package needs to be in C or C++ interface. Ordinarily, a scanning process with CLI (Command Line Interface) has three steps: initialization (including loading the entire database), scanning, and deinitialization. The whole process occurs for every scanned file which slows down the scanning process. Whereas with C++ integration, the system needs to be initialized just once and waits for incoming files to scan. We only have to deinitialize the system when stopping the entire product service.
- The qualified engine should have separate engine module files and definition files to facilitate delivery as a small package and should not be updated unintentionally.
- Additionally, we serve many critical infrastructure industries with air-gapped environment, so the provided engines must support offline update of definition files.
- To provide an advanced threat prevention solution to our customers, we necessitate several other requirements for the added AVs such as thread-safe, high throughput, and a standalone SDK without installation process.
If all the package requirements are met, we move to the second phase of evaluation which is examining the compliance of the package. It is scanned with a third-party tool to detect all vulnerabilities or license issues. If any issues are found, we notify the AV vendor to solve it before continuing the evaluation process.
The third phase of the requirement is an integration check to see if the engine can be seamlessly integrated and runs smoothly. Based on the sample code or integration guide, we start very basic functions like initialization and scanning. Then a quick test is conducted to assure that the integration is correct by scanning EICAR anti-virus test file and clean file. For data security control, we utilize a network monitoring program during the test to ensure that the engine doesn't send any data to their home server.
Furthermore, we have developed a comprehensive test framework to measure performance metrics including throughput, memory leaking, CPU consumption and thread safety. As shown in the figure hereunder, we conducted a test with 2 scenarios: single-thread scanning and multiple-thread scanning (20 threads in this test). Based on the performance measurement, we can identify existing errors or issues made by the AV during the scanning process.
We use thousands of sample files including both known malicious and benign files and get them scanned by the engine under evaluation to gauge the detection rate (for both false-positives and false-negatives). The framework also monitors the AV’s footprint to discover potential memory leaks or higher-than-desired CPU consumption. For instance, in the test demo above, the memory usage increased in four different inspections that reveals a possible memory leak. Likewise, the testing result revealed the engine throughput as well as any failures during the scanning process, which were logged for further investigation.
Subsequently, we executed a stress test, which was run for a day with a much bigger dataset, to further investigate the AV’s performance and stability. We built an integration test environment in a docker container. If any issues are found during this phase, we share the identified problems with the AV vendor together with the test container to maintain consistent test environments.
After carefully assessing the AV’s integration and performance, we deploy the official integration with Metascan if it passes all our rigorous testing, confirm the partnership agreement and announce the addition of a new anti-malware engine to our customers.
Our meticulous AV evaluation process is to assure that an impeccable, dynamic and efficient security product is provided to our customers. It also establishes a close-knit and successful collaboration between OPSWAT and our technology partners to together protect our customers against increasingly advanced cyber-attacks. We constantly hunt for new AV suppliers to join our multiscanning solution. For possible partnership with OPSWAT, please contact us now. We’re always happy to answer any questions.
