Standalone DB - Linux

Disclaimer

These results should be viewed as guidelines and not performance guarantees, since there are many variables that affect performance (file set, network configurations, hardware characteristics, etc.). If throughput is important to your implementation, OPSWAT recommends site-specific benchmarking before implementing a production solution.

Environment

Using AWS environment with the specification below:

MetaDefender Core

OS	AWS instance type	vCPU	Memory (GB)	Network bandwidth (Gbps)	Disk type	Benchmark
Ubuntu 22.04	c5.4xlarge	16	32	Up to 10	SSD	Amazon EC2 c5.4xlarge - Geekbench

Client tool

A simple tool written in Python to collect files in a designated folder and submit requests to MetaDefender Core.

files_to_scan = list of files to scan scan_futures = [asyncio.create_task(self.scan_file(file_path)) for file_path in files_to_scan] async def scan_file(self, file_path): api_url = f'{self.core_url}/file' with open(file_path, 'rb') as file: file_content = file.read() headers = {'Content-Type': 'application/octet-stream', 'filename': file_path} max_retries = 20 # Maximum number of retry attempts retry_delay = 120 # Delay in seconds before retrying for attempt in range(max_retries + 1): try: response = requests.post(api_url, data=file_content, headers=headers) status_code = response.status_code if status_code == 200: response_json = response.json() data_id = response_json.get('data_id') return status_code, response_json, data_id if status_code == 503: print(f"Received status code {status_code}. Retrying in {retry_delay} seconds...") await asyncio.sleep(retry_delay) else: return None, None, None except requests.RequestException as e: return None, None, None return None, None, None

OS	AWS instance type	vCPU	Memory (GB)	Network bandwidth (Gbps)	Disk type
CentOS 7	c5.4xlarge	16	32	Up to 10	SSD

Dataset

Detailed information of dataset below will be used for testing:

File category	File type	Number of files	Total size (MB)	Average file size (MB)
Adobe	PDF	370	385 MB	1.0 MB
Executable	EXE	45	309.5 MB	6.9 MB
	MSI	15	45.75 MB	3.1 MB
Image	BMP	80	515 MB	6.4 MB
	JPG	420	237.5 MB	0.6 MB
	PNG	345	169 MB	0.5 MB
Media	MP3	135	865 MB	6.4 MB
	MP4	50	500 MB	10.0 MB
Office	DOCX	235	190 MB	0.8 MB
	DOC	225	486 MB	2.2 MB
	PPTX	365	860 MB	2.4 MB
	PPT	355	1950 MB	5.5 MB
	XLSX	340	283.5 MB	0.8 MB
	XLS	335	284.5 MB	0.8 MB
Text	CSV	100	236 MB	2.4 MB
	HTML	1075	76 MB	0.1 MB
	TXT	500	210 MB	0.4 MB
Archive	ZIP	Compressed files: 10 Extracted files: 270	Compressed size: 125.5 MB Extracted size: 156.5 MB	Avg compressed size: 12.6 MB Avg extracted size: 0.6 MB
Summary (compressed)		5000	7728.5 MB	1.55 MB average file size
Summary (extracted)		5260	7759.5 MB	1.48 MB average file size

Product Information

Product versions:

MetaDefender Core version 5.12.1
Engines:
- 10 AV engines: Ahnlab, Avira, ClamAV, ESET, Bitdefender, K7, Quick Heal, IKarus, Cyren, Tachyon (nProtect)
- Deep CDR: 7.2.3
- Proactive DLP: 2.21.1
- Archive: 7.2.1
- File type analysis: 7.2.2
- File-based vulnerability assessment: 4.57-236

MetaDefender Core settings

General settings

Turn off data retention
Turn off engine update

Archive extraction

Max recursion level: 99999999
Max number of extracted files: 99999999
Max total size of extracted files: 99999999
Timeout: 10 minutes
Handle archive extraction task as Failed: true
- Extracted partially: true

Metascan

Max file size: 99999999
Scan timeout: 10 minutes
Per engine scan timeout: 1 minutes

Performance test results

Summary

MetaDefender Core with single engine (technology):

Use case	Scan duration	Avg./Max CPU usage	Avg./Max Memory usage	Avg. Network speed	Throughput	Avg. processing time
	(minutes)	(%)	(%)	(KB/s)	(objects/hour)	(seconds/object)
Metascan (10 AV)	30.2	69 / 98	29 / 32		252,666	0.0142
Deep CDR	19.4	48 / 62	36 / 49		393,325	0.009
Proactive DLP	16.5	34 / 39	32 / 48		462,455	0.008
Vulnerability	12	50 / 65	46 / 46		635,875	0.006

MetaDefender Core with common engine packages:

Use case	Scan duration	Avg./Max CPU usage	Avg./Max Memory usage	Avg. Network speed	Throughput	Avg. processing time
	(minutes)	(%)	(%)	(KB/s)	(objects/hour)	(seconds/object)
Metascan (10 AV) + Deep CDR	39.8	78 / 99	62 / 73		191,721	0.0188
Metascan (10 AV) + Deep CDR + Proactive DLP	43.5	61 / 84	47 / 67		175,414	0.0205
Metascan (10 AV) + Deep CDR + Proactive DLP + Vulnerability	44.8	62 / 88	69 / 83		170,324	0.0211

System resource utilization charts

MetaDefender Core with Metascan (10 AV engines) :

	Utilization charts
CPU usage
Memory usage
Network usage
Disk usage

MetaDefender Core with Metascan (10 AV engines) + Deep CDR :

	Utilization charts
CPU usage
Memory usage
Network usage
Disk usage

MetaDefender Core with Metascan (10 AV engines) + Deep CDR + Proactive DLP :

	Utilization charts
CPU usage
Memory usage
Network usage
Disk usage

MetaDefender Core with Metascan (10 AV engines) + Deep CDR + Proactive DLP + Vulnerability :

	Utilization charts
CPU usage
Memory usage
Network usage
Disk usage

MetaDefender Core with Deep CDR :

	Utilization charts
CPU usage
Memory usage
Network usage
Disk usage

MetaDefender Core with Proactive DLP :

	Utilization charts
CPU usage
Memory usage
Network usage
Disk usage

MetaDefender Core with Vulnerability :

	Utilization charts
CPU usage
Memory usage
Network usage
Disk usage

Last updated on

Was this page helpful?