NGINX

This documentation is based on NGINX v1.20.0. and is the minimum configuration required to integrate NGINX and MetaDefender ICAP Server v4.x. The integration is enabled via a certified NGINX Dynamic Module and is available to early adopters. If you are interested in performing a POC with this module, please contact your account manager. This integration has been validated with NGINX versions 1.20.0, 1.20.2, and 1.21.4.

MetaDefender ICAP Server integrates with NGINX via the upstream module (reverse proxy) and related configuration directives.

Prerequisites

Integration Steps

Command Line Install

CentOS

Bash
Copy

Debian

Bash
Copy

nginx.conf - Configuration Directives

The configuration file for NGINX is located at /etc/nginx/nginx.conf.

Syntaxometascan_pass URL;
Default
Contexthttp, server, location

Sets the protocol, address and optional URI of the proxied ICAP server. “http” or “https” protocols can be specified. The address can be specified as a domain name or IP address, and an optional port:

ometascan_pass http://icap_server:8080;

If an error occurs when sending a sub-request to the ICAP Server (timeout, network issue, etc) an error will be sent back to the ICAP client, and the request will be blocked.

When using ometascan_pass, the proxy_request_buffering of the Nginx proxy module will be disabled.

Syntaxometascan_methods <list methods>;
Defaultometascan_methods GET HEAD POST PUT PATCH DELETE;
Contexthttp, server, location

This directive specifies HTTP request methods that are considered by ometascan_pass. HTTP request methods not listed will be ignored. The following HTTP methods are allowed: GET, HEAD, POST, PUT, PATCH, and DELETE

Syntaxometascan_send_timeout time;
Defaultometascan___send_timeout 60s;
Contexthttp, server, location

Sets a timeout for transmitting a request to the proxied server. The timeout is set only between two successive write operations, not for the transmission of the whole request. If the proxied server does not receive anything within this time, the connection is closed.

Time intervals can be specified in milliseconds, seconds, minutes, hours, days and so on. Refer here for more information.

Syntaxometascan_connect_timeout time;
Defaultometascan_connect___timeout 60s;
Contexthttp, server, location

Defines a timeout for establishing a connection with a proxied server. Note that this timeout should not exceed 75 seconds.

Syntaxometascan_read_timeout time;
Defaultometascan_read___timeout 60s;
Contexthttp, server, location

Defines a timeout for reading a response from the proxied server. The timeout is set only between two successive read operations, not for the transmission of the whole response. If the proxied server does not transmit anything within this time, the connection is closed.

Syntaxometascan_pre_cache on/off;
Defaultometascan_pre___cache off;
Contexthttp, server, location

Turn on/off pre-caching request when sending to ICAP Server.

Syntaxometascan_pre_cache_size size;
Defaultometascan_pre_cache_size 9223372036854775807;
Contexthttp, server, location

Config maximum caching size per request. Sizes can be specified in bytes, kilobytes (suffixes k and K) or megabytes (suffixes m and M), for example, “1024”, “8k”, “1m”.

Syntaxproxy_pass URL;
Default
Contexthttp, server, location

Defines the IP and port for the web application back-end.

Example Configuration File

Copy
  • client_max_body_size Sets the maximum allowed size of the client request body. If the size in a request exceeds the configured value, a 413 error (Request Entity Too Large) is returned to the client. Please be aware that browsers cannot correctly display this error. Setting size to 0 disables checking of client request body size.
  • ometascan_pass Set URL of the ICAP Server
  • ometascan_methods POST PUT; (only enables scans for POST and PUT methods)

Load Balancing - Example Config

Copy

Configure HTTPS

Generate the certificate

Copy

HTTPS Example Config

Bash
Copy

Add ssl, ssl_certificate and ssl_certificate_key variables. Refer here for more information regarding nginx HTTPS configuration.

Flow Diagrams

When CDR is enabled, the filename parameter of the Content-Disposition header will be renamed corresponding to Metadender Core's CDR setting. Default renaming configuration is reflected below.

Performance Testing Results

Performance test results are performed in a controlled environment and serve only as a reference. The results demonstrate the raw throughput capacity for the MD ICAP Server, outside of any scanning activity performed by MetaDefender Core. Overall solution performance will depend on several factors including available system resources, file content (dataset) and network performance.

Test Environment

ComponentValue
CPU cores8
RAM8 GB
Storage50 GB
CPU limit4000MHz
Squid cachingNo caching
# files in dataset542 (500KB-1MB)
Dataset size (run 10x)378MB
Simultaneous users (threads)100
Total requests542000

Results

MetricClient - BackendClient - Squid - Backend*Client - Squid - MD ICAP - Backend *Client - Nginx - MD ICAP - Backend
Request throughput (req/s)105.991.817.865.6
Data throughput (kB/s)75629655491300046861
Total time (sec)51185905297748260
Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard
  Last updated