NGINX

This documentation is based on NGINX v1.20.0. and is the minimum configuration required to integrate NGINX and MetaDefender ICAP Server v4.x. The integration is enabled via a certified NGINX Dynamic Module and is available to early adopters. If you are interested in performing a POC with this module, please contact your account manager. This integration has been validated with NGINX versions 1.20.0, 1.20.2, and 1.21.4.

Info

MetaDefender ICAP Server integrates with NGINX via the upstream module (reverse proxy) and related configuration directives.

Prerequisites

Integration Steps

Command Line Install

CentOS

sudo rpm -i nginx-module-ometascan-1.20.0+0.1.0.50d7c3f-1.el7.ngx.x86_64.rpm

Debian

sudo dpkg -i nginx-module-ometascan-dbg_1.20.0+0.1.0.50d7c3f-1~buster_amd64.deb

nginx.conf - Configuration Directives

The configuration file for NGINX is located at /etc/nginx/nginx.conf.

Syntax

ometascan_pass URL;

Default

Context

http, server, location

Info

Sets the protocol, address and optional URI of the proxied ICAP server. “http” or “https” protocols can be specified. The address can be specified as a domain name or IP address, and an optional port:

ometascan_pass http://icap_server:8080;

Info

If an error occurs when sending a sub-request to the ICAP Server (timeout, network issue, etc) an error will be sent back to the ICAP client, and the request will be blocked.

Warning

When using ometascan_pass, the proxy_request_buffering of the Nginx proxy module will be disabled.

Syntax

ometascan_methods <list methods>;

Default

ometascan_methods GET HEAD POST PUT PATCH DELETE;

Context

http, server, location

Info

This directive specifies HTTP request methods that are considered by ometascan_pass. HTTP request methods not listed will be ignored. The following HTTP methods are allowed: GET, HEAD, POST, PUT, PATCH, and DELETE

Syntax

ometascan_send_timeout time;

Default

ometascan___send_timeout 60s;

Context

http, server, location

Info

Sets a timeout for transmitting a request to the proxied server. The timeout is set only between two successive write operations, not for the transmission of the whole request. If the proxied server does not receive anything within this time, the connection is closed.

Time intervals can be specified in milliseconds, seconds, minutes, hours, days and so on. Refer here for more information.

Syntax

ometascan_connect_timeout time;

Default

ometascan_connect___timeout 60s;

Context

http, server, location

Info

Defines a timeout for establishing a connection with a proxied server. Note that this timeout should not exceed 75 seconds.

Syntax

ometascan_read_timeout time;

Default

ometascan_read___timeout 60s;

Context

http, server, location

Info

Defines a timeout for reading a response from the proxied server. The timeout is set only between two successive read operations, not for the transmission of the whole response. If the proxied server does not transmit anything within this time, the connection is closed.

Syntax

ometascan_pre_cache on/off;

Default

ometascan_pre___cache off;

Context

http, server, location

Info

Turn on/off pre-caching request when sending to ICAP Server.

Syntax

ometascan_pre_cache_size size;

Default

ometascan_pre_cache_size 9223372036854775807;

Context

http, server, location

(((Info)))

Config maximum caching size per request. Sizes can be specified in bytes, kilobytes (suffixes k and K) or megabytes (suffixes m and M), for example, “1024”, “8k”, “1m”.

Syntax

proxy_pass URL;

Default

Context

http, server, location

Info

Defines the IP and port for the web application back-end.

Example Configuration File

load_module modules/ngx_http_ometascan_module.so; events { worker_connections 4096; } http { client_max_body_size 1000M; server { listen 80; server_name localhost; location ^~ / { ometascan_pass http://icap_server.domain:8080; ometascan_methods POST PUT; ometascan_read_timeout 1d; proxy_pass http://back_end_server.domain:8000; } } }
Info
  • client_max_body_size Sets the maximum allowed size of the client request body. If the size in a request exceeds the configured value, a 413 error (Request Entity Too Large) is returned to the client. Please be aware that browsers cannot correctly display this error. Setting size to 0 disables checking of client request body size.

  • ometascan_pass Set URL of the ICAP Server

  • ometascan_methods POST PUT; (only enables scans for POST and PUT methods)

Load Balancing - Example Config

load_module /etc/nginx/modules/ngx_http_ometascan_module.so; http { client_max_body_size 1000M; upstream icap { server icap_server.first.domain:8043; server icap_server.second.domain:8043; } server { listen 80; server_name localhost; location ^~ / { ometascan_pass http://icap; ometascan_methods POST PUT; ometascan_read_timeout 1d; proxy_pass http://back_end_server.domain:8000; } } }

Configure HTTPS

Generate the certificate

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout certs.key -out certs.pem

HTTPS Example Config

load_module /etc/nginx/modules/ngx_http_ometascan_module.so; http { client_max_body_size 1000M; upstream icap { server icap_server.first.domain:8043; server icap_server.second.domain:8043; } server { listen 443 ssl; server_name localhost; ssl_certificate /path/to/certs.pem; ssl_certificate_key /path/to/certs.key; location ^~ / { ometascan_pass http://icap; ometascan_methods POST PUT; ometascan_read_timeout 1d; proxy_pass http://back_end_server.domain:8000; } } }
Info

Add ssl, ssl_certificate and ssl_certificate_key variables. Refer here for more information regarding nginx HTTPS configuration.

Flow Diagrams


Info

When CDR is enabled, the filename parameter of the Content-Disposition header will be renamed corresponding to Metadender Core's CDR setting. Default renaming configuration is reflected below.


Performance Testing Results

Info

Performance test results are performed in a controlled environment and serve only as a reference. The results demonstrate the raw throughput capacity for the MD ICAP Server, outside of any scanning activity performed by MetaDefender Core. Overall solution performance will depend on several factors including available system resources, file content (dataset) and network performance.

Test Environment

Component

Value

CPU cores

8

RAM

8 GB

Storage

50 GB

CPU limit

4000MHz

Squid caching

No caching

files in dataset

542 (500KB-1MB)

Dataset size (run 10x)

378MB

Simultaneous users (threads)

100

Total requests

542000

Results

Metric

Client - Backend

Client - Squid - Backend

*Client - Squid - MD ICAP - Backend *

Client - Nginx - MD ICAP - Backend

Request throughput (req/s)

105.9

91.8

17.8

65.6

Data throughput (kB/s)

75629

65549

13000

46861

Total time (sec)

5118

5905

29774

8260


  Last updated