Container-Based Setup

Before running the setup, please check (System Requirements) to install all required dependencies of MetaDefender Cluster (MD Cluster).

Setup order requirement

Please follow the installation order to complete the system setup properly.

OrderServiceNotes
1Redis, RabbitMQ, PostgreSQL and MD Cluster Identity Service
  • Could be setup in parallel in any order among them.
  • Make sure they are all fully functional and accessible before proceeding to the next setup order #2.
2MD Cluster Control Center
  • Ensure it's able to connect to those services in #1
  • Make sure it is fully functional and accessible.
3MD Cluster File Storage
  • Ensure they're able to connect to MD Cluster Control Center
  • Make sure it is fully functional and accessible.
4MD Cluster Worker for MD Cluster API Gateway and MD Cluster Worker for MetaDefender Core
  • Could be setup in parallel in any order among them.
  • Ensure they're able to connect to MD Cluster Control Center
  • Make sure they are all fully functional and accessible.

Image name and version

All the images can be found at OPSWAT Docker Hub with the following information:

version is the currently release version.

MD Cluster Identity Service

Docker image
Copy

MD Cluster File Storage

Docker image
Copy

MD Cluster Control Center

Docker image
Copy

MD Cluster Worker for MD Cluster API Gateway

Docker image
Copy

MD Cluster Worker for MetaDefender Core

Docker image
Copy

Environment variables

1. MD Cluster Identity Service

Environment VariableNecessityDescription
MDDC_IDENTITY_SERVICE_DB_HOSTRequiredProvide the database host for MD Cluster Identity Service
MDDC_IDENTITY_SERVICE_DB_PORTOptionalProvide the database port for MD Cluster Identity Service Default: 5432
MDDC_IDENTITY_SERVICE_DB_USERRequiredProvide the database user for MD Cluster Identity Service
MDDC_IDENTITY_SERVICE_DB_PASSWORDRequiredProvide the database password for MD Cluster Identity Service
MDDC_USERRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_PASSWORDRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_EMAILRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_APIKEYOptional

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_IDENTITY_SERVICE_CONNECTION_KEYRequiredDefine the connection key in order to register to Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_IDENTITY_SERVICE_PORTOptionalDefine the expose port for MD Cluster Identity Service Default: 8891
LOG_LEVELOptional

Define the log level. Default value: info

Accepted values: info/debug/error/warning

Start MD Cluster Identity Service container with docker run:

Bash
Copy

2. MD Cluster File Storage

Environment VariableNecessityDescription
MDDC_FILE_STORAGE_CONNECTION_KEYRequiredDefine the connection key in order to register to MD Cluster Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_FILE_STORAGE_PORTOptionalDefine the expose port for MD Cluster File Storage. Default is 8890.
MDDC_FILE_STORAGE_HOSTOptionalDefine the MD Cluster File Storage's host address. If it's not specified, it will get the container's internal IP address.
LOG_LEVELOptional

Define the log level. Default value: info.

Accepted values: info/debug/error/warning.

MDDC_CONTROL_CENTER_HOSTRequiredProvide the MD Cluster Control Center's host address.
MDDC_CONTROL_CENTER_PORTOptionalProvide the MD Cluster Control Center's port. Default is 8892.
MDDC_USERRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_PASSWORDRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_APIKEYOptional

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.

Persistent storage is located at /opt/opswat/mddc-file-storage. If end-users require data to be retained across container lifecycles, they must mount a volume to this path with 777 permissions to ensure full read/write access for all processes.

Start MD Cluster File Storage container with docker run.

Bash
Copy

3. MD Cluster Control Center

Environment VariableNecessityDescription
MDDC_LAKE_DB_HOSTOptionalProvide the database host for Data Lake. In case that the end-user does not have the Data Lake, it's required to provide this variable to automate the database preparation.
MDDC_LAKE_DB_PORTOptionalProvide the database port for Data Lake. Default is 5432.
MDDC_LAKE_DB_USEROptionalProvide the database user for Data Lake. In case that the end-user does not have the Data Lake, it's required to provide this variable to automate the database preparation.
MDDC_LAKE_DB_PASSWORDOptionalProvide the database password for Data Lake. In case that the end-user does not have the Data Lake, it's required to provide this variable to automate the database preparation.
MDDC_WAREHOUSE_DB_HOSTOptionalProvide the database host for Data Warehouse. In case that the end-user does not have the Data Warehouse, it's required to provide this variable to automate the database preparation.
MDDC_WAREHOUSE_DB_PORTOptionalProvide the database port for Data Warehouse. Default is 5432.
MDDC_WAREHOUSE_DB_USEROptionalProvide the database user for Data Warehouse. In case that the end-user does not have the Data Warehouse, it's required to provide this variable to automate the database preparation.
MDDC_WAREHOUSE_DB_PASSWORDOptionalProvide the database password for Data Warehouse. In case that the end-user does not have the Data Warehouse, it's required to provide this variable to automate the database preparation.
MDDC_CACHE_HOSTOptionalProvide the caching host (Redis).
MDDC_CACHE_PORTOptionalProvide the caching port (Redis).
MDDC_CACHE_USEROptionalProvide the caching username (Redis). If the end-user does not provide it, Redis will be added without authentication.
MDDC_CACHE_PASSWORDOptional

Provide the caching password (Redis). If the end-user does not provide it, Redis will be added without authentication.

Do not support double quotes (") and backslash (\) in the password.

MDDC_BROKER_HOSTOptionalProvide the broker host (RabbitMQ).
MDDC_BROKER_PORTOptionalProvide the broker port (RabbitMQ).
MDDC_BROKER_USEROptionalProvide the broker username (RabbitMQ).
MDDC_BROKER_PASSWORDOptionalProvide the broker password (RabbitMQ).
MDDC_CONTROL_CENTER_DB_HOSTRequiredProvide the database host for MD Cluster Control Center.
MDDC_CONTROL_CENTER_DB_PORTOptionalProvide the database port for MD Cluster Control Center. Default is 5432.
MDDC_CONTROL_CENTER_DB_USERRequiredProvide the database username for MD Cluster Control Center.
MDDC_CONTROL_CENTER_DB_PASSWORDRequiredProvide the database password for MD Cluster Control Center.
MDDC_USERRequired

Provide the administrator account that is defined in MD Cluster Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_PASSWORDRequired

Provide the administrator account that is defined in MD Cluster Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_APIKEYOptional

Provide the administrator account that is defined in MD Cluster Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_IDENTITY_SERVICE_HOSTRequiredProvide the MD Cluster Identity Service host in order to add it to MD Cluster Control Center.
MDDC_IDENTITY_SERVICE_PORTOptionalProvide the IMD Cluster dentity Service port in order to add it to MD Cluster Control Center. Default is 8891.
MDDC_IDENTITY_SERVICE_CONNECTION_KEYRequiredProvide the MD Cluster Identity Service connection key in order to add it to MD Cluster Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_CONTROL_CENTER_ENCRYPTION_KEYRequiredDefine the encryption key for communication between MD Cluster Control Center and the services. Must be 32 characters long and contain only lowercase letters (a–z) and digits (0–9).
MDDC_CERT_PATHOptional

Provide the directory path that contains the certificate and private key in order to enable https Note: when provide this variable, it's supposed to mount this path to /certs/ as volume For example: --volume /your-path:/certs

Note: In cases where SSL fails to enable due to the File Storage service not being ready, the end-user can either restart the MD Cluster Control Center or manually activate SSL as a workaround.

LOG_LEVELOptional

Define the log level. Default value: info.

Accepted values: info/debug/error/warning.

Start MD Cluster Control Center container with Docker run.

Bash
Copy

4. MD Cluster Worker for API Gateway

Environment VariableNecessityDescription
MDDC_WORKER_CONNECTION_KEYRequiredDefine the connection key in order to register to MD Cluster Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_WORKER_PORTOptionalDefine the expose worker's port. Default is 8893.
MDDC_WORKER_HOSTOptionalDefine the worker's host address. If it's not specified, it will get the container's internal IP address.
MDDC_CONTROL_CENTER_HOSTRequiredProvide the MD Cluster Control Center's host address.
MDDC_CONTROL_CENTER_PORTOptionalProvide the MD Cluster Control Center's port Default is 8892.
MDDC_USERRequired

Provide the administrator account that is defined in MD Cluster Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_PASSWORDRequired

Provide the administrator account that is defined in MD Cluster Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_APIKEYOptional

Provide the administrator account that is defined in MD Cluster Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_API_GATEWAY_PORTOptionalDefine the expose port to scan files via MD Cluster API Gateway. Default is 8899.
LOG_LEVELOptional

Define the log level. Default value: info.

Accepted values: info/debug/error/warning.

If multiple MD Cluster API Gateway containers are deployed on the same host, make sure their ports are configured to avoid conflicts.

Start MD Cluster Worker for MD Cluster API Gateway container with Docker run.

Bash
Copy

5. MD Cluster Worker for Core

Environment VariableNecessityDescription
MDDC_WORKER_CONNECTION_KEYRequiredDefine the connection key in order to register to MD Cluster Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_WORKER_PORTOptionalDefine the expose worker's port. Default is 8893.
MDDC_WORKER_HOSTOptionalDefine the worker's host address. If it's not specified, it will get the container's internal IP address.
MDDC_CONTROL_CENTER_HOSTRequiredProvide the MD Cluster Control Center's host address.
MDDC_CONTROL_CENTER_PORTOptionalProvide the MD Cluster Control Center's port. Default is 8892.
MDDC_USERRequired

Provide the administrator account that is defined in MD Cluster Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_PASSWORDRequired

Provide the administrator account that is defined in MD Cluster Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
MDDC_APIKEYOptional

Provide the administrator account that is defined in MD Cluster Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MD Cluster Control Center if specified.
  • Add RabbitMQ to MD Cluster Control Center if specified.
  • Add Data Lake to MD Cluster Control Center if specified.
  • Add Data Warehouse to MD Cluster Control Center if specified.
  • Add MD Cluster File Storage to MD Cluster Control Center if specified.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MD Cluster API Gateway installer to MD Cluster Control Center, and deploy MD Cluster API Gateway to MD Cluster Worker.
  • Add MD Cluster Worker to MD Cluster Control Center, upload MetaDefender Core installer to MD Cluster Control Center, and deploy MetaDefender Core to MD Cluster Worker.
LOG_LEVELOptional

Define the log level. Default value: info.

Accepted values: info/debug/error/warning.

MDDC_LICENSE_KEYOptionalProvide the license key to activate MetaDefender Core.
MDDC_LICENSE_DESCRIPTIONOptionalDefine description of the license key.

If multiple MetaDefender Core containers are deployed on the same host, make sure their ports and hosts are configured to avoid conflicts.

Start MD Cluster Worker for MetaDefender Core container with Docker run.

Bash
Copy

Start MetaDefender Cluster with Docker Compose

  1. Create a local file named docker-compose.yaml and copy the following content to this file:
yaml
Copy
  1. Prepare an environment variable file named .env.example and provide with your own values
  2. Run the application with the command:
YAML
Copy

Known limitation

  • When the host experiences resource limitations or degraded performance, some containers may fail to start properly. In such cases, restarting the container is recommended to restore normal operation.
Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard