Container-Based Setup

Before running the setup, please check Recommended System Configuration to install all required dependencies of MetaDefender Distributed Cluster (MDDC).

Setup order requirement

Please follow installation steps in order to complete the system setup properly:

Order no.	Service	Notes
1	Redis, RabbitMQ, PostgreSQL, Identity Service and File Storage	Could be setup in parallel in any order among them. Make sure they are all fully functional and accessible before proceeding to the next setup order #2.
2	Control Center	Ensure it's able to connect to those services in #1 Make sure it is fully functional and accessible.
3	Worker for API Gateway and Worker for MetaDefender Core	Could be setup in parallel in any order among them. Ensure they're able to connect to Control Center Make sure they are all fully functional and accessible.

Image name and version

All the images can be found at OPSWAT Docker Hub with the following information:

Note: version is the currently release version.

MetaDefender Distributed Cluster Identity Service

Docker image
    
 
opswat/metadefender-distributed-cluster:identity-service-<version>-debian-12
Copy

MetaDefender Distributed Cluster File Storage

Docker image
    
 
opswat/metadefender-distributed-cluster:file-storage-<version>-debian-12
Copy

MetaDefender Distributed Cluster Control Center

Docker image
    
 
opswat/metadefender-distributed-cluster:control-center-<version>-debian-12
Copy

MetaDefender Distributed Cluster Worker for API Gateway

Docker image
    
 
opswat/metadefender-distributed-cluster:worker-api-gateway-<version>-debian-12
Copy

MetaDefender Distributed Cluster Worker for MetaDefender Core

Docker image
    
 
opswat/metadefender-distributed-cluster:worker-core-<version>-debian-12
Copy

Environment variables

1. MDDC Identity Service

Environment Variable	Necessity	Description
`MDDC_IDENTITY_SERVICE_DB_HOST`	Required	Provide the database host for Identity Service
`MDDC_IDENTITY_SERVICE_DB_PORT`	Optional	Provide the database port for Identity Service Default: 5432
`MDDC_IDENTITY_SERVICE_DB_USER`	Required	Provide the database user for Identity Service
`MDDC_IDENTITY_SERVICE_DB_PASSWORD`	Required	Provide the database password for Identity Service
`MDDC_USER`	Required	Define the information to initiate the administrator account. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_PASSWORD`	Required	Define the information to initiate the administrator account. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_EMAIL`	Required	Define the information to initiate the administrator account. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_APIKEY`	Optional	Define the information to initiate the administrator account. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_IDENTITY_SERVICE_CONNECTION_KEY`	Required	Define the connection key in order to register to Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
`MDDC_IDENTITY_SERVICE_PORT`	Optional	Define the expose port for Identity Service Default: 8891
`LOG_LEVEL`	Optional	Define the log level. Default value: info Accepted values: info/debug

Start Identity Service container with docker run:

Bash
    
 
docker run -d --name mddc-identity-service \  -e MDDC_IDENTITY_SERVICE_DB_HOST=postgres-is \  -e MDDC_IDENTITY_SERVICE_DB_USER=admin \  -e MDDC_IDENTITY_SERVICE_DB_PASSWORD=admin \  -e MDDC_IDENTITY_SERVICE_CONNECTION_KEY=1234abcd \  -e MDDC_USER=admin \  -e MDDC_PASSWORD=admin \  -e MDDC_EMAIL=admin@local \  -p 8891:8891 opswat/metadefender-distributed-cluster:identity-service-2.0.0-debian-12
Copy

2. MDDC File Storage

Environment Variable	Necessity	Description
`MDDC_FILE_STORAGE_CONNECTION_KEY`	Required	Define the connection key in order to register to Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
`MDDC_FILE_STORAGE_PORT`	Optional	Define the expose port for File Storage Default: 8890
`LOG_LEVEL`	Optional	Define the log level. Default value: info Accepted values: info/debug

Start File Storage container with docker run

Bash
    
 
docker run -d --name mddc-file-storage \  -e MDDC_FILE_STORAGE_CONNECTION_KEY=1234abcd \  -e MDDC_FILE_STORAGE_PORT=8890 \  -p 8890:8890 opswat/metadefender-distributed-cluster:file-storage-2.0.0-debian-12
Copy

3. MDDC Control Center

Environment Variable	Necessity	Description
`MDDC_LAKE_DB_HOST`	Optional	Provide the database host for Data Lake. In case that the end-user does not have the Data Lake, it's required to provide this variable to automate the database preparation
`MDDC_LAKE_DB_PORT`	Optional	Provide the database port for Data Lake Default: 5432
`MDDC_LAKE_DB_USER`	Optional	Provide the database user for Data Lake. In case that the end-user does not have the Data Lake, it's required to provide this variable to automate the database preparation
`MDDC_LAKE_DB_PASSWORD`	Optional	Provide the database password for Data Lake. In case that the end-user does not have the Data Lake, it's required to provide this variable to automate the database preparation
`MDDC_WAREHOUSE_DB_HOST`	Optional	Provide the database host for Data Warehouse. In case that the end-user does not have the Data Warehouse, it's required to provide this variable to automate the database preparation
`MDDC_WAREHOUSE_DB_PORT`	Optional	Provide the database port for Data Warehouse Default: 5432
`MDDC_WAREHOUSE_DB_USER`	Optional	Provide the database user for Data Warehouse. In case that the end-user does not have the Data Warehouse, it's required to provide this variable to automate the database preparation
`MDDC_WAREHOUSE_DB_PASSWORD`	Optional	Provide the database password for Data Warehouse. In case that the end-user does not have the Data Warehouse, it's required to provide this variable to automate the database preparation
`MDDC_CACHE_HOST`	Optional	Provide the caching host (Redis).
`MDDC_CACHE_PORT`	Optional	Provide the caching port (Redis)
`MDDC_BROKER_HOST`	Optional	Provide the broker host (RabbitMQ)
`MDDC_BROKER_PORT`	Optional	Provide the broker port (RabbitMQ)
`MDDC_BROKER_USER`	Optional	Provide the broker username (RabbitMQ)
`MDDC_BROKER_PASSWORD`	Optional	Provide the broker password (RabbitMQ)
`MDDC_CONTROL_CENTER_DB_HOST`	Required	Provide the database host for Control Center
`MDDC_CONTROL_CENTER_DB_PORT`	Optional	Provide the database port for Control Center Default: 5432
`MDDC_CONTROL_CENTER_DB_USER`	Required	Provide the database username for Control Center
`MDDC_CONTROL_CENTER_DB_PASSWORD`	Required	Provide the database password for Control Center
`MDDC_USER`	Required	Provide the administrator account that is defined in Identity Service. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_PASSWORD`	Required	Provide the administrator account that is defined in Identity Service. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_APIKEY`	Optional	Provide the administrator account that is defined in Identity Service. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_IDENTITY_SERVICE_HOST`	Required	Provide the Identity Service host in order to add it to Control Center
`MDDC_IDENTITY_SERVICE_PORT`	Optional	Provide the Identity Service port in order to add it to Control Center Default: 8891
`MDDC_IDENTITY_SERVICE_CONNECTION_KEY`	Required	Provide the Identity Service connection key in order to add it to Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
`MDDC_CONTROL_CENTER_ENCRYPTION_KEY`	Required	Define the encryption key for communication between Control Center and the services. Must be 32 characters long and contain only lowercase letters (a–z) and digits (0–9)
`MDDC_FILE_STORAGE_HOST`	Required	Provide the File Storage host in order to add it to Control Center
`MDDC_FILE_STORAGE_PORT`	Optional	Provide the File Storage port in order to add it to Control Center Default: 8890
`MDDC_FILE_STORAGE_CONNECTION_KEY`	Required	Provide the File Storage connection key in order to add it to Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
`MDDC_CERT_PATH`	Optional	Provide the directory path that contains the certificate and private key in order to enable https Note: when provide this variable, it's supposed to mount this path to `/certs/` as volume For example: `--volume /your-path:/certs`
`LOG_LEVEL`	Optional	Define the log level. Default value: info Accepted values: info/debug

Start Control Center container with Docker run

Bash
    
 
docker run -d --name mddc-control-center \  -e MDDC_CONTROL_CENTER_DB_HOST=postges-cc \  -e MDDC_CONTROL_CENTER_DB_USER=admin \  -e MDDC_CONTROL_CENTER_DB_PASSWORD=admin \  -e MDDC_IDENTITY_SERVICE_HOST=mddc-identity-service \  -e MDDC_USER=admin \  -e MDDC_PASSWORD=admin \  -e MDDC_IDENTITY_SERVICE_CONNECTION_KEY=1234abcd \  -e MDDC_CONTROL_CENTER_ENCRYPTION_KEY=12345678123456781234567812345678 \  -e MDDC_FILE_STORAGE_HOST=mddc-file-storage \  -e MDDC_FILE_STORAGE_CONNECTION_KEY=1234abcd \  -e MDDC_CERT_PATH=/certs \  -v /new-certificates:/certs  -p 8892:8892 opswat/metadefender-distributed-cluster:control-center-2.0.0-debian-12
Copy

4. MDDC Worker for API Gateway

Environment Variable	Necessity	Description
`MDDC_WORKER_CONNECTION_KEY`	Required	Define the connection key in order to register to Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
`MDDC_WORKER_PORT`	Optional	Define the expose worker's port. Default: 8893
`MDDC_WORKER_HOST`	Optional	Define the worker's host address. If it's not specified, it will get the container's internal IP address
`MDDC_CONTROL_CENTER_HOST`	Required	Provide the Control Center's host address
`MDDC_CONTROL_CENTER_PORT`	Optional	Provide the Control Center's port Default: 8892
`MDDC_USER`	Required	Provide the administrator account that is defined in Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_PASSWORD`	Required	Provide the administrator account that is defined in Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_APIKEY`	Optional	Provide the administrator account that is defined in Identity Service. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_API_GATEWAY_PORT`	Optional	Define the expose port to scan files via API Gateway Default: 8899
`LOG_LEVEL`	Optional	Define the log level. Default value: info Accepted values: info/debug

Start Worker for API Gateway container with Docker run

Bash
    
 
docker run -d --name mddc-worker-api-gateway \  -e MDDC_WORKER_CONNECTION_KEY=1234abcd \  -e MDDC_WORKER_HOST=mddc-core-api-gateway \  -e MDDC_CONTROL_CENTER_HOST=mddc-control-center \  -e MDDC_USER=admin \  -e MDDC_PASSWORD=admin \  -e MDDC_API_GATEWAY_PORT=8899 \  -p 8893:8893 -p 8899:8899 opswat/metadefender-distributed-cluster:worker-api-gateway-2.0.0-debian-12
Copy

5. MDDC Worker for Core

Environment Variable	Necessity	Description
`MDDC_WORKER_CONNECTION_KEY`	Required	Define the connection key in order to register to Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
`MDDC_WORKER_PORT`	Optional	Define the expose worker's port. Default: 8893
`MDDC_WORKER_HOST`	Optional	Define the worker's host address. If it's not specified, it will get the container's internal IP address
`MDDC_CONTROL_CENTER_HOST`	Required	Provide the Control Center's host address
`MDDC_CONTROL_CENTER_PORT`	Optional	Provide the Control Center's port. Default: 8892
`MDDC_USER`	Required	Provide the administrator account that is defined in Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_PASSWORD`	Required	Provide the administrator account that is defined in Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_APIKEY`	Optional	Provide the administrator account that is defined in Identity Service. This account is to automatically do the following tasks: Add Redis to Control Center if specified Add RabbitMQ to Control Center if specified Add Data Lake to Control Center if specified Add Data Warehouse to Control Center if specified Add File Storage to Control Center if specified Add Worker to Control Center, upload API Gateway installer to Control Center, and deploy API Gateway to Worker Add Worker to Control Center, upload MetaDefender Core installer to Control Center, and deploy MetaDefender Core to Worker
`MDDC_CORE_PORT`	Optional	Define the expose port for MD Core Default: 8008
`LOG_LEVEL`	Optional	Define the log level. Default value: info Accepted values: info/debug
`MDDC_LICENSE_KEY`	Optional	Provide the license key to activate MetaDefender Core
`MDDC_LICENSE_DESCRIPTION`	Optional	Define description of the license key

Start Worker for Core container with Docker run:

Bash
    
 
docker run -d --name mddc-worker-core \  -e MDDC_WORKER_CONNECTION_KEY=1234abcd \  -e MDDC_WORKER_HOST=mddc-core-core \  -e MDDC_CONTROL_CENTER_HOST=mddc-control-center \  -e MDDC_USER=admin \  -e MDDC_PASSWORD=admin \  -p 8893:8893 opswat/metadefender-distributed-cluster:worker-core-2.0.0-debian-12
Copy

Start MetaDefender Distributed Cluster with Docker Compose

Create a local file named docker-compose.yaml and copy the following content to this file:

yaml
    
​x
 
services:
  redis:...​
  rabbitmq:...​
  postgres:...​
  identity-service:...​
  file-storage:...​
  control-center:...​
  worker-api-gateway:...​
  worker-core:...​
​networks:...​
Copy

Prepare an environment variable file named .env.example and provide with your own values
Run the application with the command:

YAML
    
 
docker compose up -d
Copy

Known limitation

When the host experiences resource limitations or degraded performance, some containers may fail to start properly. In such cases, restarting the container is recommended to restore normal operation.

Last updated on

Was this page helpful?