Title
Page icon
Create new category
Edit page index title
Edit category
Edit link
Storage configuration
Two components in MetaDefender Cluster hold data that must survive a pod restart: MetaDefender Cluster File Storage, which holds scanned files and their metadata, and PostgreSQL, which holds the four cluster databases.
Both default to ephemeral emptyDir volumes. That default is convenient for evaluation and wrong for anything else: an emptyDir is deleted with its pod, so a restart loses every scanned file and the entire database.
Enable persistence before putting real data in the cluster.
Supported storage
Persistent volumes must support the ReadWriteOnce access mode and honour POSIX file ownership.
Do not use Network File System (NFS) for persistent volumes. NFS can corrupt the MetaDefender Cluster File Storage metadata database and the PostgreSQL data directory. This is not a performance recommendation — it is a data-integrity requirement.
Suitable options:
Environment | Storage |
|---|---|
On-premises Kubernetes | Longhorn, or any CSI driver backed by block storage |
AWS | EBS ( |
Azure | Azure Disk (Premium SSD) |
Google Cloud | Persistent Disk (SSD) |
A default StorageClass must exist, or you must name one explicitly in each persistence block.
PostgreSQL
This creates a single ReadWriteOnce PersistentVolumeClaim named postgres, mounted at /var/lib/postgresql/data. The data directory is placed in a pgdata subdirectory of the volume, so filesystem artefacts at the volume root — such as lost+found — do not clash with PostgreSQL's initialisation check.
The PostgreSQL Deployment uses the Recreate strategy: the old pod is fully terminated before the new one starts. Two PostgreSQL pods can never contend for the same ReadWriteOnce volume, but expect a brief outage on any change to the PostgreSQL pod spec.
The PVC outlives the release
The PostgreSQL PVC carries the helm.sh/resource-policy: keep annotation, so helm uninstall leaves it behind along with your data. This is deliberate — an accidental uninstall should not destroy the databases.
It also means a reinstall reattaches to the existing volume and finds the previous databases. To start genuinely fresh, delete the PVC explicitly:
Sizing
The published baseline is 1 TB on SSD or NVMe, per MetaDefender Cluster system requirements. The 100Gi in the examples on this page is a starting point for a small or evaluation deployment, not a production figure.
Demand is driven mostly by processing history in the datalake and data warehouse databases, which grows with scan volume and your retention policy. A useful rule of thumb from the requirements page: roughly 4 GB per million objects in a five-instance, eight-engine configuration.
Growing a volume later depends on your StorageClass supporting volume expansion.
MetaDefender Cluster File Storage
MetaDefender Cluster File Storage is a StatefulSet, so persistence works differently from PostgreSQL: each replica gets its own PersistentVolumeClaim, provisioned from a volume claim template and mounted at /app/storage.
size is per replica. Three replicas at 1000Gi provision 3000Gi in total.
Give each replica its own ReadWriteOnce volume. Do not point the replicas at one shared ReadWriteMany volume. Each MetaDefender Cluster File Storage instance owns its own metadata database and expects exclusive access to its directory. Sharing one volume corrupts that metadata.
The MetaDefender Cluster Control Center distributes files across the instances in its endpoint list; it does not expect any instance to hold a complete copy. How many copies of each file are kept is configured separately — see MetaDefender Cluster File Storage.
Sizing MetaDefender Cluster File Storage
The published baseline is 1 TB on SSD or NVMe, per System requirements,with two rules that matter more than the flat number:
At least twice the total size of the files submitted concurrently. MetaDefender Cluster File Storage holds both the submitted file and the working data derived from it, so peak concurrent submission volume — not daily throughput — sets the floor.
Double it again if you enable CDR or DLP engines on the MetaDefender Core instances. Sanitisation writes rebuilt copies alongside the originals.
File ownership
MetaDefender Cluster File Storage runs as an unprivileged user with UID and GID 1000. The chart sets fsGroup: 1000 on the pod so the mounted volume is group-writable by that process, with fsGroupChangePolicy: OnRootMismatch so ownership is only recursively applied when the volume root does not already match — important on large volumes, where an unconditional chown would delay every pod start.
PVCs outlive the StatefulSet
Kubernetes does not delete volume-claim-template PVCs when a StatefulSet is removed or scaled in. After helm uninstall services, the MetaDefender Cluster File Storage PVCs remain:
Keep them if you intend to reinstall and retain the stored files. Delete them if you are decommissioning the deployment.
Ephemeral storage is configured elsewhere
MetaDefender Cluster Worker pods also need disk — for the deployed engine and scan temporary files — but that space comes from the node's ephemeral filesystem rather than a persistent volume, and is configured as a resource limit. See Disk space: ephemeral storage.