ArangoDB corruption issue
In some rare cases, ArangoDB might fail to restart after a compaction operation.
In this situation, docker ps
shows that the graphdb
(arangodb) container is restarting continuously and the webservice
container becomes unhealthy
because of that:

To see the underlying issue, please check the logs of the graphdb
container and look for the fatal error: "Corruption: Compaction sees out-of-order keys."
This is a known issue in ArangoDB: https://github.com/arangodb/arangodb/issues/20841
Until ArangoDB releases an automated solution to recover from this corrupted state, the simplest solution is purging the Arango database files from the /data/graphdb/fsio
folder (after stopping the sandbox service):
Purging the database files will impact the Advanced Search (Hunting) functionality: All previous scan reports will remain available in the system, but older scans will be excluded from advanced search operations.
sudo service sandbox stop
sudo rm -rf /data/graphdb/fsio/*
sudo service sandbox start
After this purge, ArangoDB will start from a clean slate and the Sandbox webservice
can function properly.