Why are batch operations slow in MetaDefender Core and how are they resolved in version 5.14.0?

This article applies to all MD Core prior to version 5.14.0

Issue:

Significant delays during batch operations, specifically when executing API calls such as:

  • POST /file/batch/<batch_id>/close
  • GET /file
  • POST /admin/batch/closeall

PostgreSQL logs revealed that the function scan.getFirstRcFile(BIGINT) was executing repeatedly and taking longer time to query:

[11076] LOG: duration: 6552.132 ms execute : SELECT * FROM scan.getFirstRcFile($1::BIGINT); 2025-10-27 00:00:17.705 IST [9804] LOG: duration: 10619.755 ms execute : SELECT * FROM scan.getFirstRcFile($1::BIGINT);

These delays led to bottlenecks in the batch-processing workflow.

Root Cause:

In earlier versions, batch-scan requests triggered the scan.getFirstRcFile(BIGINT) query multiple times for each file within a batch. When processing large numbers of files, this resulted in cumulative delays and degraded overall performance.

Resolution:

The issue has been addressed in version 5.14.0

https://www.opswat.com/docs/mdcore/v5.14.0/release-notes

The Close Batch workflow has been refactored so that it no longer invokes scan.getFirstRcFile(BIGINT). The GET /file API has been optimized to reduce query overhead and avoid repeated calls to this function.

As a result, PostgreSQL logs in version 5.14.0 no longer display long-running scan.getFirstRcFile($1::BIGINT) queries during the impacted APIs.

If you require further assistance, please follow these instructions on How to Create Support Package?, before creating a support case or chatting with our support engineer.

Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard