Archived release notes
Version v5.7.1
Release Date: 09 November 2023
New features:
MetaDefender Core supports to receive engine update packages from Central Management v8.
Custom time range for Executive Report.
Upgraded bundled PostgreSQL to version 14.9 for vulnerability (CVE-2023-39417).
Improved the integration between MetaDefender Core and Deep CDR to avoid wasting system resource working on timed-out tasks.
Change Deep CDR retry count default value to 1.
Introduce Sandbox engine file size limit.
User password minimum length enforcement from 8 to 30 characters.
Whenever an user modifies UI settings such as time range or processing history columns, the modified setting value will be remembered and retained for the next user login, without having to re-select again.
Other blocked reasons added for batch processing.
Decrypted time in the JSON scan result.
Logging improvements.
Fixes:
- Submiting gzipped file failed with "Content-Encoding: gzip" if the file size is a power of 2 (2^n) bytes.
- Password protection was not retained when scanning a file without file name.
- Wrong status displayed for Threat Intelligence when processing a file bigger than 140 MB and timed out.
- Installation failed on Red Hat Enterprise 8 due to missing wkhtmltopdf library.
- The number of sanitization tasks might exceed than allowed threshold under high load.
- Failed to close large batch when file type detection timed out on files in batch.
- Temporary files generated by Deep CDR and Proactive DLP engines were not cleaned up properly when cancelling scan request.
- Failed to export scan result PDF report on Red Hat Enterprise.
Version v5.7.0
Release Date: 09 October 2023
New features:
SBOM engine integration
Upgraded NGINX to version 1.22.1 for vulnerability remediation
Central Hub enhancements
- Version 2 for MetaDefender Core NAS (file storage server) that supports continuous file streaming directly to MetaDefender Core instance over NAS (without waiting for file streaming to finish at NAS first), and data encryption capacity.
- Optimized processing for archive extraction load distribution.
Optimized scan result polling query (GET /file API) for faster response
New scan results when engine stopped executing due to hitting global processing timeout
Separated processing time for archive compression
Separated processing time for external scanner
Various improvements in MetaDefender Core log for a better visibility when processing files and troubleshooting.
UI updates
- Continuous improvements for UI accessibility for WCAG 2.0 and 2.1 level A, AA compliance.
- Hash comparison after sanitization
- In-product tagline, and other UI cosmetic updates.
Fixes:
- Deep CDR engine failed to install and became permanently failed when MetaDefender Core contains a lot of custom worflow rules.
- File processing could hang up when timing out during hash lookup with File Reputation.
- DLP result of archive file was "No Specific Configuration" instead of "Unsupported File Type" when containing allowed file.
- "Not scanned" returned by AV engines when processing archive file but disabling extraction mode.
- "Others_time" in the JSON scan result was mistakenly set to negative when a file is quarantined due to global processing timeout exceeded.
- "start_time" in the JSON scan result was set to 1970-01-01 for in-progress file processing.
Version v5.6.1
Release Date: 28 August 2023
New features:
Integration with My OPSWAT portal
- Get ready for My OPSWAT integration where MetaDefender Core can connect and be managed in My OPSWAT product inventory.
- Stay tuned for My OPSWAT further updates (end of September) to get hold of this integration on My OPSWAT.
Continuous improvement on UI accessibility
- Various UI updates and remediations for UI accessibility support.
New filtration sub-setting for file type detection fallback to extension
- The new setting is under MetaDefender Core workflow - File Type tab. This new setting can be only enabled when the existing parent setting "Fallback file type detection to current extension if needed" is checked.
- When enabled, MetaDefender Core will avoid using current file extension as a fallback when actual file type is not enabled for Deep CDR engine's setting.
More comprehensive details for processing time in the JSON scan result
- Introduce a new key
post_action_time
(separated fromothers_time
) for actual time spent on post action (if applicable). This key is underprocessing_time_details
key in the JSON scan result. - Introduce a new key
cdr_wait_time
to determine actual time spent when request is sitting in queue waiting for Deep CDR available to start processing. This new key is undersanitization_details
in the JSON scan result. - Introduce new keys
cdr_storage_time
/dlp_storage_time
/quarantine_storage_time
to determine actual time spent on output file being saved in storage for sanitization / sensitive data processing / quarantine processing (applicable when Deep CDR, Proactive DLP and Quarantine is enabled for that scan). These new keys are underprocessing_time_details
key in the JSON scan result. - Applicable to all of those above new processing time, we also support for data statistics for admins to view under Executive report UI when needed.
- Introduce a new key
Enhancement on database connection pool
- Revamped back-end logic for database connection pool to mitigate potential bottleneck on database queries under extreme load.
Fixes:
- Sometimes failed to close batch under high load and problematic circumstances with files inside batch.
- Batch signature was not generated when batch contains hundreds thousands of files inside.
- Failed to send callback for sanitizeurl webhook requests.
- MetaDefender Core could rarely crash when repeatedly processing .docx large files.
- Temporary files could not be cleaned up when hitting global timeout.
- Memory increasing abnormally and leaking with data statistics against PostgreSQL.
- Other minor stability issues.
Version v5.6.0
Release Date: 14 July 2023
New features:
Support PostgreSQL 14.8
- MetaDefender Core now also supports PostgreSQL 14 (verified with 14.8).
- Bundled PostgreSQL server comes with MetaDefender Core standalone deployment also comes with this new version.
- NOTE: For MetaDefender Core upgrade from older version 5.5.1 or older (using PostgreSQL 12) to version 5.6.0 (using PostgreSQL 14), it is required to run MetaDefender Core installer as admin privileges to ensure the database migration task finish successfully.
Priority for workflow
- Provide better option for Quality of Service on workflow, MetaDefender Core now supports users to define certain priority for each workflow rule. This enhancement will help users control better business priority in file processing.
- You can find this new setting under "General" workflow tab.
Time availability for workflow
- Only allow scan requests using certain workflow during pre-defined time windows. You can find this new setting under "General" workflow tab.
Global processing timeout per workflow
- MetaDefender Core provides one global timeout for entire processing applied to every scan request. This is expected to help client integration control its SLA for every scan at ease.
- This new setting could be configured separately for each workflow rule.
- When a scan request is timed out by this global processing timeout setting, the scan request will be blocked with "Global timeout exceeded" as blocked reason.
- NOTE: This global processing timeout, when enabled, it should be greater or equal to the sum of all other engine related timeout settings such as Metascan timeout, Deep CDR timeout etc.
- You can find this new setting under "General" workflow tab.
Override global processing timeout for specific scan requests
- Client also can override the new setting of global processing timeout per specific scan requests when needed, by using a new request header
global-timeout
for the file submission request. - Details: POST - Analyze File (Asynchronous mode)API
- If this header is not set, or its value is empty, then MetaDefender Core will use the pre-defined global processing timeout in corresponding workflow rule (only when enabled).
- Client also can override the new setting of global processing timeout per specific scan requests when needed, by using a new request header
Hash lookup supported for in-progress scan
- Before having to submit another scan request for a same file to MetaDefender Core and duplicate the effort of scanning, client could now utilize the hash lookup API GET - Fetch Analysis Result By HashAPI to be acknowledged that a same file is actually being processed, so client could waive the same request to MetaDefender Core.
- Introduce a new header
include-inprogress
for GET - Fetch Analysis Result By HashAPI to indicate whether or not client would want to receive "In-progress" (instead of not found result) in such circumstance.
Allowlist by vendor filtration
- Allow users to create a filter on MetaDefender Core to add PE (Portable Executable) files tied to certain vendors into the allowlist.
- Furthermore, users can be supported to block those PE files when unsatisfied the pre-defined conditions. You can find this new setting under "Allowlist" workflow tab.
Continuous improvement for Sandbox (formerly OPSWAT Filescan) engine integration
- Rename "OPSWAT Filescan" engine to "Sandbox".
- Add
extraction_info
to the scan result JSON of GET - Fetch analysis reports containing all files in archiveAPI - Adjust default enabled file types for Sandbox file processing under workflow:
- Remote Sandbox: all file types are selected by default.
- Local Sandbox: all file types except archive are selected by default.
New option to skip sanitizing empty files
- When enabled, MetaDefender Core will avoid sanitizing empty file (zero bytes). By default, it is disabled. You can find this new setting under "Deep CDR" workflow tab.
Attach MetaDefender Core server information in scan result fetching response
- When enabled via using a new supported API endpoint PUT - Update list of custom response headersAPI, MetaDefender Core will return allowable server information (deployment ID, server IP address and port) in designated header
X-Core-Id
in the scan result fetching response (e.g. GET /file/<data_id>). - This new API endpoint also supports users to only return those pre-defined data to authenticated scan requests (where uses API key in the request).
- When enabled via using a new supported API endpoint PUT - Update list of custom response headersAPI, MetaDefender Core will return allowable server information (deployment ID, server IP address and port) in designated header
Throttling option for scanning webhook
Support new parameters
workercount
,requestqueue
in the existing API endpoint PUT - Webhook set configurationAPI to define better flexibility for scan webhook:- workercount: the total number of workers for the webhook responser (for sending callbacks to the webhook client). Default is 1. Note: a service restart is required for changes to take effect.
- requestqueue: the maximum number of requests allowed for each worker. Default is 100.
Besides that, now users can set zero (0) for the existing parameter transfertimeout for the PUT - Webhook set configurationAPI, that means, disable timeout for sending callbacks to webhook client.
NOTE: These throttling options only apply to webhook mechanism that notify processing result, do not apply to the mechanism where MetaDefender Core sends sanitized file back to the client just yet.
Enhance sorting feature in processing history UI
- Enhance the existing feature to allow sorting entire scan history across all pages (previously only allow sorting current UI page on MetaDefender Core version 5.5.1).
Continuous improvement for UI accessibility
- Address issues and enhance GUI for better accessibility readiness.
Display file size and file type even when file is being processed
- On processing result UI screen. Including when the file is being processed, or already finished.
Display setting for scan duration on the UI
- Allow users to specify preferred time format for scan duration to be displayed on the UI.
Other UI enhancements
- Move max file size setting from "Metascan" tab to "General" time in workflow.
- Update to rename and logo from Cyren (AV engine) to Varist.
- Other enhanced UI cosmetics.
Fixed:
Callback responses got stuck
- Under high load, MetaDefender Core could fail to send callback responses to webhook client.
Health check API returned out of date data
- Under high load, the heath check API GET - Get health check statusAPI could return out-of-date data for scan queue and engine status.
Inconsistent scan result with Threat Intelligence under Quarantine
- Scanning with Threat Intelligence under Quarantine could return inconsistent result.
Leftover temporary files when timing out with Sandbox (formerly OPSWAT Filescan) engine
- When a scan is done with timeout for Sandbox (formerly OPSWAT Filescan), temporary file was not cleaned up properly.
Other minor bug fixes
Version v5.5.1
Release Date: 18 May 2023
New features:
Continuous improvements on OPSWAT Filescan engine integration
- New configuration to trigger OPSWAT Filescan processing when number of scan failures from AV engines' results exceeded certain amount of threshold.
- Behavior changes: skip sending extracted child files of MS Office files to OPSWAT Filescan engine for processing.
- Fixed some minor bugs related to UI display.
Blocking file processing based on Deep CDR forensic analysis policy
- New configuration to allow authorized users to block file processing based on Deep CDR forensic analysis policy. The policy will be managed and defined under Deep CDR engine advance settings.
- Requires Deep CDR 6.6.0 or above (to be released at the end of Q2-2023).
Workflow rule cloning
- Similar to workflow template, now allowing authorized users to clone / duplicate existing workflow rule.
UI accessibility continuous improvements
- Enormous changes and improvements towards UI accessibility support. This UI accessibility mode can be enabled on the top section of every UI page.
Auto enabling any newly supported file types for sanitization
- New setting to allow MetaDefender Core to auto enable any file option that newly supported by Deep CDR for sanitization.
- This option is unchecked by default to keep backward compatibility. You can enable it under Deep CDR tab in workflow rule > Enable for file types section.
New attribute added into existing key
extraction_info
in the JSON scan result structure- is_encrypted_file (optional, only accept "true"/"false"): a flag to indicate a file is encrypted or not.
- See more details on GET - Fetch Analysis ResultAPI
Handling MSG file with archive sanitization use-case
- When enabling "Block unsupported file type" under Deep CDR section, then sanitization on MSG file failed due to msg.data file (blocked as unsupported).
- This change on MetaDefender Core to avoid that situation, and instead let the archive compression engine take care of the MSG archive sanitization properly.
- Requires archive compression engine version 6.2.3 or above.
Optimizing memory consumption by ometascan service under load
- Possibly helped reduce up to 50% memory consumption with ometascan service.
Sorting columns in table under processing history UI
- Currently supported users to sort processing history UI based on start time and scan duration.
Adding parsing digital signature processing time
- New key named
parse_dgsg_time
added into the existingprocessing_time_details
__to indicate actual processing time spent on parsing digital signature against file. - See more details on GET - Fetch Analysis ResultAPI
- New key named
Support retry configurations for webhook post action
- Users now can customize retry mode and timeout setting for webhook post action in Workflow Rule.
Including current process name, ram usage, cpu usage into the support package
- New data will be collected and stored in new CSV files located in the support package.
Improve query time for filter labels API
- Helped improve search response time on the processing history UI page.
UI enhancements
Fixed:
Handle processing properly with content-encoding of gzip or x-gzip
- When submitting file request, if mistakenly specified Content-Encoding header value = gzip or x-gzip, MetaDefender Core returned HTTP 500 error with unexpected JSON output.
- For example: {"err":"Invalid gzipped body"}{"data_id":"a3179bc183a6494694768c315a21867d"}
- Now addressed the issue to let MetaDefender Core return HTTP 500 error with proper JSON output, for example: {"err":"Invalid gzipped body"}
Timeout occurred with synchronous submission requests
- Addressed the issue when MetaDefender Core returned scan timeout immediately against synchronous scan requests.
URL encoded "rule" header in scan request
- MetaDefender Core did not decode values in the "rule" header when it is URL encoded (for example: "rule" = "File%20process")
Version v5.5.0
Release Date: 06 April 2023
New features:
OPSWAT Filescan integration
OPSWAT Filescan is an unique adaptive threat analysis technology, enabling zero-day malware detection and more Indicator of Compromise (IOCs) extraction. Learn more at https://docs.opswat.com/filescan
MetaDefender Core now fully integrate with OPSWAT Filescan (as an engine module), and provide two separate engines:
- OPSWAT Filescan Embedded engine: bundled in MetaDefender Core server, and the engine will process files locally itself.
- OPSWAT Filescan Remote engine: the engine is supposed to send file requests to another remote OPSWAT Filescan system for processing. Requiring users to provide proper remote OPSWAT Filescan URL and API key.
We need separate MetaDefender Core license key for each engine option (Embedded vs. Remote).
Both OPSWAT Filescan engines require Java 8 JRE x64 (Windows and Linux) and Microsoft Visual C++ Redistributable for Visual Studio 2015 x64 (Windows only) as dependencies. Please make sure to have them installed on MetaDefender Core server before deploying OPSWAT Filescan engine.
The integration comes with two modes:
- Inline: working as a part of MetaDefender Core processing workflow (real-time processing). Allowing users to block entire processing based on OPSWAT Filescan engine's outcome and decision.
- Out of band: working as a part of MetaDefender Core quarantine (post-investigation processing).Providing additional option to analyze quarantined items along with the existing Threat Intelligence technology.
OPSWAT Filescan engine (both Embedded and Remote) provides configurations under workflow rule (for inline mode), and under engine module - Inventory (for out of band mode).
UI accessibility
- MetaDefender Core provides a new UI mode to support accessibility view. By default, this option is disabled.
Licensing with On-prem License Management Server (OLMS)
- This feature requires the upcoming new product (On-prem License Management Server) and our existing cloud-based service (Activation Server) to operate.
- New licensing managment model for MetaDefender Core to allow On-prem license management server (OLMS, to be released later) to manage the product's license status including activation, deactivation.
Shared Authentication Session
- In the shared database model, MetaDefender Core instances now can share same authentication session among them. That helps eliminate the need to manage separate authentication sessions on load balancer side.
This new feature supports local authentication, AD / LDAP authentication and also Single Sign-On (SSO).
Customized Proxy Configurations
- In constrast to what we provided in the past for proxy configuration, which is only one setting for all network involved modes on MetaDefender Core, now we provides a much more flexible way to define separate proxy configurations for each MetaDefender Core functionalities that require network connection.
Webhook for post action
- Besides the existing option to allow MetaDefender Core trigger each separate sub-processes to run your pre-configured post action script, now you are provided with another new option to run post action in multi-threading mode, as a part of workflow processing.
- When configured under post action page (Inventory menu), for each file processing, MetaDefender Core will send pre-configured callback URL to your designated webhook server, and you can define to run series of actions on webhook server asynchronously. Please note you will want to setup webhook server on your own.
- This new post action model helps improve overall performance.
- Note: Currently, only HTTP protocol is supported for callback URL.
SHA256 checksum for sanitized file
- MetaDefender Core provides SHA256 checksum for sanitized file in the final JSON scan result. You can find it at
sanitization_details.sanitized_file_info.sha256
- MetaDefender Core provides SHA256 checksum for sanitized file in the final JSON scan result. You can find it at
UI enhancements
Fixed:
Archive engine integration issue
- Fixed issues caused by archive engine integration that could make negative impact on archive extraction performance, as well as cause memory leak issue.
MetaDefender Core service crashed under certain circumstance
- When scanning some special files, MetaDefender Core could crash due to its file signature analysis feature.
RootCA ceritificate store could not be loaded in Linux
- When having *.0 files in rootCA trusted store.
Some UI cosmetics and minor bugs are addressed.
Version v5.4.1
Release Date: 28 February 2023
New features:
[Central Hub] High availability for distributed archive extraction
- While handling distributed extraction, if a MetaDenfeder Core instance dies for whatever reasons, then distributed extraction task should keep running on another alive MetaDefender Core instance.
[Central Hub] New setting to set file size limit (min - max) for distributed archive extraction
Exception for file types based blocklist
- A new setting under "Blocklist by file type" option for Blocklist configuration to allow users configure to block everything but excluding certain file types.
Comprehensive session timeout settings on UI
Security enhancements
- Support TLS v1.3: Harden security policy with TLS v1.3 supported for HTTPS connection. By default, both TLS v1.2 and v1.3 are allowed for client handshake.
- Upgraded to OpenSSL 3.0.8 for vulnerabilities.
- Addressed vulnerability with security through obscurity, and information leakage.
- Improved license enforcement mechanism for corner cases while communicating with the licensing server.
Retain original file name with special encoding for archive extraction
- Note: This feature requires Archive engine version 6.2 or newer.
- By specifying correct encoding of original file name via engine-metadata header while doing file submission POST /file, MetaDefender Core will work with archive engine to retain original filename with the same designated encoding after extracted.
- For example: engines-metadata={ "extraction": { "charset": "SJIS" } }
Support to sign in MetaDefender Core via Identity Provider site (also known as IdP-initiated SSO)
- IdP-initiated SSO allows users to login to MetaDefender Core dashboard directly from IdP homepage.
- Users must enable this new setting under SSO user directory page on MetaDefender Core UI.
Support proxy_pass for NGINX web server on MetaDefender Core
- Use proxy_pass to redirect the URL to another designated location.
- For example - redirect https://localhost:443 to https://localhost:8008 Add file *.conf under C:\Program Files\OPSWAT\MetaDefender Core\nginx
Supported same date selection for processing history filtering
Display "In-progress" label in processing history UI
UI enhancements
- Addressed UI accessibility issues
- Sensitive Data Protection statistics on the Dashboard UI: Changed to calculate and display total number of sensitive result detections (previously: total number of hit objects).
- Update UI labels and hyperlinks for My OPSWAT portal.
Fixed:
[Central Hub] Hub service could be crashed while running under high load circumstance.
[Central Hub] Failed to link files in batch
- Hub refused to correlate files to a batch which was initiated by a MetaDefender Core instance (returned HTTP error 400 - Cannot scan in given batch)
Deep CDR engine process could be crashed on Linux based system
- While processing MS Office 2003 files, only impacted Linux based MetaDefender Core.
Synchronous scan submission issue
- "Require a min number of AV engines for the whole file processing" setting, if enabled and configured, did not work as expected for synchronous scan submission with local file scan feature.
Data warehouse failed to refresh, when data contains non-Unicode characters.
File type mismatched result was not displayed
- When there was another higher blocked verdict (e.g. Sensitive Data Found) is also available.
Version v5.4.0
Release Date: 04 January 2023
New features:
Central Hub enhancements
- Applicable to Central Hub version 1.1.0 or above.
- Load shared among MetaDefender Core instances for archive processing (extraction).
- Shared authentication session among MetaDefender Core instances using Redis caching server.
- Batch processing supported.
Docker containerization enhancements
- Supported MetaDefender Core upgrade with zero down time on containerization environment Kubernetes (blue/green and rolling strategies).
- Supported MetaDefender Core upgrade with data retention.
- Log rotate service built in docker base image.
- Supported proxy configuration for docker run.
- Supported new option to enable "allowCrossIPSessions" when starting MetaDefender Core on container environment.
UI enhancements
- UI remediations and enhancements for accessibility support and compliance.
- Backup license status displayed on UI.
- Sanitized label as indicator in the processing history.
- Search by name supported for workflows.
- Appropriate UI result for vulnerability assessment when "skip hash calculation" setting is enabled.
- Descriptive tooltip for "Fallback file type detection to current extension if needed" setting.
- Changed setting name "Blocklist unsupported file types" to "Block unsupported file types".
Security enhancements
- Upgraded 3rd party libraries (gperftool, libxml2) for vulnerability risk.
- Digest verification supported in FIPS enabled environment while installing MetaDefender Core.
Logging enhancements
- Avoided misleading (and unharmful) error messages in PostgreSQL logs.
- Added batch ID and data ID information in NGINX access logs
- Removed misleading warning messages in MetaDefender Core logs while updating engines via folder.
Hash lookup performance improvement
Support Windows 11
New configuration to reject file submission based on engine availability
- Only allowing MetaDefender Core to accept file processing submissions when certain crucial engines are available.
- This setting can be found under workflow rule - General tab.
New configuration to add exception for unsupported file types for sanitization
- Adding exception list for unsupported file types for sanitization. By doing so, those file types configured in the exception list will not be blocked.
- This setting can be found under workflow rule - Deep CDR tab.
New configuration to block or allow file processing if smaller or bigger than certain file size
- Allowing MetaDefender Core to block or allow file processing submission if its payload exceeds or smaller certain size.
- This setting can be found under workflow rule - Blocklist and Allowlist tab.
New configuration to skip processing fast symlink in archive file
- By default, MetaDefender Core will try to process any files, including fast symlink in archive. However that could cause sanitization failure due to unsupported.
- This setting can be found under workflow rule - Genertal tab.
Import full configuration package (.zip) via the ignition file
- Supported automation better with full configuration package (.zip) via the ignition file. Example: eula=true [config] import=settings.zip import_password=123 ....
- Details: Ignition file
Webhook callback retry mechanism enhancement
- Enhanced webhook retry mechanism for resending callback result in case of either timeout or not receiving webhook server’s response.
- Timeout threshold could be defined via new key
transfertimeout
(by default, 30000 ms) with PUT - Webhook set configurationAPI
New CLI tool to clean up database created by no-longer-used MetaDefender Core instance in the shared DB mode
Using this tool to clean all scan data from MetaDefender Core instance(s) that you no longer use, applicable in the shared DB mode.
This tool can be found:
- Windows: C:\Program Files\OPSWAT\MetaDefender Core\ometascan-db-cleaner.exe
- Linux: /usr/sbin/ometascan-db-cleaner
New CLI tool to check remote PostgreSQL user compliance
- This tool is to help your database system admin quickly determine whether or not your remote PostgreSQL database have sufficient user privileges as described at PostgreSQL User Privilege Requirements to work properly with MetaDefender Core application.
- Details: Remote PostgreSQL User Privileges Checking Tool
Fixed:
Retrieving file processing result hit 404 not found
- Could happen when using archive compression with Proactive DLP
Incorrect memory resource utilization report
- Updated to recalculate memory available reported on the UI.
Undesired behavior on the UI with API rate limiting
- When hitting the limit, the UI hung at the uploading screen.
Some UI cosmetics and minor bugs are addressed.
Version v5.3.1
Release Date: 07 November 2022
New features:
- Upgraded OpenSSL framework from version 3.0.5 to version 3.0.7 for its recent vulnerability patches (CVE-2022-3358, CVE-2022-3786 and CVE-2022-3602). More details: https://www.openssl.org/news/openssl-3.0-notes.html
Version v5.3.0
Release Date: 27 October 2022
New features:
Central Hub new architecture (BETA)
Serving large scale deployments with better dynamic auto-scaling, high availability and fault tolerant.
The Central Hub architecture incorporates multiple newly crafted components running as service, and also containerization environment friendly:
- MetaDefender Core.
- PostgreSQL database server.
- Central hub (known as "MetaDefender Core Hub").
- File storage server (known as "MetaDefender Core NAS").
- Redis (the open-source caching server).
- RabbitMQ (the open-source messaging broker).
More details: Central Hub (BETA) Overview
PostgreSQL bundle upgrade
- Applicable to PostgreSQL server bundled with MetaDefender Core installation. It is upgraded to version 12.12 (from old version 12.11) for vulnerability fixes.
Engines integration improvements
- Back-end related improvements against engines integration for more scalable, flexible and robust engines' new features and furture changes without MetaDefender Core upgrade requirement.
New workflow setting for Proactive DLP
- Allow users to configure MetaDefender Core to block file processing (final verdict = Blocked) if the Proactive DLP processing fails or times out.
Vendor information added into processing result PDF report
- Only applicable to files which are detectable for file signature.
Fixed:
Product stability improvement
- MetaDefender Core service could crash under certain circumstance (modifying workflow rules while actively scanning files).
- Integration issue with Deep CDR that could possibly wipe up sanitized files by mistake when Deep CDR engine is stopped.
- Engine ID was wrongly displayed in workflow rule for "excluded engines" section after removing that engine.
Product UI fixes
- Custom date and time feature: users could not set PM for start time, and AM for stop time as a range. For example: Sep 07 12:00:00 PM to Sep 08 12:00:00 AM
Version v5.2.2
Release Date: 06 October 2022
New features:
External scanner and post action are now available configurable for the workflow
Use certain external scanners and post actions with corresponding settings in each workflow rule.
The import & export configuration feature will also support to cover external scanners and post actions defined in the workflow.
For MetaDefender Core upgrade scenario:
- When you already have external scanners and post actions defined in the Inventory in older product version, then all of them will be automatically added to every available workflow rule (not template) in the new product version once upgraded.
- Applied to shared DB mode, all external scanners and post actions settings will be auto shared among MetaDefender Core instances, unless the license for external scanner on each MetaDefender Core is different.
Details: External Scanner and Post Action
File digital signature vendors in the executive report
- Vendor statistics information for file digital signature is now calculated and displayed in the executive report.
More details for file digital signature analysis information
- Detect multiple signers (co-signers).
- Verbose information of digital signature: version, issuer, serial number, digest algorithm, digest encryption algorithm.
Archive extraction performance improvement
- When processing archive file that includes empty folders.
UI updates
- Vulnerabilities Detection now is clickable.
- Support to check enabled file types for sanitization in the workflow rule when MetaDefender Core is managed by OPSWAT Central Management.
- Some other minor UI changes.
Fixed:
Product stability improvement
- Iterrupted connection with syslog server could crash the MetaDefender Core service.
- Nested files was not be extracted correctly if its filename and its parent folder name are identical.
- Did not display the result and error correctly when processing unsupported file types with Threat Intelligence.
- User management UI loaded very slowly while handling numerous (thousands+) users.
Product UI fixes
- Too aggressive polling for file processing result when users navigated to some last pages.
- UI cosmetic bug fixes.
Version v5.2.1
Release Date: 30 August 2022
New features:
Compression tab in workflow
- MetaDefender Core workflow is organized better with new compression tab to provide the archive compression related configuration for both existing Deep CDR use-case, and now, for the new-feature Proactive DLP use-case as well on archive files.
- More context: archive compression technically covers the archive sanitization feature that we know so far, but now we take them out from Deep CDR workflow tab to mange everything relevant under the new Compression tab for both Deep CDR and Proactive DLP use-case.
Archive processing (Compression) with Proactive DLP
- MetaDefender Core now supports the archive compression for Proactive DLP use-case, along with already-supported Deep CDR so far.
- That means, MetaDefender Core will also pack applicable output files produced by Proactive DLP as well for the new compressed archive.
- By default to keep backward compatibility, this feature is turned off for Proactive DLP use-case, and you must enable it under the new Compression workflow tab.
Hash lookup API new features
Hash lookup API is prefered to optimize the file processing especially when same files to be processed over again within a short time period.
MetaDefender Core now supports 2 new modes for the hash lookup API via new headers called
timerange
andselfonly
:timerange
(number of hours): allow scoping down the recent number of hours that hash lookup task should start from till now, instead of searching the entire scan history in MetaDefender Core database.selfonly
(true/false): useful to archive hash lookup, allow specifying to only perform hash lookup against the original archive file self only, and skip searching all child files result within the original archive.
More details: Fetch Analysis Result By HashAPI
Verbose file type analysis data
- Support to expose verbose file type analysis data in JSON scan details. This can be done via setting
include_
file___details
in theengines-metadata
header for POST file submission request.
- Support to expose verbose file type analysis data in JSON scan details. This can be done via setting
Expose file type ID information
- The info can be found in both JSON scan details, and on the scan result page UI. The file type ID will help indicate unique file type identification, e.g. PDF / TXT
Engine dependency missing error response
- Certain engines on MetaDefender Core require specific dependencies to run properly, otherwise failing on engine is expected. MetaDefender Core now make the dependency missing error more visible in both UI and log, and help troubleshooting easier and faster.
Custom internal PostgreSQL user
- MetaDefender Core once installed, it will automatically create an internal PostgreSQL user for operation purpose, and so far that user name is auto randomized by MetaDefender Core and non adjustable (
usr_<random characters>
). - Now MetaDefender Core supports database system admin to define and change to their own PostgreSQL user instead.
- More details: Customize Internal PostgreSQL User
- MetaDefender Core once installed, it will automatically create an internal PostgreSQL user for operation purpose, and so far that user name is auto randomized by MetaDefender Core and non adjustable (
Security enhancements
- Continuing to harden and secure MetaDefender Core even better to prevent potential HTML injection and XSS attacks.
UI updates
- The setting page is auto refreshed itself after importing configurations.
- Added copy button to copy entire file path on the scan result UI.
- Pagination supported for "Skip by hash" UI.
- Numbers on the dashboard are now clickable.
Fixed:
Product stability improvement
- MetaDefender Core on Linux crashed randomly while processing archive files.
- Very long list of hashes in the Skip by hash caused slow response time on some admin API endpoints.
- Regular expression validation failed with mode modifier (?i).
- Display issue with pagination on the processing history UI.
- Engine download could be hung up if keep hitting download error.
- Failed to show blocked leaf files if searching by hash.
- Filtering by post action was not correct on the processing history UI.
Product UI fixed
- UI cosmetic bug fixes.
Version v5.2.0
Release Date: 12 July 2022
New features:
New OS support
- Windows Server 2022
- Debian 11
Security enhancements
- As a regular security practice, we upgraded 3rd party dependencies and development framework to prevent critical vulnerabilities:
- Upgraded OpenSSL 3
- Upgraded Qt framework 6
- As a regular security practice, we upgraded 3rd party dependencies and development framework to prevent critical vulnerabilities:
Executive report in PDF
- The executive report is now supported for PDF file export. Details: Executive Report
File digital signature verification
- File digital signature is now analyzed and recognized by MeDefender Core while processing. The vendor digitial signature is available on both processing details UI page and JSON scan result, and also calculated for the statistics found on Executive Report. Also supporting users to allowlist files based on specific vendor names (using regular expression) configured in the workflow rule (under "Allowlist" tab).
- Currently only PE files are supported
.
acm,.ax, .cpl, .dll, .drv, .efi, .exe, .mui, .ocx, .scr, .sys, .tsp
(Portable Executable)
Blocked leaf files filtered in archive scanning
- Leaf file (in archive scanning context): file that not having any successful extracted file inside.
- Support to filter and display blocked leaf files in archive scanning on the processing details UI page ("Show blocked leaf files" checkbox). That helps narrow down the searching scope, and spot out exactly blocked files that should be investigated further.
Blocked reasons exposure
- All blocked reasons are exposed on the processing details UI page (if archive file, then also inheriting from child files' blocked reasons if available), that helps with further context on unclear failures, and troubleshooting.
A setting to exclude sanitize version of blocked child files from archive sanitization
By default (Archive Sanitization) feature will pack following into sanitized archive file:
- All allowed child files (or their sanitized version if available).
- Sanitized versions of blocked child files (if blocked child files are sanitized successfully).
A new setting "Include sanitized version of blocked child files" under Deep CDR tab in MetaDefender Core workflow rule will allow you skipping not to include sanitized versions of blocked child files, so that you can conduct further investigation on those blocked files first.
Sanitization tag on processing details UI page
- Especially helpful to archive processing when you can tell which child files are sucessfully sanitized, and which files are not on one UI screen (without checking every single child file for details).
Data retention for statistics
- The new setting can be found under Settings page - Data Retention tab with named "Statistics". By default, the retention range is set to 6 months.
- Details: Data Retention
An installation option to skip startup UI screens
- Better support for automation environments.
A new setting
skip
_startup _pages
supported in the ignition file to skip all welcome and other startup pages (e.g. upgrade result) when installing / upgrading MetaDefender, and going straight to the product index default page. - Details: Ignition file
- Better support for automation environments.
A new setting
Skipped sanitization result
- A new sanitization result to indicate the santization is skipped due to certain reasons ruled by Deep CDR configurations.
Child files processing time displayed in achive processing JSON result
- Original archive file will want to wait for all child files processing to be finished before finalizing the overall processing.
- Previously
others_time
covered that as well in JSON result, but now we make a dedicated new field for that in JSON result calledwait
`_
child_``files_time`
HTTPS configuration removal tool
- Just in case you configured HTTPS via MetaDefender Core UI but due to unexpected failures on certificates, and thus being blocked to make any change because MetaDefender Core service is not running, thus you need to disable HTTPS configuration on MetaDefender Core using a dedicated CLI tool - (HTTPS Configuration Removal Tool
Minor UI changes
- Some UI updates on the executive reports, modules.
Fixed:
Product stability improvement
- Failed to process same files repeatedly with scan-by-download-link feature.
- Failed to extract archive files occasionally.
- SSL certificate issue on RedHat 8.
- Memory leak during PostgreSQL disconnection and more scan requests are still coming.
- Other minor product bug fixes.
Product UI fixes
- UI cosmetics bug fixes.
Version v5.1.2
Release Date: 27 April 2022
New features:
Vulnerability CVSS v3 scoring
- File-based Vulnerability Assessment now supports CVSS v3 scoring, also comes with attack classification and impact details.
Fallback for engine (module) deployment
- Engine deployment on MetaDefender Core is now fault-tolerant support i.e. just in case a new version of engine ever failed to be deployed, the current (older) working engine version will be re-used and kept running well.
- Notes:
- Disable and re-enable engine: MetaDefender Core will try to re-deploy the new engine version again, if still failing, then the old-working engine version build will be re-used (fallback).
- Remove engine: MetaDefender Core will remove all engine package(s) entirely, including new and old-working versions.
- Not supported for custom engines which require separate engine installation setup
Fixed:
Product stability improvement
- Faster API responses when processing very long list of users (>= 1000 users).
- Custom engines sometimes could be inactive after being updated.
- Even when enabling processing of allowlisted file in Workflow Rule, the file could not be processed properly.
- Fixed minor product bugs.
Fixed some security vulnerabilities on the product:
- Privilege escalation: allows non-admin users to gain unauthorized privileges on the product to execute harmful actions via the user management configuration. Details: CVE-2022-32272
- Information leakage: an MetaDefender Core user without permission of accessing to the MetaDefender Core server could leverage the observable discrepancy attack method to expose file system structure on where the product resides via the certification configuration. Details: CVE-2022-32273
Product UI fixes
- The pie chart in the Executive Report page was not rendered and displayed correctly occasionally.
- Some other UI cosmetic minor fixes.
Version v5.1.1
Release Date: 31 March 2022
New features:
Product performance improvement
- Up to 50% throughput increased while processing archive / office files. Applicable under certain circumstances, configurations and data.
Product security improvement
- Remediate some potential security related issues while operating MetaDefender Core.
Optimize processing flow for office files
- All child nested files inside original office file are no longer sanitized and DLP processed separately.
Bundle PostgreSQL upgrade (v12.10)
- Applicable to the local PostgreSQL server bundled with MetaDefender Core installation.
User's API key redeployment
- Allow API key of an already-removed user to be assigned to another user, this is to ensure no interruption or changes in the cross-system integration.
Health check settings supported for the configurations export/import feature
- Health check settings (Health Check API Configuration) is now included in the MetaDefender Core configurations export/import feature
Auto upgrade when connecting to an older database version
- Allow MetaDefender Core auto detect and upgrade its database when connecting to an older database version as expected.
Single Sign On (SSO) SAML enhancements
- Support AES128-GCM and AES256-GCM encryption algorithms.
Product UI/UX enhancements
- Add OS information to the dashboard UI
- Resizable column width on tables (workflow rules, workflow templates, workflow zones, modules)
- Toggle configuration for proxy credentials
Fixed:
Product stability improvement
- Avoid potential Nginx socket read timeout, and MetaDefender Core crash issue under certain memory shortage circumstances.
- MetaDefender Core service crashed when trying to load an empty advanced configuration of engines.
- Some minor bugs.
Product UI fixes
- Pagination is broken when the page number is too large.
- Inconsistency multiscanning engine names on the module page.
- User Management page didn't display any record.
- Cannot traverse the list of files in a batch on the scan result page.
- Incorrect results are returned when filtering the datetime on Statistic page.
- Scan result report in PDF text alignment issue.
- Cosmetic minor fixes.
Incorrect callback messages mistakenly sent
- "In-progress" scan result callback message was mistakenly sent to webhook server while the scan is not yet finished.
Disk space availability check omitted before processing file request
- MetaDefender Core is supposed to check for disk space availability on where the product temporary folder is configured to ensure it could store file and serve processing request.
Kaspersky and NanoAV engine failed to load on MetaDefender Core Linux
- The issue was due to incorrect handling within symlink in the engine package.
Engine manual upload issue
- Re-uploading the same engine package again immediately after a failed attemp on the same engine might cause issue on engine upload feature until MetaDefender Core service is restarted.
Processing failure with the encrypted archive ALZ/EGG file
- Failed to sanitize and re-compress those file types.
Failed to remove AD/LDAP users
- Hit error message "Item does not exist" on the UI when trying to remove AD/LDAP users.
Temporary files and folders cleanup
- Obsolete temporary files and folders were not cleaned up when user configured to set a new tempdirectory location.
- Temporary files downloaded from a link were not cleaned up when disabling engines during scan.
Version v5.1.0
Release Date: 23 February 2022
New features:
New statistics data for executives report
- Average file size based on processed file types
- Hash calucaltion time
- Archive extraction time
- Per AV engine processing time
- Details: Executives Report
Exclusion list for proxy server configuration
Define an exclusion list of destination addresses to bypass going through proxy server configured on MetaDefender Core. Applied to connections triggered by MetaDefender Core for following activities:
- Callback (webhook)
- Engine update
- Scan by download link
- Single Sign On integration (Fetch metadata from URL and login)
- Remote PostgreSQL database connection
Details: Exclusion List
Support Ubuntu 20.04
- See the list of supported OS at Recommended System Configuration
API key in the ignition file
- Local admin user's API key could be now definied in the ignition file for automation enhancement.
- Details: Ignition file fields
Network drive and UNC path for update folder pickup
- Allow users to define UNC path and network drive for update folder pickup setting.
- Details: Local folder update
Timeout indication for Proactive DLP processing
- Whenever Proactive DLP processing timed out, a clear and explicit timeout result is now used (instead of a generic failure reason).
More logging comprehensive information for scanning activity
- Absolute submitted file path for local scan feature.
- Clear indication to distinguish between asynchronous and synchronous scan.
- File path, file name, callback URL, sanitize URL, download-from URL for both asynchronous and synchronous scans.
New simplified REST API to download output file after sanitized and Proactive DLP processed
Clear indication for archive extraction in STDIN for post action
- A new key of "extracted_files" is added into the STDIN for post action to indicate whether MetaDefender Core indeed extracted to child files from original file or not.
- Details: Post Actions
Resizable table column on the UI
- Applied to tables on the UI including processing history, quarantine, user management, engine list, quarantine etc.
Security enhancements
- UI setting input validation.
Fixed:
Product stability improvement
- Fixed minor bugs regarding to product stability.
Samesite cookie modification with SAML based SSO
- Value of samesite cookie was not updated correctly while using SSO with SAML integration.
Wrong error code and error message with user role and directory removal
- Wrongly returned 404 (item does not exist) instead of 400 when attempting to remove non-existed user role and directory.
Custom data directory configuration
- MetaDefender Core did not point to custom data directory after upgrading on CentOS / RHEL.
Yara engine failure
- Yara engine could become permanently failed after updating.
Minor UI fixes
- Cosmetic bug fixes
Version v5.0.1
Release Date: 28 December 2021
New features:
New fields supported for PDF scan report
- Receive data timestamp (indicating when a first byte was uploaded to MetaDefender Core from client)
- Upload timestamp (indicating when entire upload was finished)
- Upload time (indicating total time spent for upload)
Product UX improvement
- Under MetaDefender Core modules page: Different color (Orange) for engine and database when they are still being downloaded and deployed, not reached yet to final state. And some other minor UI changes.
Archive compression engine auto enabled after product upgrade from v4 to v5
- This engine is supposed to be inactive and permanently failed on MetaDefender Core v4, when upgrading to v5.0.1, now this engine will be auto re-enabled back without any user action required.
Hot backup register API behavior updated
- Not possible now to register a MetaDefender Core instance as backup when
active-deployment-id
header value equals to that own instance's deployment ID. Otherwise, expecting to hit an error:{ "error": "active-deployment-id cannot be the same with this Core's deployment ID" }
- Not possible now to register a MetaDefender Core instance as backup when
Security hardening
- 3rd party vulnerabilities check and remediation.
Fixed:
Sanitization of files without extension on Windows
- Applicable to Windows
- When configured to keep both original file name and extension
${original.basename|long}.${original.extension}
, but original file name does not have extension: Previously MetaDefender Core auto changed the dot (.) between file name and extension (which is empty) to underscore (_). Now it is no longer the case, expecting to have only original file name.
User agent "webscan" took no effect
- Adding user agent "webscan" to any workflow should now force MetaDefender Core to display that workflow for selection on processing scan UI.
Engine communication was not resumed correctly after engine timed out or crashed
- When happened, causing not scanned / not available result on engine processing.
Missing Proactive DLP failure details
- Previously it just showed failed to process for Proactive DLP, now the failure reason should be exposed correctly (e.g. file size limit exceeded).
Wrong archive handling details description
- Mistakenly changed from "Corrupted Archive" to "Unsupported Archive"
Wrong sanitization time when timed out
- When sanitization timed out, cdr_time was mistakenly set to zero (0).
Leftover Node related folder / key
- On Windows (
Computer\HKEY
_ LOCAL _ MACHINE\SOFTWARE\OPSWAT\Metascan Node`` ) - On Linux (
/etc/ometascan-node
) - They should be all cleaned up after upgrading to MetaDefender Core v5.
- On Windows (
Leftover archive engine temp folder
- Only encountered when disabling archive extraction engine during extraction.
Exporting scan history for all instances in shared database mode
- Applicable to CSV / STIX export. The report only contained scan history records made on that current MetaDefender Core instance.
UI bug fixes
- Engine status color display on UI.
- Processing scan highlighter.
- Total number of processed objects displayed on Dashboard UI when being too big.
- Error message on User Management disappeared when hitting Save changes button twice.
- Unable to view sanitization details on UI when XFA form is sanitized.
- MetaDefender Cloud API key was not redacted on Firefox.
- Not displaying error message on UI when failing to import product configurations.
- While file processing is happening, hitting details in DLP section was mistakenly redirected to AV section instead.
- Error message was misleading when modifying a certificate being used by workflow rule or HTTPS setting.
- Not able to continue testing LDAP configurations on UI after previous attempt was successful.
Version v5.0.0
Release Date: 30 November 2021
New features:
Processing performance improvement
- In typical use-cases, expecting better throughput on version 5 with architecture changes and processing flow optimization (30% - 200% depending on specific circumstances, configurations and data set)
- More details: (Link Removed)
RedHat 8.x is supported (official)
- Installation guide: Red Hat Enterprise Linux 8.x package (.rpm)
Security hardening
- Eliminated critical and major vulnerabilities found in 3rd party libraries as well as the application itself.
Hot backup mode supported for high availability
- Supporting to setup backup MetaDefender Core instance(s) to run in parallel with your production (active) instance. Backup instance(s) will be allowed to use the same license key as your active one without actually being accumulated into the activation slots, and allowed to have all engines downloaded and deployed, configurations pre-setup, share the same database with active one (except of processing files).
- Any of backup instance(s) should be ready to take over the active one anytime as a part of your disaster recovery plan to minimize the down time as much as possible.
- More details: Hot Swap Backup Configuration
Health check API
- Determining MetaDefender Core server's readiness to process upcoming files based on pre-defined criteria / policy that configurable by administrators. Allowing native support for a large pool of MetaDefender instances (load balancer in autoscaling groups. or liveness probe in Kubernetes, etc.)
- More details: Health Check API Configuration
Server active performance statistics API
- Determining MetaDefender Core performance (processing time, throughput, queue time etc.) within specific time window to help you make better decision on which server is the best to route upcoming requests. Metrics that can be also collected and reported in external tools, not just via SIEM.
- More details: GET - Active performanceAPI
Bulk hash skipping modification API
- No longer need to import one by one hash for either or both whitelisting and blacklisting. Allows you to import a list of hashes instead in a form of file.
- More details: POST - Add new hashes to 'skip by hash' listAPI
Parallel engine deployment
- Only applicable to bundled engines for now. Bundled engines will be deployed and initialized in parallel, to make MetaDefender Core get to ready state faster, and possibly reduce down time in dynamic environments (Containers, Cloud Autoscaling, etc.).
More data points in log message for processing finished event
- Applicable to log event generated upon a processing analysis task is finished.
New information added:
- Parent data ID
- File size
- File type description
- Total processing time
- Applicable to log event generated upon a processing analysis task is finished.
New information added:
Estimated completion time on product upgrade
- Upgrading time depends on various conditions including system specs itself, data size, characteristics etc. thus a roughly upgrade estimation time would be useful to set right expectation, especially to big size database that typically requires hours in upgrade. As a reminder, there is already an existing option on product to skip migrating processing data for upgrade when you do not really need it.
Preferred encoding supported for sanitization and DLP processing
- A new header
engines-metadata
forPOST /file
(asynchronous scan) andPOST /file/sync
(synchronous scan) to specify explicitly payload's preferred encoding (e.g. ISO-2022-JP) for sanitization and DLP processing. If not specified, then default encoding (UTF-8) will be applied. - More details: POST - Analyze File (Asynchronous mode)API
- A new header
Proactive DLP integration enhanced
- Helped expose new Proactive DLP features on MetaDefender Core workflow rules, and make the product workflow flexible for changes and improvements.
- More details: Proactive DLP release notes
SHA256 checksum of objects in Deep CDR analysis result details displayed
- SHA256 checksum calculated on every sanitized objects processed by Deep CDR is now displayed on MetaDefender Core's processing result UI.
Handle unsupported file type for archive compression
- Under MetaDefender Core workflow rule - Archive tab. A new setting option where you can select to fail the scan with unsupported file type for archive compression / sanitization.
Enhanced logic to help avoid MetaDefender Core initialization failed due to database failure
- Possibly encountered in container based environment with remote PostgreSQL database in certain circumstances.
A new option for unsupported compression method handling
- Under Archive handling settings in MetaDefender Core workflow, a new option to allow failing archive extraction task for unsupported compression method.
Fixed:
Advanced Deep CDR settings were reverted under certain circumstances
- Only encountered when using workflow rule based on custom workflow template (this issue had happened since MetaDefender Core version 4.20.0) Changes made on Deep CDR advanced settings were reverted after MetaDefender Core service restarted or Deep CDR engine re-enabled.
Vulnerability processing result was wrong in PDF scan report
- When actual result was a potentially vulnerable file, but reported as no vulnerability found mistakenly in the PDF report.
Fix memleak when handling Proactive DLP errors
- Possibly happened when Proactive DLP encountered errors while processing files.
Reinstalling MetaDefender Core could be failed
- Uninstalling the product and then reinstalling back on the same machine but with different deployment ID.
Version v4.21.2
Release Date: 30 August 2021
New features:
Mutual authentication for secured webhook (callback) mode
- An additional security mode for callback HTTPS to authenticate requests indeed sent by MetaDefender Core, and help against man-in-the-middle attack. Details could be found at: Webhook Authentication
Webhook (callback) for sanitized file download (BETA feature - not recommend to be used on production)
- When registered by client, whenever a sanitized file is produced and ready, a new callback with sanitized file content will be sent back to client.
Configurable relevant session cookie attribute aided in CSRF attack prevention
- "SameSite" attribute on the session cookie "session_id_ometascan" with supported values: Lax (default), Strict, None (i.e. empty) to be configurable. Supported to configure via either administrative REST API, or ignition file. Details could be found at: Configure session cookie attributeAPI
Support SSL connection between MetaDefender Core and PostgreSQL
- Along with a setup guidance for SSL database connection. Details could be found at: SSL connection for PostgreSQL communication
SQL Injection prevention check and ensurement
- On all supported REST APIs
Adaptive mode when moving AD / LDAP users to other OU (Organization Unit)
- Allowed users to change DN information in AD / LDAP directories
Logging details enhancement on Webhook (callback) failures
- Enhanced on generic failure (error code = 444) to expose more details in WARNING log level. Details could be found at: Webhook error message descriptionAPI
Logging details enhancement when failing to open PostgreSQL service
- In a circumstance of when failing to open PostgreSQL configuration file.
Separate JSON configuration file now collected in the support package
- For easier analysis and import back into MetaDefender Core when needed.
Preinstall Deep CDR and Proactive DLP dependencies in MetaDefender Core docker image (libgomp, libgdiplus, mscorefonts)
Fixed:
MetaDefender Core upgrade failures
- Handled duplicated sanitization detail records
- Handled empty key in registry when installing by commandline
- Handled invalid value of type in ignition file
- Handled invalid Unicode characters
Initialization failure of Threat Intelligence engine on Ubuntu
- Updated openssl linkage to address the issue
Improper value type of "archive_handling_details" in JSON response
- In a case of processing password-protected archive file, the value was set incorrectly, it should always be a map data type (not string).
Lacked of compression failure detail (archive sanitization)
- When MetaDefender Core fails to re-construct email file (.eml) in archive sanitization.
Incorrect error message when recovering user password
- In a case of the email server address configuration is not setup correctly for password recovery.
Out of bound value of session timeout handling
- Setting value on session timemout to 1 million or above resulted in authentication failure.
Update hang up on engine in certain circumstance
- Engine remained in "staging" status indefinitely if the configuration validation failed for engine.
Missing Proactive DLP engine processing result
- Making sure all engine results return correctly in both failure and success circumstance.
Incorrect scan result in PDF report
- In a case of using skip scan on engine setting
License auto deactivation did not work properly in docker container deployment
- When stopping docker container, the docker automation script in the toolkit (entrypoint.sh) did not handle license deactivation properly.
Retry mechanism of Webhook (callback) did not work as expected in special circumstance
- When MetaDefender Core service is restarted while operating, the callback was not sent as designated in callback retry configuration.
Intermediate certificate was stripped out
- HTTPS configured with full chain certificate scenario, the intermediate certificate could be stripped out before importing into the product
Version v4.21.1
Release Date: 13 July 2021
New features:
A new setting to exclude password-protected documents from being blacklisted due to unsupported sanitization
- The password-protected document is not supported for sanitization and possibly blacklisted when enabling "BLACKLIST UNSUPPORTED FILE TYPE" option under Deep CDR workflow rule.
- Now you can exclude it via a new setting. By default, the new setting is not enabled to keep backward compatibility.
Upgrade 3rd party libraries for security enhancement
- (Built-in) PostgreSQL from 12.6 to 12.7
- NGINX web server from 1.18.0 to 1.20.1
- Remove outdated jsPDF 1.5.3
A new CLI tool to analyze and deeply vacuum database
- PostgreSQL database could be more bulky over time with high scanning traffic. The tool which is bundled with MetaDefender Core installation is to analyze database and vacuum database effectively, and simplify IT administration. Even though it is not required to stop MetaDefender Core service while running, but still recommended to run the tool when the scanning service is at rest, or not in peak hours to avoid scanning performance impact.
- Details: Database Maintenance | MetaDefender Core 4.19.0 or above
More data points collected for the support package
- Including size of objects from MetaDefender Core database to help us determine better the characteristics of the data for troubleshooting purpose.
Other minor changes
- Enhanced integration logic on engine configuration handling to avoid potential configuration corruption.
- Enhanced installation script in Linux-based installer to avoid misleading warning.
- When the Vulnerability assessment module is not enabled, the relevant vulnerability result on the MetaDefender Core scan result UI will be now as "not configured" (instead of no vulnerability found).
Fixed:
OPSWAT Central Management integration issues
- Workflow rule list order was different between MetaDefender Core and OPSWAT Central Management UI.
- Last configuration import timestamp was invalid.
Deep CDR's advanced settings was not applied
- Occurred when creating a new workflow rule on MetaDefender Core.
Negative duration processing time
- Under certain corner circumstances, the wait_time and queue_ time in JSON scan result could be mistakenly calculated.
User on old database was left out during the MetaDefender Core upgrade
- When upgrading MetaDefender Core with standalone database mode to shared mode.
Leftover files when canceling batch
- Occurred when handling archive files.
Version v4.21.0
Release Date: 14 June 2021
New features:
Docker container support
- A new deployment support for MetaDefender Core on docker container environment.
- Supporting for Linux based environment (CentOS / RedHat, Ubuntu / Debian). Windows based more to come later.
- Comes with 2 options: (1) Docker images hosted by OPSWAT (2) A build kit with docker file and scripts to use with your own docker image.
Synchronous scan
- One more new scan API interface for Core with a sync mode (blocking mode). That helps ease integration hassles on some of your web applications.
- The connection of file submission request must be held by client side until MetaDefender Core returns processing result of individual scan when finished, or return data ID associated to that request if timed out (which is configurable).
Scan performance improvement
- Optimized for resource conflict handling, even better under high scan rate with many AV engines.
Azure AD supported with OIDC
- Continuous Sigle-Sign-On (SSO) improvement with OIDC supported for Azure AD.
New scan result "Potential Unwanted" for PUA/PUP
- Applicable to AV engine scan result.
Crash dump auto collected for support package
- Only if crash dump is available for MetaDefender Core related processes.
Statistics page enhancement
- New section"Custom Regular Expression" applicable to Proactive DLP.
Proactive DLP settings limit range extended
- Extended maximum character length limitation of Proactive DLP settings under workflow rule.
Secure the integration with Central Management product
- The integration credentials meet complexity requirement.
Retain password protection for RAR file sanitization
- A new option for RAR file could be found under CDR tab in workflow rule, applicable to password protection for archive sanitization.
More data points for /stat/nodes API endpoint
- Including:
- Total disk space
- Total scan queue
- Service uptime
- Including:
Upgrade 3rd party libraries
- Built-in PostgreSQL 12.3 to 12.6
- Lodash 4.17.10 to 4.17.21
Product upgrade process enhancement
- For back-end processing efficiency when upgrading MetaDefender Core from standalone to shared DB mode.
Fixed:
Product upgrade issues
- Fixed some failures when upgrading the product with malform data in database.
Default workflow template mistakenly modified with Central Management integration
- On Central Management, changing any setting in a new workflow rule unexpectedly resulted in changing associated workflow template.
Wrong data in PDF report for processing result
- Under certain circumstances, some information in PDF report could be invalid / misleading.
Incorrect blocked_reason in tombstone file
- When processing an archive file contains at least one child file returning Sanitization Timed Out / Sanitization Failed.
Minor fixes
- Some other minor bug fixes.
Version v4.20.1
Release Date: 31 Mar 2021
New features:
MetaDefender Core's PostgreSQL database admin credentials management
- Supported via both web management UI and CLI tool to change Core's PostgreSQL database information.
- The CLI tool could be used for older PostgreSQL based Core versions as well (4.19.0 to 4.20.0)
MetaDefender Core upgrade mechanism changes
- Ensure the old product database is always kept regardless of upgrade result, and easier to rollback when needed.
Fixed:
MetaDefender Core upgrade failures
- Addressed various upgrade failures occurred with Core version 4.20.0
Scan stuck until timeout hit under certain circumstances
- Possibly happened with Core 4.20.0 when the data retention is enabled for sanitized / DLP processed file, and only when the system keeps processing the same file over again while running the data retention.
- This issue could only be applicable when CDR and(or) DLP feature is used.
MetaDefender Core service failed to start
- Encountered when the Core's PostgreSQL database admin password contains special characters /-+"?<>:{}&^%$#@!~`';.,\
Invalid batch signature
- Due to a bug in 3rd party library, batch signature could be invalid.
MetaDefender Core service issue on docker
- Only applicable when replacing systemctl to start Core service within docker container.
MetaDefender Core service failed to start automatically after reboot
- Thus sometimes required manual start action.
Minor fixes
- Some other minor bug fixes.
Version v4.20.0
Release Date: 08 Feb 2021
New features:
Centralized (or shared) database mode
- A complete new database design to support to use single PostgreSQL database shared among all connected MetaDefender Core applications. That helps auto-sync all product configurations (except Single-Sign-On that requires unique settings for each Core), processing histories, live/on-going scan result query, audit, statistics data. All product functionalities are updated to support both standalone and shared database deployment model.
- Database migration (configurations and processing history) is also supported for older Core versions upgrade.
Workflow based for Deep CDR advanced configurations
- Deep CDR advanced configurations now could be defined differently for each workflow rule on MetaDefender Core.
HEAD request is now optional for the scan-from-link feature
- MetaDefender Core no longer requires to send HEAD request to check for metadata of file before downloading with scan-from-link feature. Some storage web services such as AWS S3 with signed URL, it does not support unauthorized HEAD request.
- When HEAD request is omitted, expecting no support for download progress status and insufficient disk space pre-checking.
Fixed:
Statistics data failed to migrate after Core ugprade
- Upgrading from old Core version 4.17.3 or below failed at statistics data migration.
Processed file content duplicated
- The issue could sometimes occur when we have two or more same file to be processed.
Minor UI fixes
- Tooltip for processed file cleanup explanation
Version v4.19.2
Release Date: 07 Dec 2020
New features:
Non-persistent scan mode
- A stateless scanning solution to improve performance. When triggered, MetaDefender Core will not write any scan result into its database.
- Note: Webhook scanning fashion must be used from client side to retrieve scan result back from MetaDefender Core.
Interactive engine removal on UI
- Any engine module could be removed on the management console (instead of using CLI tool). No service restart is required, no command to run.
New Identity Provideres (IDP) supported and verified for Single Sign On (SSO)
- Microsoft Azure AD Connect
- Ping Identity
- Idaptive
- Xecurity
- OneLogin
Data retention supported for statistics
- Data stored in statistics warehouse could be configured to clean up.
Minor supports / changes
- File type processing time included in final JSON response
- DLP image cropping feature support (Only available starting Proactive DLP version 2.6.0)
- Remove BETA from Statistics page
- UI clarity with scan result page
- Unicode characters better support for DB migration (product upgrade)
- Better performance for statistics page
- Support package to include engine logs (only available when engines start writing it)
- Workflow rule setting values validation
- Case sensitive support in header "filepath" for POST /file request
Fixed:
Performance degradation issue mitigation
- Archive scanning performance
- Hash lookup slow response
- Batch close request potential bottleneck
- Database dynamic query optimization
Memory leak
- Both could happen on ometascan and ometascan-node processes under certain circumstances.
Incompatible Nginx cipher algorithm
- Default acceptance cipher algorithm list modified to be less strict.
Missing vulnerability detailed result
- Vulnerability detailed result could be missing from final scan report in some cases.
Very high CPU continuous usuage occupied by node process
- Might happen when extraction failed and then affecting to the completion of hash calculation process, result in high CPU usuage continously.
Potential service crash
- While cancelling archive scanning
- Engine package download
Issues with scan result report
- Wrong AV and Deep CDR result
- Missing Proactive DLP section
- User right issue (i.e. who is allowed to download the scan report)
Proactive DLP setting issue with unicode characters
- The unicode setting value could be malformed after Core / Proactive DLP upgrade
Other product functional issues
- Blocked files was not stored in quarantine with local file path scan, or scan from link fashion.
- Wrong data calculation on statistics page
Version v4.19.1
Release Date: 21 Oct 2020
New features:
Processing file with input as a download link
- Supporting users to process file by just specifying a direct download link, which means no required for the payload in HTTP(S) request body.
Tombstone file supported for archive sanitization
- Whenever any child file in original archive file are not sanitized successfully, the product supports to append tombstone file into sanitized archive file to indicate which original child file(s) are removed from sanitized archive output file with details.
No hash calculation configurable setting
- Support an option to skip hash calculation on every processing file (using dummy hash values instead) regardless it is individual file or a part of archive.
- Expecting to be used against giant file processing to significant reduce overall processing time.
Nginx web server component ugprade
- Nginx web server has been upgraded to its latest stable version 1.18.0.
Comprehensive individual scan report in PDF
- Support users to download a detailed scan report in PDF format on any processed individual file.
PostgreSQL performance improvement on scan result query
- Improved scan result query against PostgreSQL server in certain circumstance (query by hash value).
- Enhanced mechanism in selecting relevant database connections for serving data related requests.
PostgreSQL connections configurable setting is supported (for scaling up)
- A new setting called " db_connection" to define a maximum number of concurrent connections allows MetaDefender Core to open and work with PostgreSQL database server.
UNC path supported for local file scan
- Local file scan feature now accepts UNC path specified on desired workflow rule's setting
New processing time in JSON scan report for each stage in workflow processing
- The JSON response for scan result will include processing time on each major workflow processing step.
New workflow rule configuration supported for file-based vulnerability assessment
- File-based vulnerability assessment will be supported to be managed under each workflow rule, comes with new configurable settings.
New workflow rule configuration supported for file type analysis
- File type analysis will be supported to be managed under each workflow rule, comes with new configurable settings.
New configuration to skip further processing (and block) in certain malware scan results
- Support an option to skip any further processing (e.g. sanitization) EXCEPT post action run (if configured) on every processing file if the AV scan engines' final result matched to any supported and checked scan result in the list.
New configuration to skip further processing (and block) if file type detection failed
Enabling it to let MetaDefender Core skip further processing if actual file type could not be detected by file type engine for following reasons:
- File type engine process crashed
- Waiting tasks in queue aborted when file type engine process terminated
- File type analysis task timed out
- User cancelled entire processing while analyzing file type
When triggered, the final scan verdict will be “Not scanned”.
Enhance existing configuration to skip further processing (and block) if archive engine is not available
- Support an option to skip any further processing (e.g. sanitization) and fail any processing archive file if archive engine was not available for any reasons (disabled, crashed, timed out…) before trying to extract that archive file. The final processing verdict will be “Failed”.
Proper blocked reason in JSON scan result when sanitization failed or timed out
- With Deep CDR's workflow setting "Block files if sanitization fails or times out" is enabled, when the sanitization result meets those conditions, then now the "blocked_reason" key value would represent actual sanitization failure result (instead of other scan results such as No Threat Detected which could cause misleading to client integration).
Override whitelist certain file types in archive file scanning
- When triggered, the whitelist setting will be ignored if pre-configured file types belong to archive file processing.
Proactive DLP default threshold value changed to high
- The previous threshold value was low under Proactive DLP workflow rule settings. For those customers already tweaked this setting value, this change will not revert their pre-configured setting.
- This change is to address false positive concern on DLP processing against files inside document by default.
Email server configuration now accepts no authentication option
- That helps integrate with external email server without authentication required.
MetaDefender Core could be activated via offline mode without any network card enabled
- In an isolated environment, we understand that your machine's network card could be disabled completely for security reasons. Hence now MetaDefender Core could be activated under that circumstance with offline yml file upload as usual.
Log enhancement on webhook callback failure on HTTPS mode
- When the callback failed to send to client side over HTTPS, the status code will be 444, and now the product log is enhanced to reveal more why the secure connection was not made successfully.
Improved search performance on MetaDefender Core processing history page
- Earlier users could observe the slowness in search by username while MetaDefender Core was running data retention.
Fixed:
Advanced engine settings mistakenly reset back to default values
- The issue could encounter in following circumstances:
- Upgrading MetaDefender Core
- Engine reloaded (disable and enable back)
- Engine updated (and then reloaded)
- The issue could encounter in following circumstances:
Ignition file with configuration file location import issue
- Using ignition file for configuration file location import was not working.
Sanitized archive compression failed with same password retainment
- Archive sanitization could be failed with same password retain feature enabled under Deep CDR workflow settings.
Potential memory leak issue
- Leaking on Node component (ometascan-node) while updating engines online, or via Central Management.
Input validation
- Enhanced and fixes bad input validation for data queries.
Service crashed in certain circumstance
- Core component could be crashed when stopping the service immediately upon the product service start command.
Update from folder feature issue
- Update from folder trigger failed to skip in what pre-configured for update pause time frame.
Failed to download big sanitized files
- Failure could be exposed (failed to download) against big sanitized files in GB in size.
Product database migration issue
- Failed for statistics data migration.
Proactive DLP regex validation failed for Unicode characters
- Fixed to support encoded Unicode characters in Proactive DLP's regex (Proactive DLP version 2.5.0 or above is required)
Sanitized archive file contained blocked watermarked file
- The issue could only happen when Proactive DLP was used with watermarking triggered on child files inside archive sanitization.
Version v4.19.0
Release Date: 27 Aug 2020
New features:
New Database Management System (PostgreSQL) to replace SQLite
- PosgreSQL is now MetaDefender Core's new database management system to replace its predecessor SQLite. That expects to step by step help the product easily scale out, network based database support, gain better performance, migrate high load bottleneck and native high availability. MetaDefender Core supports users to create a local PostgreSQL server running in the box, or allow leveraging a pre-installed remote PostgreSQL server.
- Data migration auto runs in background upon product uprade.
- For large database migration, MetaDefender Core comes with a web-based data migration to walk users through quick steps to move all your SQLite data to PostgreSQL at ease.
FIPS-140 security compliant
- We are now FIPS-140 compliant with a new support for RSA186-4 on OpenSSL
Native proxy management with authentication support
- MetaDefender Core now will allow users to control proxy settings for product (instead of using system configuration), and also support authentication for proxy which is not possible on older Core versions.
- That comes with a UI configuration support on MetaDefender Core management console.
Harden Nginx web server settings for security
- Secure MetaDefender Core web server even more based on nginx vendor guideline to protect your MetaDefender server from being vulnerable (i.e. cross site scripting, MIME sniffing, TLSver 1.1 or below forbidden)
- Still we keep all supported functionalities working as expected.
Nginx web server statistics (on web server report)
- Support to enable better statistics for the HTTP Server for web server healthcheck and debugging.
Origin client source address retrieval when running via load balancer
- MetaDefender Core now will be able to retrieve your origin client source address even when the client communicates over a load balancer.
Enhanced search for processing history page
- Searching by attributes represented via corrensponding column on the list.
Enhanced user experience on statistics page
- Instant statistics processing data calculated and visualized on UI
Logic improvement to handle better against sanitization timed out
- New logic implemented on MetaDefender Core to offload concurrent tasks on Deep CDR engine, and to reduce sanitization timed out as a result.
Enhanced log messages
- Log events enhanced with more sufficient and clear information, easier for traceability while troubleshooting.
Pre-check mode for file submission
- Refuse file upload immediately when MetaDefender Core does not have enough disk space to handle, expecting to hit error 400. This is to avoid wasting upload time on big files.
Sanitized file information appended into JSON scan result
- Including sanitized file size and its SHA-256 hash value.
Blacklist overidden on nested files within archive
- New configurable setting to allow overridening blacklist enablement on nested files within an archive.
New engine sweeper tool bundled into the product
- New engine sweeper troubleshooting tool tailored for PostgreSQL, and now it is bundled into MetaDefender Core product (not a separate download tool).
Fixed:
Setting inputs validation
- Threat detection threshold, wizard password and SSO profile settings affected.
Batch signature sometimes contained redundant characters
- That could make signature becomes invalid.
Processing time of nested files in archive not calculated correctly
- Processing time of nested files could be very big due to incorrect caculation (but actually they are processed much faster).
Statistics page to support multiple users simultaneously
- The statistics UI now could handle multiple users query at the same time.
Proactive DLP timeout setting mistakenly reset upon engine restart
- The setting was reset back to default value (3 minutes).
Version v4.18.0
Release Date: 26 May 2020
New features:
Single Sign On (SSO) Authentication
- Additional to already-supported various authentication models (Local, Active Directory, LDAP), now MetaDefender Core also supports authentication using SSO with widen integration coverage for most of Identity Providers (IDP) via SAML 2.0 and OpenID Connect 1.0 standard support.
Brand New MetaDefender Core API Guide (Sample Codes Available)
- Brand new design and standardized API documentation (following OpenAPI V3 specification), auto-generated sample codes on various programming languages supported helps your API integration even easier.
Database Defragmentation and Optimization
- When your scan database grows big, it might cause performance degradation (e.g. timeout on client requests). Now MetaDefender Core administrators can be notified on the UI (also warning logs), and you are supported to perform database defragmentation and optimization including multiple stages to vacuum and defrag your database without loss of actual scan data.
- As a result, your database file size could be reduced which helps boost processing performance tremendously over usage time.
Comprehensive Statistics On Processing Data
- Featured in an interactive UI help you gain deeper insights on your processing filtered by every workflow rule, breaking down into each file type. Last but not least, you are also supported to select time range to calculate statistics data.
Data Reporting (Business Intelligence)
- When enabled, MetaDefender Core will auto-sync your historic processing data to OPSWAT dedicated servers. That helps us gain more visibility on your processing load and how our product is being used, and thus we could improve our product to accommodate your use-case better. You are supported to customize which piece of information should be shared with OPSWAT, and when to share.
- By default, this feature is disabled to respect your privacy rights and save performance impact.
Webhook Continuous Improvement
- New setting mode to control callback timeout and retry (configurable via REST API)
- Stability improvement to avoid being stuck on callback and crashing on Node service
High Load Processing Improvement
- When running under high load, file type usually returns "Not Available" caused by various reasons. Product logic enhanced to elaborate causes, and improved stability on the product.
- We keep working on this matter to ensure our customers have the most stable product running under high load as much as possible.
Configurable Behavior On Archive Extraction Failure
- Configurable settings on workflow rule to let you tweak and decide MetaDefender Core final scan verdict when a processing archive file failed to extract for some reasons.
- The default selection on each designated extraction failure reasons (invalid file structure, extracted partially,...) will be different on each workflow rule depending on use-case characteristics. Please make sure you are aware of the new settings and adjust them accordingly tailored to your security demand.
Archive Extraction Failure Exposure
- Archive extraction failure reasons exposed to both REST API response and UI.
Encryption on Archive & Document Sanitized Files
- Help retain password protection on supported archive and document files (.zip, .7z, .pdf, MS Offices) upon sanitized successfully.
Sanitization Forensic Details Enhancement
- When sanitized successfully, an even more comprehensive forensic available on both UI and REST API level letting you know all processed object details (e.g. what exact hyperlink was sanitized).
Processing File Information Enhancement (File Type Category)
- File type category is now available on REST API response along with other already-supported file information.
MetaDefender Core Log Rotation Experience Improvement
- This feature is now enabled by default applicable to both upgrade and fresh install scenario.
Workflow Rule For MetaDefender For Secure Storage
- With the best practice to serve MetaDefender For Secure Storage use-case, we have a new dedicated workflow rule with designated configurations.
Archive Processing Result Retrieval API Enhancement
- Applicable to pagination fashion polling GET /stat/log/scan?first={start_item}&size={number_of_items_next} , now the action ran information available in JSON response
FIPS Object Module 2.0 Bundled
- Operating product in FIPS mode enabled on Operating System
Fixed:
File Scanning Process Stuck
- When the custom engines stopped its process for reasons (updating while scanning / crashed), the running scans on Core could not be finished and stayed at 95% forever.
MetaDefender Core Service Crashed (Webhook Mode)
- When using webhook mode, and callback can't be sent back to client, the MetaDefender Core service could be crashed.
Memory Leak While Updating Engines Automatically
- The memory could be leaked on ometascan-node process while updating engines in online mode.
Scan Details Missing From Recursive Scan Results
- While fetching scan results on all nested files in big archive file GET /archive/{data_id} the "scan_details" field from the top-level root archive was empty.
Input Field Overflow On Management Console UI
- Preventing invalid values putting in UI configuration controls (Deep CDR, Archive)
Archive Timeout File Skipped For Scanning
- None of AV engines actually scanned archive file when archive timeout occurred
Version v4.17.3
Release Date: 06 Apr 2020
New features:
Configurable setting to run database optimization
- Database optimization has been introduced since Core 4.17.0 to help run database queries faster. The downside is while running (for a few seconds), Core queries hold up causing possibly timeout on client side.
- This new setting allows users setting specific time to run database optimization task (to avoid peak hours), or just disable to prevent this task from running (to avoid performance degradation while running). Learn more how to configure: Startup Core Configuration)
Scan database rollback mechanism
- In some circumstances (e.g. Core crashes, out of disk/memory etc.), the atomicity of product database could be compromised causing inconsistent processing scan history returned. Rollback mechanism helps retain that atomicity of database.
Logging improvement with configurable settings
- Log rotation for Core, Node, Nginx web server logs (Configurable settings supported).
- More comprehensive support package (to include engine and database info, Nginx web server info).
- More informative log message on sanitization related tasks.
- Sensitive info redacted (on debug level logging mode).
- Performance impacted warning on both MetaDefender Core GUI and logs when the scan database (ometascan.db.sqlite) starts growing up big (>10 GB).
Webhook mode continuous refinement
- Retry to send scan results to client upon network interim disconnected.
- Resend file scan results to client after Core service restarted.
New download mechanism for Processing History on MetaDefender Core management console
- Support IE / Edge web browsers to download processing history report.
MetaDefender Drive use-case better support when engine packages corrupted
- Support to re-new engine packages to re-download engine packages again when corrupted (due to upon unexpected reboot).
Central Management v7 support to revert download source when unhooked
- Respect skipped scan settings (Whitelist / Blacklist) to keep backward compatibility, and also save Core resource for processing files as well.
RoleIDs JSON field validated when creating / modifying user
- Effective to POST__/admin/user and POST /user endpoint REST APIs. Role ID value must be an array of strings according to current user guide.
Account name value validated on Core wizard setup
- Effective to "Admin User Setup" screen during wizard setup, "Account name" validated against special characters (e.g. @ & )
Better support for sanitized file download when under load
- When under load and certain circumstance with system write failed, the sanitized download on the same file might return 404 HTTP response (not found) to client. Enhanced our Core caching mechanism to ensure next time sanitization on the same file will not rely on the previous failed time.
Minor UI changes
- Hide "Edit Workflow" button in "Workflow Templates Management" screen
- Remove space between date and time in "Definition date" field on "Modules" screen
Fixed:
Data tunnel between Node and engines could be lost under high load
- When occurred, expecting to see " process communication timed out" message repeatedly in Node log, and none of engines could be able to scan files.
Node crashed when swapping engines during update
- Node could be crashed under certain circumstances, applied to swapping engine instances during update.
File processing was stuck at 95%
- Encountered when custom engine stopped its process, all of running tasks on that engine becomes stuck, or when ClamAV engine can't return consistent scan result during its engine update.
Overflow issue with unexpected inputs
- Overflow value issue could occur within product causing unexpected behavior or result.
Unexpected result with non-ASCII password protected document scanning
- When occurred, document file could not be sanitized properly.
Proactive DLP displayed wrong result within archive scanning
- When occurred, Proactive DLP engine could return misleading result (Not scanned) while archive file processing result is "Sensitive Data Found".
Session expired on IE / Edge web browser
- When session cleared out, authenticated users could be logged out repeatedly due to session expired error on MetaDefender Core management console.
Version v4.17.2
Release Date: 03 Mar 2020
New features:
Quarantine cleanup task no longer blocks Core service starting procedure
Empty file submission is no longer be blocked at REST API level
- Retain same behavior on Core 4.16.3 or older, to support back some corner use-cases from MD Kiosk and ICAP
Custom engine initialization enhancement
- Increased timeout to 10 minutes to support engine deployment on under-specs hardware (formerly 1 minute)
Processing history report enhancement
- Added "username" column to the processing history export from MD Core
Validation mechanism on file scan and batch init REST API changed
- When using via REST API, no longer validate session cookie, only API key header is validated when exists (same behavior on Core 4.16.3 or older)
Configurable Proactive DLP timeout is supported
- Support to adjust timeout for Proactive DLP handling (formerly fixed on 3 minutes)
Respecting whitelist and blacklist configurations
- Respect skipped scan settings (Whitelist / Blacklist) to keep backward compatibility, and also save Core resource for processing files as well.
Response for POST /login no longer returns cookie back to client
- When using via REST API, by default the response for POST /login no longer returns cookie back to client (same behavior on Core 4.16.3 or older, to avoid breaking F5 LTM scenario with cookie header is auto added)
Removing failed dummy scan results on Core processing history UI due to upload failure
- When file upload is failed for some reasons (e.g. network corrupted) between clients and MD Core, dummy record results are still available and displayed on Core processing history UI, but actually MD Core never processed those files, and client never got results from MD Core on those files. Those dummy records will be removed since this version to avoid misleading.
Fixed:
Deadlock could possibly happen when engine update task is timed out
- When encountered, all files happens in "Failed" result with "Not available" result for file type analysis after timeout hit (~ 70 seconds), and only Node service relaunched can bring the scanning be operational back.
Node service could be crashed when archive engine crashed
- When archive engine crashed for some reasons, Node service could be crashed as well (but not happened all the time)
Core and Node service could be crashed when under high load
- Core and Node service could be crashed when under high load
Archive file extraction when timed out, or failed to extract, the original archive itself could not be scanned by AV engines
- When archive extraction hits timeout or failed to extract, the original archive itself could not be scanned by AV engines
Core could return 404 not found HTTP(S) response to client for sanitized file download API request
- When processing the same file many times on Core, it could return 404 not found HTTP(S) response to client (e.g. MD Email) due to file sync issue between Core and Node
Memory leak issue on Core process
- The process ometascan could be leaked on memory with auto update mode enabled and Proactive DLP engine is enabled
Anonymous user can't submit file scan to MD Core web scan UI
- When not logged in, anonymous user can't submit file scan to MD Core web scan UI (error: Invalid session ID given)
Core service can't restart due to configurations corrupted while running Proactive DLP engine
- Core service can't restart due to configurations corrupted while running Proactive DLP engine
Document files inside sanitized password protected archive file could not be sanitized
- When document file is also treated as an archive file, there was a bug on archive compression level calculation to prevent document files not being sanitized inside original archive file
Timeout on hash calculation task resulted as Blocked regardless of "override scan results classified as allowed" setting
- When this task timed out, regardless what users set on "override scan results classified as allowed" setting, the final verdict were "Blocked"
Webhook continuous fixes and updates
- Duplicated callbacks returned to client
- Support retry mode for sending callback to client (when client is temporarily unresponsive etc.)
- Enhance validation callbackurl header against IP version 6 and domain format
Minor UI fixes
- Added margin to bottom edge of scan result UI
- Name of rights under user management did not match to Inventory
Version v4.17.1
Release Date: 06 Jan 2020
New features:
Archive extraction details
- Available on both scan result UI and
Advanced engine configurations enhancement
- UI interactive and schema based for advanced engine settings
Proactive DLP engine integration enhancement
MetaDefender Cloud integration enhancement
- MetaDefender Cloud API version 4 upgraded
Engine integration enhancement to avoid product crash
User validation update for file and batch processing
- File scan and batch init endpoint API is now validated on API key input when that key information is available.
More ready for adding password back to sanitized archive and document files
Minor UI update
Fixed:
Core could become unavailable to clients when under high load
- Data communication channel between Core and Node service could be broken when under high load
Processing giant files (> 50 GB) could be stuck at hash calculation
- Hash calculation with pre-set timeout value (10 minutes) could be exceeded when processing giant files (> 50 GB) and then stuck at 5% forever
Scan could be failed with "not available" result for File type analysis when under certain circumstances
Memory leak issue on Core process
Temporary files not cleaned up when archive extraction timed out
Webhook continuous fixes and updates
- Redundant warning log messages populated even when not using webhook mode
- Core could be crashed itself when trying to close a not-found bath with callback
- Callback sent to client with wrong status when Core is restarted
Visibility level smaller than full details might break batch result display
Minor UI fixes
- The field "File Password" on the file processing UI not cleared up after empty file selected
Version v4.17.0.1
Release Date: 27 Nov 2019
Fixed:
- Deadlock issue on batch handling
- Under certain circumstances, a deadlock issue could encounter locking database from being queried (timeout on REST requests)
Version v4.17.0
Release Date: 14 Nov 2019
New features:
Callback URL (Webhooks) for file and batch scanning (to avoid polling result from client)
- Support for individual file and batch scanning to eliminate polling mechanism i.e. MetaDefender Core will notify client based on designated / configurable callback URL whenever an individual scan finished or a batch can be closed.
Security enhancements
- Harden MetaDefender Core management console against security vulnerabilities found on pen-test's result
Log correlation from parent archive file to child files
Comprehensive failure reason on archive extraction (available on JSON response)
Sanitization output name on password protected document fully respects value set on UI
- No longer appended with fixed value "decrypted_document" in output name
UTF8-encoding password for file scan request via REST
- File scan REST API now supports "archivepwd" header with encoding password
Total number of files inside archive (all recursive levels), available on scan result UI
Archive scanning enhancements
- Better integration logic with archive engine
- Support empty folder inside archive engine
- Not try to extract archive file if extracted size exceeded is anticipated
Database query optimization
Processing input refinement
- Empty file scan request no longer is accepted at API level
MetaDefender Core's nginx log location no longer requires double backslashes
UI enhancements
- Password field supported for password-protected archives or documents on the UI (web scan)
Fixed:
Relaunching Proactive DLP engine process after timeout could crash Node service
Nginx custom configuration file and certificates is unexpectedly erased when upgrading MetaDefender Core
Overall failure on scan could encounter when system goes wrong while analyzing file type
- When something wrong occurred while analyzing file type, the scan process could be immediately stopped and ended up as overall failed.
Override scan result setting did not apply properly to empty batch
Changes on workflow template could interfere Core service
- Excluding engines in workflow template could make Core service failed to start
Return incorrect REST response code when closing batch with invalid API key
Fail to create local user directory under certain settings
- Creating "Local" user directory type could be failed when "Enhance password policy" setting is unchecked
Nginx access log location customized on registry is not retained when upgrading MetaDefender Core
- When upgrading MetaDefender Core, nginx log location (nginx_logfile) could be unexpectedly reverted back to default value, not retained to what users configured.
Minor UI fixes
- Some display and hyperlink minor issues related to table and navigation
- Hitting cancel batch button on UI causing error
Version v4.16.3
Release Date: 16 Oct 2019
New features:
- Support new header (metadata) for file submission API
- Enhance MetaDefender Core service starting procedure
- Enhance engine update procedure
- Remove restriction on Core version retrieval REST API
Fixed:
- MetaDefender Core service on Linux could not be started when running on FIPS mode
- MetaDefender Core service could be unexpectedly restarted when engines repeatedly crashed
- Uninstalling MetaDefender Core did not terminate its processes properly (nginx)
- Password protected document could not be decrypted properly for data sanitization
- Uninstalling MetaDefender Core did not clean up its leftover data folder
Version v4.16.2
Release Date: 10 Sep 2019
New features:
- Restrict APIs based on user roles (configurable)
- Support displaying and filtering username on processing history UI
- Enhance logging with Yara matched rules appended
- Upgraded nginx web server component to latest version 1.16.0
- Add new scan result - Unsupported file type
- Refined JSON output when users want to quarantine items which are already in quarantined folder
- Updated UI (minors)
Fixed:
- In-progress files could be deleted mistakenly, causing failures when scanning
- Engines repeatedly disable and re-enable
- File processing could be stuck until archive timeout value reached
- Non UTF-8 characters were not displayed correctly when exporting process history via UI
- Dependency installation issue on Ubuntu 18 & Debian 9
Version v4.16.1
Release Date: 12 Aug 2019
New features:
- Supported to pin & unpin engines and their database on the UI to prevent auto update being applied
- Gently handled timeout on Archive and Deep CDR engines
- New logging mode for archive processing troubleshooting
- Enhanced logic for non-archive file processing
- Limited number of characters on some applicable text fields on the UI
- Enhanced security with unquote service exploit
Fixed:
- Node crash issue when under high load
- Issue with resource manager with in-use temp files
- Memory leaking issue on archive engine process
- Memory leaking issue on Node process
- Batch handling issue causing failure on batch
- Stuck scan issue at 5% when parallelcount_7z_extract is set with definitive number
- Detection issue on Proactive DLP engine with regrex rule applied
- UI issue where Yara result is not displayed
- UI visibility issue on Internet Explorer (IE) web browser
- Some other minor UI issues
- Wrong timezone set on exported CSV scan report
Version v4.16.0
Release Date: 08 July 2019
New features:
- Proactive DLP engine (ver 2.0) integration
- Password policy enforcement
- Support archive partial sanitization for Vault and Email integration
- New REST API for local update server source
- Better handle archive sanitization timeout
- Support configurable settings for archive extraction and compression parallel count
- Enhance syslog message format
- Retouch UI
- Better logging with timeout on engines
- Enhance logic to apply engine definition files
Fixed:
- Wrong outcome when archive engine process unexpectedly stopped
- Wrong UI result on sanitization timeout
- Memory leak issue on engine package uploading
Version v4.15.2
Release Date: 19 June 2019
Fixed:
Stability issue
- Potential deadlock issue on batch scan handling prevents querying batch information
Usability issue
- Enhanced error log messages when the engine process is terminated due to engine timeout
- Exposed log messages on warning level when there is an archive extraction failure
Version v4.15.1
Release Date: 06 June 2019
New features:
- Partial sanitization use-case for archive file types
- Clarified error messages for terminated engine processes
- New REST API for cleaning up idle batch scans
- UI improvement
- License EULA update
Fixed:
Stability issue
- Potential memory handling issue that could cause the node service to crash
- Empty and read-only files are no longer extracted
Usability issue
- Not able to remove abandoned temp. files of archive files when they are empty and read-only
Security issue
- AD user credential is not masked properly on the audit log while sending over to AD server for authentication
Scanning batch REST API issues
Engine custom configuration
UI issues
- Dashboard refresh button sometimes did not work as expected
- List of processing records didn't show when changing "number records per page" while not staying at first page
- Not user-friendly error messages when adding duplicate hashes to a backlist
- Typos on the UI
Version v4.15.0
Release Date: 06 May 2019
New features:
Data Sanitization details displayed on Core management console
User password recovery and reset enforcement
API rate limiting
Support Windows Server 2019 (The support is still on beta)
Suspicious results returned by engines are now configurable to be handled as a different circumstance (infected, ignore)
Improve usability
- Return zero for definition dates on non-AV engines' database
- "Select all" option added to the Data Sanitization page
Improve handling on node
- Improve cleanup mechanism on nodes to avoid deleting files in use
- Improve validation process when starting the node service, support to try creating temp. folder with a configurable timeout
Fixed:
Fixed stabilization issues that possibly caused Node services to crash
Scan batch API closing issues
- No longer returns total time of -1 in response
- Should not randomly fail due to " 400 - One or more scan is still in progress" even when all linked scans already finished
Upgrading Core when installed in a non-default installation path prevented users from choosing another folder path by mistake
UI issues
- The "Process File" button no longer disappears in case of sanitization failed
- Max recursive level under archive handling tab must equal 1 or greater
- non-Unicode file name displayed on web scan UI encoded properly
Sanitizing empty archive file no longer returns failed
Version v4.14.3
Release Date: 01 Apr 2019
New features:
- Support built-in integration with OPSWAT Central
- New setting for archive sanitization timeout
- Add process time field into CSV exported history report
- Effectively wipe out necessary data from support package
- Revamp Inventory UI page with "Utilities" group
- More relevant REST error message for scan request where file is non-existed / inaccessible
- Syslog message for scan-finish event more comprehensive
- Consolidated scan info for archive scan result fetching
- Add libcurl4 as alternative dependency to libcurl3 for better support on Ubuntu 18.04
- Outputs and indicators for Threat Intelligence feature on Quarantine UI page more relevant and informative
Fixed:
- Node becomes unstable under high load processing
- Closing batch with ongoing scans could result in failed verdict on batch
- Inconsistent behavior with password protected document
- Temporary files are not cleaned up when cancelling an ongoing scan
- Inconsistent returned error message between batch and file scanning via REST
- DLP verdict returns incorrect value for some cases
- Logs in support package did not handle non-Unicode characters
Version v4.14.2
Release Date: 28 Feb 2019
New features:
- New result page, n ew look and more informative badge
Fixed:
- Engine configurations could not be saved
- Make error message more relative for case where file exceeded the size limit
- Pinning engines and their databases independently
Version v4.14.1
Release Date: 31 Jan 2019
Fixed:
- Missing "pinned" option from "/stat/packages" JSON response
- Inconsistent "progress_percentage" and "result" values
- Hash validation (blacklist/whitelist)
- Upload performance
Version v4.14.0
Release date: 19 Dec 2018
New features:
Send quarantined files to MetaDefender Cloud for scanning
Automation support:
- Support ignition file to automate the welcome wizard
- Configuration API functions have been documented
Enhanced password policy can be enabled for local users
Files with Failed to sanitize result can be set to be blocked
Version v4.13.2
Release date: 21 Nov 2018
New features:
- Tiles on Dashboard are linked to the corresponding pages
- More options to filter Processing History (Post Actions and CDR)
Fixed issues:
- In case of an engine hangs, the communication channel is blocked between the Node and the Core, so more engines can time out
- Clean-up mechanism removes files still in use
- Various engine handling issues
Version v4.13.1
Release date: 31 Oct 2018
Fixed issues:
- Yara and DLP tasks are not stopped on cancelling a processing
- Batch processings cannot be cancelled via web management console
- "Can't process shared resource file" error message did not contain the file name
Version v4.13.0
Release date: 16 Oct 2018
Important:
- Yara engine integration
New features:
- Processing history entries can be colorized
- Files can be marked as suspicious if less than a given number of engine mark it as infected
- Processings can be cancelled via web management console
- Default rules are added for MetaDefender Email Security
- Bulk operations in quarantine
Fixed issues:
- Extracted files are left behind
- On Debian based systems, on upgrades, engines are deleted and disabled engines are re-enabled
Version v4.12.2
Release date: 3 Oct 2018
Fixed issues:
- In case of archive processing, sometimes clean-up mechanism removes some extracted files before processing is finished
Version v4.12.1
Release date: 26 Sept 2018
New features:
- Files can be whitelisted/blacklisted by their checksums
- More specific log entries for CDR
Fixed issues:
- Details of scan result for nested archives (for the file itself not for the content) is not propagated to the top level
- The value, set in "MAX TOTAL SIZE OF EXTRACTED FILES" is handled incorrectly
- Older configs cannot be imported into v4.12.0
Version v4.12.0
Release date: 15 Sept 2018
Important:
- Data Loss Prevention functionality
New features:
- Possibility to set the number of engines that required to start file processings (per workflow)
- Possibility to exclude engines from processings (per workflow)
- Improved user interface performance
- Possibility to blacklist/whitelist files by file types besides file type groups
- Re-designed workflow tab list appearance
- Possibility to set timeout for sessions regardless of user activity
Fixed issues:
- On Node details page, every issue appears multiple times
- Despite not detecting any vulnerabilities, the vulnerability tab appears
- On hash lookup page, empty hash can be searched
- Sanitized output file name validation can cause user interface stalled
Version v4.11.3
Release date: 30 Aug 2018
Fixed issues:
- Whitelist page under Inventory menu does not exist (only UI issue)
Version v4.11.2
Release date: 29 Aug 2018
New features:
- The access_log Nginx directive now can be overridden
- The parallel count parameter now can be set per engine
- Minor changes on user interface for better user experience
Fixed issues:
- A critical CSV injection vulnerability in the CSV export functionality (issue reported by Wojciech Reguła, SecuRing)
- Archives can be sanitized even in case of partial processing (e.g. exceeded archive size, exceeded archive file number)
- In some cases, blocked results can be overwritten by an allowed result with higher priority
- Inconsistent operation of MetaDefender Cloud integration
- Typos on the user interface
- Abandoned files left behind after processings
Version v4.11.1
Release date: 8 Aug 2018
Fixed issues:
- Unexpected Core and Node service restart in some corner cases
- Using remote syslog server slows down the product in case of missing PTR record in DNS
- Empty files are skipped in archives
- Incomplete archive extraction issue happened on heavily overloaded systems
Version v4.11.0
Release date: 11 July 2018
New Features:
- Exceptions (by mime-type) from whitelist/blacklist
- New engine page called Technologies
- Support for user-friendly engine configuration (depends on the engine version)
- Welcome wizard
Fixed issues:
- Slow clean-up mechanism
- Abandoned files after uninstall in Windows
- Temporary files are left behind after processings
- Wrong sanitized output file name in some cases
- Default workflows can be overridden on config import
- Core crashes
Version v4.10.2
Release Date: 27 June 2018
Fixed issues:
- Uninstall not properly cleans the system
- The "whitelisted" and "blacklisted" results are overriden by "infected" result
- Node crashes
- Inconsistent results in case of archive processing: In case of processing an archive more times, the result may be different by cases (infected/exceeded archive file number/exceeded archive size)
Version v4.10.1
Release Date: 23 May, 2018
New features:
- Data Sanitization engine time-out and retry count is now configurable
- REST API: process info contains the name of the last scanned file when scanning archive file types
- REST API: Configurations that may change the final scan result since the time of processing will be included in the process info response (i.e., outdated definitions)
- Hash based result lookups can be filtered by rule name
Fixed issues:
- Sanitized DB integrity issue
- On the dashboard, category names of doughnut charts were truncated
- In case of archive processing, the "Not scanned" result to a file is not propagated to a higher level (overall verdict)
Version v4.10.0
Release Date: 2 May, 2018
Important:
- Added support for the LDAP directory type
- Syslog messages can now be sent to multiple log aggregators
- MetaDefender installers no longer use eicar test files
New features:
- AD and LDAP directories can now be configured with multiple servers
- Sanitization failures are marked with a badge in the scan session summary
- Admin’s will be notified if a third party solution is blocking MetaDefender from working as expected
- Users can now be granted API keys manually
- Paginated archive results
- HTTPS can now be enabled from web management console
Fixed issues:
- Improved license status info
- In some cases, sanitized files had faulty names
- Suspicious scan results were not always at the top of the list in archive file types
- Inappropriate handling of user rights in the Whitelist page
- AD group members did not have user profiles
- Misleading license information
Version 4.9.1
Release Date: 28 February, 2018
New features:
- New-looking user interface
- Workflows based on the default one (not edited by workflow editor) will be kept and upgraded on version upgrade in the future
- It is allowed blacklisted/whitelisted files to be processed
Fixed issues:
- Security zone: IP address validation
- Cancelled batches are displayed as in-progress
- Removing certificates from the inventory caused policies to disappear
- Memory leak in Node
- Access via Active Directory is not logged
- Sluggish pages under Policy menu
Version 4.9.0
Release Date: 13 December, 2017
New features:
- IPv6 support
- Global whitelist by hash
- Whitelist by file type group
- Display more security related information on dashboard
- Changed default port for external nodes to 8007
- New default security rule for Metadefender Secure File Transfer (SFT)
- Performance tuning of processing history
- Improved resource handling on Node
- On Linux, multiple nginx worker processes for better scaling
Fixed issues:
- Upgrades overwrite existing configuration (IP, port, etc.)
- Resource folder clean up after data sanitization
- Update timing settings affect manual updates
- Poorly handled invalid update files
- Poorly handled UTF-8 characters in output file name for sanitized files
- /hash API can give "in progress" result
Version 4.8.2
Fixed issues:
- Fixed a memory leak caused by failed update download
- Fixed a possible crash issue at Scan history manual cleanup in case of high load
- Fixed a memory leak in case of recurrring failed database deployment on Node
Version 4.8.1
Release Date: 5 October, 2017
New features:
- Improved engine/database update distribution to nodes
- Improved archive extraction limit handling
- Improved engine monitoring
- More precise time duration measurement for requests
- API for canceling scans (file/batch scans)
- Option to disable archive extraction of office documents
- For batch scans, certificate validity interval can be set
- Improved scan result badge
Fixed issues:
- Fixed issue of scans stuck in "in progress" state
- Fixed possible product crash during archive scanning
- Fixed update bug where incorrect packages left behind
- Fixed failed quarantine handling
- Fixed handling unavailable engine during scans
- Scan result JSON now contains file name in UTF-8 format
- Limited number of parallel Post Action and External Scanner scripts
- Archive handling parameters now have upper bound
- Improved archive handling
- Archive related failure handling
Version 4.8.0
New features:
- Quarantine for blocked files
- Scanning files in batch (REST API)
- Certificate and key handling for scan batch signing
- Configurable sanitized file name
- Post action commands gets the result JSON with final verdict included
- Increased scan history export interval
- Improved archive bomb handling
- Added eng_id to scan_results.scan_details (REST API)
- Showing in-progress files in "extracted files" list of archives
- Added "scan_all_result_a" into "extracted_files" (REST API)
Fixed issues:
- Fixed case insensitive username comparison in Active Directory integration
- Process workflow revamped (post actions run every time)
- Fixed non-updated policy user interface after added new user roles
- Fixed handling of database upgrade errors in linux package installers
- Fixed error handling when scan target was sent in the body and via filepath (/file REST API)
- Fixed disconnected ghost node issue displayed on user interface
Version 4.7.2
Issues fixed:
- Fixed bug that could cause policies to not contain any elements and forbid user to create new items
- Fixed bug where Core could download older version of engines where newer one was already downloaded
Version 4.7.1
Issues fixed:
- Fixed upgrade of scan configuration
- Fixed ghost nodes appeared on Inventory→ Nodes page
Version 4.7.0
New features:
- Active Directory integration
- Custom post actions
- Redesigned user interface
- External (customer developed) scanner integrations
- Policies export/import
- Archive sanitization
- Individual log message level override
- Aggregated archive scan result in Scan History
- Self-lockout protection, admins can not delete themselves
- gzip and base64 encoding now supported on /file REST API
- Able to navigate through archive hierarchy
- Timezone changed to local in log messages
Issues fixed:
- Fixed scanning of .lnk files on Windows
- Fixed blacklisting of Unicode filenames
- Automatically downloads packages again if the previous download failed
- Fixed order of extracted files on scan details view
- Fixed rare temporary file leak during archive scan
Version 4.6.3
Issues fixed:
- Improved scan result fetching performance for big archives
Version 4.6.2
Issues fixed:
- Improved archive extraction performance
- Fixed a race condition in /file/<data id> REST API that could provide access error in some cases
- Fixed advanced engine config reload for Data sanitization engine
- Fixed login issue which happened when many login request was initiated concurrently
- Fixed calculation of extracted file count
Version 4.6.1
New features:
- List of path for local filescan can be blacklist / whitelist with specific error message on REST
Issues fixed:
- Invalid external Node listening IP/port config stops product startup
- Connection to remote syslog is reactivated on network error
- If user has no right to use a rule, following rules in order will still be checked
- sending HEAD request where GET should have been sent will not lead to product crash
- Ensure resource file deletion on Microsoft Windows when a scan engine locks file further than expected
- Scan history CSV export uses comma as separator
- Fixed potential Node service crash when stopping during scanning
- More specific error message when uploaded file size limit exceeded
- Fixed a rare race condition in update downloader component
- Fixed login issue when Core v3 like URL is used by the admin
Version 4.6.0
New features:
- Multiple user roles introduced with different access rights
- Scan Agent has been renamed to Scan Node
- Role (user group) based rule availability configuration
- Role based scan result visibility with different level of details exposed
- Ability to export part of scan history into STIX/Cybox format
- Ability to export part of scan history into CSV format
- Filter on rule and source added into Scan history
- Configurable lockout feature against brute force login attack
- Official support introduced for Ubuntu 16.04
- Detection threshold (suppress threat detection if less then X engines detected a threat)
- Custom engine configuration via user interface
- Free text search functionality in user guide
- Suspend engine testing/deployment to Node when 3rd party security software blocks access to malware files
- Successful login / unsuccessful login / lockout events are logged
- Option to send engine issue count info during update
- [REST API] /file/{data_id} response for scan results now contain process info block for extracted files
- Initiating local scan is faster as no wait for hashing is required
Issues fixed:
- [REST API] /file/{data_id} blocked reason change to mirror V3 API
- Fixed handling of archive extraction depth
- More flexible and stable internal database upgrade when upgrading product
- Custom engine update timeout increased to one hour to deal with slow engine updates
- Archive engine fixes (non-ASCII filenames in archive)
- Engine handling fixes, improved handling of engine deinitialization
- More precise engine cleanup when removing engines
- Fixed bug where random connections were rejected every 2 min
- Fixed bug regarding updates handling (conflicting names)
- Filesize is now correctly displayed on scan result user interface
- Support package generator now includes auditlog db
Version 4.5.1
Issues fixed:
- Fixed possible crash of Agent when there is database which is handled by engine
- Fixed possible crash of Core that could occur when updating a package
Version 4.5.0
New features:
- Data Sanitization of files to protect against unknown threats
- Filetype mismatch detection
- Improved user interface responsiveness for small screens
- Real filetype based blacklist option in rules/workflows
- Improved licensing for offline deployments
- Added product specific proxy settings in the Linux version
- Advanced configuration for allowed/blocked file scan result types
Issues fixed:
- Fixed local scan option user interface for new rules
- Fixed Scan History auto cleanup collision with manual cleanup
- Potential issue fixed for update file upload
- /apiversion interface is added to easily determine REST API compatibility level
Version 4.4.1
New features:
- Added several features/improvement for better Metadefender Kiosk integration
- Full audit log about any configuration changes via Web user interface or REST API
- Able to disable applying update in user configurable time periods
- Core can act as an update source for OESIS product line
- Detect if the analyzed binary is a part of any vulnerability detection
- Improved scan engine status monitoring and auto recovery
- Custom directory can be set for storing temporary files
- Able to set up apikey for every user for easier REST API integration
- Improved hardware detection in license component
Issues fixed:
- Fixed message content format in Windows Event log
- Fixed system wide proxy usage on Windows
- Improved browser cache handling in case of product upgrades
- Fixed a path specification issue in local file scanning feature on Windows
- Fixed engine counting on Agent details page (do not count utility type engines)
- Fixed lost agent connection handling
- Fixed handling of unsupported Transfer-Encoding on REST API
- Patched internal nginx web server to fix CVE-2016-4450
- Fixed archive timeout handling and user interface
- Fixed scan results in case of archive related findings
- Improved logging of proxy usage
- Improved handling of slow file uploads
- Detailed logging in case of SSL connection issues
- Improved auto-recovery of engines running under Emulated Windows
Version 4.3.0
New features:
- Introduced official support for Microsoft Windows 7 or newer and Microsoft Windows Server 2008 R2 or newer
- Added offline update picker feature to make it easy to apply offline updates without user interaction or scripting
- Able to scan local files stored on server without transferring the content via REST API
- Added hardware related info into generated support package
- Created a framework in Linux version to be able to run Windows scan engines on Linux server
- Option added to log to a remote syslog server
- Inventory / Scan Agents page extended with more detailed agent information
- Parameter workflow renamed to rule in some REST APIs
- Improved system issue notification on Web Management Console
- Added detection of 3rd party anti-malware products that break operation of Metadefender Core
- Improved scan performance of various engine integrations
Issues fixed:
- Improved documentation of multiple REST APIs
- Fixed failed scans during some engine or database update
- Removed unmeaningful database age display of non-anti-malware engines
Version 4.2.0
New features:
- product name has changed to Metadefender Core
- able to use scan results from metadefender.com
- workflow options can be configured from Web Management Console
- workflow options can be overridden from rule editor window
- support for system wide HTTPS proxy
- it is possible to configure maximum file size of scanned files
- filtering security rule by user agent is now possible
- eliminate limitations on the size of scanned files
- improved scan related log messages
- deployment can now be deactivated on the License page
- automatic deployment reactivation of online installations if license becomes invalid
- Metascan v3 URLs (/management and /metascan_rest) are now redirected to the proper v4 URLs
- check disk space before/during scan requests
Issues fixed:
- fixed encrypted communication error with activation server on Ubuntu 12.04
- fixed temporary folder cleanup
- fixed support data collector scripts
- do not download database without the corresponding engine package
- number of engines and maximum file size is now reflect the current status
Version 4.1.0
New features:
- https support for REST API and for Web Management Console
- update history to track every database/engine change
- new option to globally disable or enable specific scan engine
- reworked result page for archive files
- user guide is available within the product
- no scan downtime while updating engine/database (if engine supports)
Issues fixed:
- more descriptive communication error messages instead of error codes in logs
- proper handling of update download issues
- fixed handling of scan engine crashes
- fixed manual update package upload
- fixed unwanted warning message after successful activation
Version 4.0.1
New features:
- new script to help log collection for support
- inform the user if browser is not HTML5 compatible
- show a spinner if loading a page takes too much time
- support lower screen resolution for web interface
- support for non-ascii character filenames in archives
Issues fixed:
- fix stability issue in update downloader
- optimize database queries
- do not check for updates at product startup if auto update is off
- fixed a page auto refresh issue with Internet Explorer
Version 4.0.0
New features:
- Able to to monitor Metascan v4 for Linux instances
- Able to to monitor Metascan v3 for Windows instances
- Collect Files scanned and Infections found stats from managed instances
- Deploy scan engine database updates to Metascan v3 for Windows instances
- Deploy scan engine and scan engine database updates to Metascan v4 for Linux instances