Custom detection
The custom detection feature is an advanced feature that allows users to define their own rules for identifying specific patterns within files. This capability enables users to support their own file types for detection quickly, without needing to wait for official support from the FileType engine.
Enable custom detection
This feature is disabled by default. To enable this feature:
- At Inventory > Modules > Utilities > FileType, Tick Enable custom detection
- At Inventory > Modules > Utilities > FileType, section Enable custom detection, specify paths to XML rule files and/or rule directories that contain XML rule files.
When there are updates on the rule files or the rule directories, the engine needs to be restarted in order for the rules to take effective.
When new items of the configuration are added, the rules are loaded automatically along with the changes insides the existing files or directories if available.
Custom rules
Rule definitions
Info of file types detected with custom rules and the rules are defined in XML format with fields described as in the table below.
| Field | Mandatory | Meaning |
|---|---|---|
| File type info | ||
| description | Required | File type description to be used to output. |
| id | Required | File type ID. |
| mime | Optional | Mime type to be used to output. Default value: application/octet-stream. |
| group | Optional | Group ID to be used to output. Default value: O. See the list of group IDs below. |
| extension | Optional | Extension(s) for the file format. This value will be used to check mismatching. Default value: empty. |
| score | Optional | Confidence score for the custom file type. Value range [0, 1]. Default value: 0.25. |
| Patterns for detection | ||
| FrontBlock | Required | Define patterns at specific offsets |
| FrontBlock.Pattern | Required | Define offset (stored in Pos) and hex pattern to be compared (stored in Bytes). |
| GlobalStrings | Optional | Define patterns at random offsets. |
| GlobalStrings.String | Optional | Define string pattern to be matched. |
Group ID and name
| Group | Group | Group |
|---|---|---|
| A: Archive Files | G: Image Files | T: Text Files |
| AP: Application Files | I: Disk Image Files | Z: Email Files |
| D: Office Documents | M: Media Files | O: Other |
| D_ENC: Encrypted Documents | OPENSSL_ENC: OpenSSL Encrypted Files | |
| E: Executable Files | P: Adobe Files |
The current use case is to turn a unknown (DATA) or not surely (non-DATA with score < 1.0) (detected by native rules of the engine) file type into a user-custom one with higher score.
There can be cases in which a file matches both a custom and a built-in rule. In order to prioritize the detection result from the custom rule, the custom rule should be defined with a high confidence score, e.g., 1.1.
The "detection score" can be found in the JSON scan result: filetype_info.file_info.likely_type_ids.score
Example rules
Below are some XML example rules.
<CustomRule ver="1.0"> <Info> <description>FTA1 - an OPSWAT-defined file format</description> <id>FTA1</id> <mime>application/fta1-opswat</mime> <group>A</group> <extension>fta,fta1,opswat</extension> <score>0.9</score> </Info> <FrontBlock> <Pattern> <Bytes>AABBCCDDEE</Bytes> <Pos>0</Pos> </Pattern> </FrontBlock></CustomRule>