Custom detection


The custom detection feature is an advanced feature that allows users to define their own rules for identifying specific patterns within files. This capability enables users to support their own file types for detection quickly, without needing to wait for official support from the FileType engine.

Enable custom detection

This feature is disabled by default. To enable this feature:

  • At Inventory > Modules > Utilities > FileType, Tick Enable custom detection

  • At Inventory > Modules > Utilities > FileType, section Enable custom detection, specify paths to XML rule files and/or rule directories that contain XML rule files.


Info

When there are updates on the rule files or the rule directories, the engine needs to be restarted in order for the rules to take effective.

When new items of the configuration are added, the rules are loaded automatically along with the changes insides the existing files or directories if available.

Custom rules

Rule definitions

Info of file types detected with custom rules and the rules are defined in XML format with fields described as in the table below.

Field

Mandatory

Meaning

File type info



description

Required

File type description to be used to output.

id

Required

File type ID.

mime

Optional

Mime type to be used to output. Default value: application/octet-stream.

group

Optional

Group ID to be used to output. Default value: O. See the list of group IDs below.

extension

Optional

Extension(s) for the file format. This value will be used to check mismatching. Default value: empty.

encrypted

Optional

Encryption property (True/False). Default value: True.

score

Optional

Confidence score for the custom file type. Value range [0, 1]. Default value: 0.25.

Patterns for detection



FrontBlock

Required

Define patterns at specific offsets

FrontBlock.Pattern

Required

Define offset (stored in Pos) and hex pattern to be compared (stored in Bytes).

GlobalStrings

Optional

Define patterns at random offsets.

GlobalStrings.String

Optional

Define string pattern to be matched.

Group ID and name

Group

Group

Group

A: Archive Files

G: Image Files

T: Text Files

AP: Application Files

I: Disk Image Files

Z: Email Files

D: Office Documents

M: Media Files

O: Other

D_ENC: Encrypted Documents

OPENSSL_ENC: OpenSSL Encrypted Files


E: Executable Files

P: Adobe Files


Note

The current use case is to turn a unknown (DATA) or not surely (non-DATA with score < 1.0) (detected by native rules of the engine) file type into a user-custom one with higher score.

File type with highest internal match point will be picked although its defined score is higher. See example in the table below.

File type

User-defined score

Internal match point

Result

Custom type A

0.96

5600


Custom type B

0.85

8700

Custom type B

Example rules

Below are some XML example rules.

<CustomRule ver="1.0"> <Info> <description>FTA1 - an OPSWAT-defined file format</description> <id>FTA1</id> <mime>application/fta1-opswat</mime> <group>A</group> <extension>fta,fta1,opswat</extension> <encrypted>True</encrypted> </Info> <FrontBlock> <Pattern> <Bytes>AABBCCDDEE</Bytes> <Pos>0</Pos> </Pattern> </FrontBlock> </CustomRule>
<CustomRule ver="1.0"> <Info> <description>FTA2 - an OPSWAT-defined file format</description> <id>FTA2</id> <mime>application/fta2-opswat</mime> <group>O</group> <extension>fta,fta2,opswat</extension> <encrypted>True</encrypted> </Info> <FrontBlock> <Pattern> <Bytes>AABBCCDDFF</Bytes> <Pos>0</Pos> </Pattern> <Pattern> <Bytes>FAFAFAFAFAFAFAFAFAFAFAFAFAFAFA</Bytes> <Pos>16</Pos> </Pattern> </FrontBlock> </CustomRule>
<CustomRule ver="1.0"> <Info> <description>FTA3 - an OPSWAT-defined file format</description> <id>FTA3</id> <mime>application/fta3-opswat</mime> <group>O</group> <extension>fta,fta3,opswat</extension> <encrypted>True</encrypted> </Info> <FrontBlock> <Pattern> <Bytes>FF112233</Bytes> <Pos>0</Pos> </Pattern> </FrontBlock> <GlobalStrings> <String>OPSWAT</String> </GlobalStrings> </CustomRule>
<CustomRule ver="1.0"> <Info> <description>FTA4 - an OPSWAT-defined file format</description> <id>FTA4</id> <mime>application/fta4-opswat</mime> <group>O</group> <extension>fta,fta4,opswat</extension> <encrypted>True</encrypted> </Info> <FrontBlock> <Pattern> <Bytes>FF112244</Bytes> <Pos>1</Pos> </Pattern> </FrontBlock> <GlobalStrings> <String>OPSWAT</String> <String>FTA</String> </GlobalStrings> </CustomRule>