InSights TIDB and REPDB

The OPSWAT InSights enrichment is MetaDefender NDR's broad-coverage threat-intelligence layer. Where the C2 and Threat Intelligence family is tuned narrowly — exact-match lookups against the small, high-curation Eyelet C2 feed — InSights casts a much wider net: every IP address and every domain name observed on the wire is checked against two large indicator feeds (the Threat Intelligence Database and the Reputation Database), and domain matches walk up the hierarchy so a query for login.evil.example.com matches an indicator for evil.example.com. Two dedicated alert types distinguish the two intelligence tiers so analysts can treat confirmed-threat hits and reputation hits on different priority lanes.

First-use acronym expansions in this chapter: InSights (the OPSWAT-curated actionable-intelligence feed family), TIDB (Threat Intelligence Database), REPDB (Reputation Database), IOC (Indicator of Compromise), FQDN (Fully Qualified Domain Name), CNAME (DNS Canonical Name record), SNI (Transport Layer Security Server Name Indication), DNS (Domain Name System), TLS (Transport Layer Security), HTTP (Hypertext Transfer Protocol), URL (Uniform Resource Locator), CIDR (Classless Inter-Domain Routing), MVP (Minimum Viable Product), FRD (Functional Requirements Document).

What It Is

The InSights family is an enrichment service running in the MetaDefender NDR Manager pipeline. It maintains four in-memory indicator sets — a TIDB (high-confidence threats), a REPDB (reputation-driven indicators), and two whitelist sets (known-good domains and known-good Classless Inter-Domain Routing blocks) — all loaded from the OPSWAT InSights feed and refreshed on an hourly cadence. Every DNS, Transport Layer Security, Hypertext Transfer Protocol, and flow record that the sensors emit is scanned for source and destination Internet Protocol addresses, DNS query names, DNS answer names, TLS Server Name Indication values, and HTTP host and URL fields. Every extracted indicator is checked first against the inclusion filters (TIDB and REPDB) and then against the whitelists; any indicator that is in an inclusion filter and not on a whitelist produces an entity-level enrichment that the aggregator stitches back onto the originating event, and the alert engine promotes the enriched event to a typed InSights alert at the appropriate severity tier.

What It Detects

The family covers the full breadth of IP-level and domain-level observables that threat-intelligence vendors publish. Typical hits include:

• Malware staging and payload infrastructure. Hosts delivering loaders, droppers, or second-stage implants — observed in past campaigns, still tracked by OPSWAT Analyst Network researchers, and appearing as an IP in an outbound flow or as a name in an HTTP request URL.
• Phishing landing pages and credential-harvester domains. Domains set up to mimic a legitimate login surface, observed as a DNS query from an internal host or as a Hypertext Transfer Protocol hostname in an outbound request.
• Commodity command-and-control and remote-administration trojan panels. Broad commodity C2 infrastructure that the Reputation Database tracks with more historical coverage than the tight C2 feed.
• Trojanized-software update servers, exploit-kit landing hosts, sinkholes, and parked-malware domains. Published reputation data that often correlates with older or widely distributed threats.
• DNS-response reachback. A domain that itself is not on the feed but resolves to an IP that is, or whose Canonical Name chain passes through a known-bad name.
• Subdomains of flagged domains. Every subdomain of a flagged domain inherits the parent's match — a query for auth.payroll.trusted-lookalike.ru matches an indicator for trusted-lookalike.ru through the parent-hierarchy expansion described in the next section.

InSights is an indicator-match family — it does not try to infer whether a never-seen-before domain is malicious. Domain-generation-algorithm detection and other name-behavior analytics are the job of Behavioral Detections.

How It Works

The pipeline moves a candidate observation through five stages.

1. Indicator load. At startup, the InSights enrichment service loads four filter files from disk: the TIDB inclusion filter, the REPDB inclusion filter, the domain whitelist, and the CIDR whitelist. Filters are compact probabilistic data structures (XOR and Binary Fuse variants) that support constant-time membership lookups independent of indicator-list size. The filters are refreshed hourly in the background by an automatic updater that pulls the latest feed from OPSWAT; no redeployment is required to adopt a new indicator.
2. Event consumption. The service consumes DNS, Transport Layer Security, Hypertext Transfer Protocol, and flow events from the raw sensor stream. Other event types are skipped at this stage — the three protocol events plus the flow stream together cover every IP and every name the sensor observes.
3. Indicator extraction. The extractor walks the event and pulls every candidate IOC out of the relevant fields: source and destination IP from every event type; DNS query names, DNS answer names, and Canonical Name chain entries from DNS events; Server Name Indication from Transport Layer Security events; hostname and Uniform Resource Locator from Hypertext Transfer Protocol events. Every extracted IOC carries the JavaScript Object Notation path it came from so analysts can trace a match back to its source field.
4. Parent-domain expansion and filter check. For every extracted domain, the extractor adds every parent name (auth.shop.evil-vendor.net produces candidates auth.shop.evil-vendor.net, shop.evil-vendor.net, evil-vendor.net, and so on). Each candidate is looked up in TIDB and REPDB; any match is then checked against the domain whitelist, and any matching IP is checked against the CIDR whitelist. Only indicators that hit an inclusion filter and miss every whitelist are retained.
5. Entity enrichment and alert promotion. For every retained match, the service attaches a per-entity enrichment block to the matched field — dest_ip_enrichments.insights on a flow, dns.queries[].rrname_enrichments.insights on a DNS query, tls.sni_enrichments.insights on a Transport Layer Security handshake, and so on. Each per-entity block carries the flags malicious: true, has_tidb_match, has_repdb_match, and the list of feed sources that flagged the indicator (for example, ["tidb-malware", "repdb-generic"]). The aggregator stitches the enrichments back onto the event; the alert engine then evaluates two rules: the TIDB rule promotes any event with at least one has_tidb_match: true entity to a High-severity InSights alert carrying a TIDB match payload, and the REPDB rule promotes any event with at least one has_repdb_match: true entity to a Medium-severity InSights alert carrying a REPDB match payload — but only when no entity on the event carries a TIDB match.

The mutual-exclusion rule on REPDB is deliberate: if any entity on the event is in the TIDB tier, the analyst only wants to see the High-severity TIDB alert — the concurrent REPDB hit would be redundant noise. This produces a clean severity signal: an event raises at most one of the two InSights alert types.

Trigger Conditions

An InSights alert fires when the enrichment service produces at least one match and the alert engine's TIDB or REPDB rule sees the corresponding match flag on any entity in the merged event. The fields below are what analysts read when triaging the alert.

The alert also carries every standard Suricata-shaped field (timestamp, flow_id, src_ip, src_port, dest_ip, dest_port, proto, alert.signature, alert.signature_id, alert.severity, alert.category, and MITRE Adversarial Tactics, Techniques, and Common Knowledge metadata where applicable) so that analysts can pivot into protocol logs and packet capture exactly as they would on any other alert type.

Severity Classification

InSights alerts are raised at two native severity tiers that reflect the confidence difference between the two feeds. TIDB is curated for confirmed-threat use; REPDB is curated for broader reputation coverage and tolerates more historical or low-confidence indicators.

When a TIDB and a REPDB hit land on the same event — for example, the destination IP is on REPDB and the queried domain is on TIDB — only the High-severity TIDB alert is raised; the REPDB rule is suppressed by its mutual-exclusion guard. The REPDB payload still exists on the entity enrichment, so analysts can read both hits in the Hunt sidebar even though only one alert was emitted.

The IOC auto-escalation rule described in Detection Overview interacts with this family in one direction only: a behavioral or machine-learning detection that coincides with a TIDB or REPDB hit is auto-escalated to Critical / 0.99 — for example, a Beaconing detection with a TIDB hit on the destination domain becomes a Critical Beaconing alert. The InSights-native alert retains its own High or Medium severity. Analysts who see a Critical behavioral or ML alert with an Insights Enrichment sidebar block should treat the InSights match as the reason the parent detection was auto-escalated.

Confidence Scoring

InSights alerts are raised at two confidence tiers that track the severity assignment. The underlying match is binary (membership in the inclusion filter), so confidence reflects feed authority rather than a per-entity score.

The confidence scale defined in Detection overview reserves the 0.80–0.94 band for strong signals with corroborating evidence and the 0.60–0.79 band for behavioral findings that warrant context. TIDB's 0.90 reflects high-authority indicator curation; REPDB's 0.70 reflects the broader, reputation-weighted nature of the feed and gives analysts a built-in ordering cue when a queue of Medium alerts is waiting for triage.

Where It Surfaces

InSights alerts appear in four places.

• Dashboard — Recent Severities donut. TIDB alerts contribute High slices; REPDB alerts contribute Medium slices. Neither tier contributes Critical slices unless the match was absorbed into a behavioral or ML detection that was then auto-escalated. Top Signature Hits is Suricata-only and does not include InSights alerts.
• Dashboard — Recent Alerts. Newest InSights alerts flow through the cross-family feed, distinguishable by the InSights Alert type label. Clicking a row opens the same sidebar used on the Hunt page.
• Hunt page — InSights Alert sub-tab. Under the All Alerts bucket, this per-type sub-tab lists every InSights alert with columns for timestamp, alert.signature, source and destination endpoints, event_type, the severity label, and match-level columns drawn from the insights.tidb / insights.repdb payload (indicator, sources, paths). Both tiers are listed here; analysts filter by severity to split TIDB and REPDB queues.
• Hunt detail sidebar — Insights Enrichment section. This section renders on any row whose record carries an *_enrichments.insights block — not just the InSights alert rows. A DNS session row whose queried name matched the REPDB renders the DNS section alongside the Insights Enrichment section, even though the row itself is a session event rather than an InSights alert; a Transport Layer Security session whose SNI matched the TIDB does the same. The section lists each matched entity with the feed sources that flagged it and the path on the event where the indicator was extracted. When the matched entity is also geolocated, the sidebar's ASN and GeoIP enrichment renders alongside so analysts see the destination's country, ASN, and organization without leaving the record.

On the Hunt page's Sessions tab, the insights column injects a small enrichment ribbon on every DNS, Hypertext Transfer Protocol, Transport Layer Security, flow, and protocol-log row that carries a match — analysts do not have to open a record to see that it is InSights-adjacent.

Alert Payload Example

Abbreviated JavaScript Object Notation for a TIDB-tier InSights alert fired on a DNS query. The underlying event is a standard Suricata DNS query; the alert adds the insights payload and the alert block, and the alert engine sets alert_type: "insights" and severity 2 (High).

xxxxxxxxxx

{

  "timestamp": "2025-11-15T08:42:17.334821+0000",

  "flow_id": 1847291847291847,

  "event_type": "alert",

  "alert_type": "insights",

  "src_ip": "192.168.10.157",

  "src_port": 51023,

  "dest_ip": "10.0.0.53",

  "dest_port": 53,

  "proto": "UDP",

  "app_proto": "dns",

  "community_id": "1:8K3vN7pL5mQ2rT9uV1wX3yZ5",

  "alert": {

    "action": "allowed",

    "gid": 1,

    "signature_id": 1000002,

    "rev": 1,

    "signature": "MetaDefender NDR InSights TIDB Match — trustpad.work",

    "category": "A Network Trojan was Detected",

    "severity": 2

  },

  "dns": {

    "type": "query",

    "queries": [

      {"rrname": "pay.trustpad.work", "rrtype": "A"}

    ]

  },

  "insights": {

    "tidb": {

      "indicator": "trustpad.work",

      "sources": ["tidb-malware", "tidb-phishing"],

      "paths": ["dns.queries[].rrname"]

    }

  },

  "rule_name": "InSightsTIDBDetection",

  "rule_salience": 7,

  "triggered_at": "2025-11-15T08:42:17.498112Z"

}

A REPDB-tier alert looks the same with three differences: the insights.tidb block is replaced by an insights.repdb block of identical shape; the sources entries carry a repdb- prefix; and the alert.severity drops to 3 (Medium) with rule_name: "InSightsREPDBDetection" and rule_salience: 5. The payload above is a parent-domain match — the query was for pay.trustpad.work but the indicator was trustpad.work — so analysts seeing an indicator value shorter than the observed name should read that as a hierarchy hit rather than a transcription error.

Tuning Considerations

The InSights family is deliberately low-knob at the alert level; curation happens upstream at OPSWAT. The Policy-managed knobs that do exist are scoped to the service and to the feed.

• Enable or disable the InSights enrichment service. Operators toggle the enrichment on or off through Policy. Disabling the service leaves the pipeline running — no crash, no dropped events — and suppresses all downstream InSights alerts. See (Link Removed) for the per-policy enrichment toggle.
• Enable or disable individual feeds per Policy. The shipped InSights pack contains three feed classes (C2, TI, and Open-Source Intelligence) that contribute indicators to TIDB and REPDB. Each feed is independently toggled in the policy that a sensor or sensor group runs under, so operators can, for example, enable TIDB globally but restrict REPDB to a high-sensitivity subset of groups. The Updates management surface shows the current enable / disable state per policy.
• Enable or disable the whitelist. The built-in domain and Classless Inter-Domain Routing whitelists cull well-known benign infrastructure before the match reaches the alert engine. Disabling them surfaces more matches (including hits on infrastructure that has been reclaimed by legitimate tenants) and is only recommended for short tuning windows.
• Feed update cadence. The default update interval is one hour. Constrained networks may lengthen it; high-sensitivity environments may shorten it. The feed-update workflow, the manual-trigger procedure, and the distribution-status dashboard all live in Updates management.

False-positive behavior on TIDB is rare; false-positive behavior on REPDB is the family's main source of noise and is the primary reason REPDB lands at Medium rather than High. A reputation indicator that triggered on infrastructure several months ago and has since been reclaimed by a benign tenant will still produce an alert until the feed removes it; when an analyst confirms the reclaim, the correct response is to flag the indicator for upstream review rather than add a local suppression — the MetaDefender NDR pattern for this family is to keep the alert engine faithful to the feed and let curation happen at the source.

Related Runbook

High-severity TIDB alerts and Medium-severity REPDB alerts both route through (Link Removed) when they coincide with other Critical evidence, and through the follow-on (Link Removed) runbook when the indicator describes command-and-control or malware staging. Analysts triaging an isolated REPDB hit with no other corroborating signal should pivot via (Link Removed) to verify intent before escalating — the runbook walks through identifying the originating host, inspecting the full session capture, and deciding whether the match reflects an active attempt to reach the indicator or benign residual traffic.