Skip to content

Enterprise: SIEM log export — stream audit_log to a customer SIEM (#847 seam) #997

Description

@dolho

Summary

Add SIEM log export as an enterprise module on the #847 entitlement seam: ship the OSS audit_log (SEC-001) to a customer's SIEM (Splunk/Datadog/Elastic/etc.) so their security team can monitor, alert on, and retain Trinity activity. Entitlement id: siem.

Open-core split (private-only)

Per the directive that enterprise features live entirely in the private repo, the whole feature is in Abilityai/trinity-enterprise (backend/siem/), gated by requires_entitlement("siem"). It reads the OSS audit_log read-only (event source) and owns its own private config table. No OSS/public surface — it's admin config + a background pusher, so no Vue/UI is needed; config is admin-only via the gated API.

Phase 1 (this issue)

  • Generic HTTP/JSON webhook transport (configurable URL + bearer token).
  • Private enterprise_siem_config table (single-row: destination, encrypted token, format, batch size, export cursor = last audit_log.id shipped).
  • Gated endpoints: GET/PUT /api/enterprise/siem/config, POST /api/enterprise/siem/test, POST /api/enterprise/siem/flush (admin + entitlement).
  • Background pusher (daemon thread; Redis-lock-serialised across the 2 prod workers) that drains new audit rows to the destination.
  • At-least-once delivery: cursor advances only after a successful POST; downstream dedupes on event_id.
  • Destination token encrypted at rest (AES-256-GCM via the OSS CredentialEncryptionService).
  • Unit tests: format, config/validation, export-cycle cursor + at-least-once, batch size, token encryption, entitlement gate.

Later phases

  • Splunk HEC + syslog/CEF transports.
  • Event filtering / field redaction beyond the existing credential masking.
  • Backpressure/buffering tuning.

Architectural notes

  • Reads OSS audit_log; does not modify it (stays OSS).
  • Router → service → db inside the private module; entitlement gate on every endpoint; register_module("siem") in register_enterprise.
  • TRINITY_OSS_ONLY=1 and OSS-only builds (submodule absent) must hide it entirely (no import errors, endpoints 404).

Out of scope

  • Moving the audit_log itself into the private repo (stays OSS).
  • Non-audit log streams (app/container logs via Vector).

Related to #847.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions