Audit And Discovery

W5A is required for first-parity signoff.

It is not implemented yet. It is a contracted implementation workstream with fixed storage, write path, retention, export, and infrastructure rules.

Goal

GoRunChat must provide deterministic, fail-closed, machine-readable audit and discovery evidence.

The application-owned audit log and discovery journal are the request-path correctness boundary.

BigQuery and Cloud Storage are downstream query and archive paths. They do not replace application-owned writes.

Target Modules

W5A owns four implementation targets:

• internal/audit/event_ledger
• internal/audit/write_enforcement
• internal/http/audit_api
• infra/gcp/audit_pipeline

Storage Model

Primary durable storage uses:

• audit_events
• discovery_journal
• audit_outbox
• legal_holds
• audit_exports

MongoDB is the recommended first-parity durable store for canonical audit and discovery records. Redis may coordinate projector retries. Redis must never be the sole evidence store.

Audit Records

Application audit events must include:

• event_id
• occurred_at
• actor
• tenant_workspace
• request_id
• trace_id
• source_surface
• action
• target_type
• target_id
• outcome
• before_after_summary
• raw_evidence_pointer
• retention_until

Use audit events for auth and session mutations, admin changes, config changes, BYOK mutations, MCP lifecycle changes, ACL mutations, destructive content actions, hold actions, and export actions.

Discovery Records

Discovery journal records must include:

• event_id
• occurred_at
• conversation_id
• message_id
• parent_message_id
• message_role
• message_version
• content_snapshot
• attachment_metadata
• provider_model
• tool_or_mcp_action
• stream_resume_id
• legal_hold_state
• retention_until

Use discovery records for prompts, assistant outputs, attachments, tool and MCP activity, edits, deletes, retries, continue flows, imports, exports, aborts, finalization, shared publication, and user-visible lifecycle changes.

User-visible delete, edit, retry, import, and export operations must not erase the discovery record.

Write Path

Every required mutating path follows this order:

1. Validate authn, authz, and request shape.
2. Derive the audit event and discovery record when needed.
3. Write the domain mutation, audit or discovery record, and outbox item in one durable boundary.
4. Return success only after the durable boundary commits.
5. Project to BigQuery and immutable archive from the outbox.

If step 3 fails, return an explicit failure. Do not relabel the operation as clean success.

If projection fails later, the canonical evidence still exists locally. Projector lag and failure must be machine-readable and alertable.

Retention And Hold

Retention is 3 years for:

• application audit log.
• discovery journal.
• BigQuery query copy.
• immutable Cloud Storage archive.

Legal hold overrides normal expiry.

Only admin and super_admin may place legal hold, release legal hold, or initiate audit and discovery exports.

Operator API

The audit API must support deterministic search and export for audit and discovery records.

Required filters include actor, tenant, request id, trace id, source surface, action, target type, target id, conversation id, message id, time range, and legal-hold state.

Recommended routes are:

• GET /api/admin/audit/events
• GET /api/admin/audit/discovery
• POST /api/admin/audit/exports
• GET /api/admin/audit/exports/:exportId
• POST /api/admin/audit/holds
• POST /api/admin/audit/holds/:holdId/release

Hold and export actions must emit audit events.

Infrastructure

The GCP pipeline owns Cloud Audit Logs posture, Data Access enablement, BigQuery query copy, Cloud Storage archive, sink IAM, projector IAM, and replay-safe projection.

The archive bucket exists with 3-year retention and Bucket Lock. BigQuery query-copy tables, Log Router sinks, Data Access audit configs, and archive writer IAM are imported into OpenTofu. Infrastructure signoff is incomplete until the app-owned projector writes and replays real evidence.