Archive Audit Log (WORM Substrate)¶

Introduced: v1.2 (PLAN-1609)

The Archive Audit Log is a durable, append-only, tamper-resistant audit trail that records every significant event in the archive lifecycle. It uses a sealed-segment JSONL journal with SHA-256 hash chaining, so any post-hoc modification to a sealed segment is detectable.

What gets audited¶

Five event types are recorded automatically. No configuration is required.

`eventType`	Triggered by	Key fields
`ItemImported`	Archive importer completing a content item	`itemKey`, `sourceFarmId`, `principalIds`, `subject` (item title)
`ArchiveModeToggled`	`POST /_api/web/setarchivemode` or `…/setlistarchivemode`	`detail.scope` (web/list), `detail.previous`, `detail.next`, `detail.override`
`IdentitySnapshotCaptured`	`POST /_api/archive/identity-snapshots`	`itemKey` (`user:<sid>`), `principalIds` (login name), `detail.upn`, `detail.email`, `detail.group_count`
`AclFrozen`	`POST /_api/archive/acls`	`itemKey` (object key), `principalIds`, `detail.role_def_bindings`, `detail.frozen_at`
`WormConfigChanged`	System events: chain verification, retention reaper, tamper detection	`detail.kind` (`chain_verified`, `chain_break`, `segment_reaped`, `retention_changed`, `recovered_partial_segment`)

Three more types are reserved for future plans:

`eventType`	Reserved for
`HoldApplied`	PLAN-A-7 Legal Hold
`HoldReleased`	PLAN-A-7 Legal Hold
`ReadAccess`	PLAN-A-7 Legal Hold

Storage layout¶

Events are stored in sealed-segment JSONL files under <data-root>/audit/:

<data-root>/audit/
  <farmId>/
    index.json          ← segment manifest (sealed segments + active coordinate)
    retention.json      ← per-farm retention policy
    active.json         ← current writer position
    <yyyy-MM-dd>/
      segment-0000.jsonl   ← sealed (ReadOnly, 1000 events max)
      segment-0001.jsonl   ← sealed
      segment-0002.jsonl   ← active (being written)

Each line in a .jsonl file is one ArchiveImportAuditEvent JSON object.
Segments roll at 1000 events or a UTC day boundary, whichever comes first.
On roll, the segment is sealed: the file is set ReadOnly (Windows) or chmod 0444 (Linux), and its SHA-256 hash is recorded in index.json together with a prevSegmentHash pointer forming a hash chain.
The data-root is configured via Cesivi:DataRootPath in appsettings.json (default: R:/MockData).

Viewing audit events¶

ControlCenter (browser)¶

Open Cesivi Control Center → Archive → Audit Log.
Use the filter form to narrow by event type, date range, farm, item key, or user login.
Paginate through results (100 per page by default).
Click {...} in the last column to drill into the JSON detail payload.
Toggle Live tail to watch new events arrive in real time via SignalR.
Click Export CSV to download the filtered result set.

REST API¶

GET /_api/archive/audit-events?eventType=ItemImported&farmId=my-farm&fromDate=2026-01-01&limit=100
Accept: application/json
Authorization: Basic <admin-credentials>

Query parameters:

Parameter	Type	Description
`eventType`	string	Filter to one event type (e.g. `ItemImported`)
`farmId`	string	Filter to a specific source farm ID
`fromDate`	ISO 8601 date	Inclusive lower bound on `importedAt`
`toDate`	ISO 8601 date	Inclusive upper bound on `importedAt`
`itemKey`	string	Exact or prefix match on `itemKey`
`userLoginName`	string	Match in `principalIds` array
`listId`	GUID string	Match on `detail.listId` field
`offset`	integer	Pagination offset (default 0)
`limit`	integer	Page size 1–1000 (default 100)

Response envelope (OData verbose):

{
  "d": {
    "results": [ { "eventId": "...", "eventType": "ItemImported", ... } ],
    "totalEmitted": 42847,
    "__count": 100
  }
}

CSV export¶

GET /_api/archive/audit-events/export.csv?farmId=my-farm&eventType=ItemImported
Authorization: Basic <admin-credentials>

Response is text/csv with header row:

eventId,eventType,itemKey,sourceFarmId,principalIds,subject,importedBy,importedAt,detailJson

Hash-chain integrity verification¶

The segment journal maintains a SHA-256 hash chain. Each sealed segment records:

hash — SHA-256 of the segment file bytes
prevSegmentHash — hash of the preceding segment (first segment: "GENESIS")

This chain means that tampering with any single sealed segment is detectable even if the attacker also patches index.json.

Manual verification¶

POST /_api/archive/audit-events/verify-chain?farmId=my-farm
Authorization: Basic <admin-credentials>

Response:

{
  "ok": true,
  "segmentsChecked": 47,
  "brokenSegmentIds": [],
  "reapedSegmentIds": ["segment-0000", "segment-0001"]
}

If ok is false, brokenSegmentIds lists the affected segments, and a WormConfigChanged{kind: "chain_break"} event has been written to the current active segment.

Automatic background verification¶

WormChainVerificationService runs every 6 hours (configurable via Cesivi:Audit:ChainVerificationIntervalHours). On each pass it emits:

WormConfigChanged{kind: "chain_verified", segmentsChecked: N, ok: true} on success
WormConfigChanged{kind: "chain_break", ...} on failure

Retention policy¶

Default¶

7 years (RetentionYears = 7, GraceDays = 0). Applies to all farms unless overridden.

Per-farm override¶

PUT /_api/archive/audit-events/retention?farmId=my-farm
Content-Type: application/json
Authorization: Basic <admin-credentials>

{ "retentionYears": 10, "graceDays": 30 }

Each PUT emits a WormConfigChanged{kind: "retention_changed"} event.

GET /_api/archive/audit-events/retention?farmId=my-farm

Returns { "farmId": "my-farm", "retentionYears": 10, "graceDays": 30 }.

Retention reaper¶

WormSegmentReaper runs daily (configurable via Cesivi:Audit:Reaper:IntervalHours). For each farm it:

Loads the retention policy.
Scans sealed segments whose newest event timestamp + RetentionYears is in the past.
Removes the ReadOnly attribute (or runs chattr -i), deletes the file, and marks the segment reaped: true in index.json.
Emits WormConfigChanged{kind: "segment_reaped", farmId, segmentId, newest_event_ts} per deleted segment.

Reaped segments show in verify-chain output under reapedSegmentIds and do not count as chain breaks — the manifest marks the gap as intentional.

Production hardening¶

Windows (IIS / Windows Service)¶

Sealed segments have FileAttributes.ReadOnly | FileAttributes.System. An administrator with filesystem access can still clear the attribute and modify the file — the hash chain will catch this, but won't stop it.

For strict WORM compliance: - Mount the audit/ subdirectory on a WORM filesystem partition (e.g. EMC Centera, NetApp SnapLock). - Or restrict folder permissions so the service account cannot clear ReadOnly (requires a separate service account with WRITE_DATA on active segments only).

Linux (systemd / Docker)¶

chattr +i makes a file immutable at the kernel level, even against root. This requires CAP_LINUX_IMMUTABLE. If the server process lacks the capability, Cesivi falls back to chmod 0444 and logs a warning:

[WormSealer] chattr +i failed (EPERM) — using chmod 0444 fallback. Consider running with CAP_LINUX_IMMUTABLE.

To grant the capability to a systemd unit:

[Service]
AmbientCapabilities=CAP_LINUX_IMMUTABLE

For Docker:

cap_add:
  - LINUX_IMMUTABLE

For truly strict WORM: mount the audit/ directory from a WORM-compliant NAS (NetApp SnapLock, Scality RING with Object Lock, or any POSIX WORM filesystem).

Tamper-detected runbook¶

verify-chain returns ok: false with a non-empty brokenSegmentIds.
The WormConfigChanged{kind: "chain_break"} event is recorded in the current active segment — this is evidence of the detection moment.
Copy the broken segment file to immutable storage for forensic analysis.
Consult legal/compliance: the archive may need to be re-imported from the source farm if the original data is available. If not, the import timestamp + surrounding sealed segments establish the most recent known-good state.
Use POST /_api/archive/audit-events/verify-chain periodically (or rely on the background service) to detect further tampering.

v1.5 Cloud WORM Backends (PLAN-1652)¶

v1.2 shipped FileSystemWormAuditLogStore as the reference implementation. v1.5 adds two production-ready cloud adapters:

S3 Object Lock adapter (`S3ObjectLockWormAuditLogStore`)¶

Uses S3 Object Lock in COMPLIANCE mode, the strongest AWS durability guarantee. Objects cannot be deleted or overwritten even by the root account until the lock expiry.

Key implementation notes:

AppendAsync writes events to a local staging buffer.
SealActiveSegmentAsync uploads the sealed segment JSONL to S3 with ObjectLockMode=COMPLIANCE and ObjectLockRetainUntilDate = now + RetentionWindow. The returned VersionId is stored in the segment manifest so that DeleteObject can target the specific version and trigger the COMPLIANCE enforcement.
ReleaseAndDeleteSealedSegmentAsync passes VersionId to DeleteObjectRequest. Without it, S3 creates a delete marker and the underlying version stays WORM-protected.
If the delete is blocked by Object Lock, S3 returns AccessDenied or a message containing "WORM". The adapter returns -1 to signal "immutability blocked" to the reaper — the segment is not marked reaped and will be retried after the lock expires.

Configuration (appsettings.json):

"Cesivi": {
  "Archive": {
    "Worm": {
      "Backend": "S3",
      "S3": {
        "BucketName": "cesivi-worm",
        "Region": "us-east-1",
        "Endpoint": "",           // blank = AWS default; set for MinIO or other S3-compatible
        "AccessKeyId": "AKIA...",
        "SecretAccessKey": "...",
        "RetentionDays": 2555
      }
    }
  }
}

Prerequisites: - S3 bucket created with Object Lock enabled (cannot be enabled after bucket creation). - Versioning is enabled automatically when Object Lock is turned on. - The IAM policy for the Cesivi service account needs: s3:PutObject, s3:GetObject, s3:PutObjectLegalHold, s3:PutObjectRetention, s3:GetObjectRetention.

Azure Blob immutability adapter (`AzureBlobImmutableWormAuditLogStore`)¶

Uses Azure Blob Storage time-based immutability policies (WORM container-level policy).

Key implementation notes:

AppendAsync uses BlockBlob Upload with each sealed segment as a named blob.
SealActiveSegmentAsync calls SetImmutabilityPolicyAsync on the blob with ExpiresOn = now + RetentionWindow and mode Locked. In lenient mode (emulator/dev environments where the immutability API is not available), the error is swallowed so the rest of the chain continues to work.
ReleaseAndDeleteSealedSegmentAsync checks for BlobImmutabilityPolicyNotAllowed (the blob is still within its retention window) and returns -1.

Configuration (appsettings.json):

"Cesivi": {
  "Archive": {
    "Worm": {
      "Backend": "AzureBlob",
      "AzureBlob": {
        "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net",
        "ContainerName": "cesivi-worm",
        "RetentionDays": 2555
      }
    }
  }
}

Prerequisites: - Azure Blob container created with time-based immutability support enabled (Storage v2, General Purpose v2 accounts). Not available on ZRS Append blobs. - The Cesivi service principal needs: Storage Blob Data Contributor.

IWormAuditLogStore contract¶

All three adapters implement the same 7-method interface:

Method	Contract
`AppendAsync(farmId, event)`	Atomically appends one event; returns segment coordinate. Must be safe under concurrent writers to the same farm.
`QueryAsync(WormAuditQuery)`	Returns events in reverse-chronological order matching all non-null filter fields. Supports multi-farm, date range, event-type, user, and item-key filters.
`GetSegmentManifestAsync(farmId)`	Returns metadata for all segments (sealed + active) for the farm.
`VerifyChainAsync(farmId)`	Checks hash chain integrity. Reaped segments must not count as breaks.
`SealActiveSegmentAsync(farmId)`	Closes the active segment, records its hash, opens a new active.
`ReleaseAndDeleteSealedSegmentAsync(farmId, segmentId)`	Used only by the retention reaper. Must update the manifest (`reaped: true`) before physical deletion. Returns `-1` if blocked by cloud WORM lock.

The contract test suite WormAuditLogContractTests (abstract base in Cesivi.Server.Tests/Audit/Worm/) exercises all 10+ behaviours. Both cloud adapters pass this suite with Docker testcontainers (MinIO for S3, Azurite for Azure).

Migrating from FileSystem to a cloud backend¶

Use WormChainMigrator (available via the Cesivi.StorageConverter CLI):

dotnet run --project Cesivi.StorageConverter -- convert \
  --migrate-worm-chain \
  --source "C:\CesiviData" \
  --target "s3://cesivi-worm" \
  --farm-id "my-source-farm" \
  --endpoint "https://s3.eu-west-1.amazonaws.com" \
  --region "eu-west-1" \
  --access-key "AKIA..." \
  --secret-key "..."

The migrator re-appends events in original order and re-seals each segment, producing the same JSONL content → same SHA-256 hash → same prevHash chain. The chain is fully verifiable in the destination store without requiring the source files.

Configuration reference¶

All settings under Cesivi:Audit: in appsettings.json:

Key	Default	Description
`UseInMemorySink`	`false`	If `true`, uses the volatile in-memory ring buffer (test mode only)
`RecentCacheCapacity`	`1000`	In-memory window size for fast `GetRecent` reads
`ChainVerificationIntervalHours`	`6`	How often `WormChainVerificationService` runs
`Reaper:IntervalHours`	`24`	How often `WormSegmentReaper` runs
`Reaper:RequireHoldCheckBeforeReap`	`true`	If `true`, reaper skips segments until PLAN-A-7 legal hold check is wired; set `false` only in dev

Document Version: 1.0 Last Updated: 2026-05-27 (PLAN-1609 — v1.2 Audit Log WORM Substrate)

See also: Archive Admin Bundle — ControlCenter Quick Tour

See also: Archive Tools Operator Guide

See also: Tutorial G — SharePoint On-Premises Retirement Archive

See also: Cesivi Archive Variant A — Whitepaper

See also: Compliance Cookbook — HIPAA/GDPR/SOX/FRCP

See also: Archive Cluster Deployment Guide