Cesivi Server - Logging Guide¶

Overview¶

This guide documents the logging configuration and best practices for Cesivi Server. Following the LOG_REDUCTION_PLAN, logging has been optimized to minimize disk usage while maintaining diagnostic capabilities when needed.

Quick Start¶

For production: Use default settings (minimal logging, 90-97% reduction from previous volume) For debugging: Enable verbose logging options as needed

Log Levels¶

Standard Log Levels (Microsoft.Extensions.Logging)¶

Level	Value	Usage	Default Status
Trace	0	Detailed diagnostic info (XML, full objects)	DISABLED
Debug	1	Detailed execution flow (method calls, IDs)	Development only
Information	2	Important events (request completed, object created)	Production
Warning	3	Unexpected but recoverable (fallbacks, retries)	Always
Error	4	Failures requiring attention	Always
Critical	5	System failures	Always

Default Configuration (appsettings.json)¶

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Warning",
        "Microsoft.AspNetCore": "Warning",
        "System": "Warning",
        "Cesivi": "Warning",
        "Cesivi.Services.CsomProcessor": "Warning",
        "Cesivi.Services.ClientObjectFactory": "Warning",
        "Cesivi.Middleware.RequestLoggingMiddleware": "Warning"
      }
    }
  }
}

Verbose Logging Configuration¶

Configuration Options¶

Located in appsettings.json under Cesivi:Logging:

{
  "Cesivi": {
    "Logging": {
      "EnableVerboseXmlLogging": false,
      "EnableVerboseQueryLogging": false,
      "EnableVerboseMethodLogging": false,
      "EnableVerboseObjectLogging": false,
      "LogRotation": {
        "Enabled": true,
        "MaxAgeDays": 7,
        "MaxTotalSizeMB": 1000,
        "MaxFileSizeMB": 100,
        "CleanupIntervalMinutes": 60
      },
      "Telemetry": {
        "Enabled": true,
        "SampleIntervalSeconds": 60,
        "AlertThresholdBytesPerSecond": 1048576
      }
    }
  }
}

Option Descriptions¶

Option	Default	Description
`EnableVerboseXmlLogging`	false	Log full XML request/response bodies (WARNING: 10-100x more data)
`EnableVerboseQueryLogging`	false	Log full CAML queries with ViewXml (5-50KB per query)
`EnableVerboseMethodLogging`	false	Log full method parameters and invocation details
`EnableVerboseObjectLogging`	false	Log full object serialization (JSON/XML of created objects)

When to Enable Verbose Logging¶

Enable specific verbose options when:

XML Logging: Debugging CSOM request/response parsing issues
Query Logging: Debugging CAML query execution or filtering issues
Method Logging: Debugging specific SharePoint API method calls
Object Logging: Debugging object serialization or property mapping

Always disable verbose logging in production!

Log Rotation and Cleanup¶

Automatic Cleanup¶

The LogCleanupService runs as a background service, automatically:

Deleting log files older than MaxAgeDays (default: 7 days)
Deleting oldest files when total size exceeds MaxTotalSizeMB (default: 1GB)
Running cleanup every CleanupIntervalMinutes (default: 60 minutes)

Manual Cleanup¶

For manual cleanup during development:

# Using the logclean skill
# In Claude Code: /logclean

# Or manually:
Remove-Item -Path ".\MockData\Logs\*" -Recurse -Force

Log Telemetry¶

Monitoring Log Volume¶

The LogTelemetryService tracks:

Total bytes written
Current bytes per second
Per-category breakdown (XML, Query, Method, Object)
Alert when volume exceeds threshold

Prometheus Metrics¶

The following metrics are exposed at /metrics:

log_bytes_total - Total bytes written to logs
log_bytes_per_second - Current logging rate
log_alert_active - 1 if high volume alert is active
log_category_bytes{category="XML"} - Bytes by category
log_category_count{category="XML"} - Log entries by category
cleanup_files_deleted_total - Total files deleted by cleanup
cleanup_bytes_freed_total - Total bytes freed by cleanup

Alert Threshold¶

Default: 1 MB/s (AlertThresholdBytesPerSecond: 1048576)

When log volume exceeds this rate for more than the sample interval, a warning is logged:

[LogTelemetry] High log volume detected: 2,500,000 bytes/sec (threshold: 1,048,576 bytes/sec)

Environment-Specific Configuration¶

Development (appsettings.Development.json)¶

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Debug",
      "Override": {
        "Cesivi": "Debug"
      }
    }
  }
}

Testing (appsettings.Test.json / appsettings.Testing.json)¶

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Warning",
      "Override": {
        "Cesivi": "Warning"
      }
    }
  }
}

Production (appsettings.Production.json)¶

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Warning",
      "Override": {
        "Cesivi": "Warning"
      }
    }
  }
}

Best Practices¶

DO¶

Use Warning level as default in production
Enable verbose logging only when debugging specific issues
Monitor log volume using telemetry
Clean up old logs regularly
Use structured logging with parameters

DON'T¶

Enable verbose logging in production
Log full XML/JSON bodies at Information level
Log in tight loops without rate limiting
Ignore log volume alerts

Logging Pattern Examples¶

Good (compact):

_logger.LogDebug("[QUERY] List={ListTitle}, RowLimit={RowLimit}", list.Title, query.RowLimit);

Bad (verbose):

_logger.LogDebug($"[QUERY] Full CAML: {query.ViewXml}"); // 5-50KB per log entry!

Conditional verbose logging:

if (IsVerboseQueryEnabled)
{
    LogVerboseQuery("[QUERY] Full CAML:", query.ViewXml);
}

Troubleshooting¶

Log Files Growing Too Large¶

Check if verbose logging is accidentally enabled
Review LogRotation settings
Manually trigger cleanup or reduce MaxAgeDays
Check for runaway logging (look for loops or high-frequency events)

Missing Log Entries¶

Verify log level configuration allows the level you expect
Check Serilog override settings for specific namespaces
Ensure log file sink is configured correctly

High Log Volume Alert¶

Identify the source using category metrics
Consider reducing log level for that component
Review if verbose logging was accidentally enabled
Check for infinite loops or high-frequency operations

Log File Locations¶

Purpose	Default Path	Configuration
Server logs	`{DataRootPath}/Logs/Server/cesivi-{date}.log`	`Cesivi:LogPath`
Test logs	`{TempPath}/Cesivi_Tests/`	Test configuration
Proxy logs	`MockData/Logs/Proxy/`	Proxy settings

Version History¶

2026-02-03: Initial documentation following LOG_REDUCTION_PLAN completion
Log levels optimized for 90-97% reduction in log volume
Added verbose logging configuration options
Added automatic log rotation and cleanup
Added log volume telemetry and alerting