Plugin System Guide

TL;DR - Quick Start

Want to get started fast? Jump to: - 📖 Quick Start Guide - 5-minute setup - 🏗️ Architecture Overview - How it works - ⚙️ Configuration - appsettings.json setup - 📝 Example Plugins - Copy-paste ready examples - 🐛 Troubleshooting - Common issues

Key Concept: Plugins are HTTP services that receive Before/After events for SharePoint operations. Before hooks can abort, After hooks can audit/sync.

Decision Tree:

Need to PREVENT invalid operations? → Use Before hooks (can abort)
Need to AUDIT/SYNC completed operations? → Use After hooks (fire-and-forget)
Need both? → Use both Before AND After hooks

Minimal Example:

{
  "Cesivi": {
    "Plugins": [{
      "Id": "my-plugin",
      "ObjectType": "Microsoft.SharePoint.SPList",
      "EventType": "After",
      "EndpointUrl": "http://localhost:5000/hook"
    }]
  }
}

---

Overview

Cesivi Server provides a flexible plugin system that allows you to intercept storage operations with HTTP hooks. This enables scenarios like:

• Auditing: Log all SharePoint object modifications
• Validation: Enforce custom business rules before writes
• Synchronization: Replicate changes to external systems
• Transformation: Modify data before/after storage operations
• Monitoring: Track usage patterns and performance

Plugin Architecture

The plugin system uses an event-driven architecture with BEFORE and AFTER hooks:

1. BEFORE Event (GET request): Triggered before reading/writing data
2. Can inspect incoming operations
3. Can modify data before it's processed

4. Can abort operations

5. AFTER Event (POST request): Triggered after reading/writing data

6. Can inspect completed operations
7. Can trigger side effects
8. Cannot modify the result

Configuration

Basic Configuration

Add plugins to appsettings.json:

{
  "Cesivi": {
    "Plugins": [
      {
        "Id": "list-audit",
        "ObjectType": "Microsoft.SharePoint.SPList",
        "EventType": "After",
        "EndpointUrl": "http://localhost:5000/api/hooks/list",
        "Headers": {
          "X-API-Key": "{API_KEY}"
        },
        "TimeoutMs": 5000,
        "ContinueOnError": true,
        "Enabled": true,
        "Priority": 100,
        "FireAndForget": false
      }
    ]
  }
}

Configuration Properties

Property	Type	Description
Id	string	Unique identifier for the plugin
ObjectType	string	SharePoint object type (supports wildcards)
EventType	string	"Before" or "After"
EndpointUrl	string	HTTP endpoint to call
Headers	object	Additional headers (supports env vars)
TimeoutMs	number	Request timeout in milliseconds (default: 5000)
ContinueOnError	boolean	Continue if plugin fails (default: true)
Enabled	boolean	Whether plugin is active (default: true)
Priority	number	Execution priority — lower values execute first (default: 100)
FireAndForget	boolean	Enqueue After hooks to background queue instead of awaiting (default: false)
MaxRetries	number	Max retry attempts for fire-and-forget failures (default: 3)
RetryDelayMs	number	Base delay in ms between retries, with exponential backoff (default: 1000)
ConfigSource	string	Read-only: "appsettings" or "api" — indicates where plugin was registered

Object Types

Supported Object Types

• Microsoft.SharePoint.Administration.SPWebApplication
• Microsoft.SharePoint.SPSite
• Microsoft.SharePoint.SPWeb
• Microsoft.SharePoint.SPList
• Microsoft.SharePoint.SPListItem
• Microsoft.SharePoint.SPFile
• Microsoft.SharePoint.SPFolder
• Microsoft.SharePoint.SPUser
• Microsoft.SharePoint.SPGroup
• And more...

Wildcard Matching

Use wildcards to match multiple object types:

{
  "ObjectType": "Microsoft.SharePoint.*",
  "EventType": "Before"
}

This matches all SharePoint objects.

Environment Variables

Plugin headers support environment variable expansion:

Syntax

• {VAR_NAME} or ${VAR_NAME}
• If env var not found, original value is kept

Example

{
  "Headers": {
    "Authorization": "Bearer {AUTH_TOKEN}",
    "X-API-Key": "${API_KEY}",
    "X-Static-Header": "static-value"
  }
}

Set environment variables:

# Windows
set AUTH_TOKEN=your-token-here
set API_KEY=your-api-key

# Linux/Mac
export AUTH_TOKEN=your-token-here
export API_KEY=your-api-key

Plugin Implementation

BEFORE Hook (GET)

Receives context as query parameters:

GET /api/hooks/list/before?webApp=webapp&site=site&list=Documents
X-API-Key: your-api-key

Response Format:

{
  "success": true,
  "message": "Optional message",
  "modifiedData": { ... },  // Optional: modify data before operation
  "abort": false,            // Set true to abort operation
  "additionalData": {
    "key": "value"
  }
}

AFTER Hook (POST)

Receives full event data as JSON body:

POST /api/hooks/list/after
Content-Type: application/json
X-API-Key: your-api-key

{
  "objectType": "Microsoft.SharePoint.SPList",
  "eventType": "After",
  "timestamp": "2025-01-15T10:30:00Z",
  "operation": "Save",
  "context": {
    "webApp": "webapp",
    "site": "site",
    "list": "Documents"
  },
  "data": {
    "title": "My List",
    "baseType": 0
  },
  "metadata": {}
}

Response Format:

{
  "success": true,
  "message": "Audit logged successfully",
  "additionalData": {
    "auditId": "123456"
  }
}

Example Implementations

Example 1: Audit Logger (Node.js/Express)

const express = require('express');
const app = express();
app.use(express.json());

// AFTER hook: Log all list operations
app.post('/api/hooks/list/after', (req, res) => {
  const { operation, context, data } = req.body;

  console.log(`[AUDIT] ${operation} on list ${context.list}`);
  console.log(`  WebApp: ${context.webApp}, Site: ${context.site}`);
  console.log(`  Data:`, JSON.stringify(data, null, 2));

  // Save to audit database
  // db.audits.insert({ timestamp: new Date(), ...req.body });

  res.json({
    success: true,
    message: 'Audit logged'
  });
});

app.listen(5000, () => {
  console.log('Audit plugin listening on port 5000');
});

Example 2: Validation Plugin (Python/Flask)

from flask import Flask, request, jsonify
import re

app = Flask(__name__)

@app.route('/api/hooks/list/before', methods=['GET'])
def validate_list():
    """BEFORE hook: Validate list names"""
    list_name = request.args.get('list', '')

    # Enforce naming convention
    if not re.match(r'^[A-Z][a-zA-Z0-9_]+$', list_name):
        return jsonify({
            'success': False,
            'message': 'List names must start with capital letter',
            'abort': True
        }), 400

    return jsonify({
        'success': True,
        'message': 'Validation passed'
    })

if __name__ == '__main__':
    app.run(port=5000)

Example 3: Data Transformation (C#/ASP.NET Core)

[ApiController]
[Route("api/hooks")]
public class PluginController : ControllerBase
{
    [HttpGet("list/before")]
    public IActionResult BeforeListOperation([FromQuery] string webApp,
                                             [FromQuery] string site,
                                             [FromQuery] string list)
    {
        _logger.LogInformation($"BEFORE: {webApp}/{site}/{list}");

        return Ok(new
        {
            success = true,
            message = "Preprocessing complete"
        });
    }

    [HttpPost("list/after")]
    public IActionResult AfterListOperation([FromBody] PluginEventData eventData)
    {
        // Synchronize to external system
        if (eventData.Operation == "Save")
        {
            _ = SyncToExternalSystemAsync(eventData.Data);
        }

        return Ok(new
        {
            success = true,
            message = "Sync initiated"
        });
    }

    private async Task SyncToExternalSystemAsync(object data)
    {
        // Your sync logic here
        await Task.CompletedTask;
    }
}

Example 4: Conditional Processing

{
  "Id": "production-only-audit",
  "ObjectType": "Microsoft.SharePoint.*",
  "EventType": "After",
  "EndpointUrl": "http://audit-service.prod:5000/api/audit",
  "Headers": {
    "X-Environment": "{ENVIRONMENT}",
    "Authorization": "Bearer {PROD_TOKEN}"
  },
  "TimeoutMs": 3000,
  "ContinueOnError": true,
  "Enabled": true
}

Plugin endpoint can check environment and act accordingly:

app.post('/api/audit', (req, res) => {
  const env = req.headers['x-environment'];

  if (env === 'production') {
    // Full audit logging
    saveToAuditLog(req.body);
  } else {
    // Light logging for dev/test
    console.log('DEV:', req.body.operation);
  }

  res.json({ success: true });
});

Multiple Plugins

You can register multiple plugins for the same object type. They execute in ascending Priority order (lower values first), with ties broken alphabetically by Id.

{
  "Plugins": [
    {
      "Id": "01-validate",
      "ObjectType": "Microsoft.SharePoint.SPList",
      "EventType": "Before",
      "EndpointUrl": "http://localhost:5000/api/validate"
    },
    {
      "Id": "02-transform",
      "ObjectType": "Microsoft.SharePoint.SPList",
      "EventType": "Before",
      "EndpointUrl": "http://localhost:5000/api/transform"
    },
    {
      "Id": "03-audit",
      "ObjectType": "Microsoft.SharePoint.SPList",
      "EventType": "After",
      "EndpointUrl": "http://localhost:5000/api/audit"
    }
  ]
}

Error Handling

ContinueOnError = true (Default)

Plugin failures are logged but don't abort the operation:

[WARN] Plugin list-audit returned failure: Connection refused
[INFO] Operation completed despite plugin failure

ContinueOnError = false

Plugin failures abort the operation:

[ERROR] Plugin critical-validator failed: Validation error
[ERROR] Operation aborted due to plugin failure

Best Practices

1. Use Appropriate Event Types

• BEFORE for validation, transformation, access control
• AFTER for auditing, synchronization, notifications

2. Keep Plugins Fast

• Aim for <100ms response time
• Use async operations for slow work
• Set appropriate timeouts

3. Handle Failures Gracefully

• Always return valid JSON
• Use ContinueOnError: true for non-critical plugins
• Log errors in your plugin endpoint

4. Security

• Use HTTPS for production
• Validate API keys/tokens
• Don't log sensitive data

5. Testing

• Test with Enabled: false first
• Monitor logs for errors
• Use staging environment

Troubleshooting

Plugin Not Called

Check: 1. Is Enabled: true? 2. Does ObjectType match? 3. Is endpoint accessible? 4. Check logs for registration errors

Timeout Errors

[WARN] Plugin list-audit timed out after 5000ms

Solution: - Increase TimeoutMs - Optimize plugin endpoint - Use async processing

Authentication Failures

[WARN] Plugin returned 401: Unauthorized

Solution: - Verify environment variables are set - Check header configuration - Test endpoint manually

Data Not Modified

For BEFORE events, ensure you're returning modifiedData:

{
  "success": true,
  "modifiedData": {
    "title": "Modified Title"
  }
}

Advanced Scenarios

Chaining Plugins

Plugins can call other plugins, creating a pipeline:

BEFORE Plugin 1 → BEFORE Plugin 2 → Storage → AFTER Plugin 1 → AFTER Plugin 2

Conditional Abortion

{
  "success": false,
  "message": "List name conflicts with reserved word",
  "abort": true
}

Data Enrichment

app.get('/api/hooks/list/before', (req, res) => {
  res.json({
    success: true,
    modifiedData: {
      ...req.body.data,
      enrichedField: 'Added by plugin',
      timestamp: new Date().toISOString()
    }
  });
});

Performance Impact

Scenario	Impact	Mitigation
No plugins	0ms	N/A
1 fast plugin (<50ms)	~50ms	Acceptable
3 plugins (~100ms each)	~300ms	Use async, increase timeout
Slow plugin (>1s)	High	Optimize plugin, use queue

Recommendation: Keep total plugin execution under 200ms.

Example Configurations

Minimal Audit

{
  "Id": "simple-audit",
  "ObjectType": "Microsoft.SharePoint.SPList",
  "EventType": "After",
  "EndpointUrl": "http://localhost:5000/api/audit",
  "Headers": {},
  "Enabled": true
}

Comprehensive Monitoring

{
  "Plugins": [
    {
      "Id": "all-before",
      "ObjectType": "Microsoft.SharePoint.*",
      "EventType": "Before",
      "EndpointUrl": "http://monitor.local/api/before",
      "Headers": { "X-API-Key": "{MONITOR_KEY}" },
      "TimeoutMs": 2000,
      "ContinueOnError": true,
      "Enabled": true
    },
    {
      "Id": "all-after",
      "ObjectType": "Microsoft.SharePoint.*",
      "EventType": "After",
      "EndpointUrl": "http://monitor.local/api/after",
      "Headers": { "X-API-Key": "{MONITOR_KEY}" },
      "TimeoutMs": 2000,
      "ContinueOnError": true,
      "Enabled": true
    }
  ]
}

Conditional Processing by Object

{
  "Plugins": [
    {
      "Id": "list-specific",
      "ObjectType": "Microsoft.SharePoint.SPList",
      "EventType": "After",
      "EndpointUrl": "http://localhost:5000/api/list-audit"
    },
    {
      "Id": "file-specific",
      "ObjectType": "Microsoft.SharePoint.SPFile",
      "EventType": "After",
      "EndpointUrl": "http://localhost:5000/api/file-audit"
    }
  ]
}

Fire-and-Forget After Hooks

By default, After hooks are called synchronously in the request path — the server waits for each plugin to respond before returning to the client. For non-critical After hooks (audit logging, analytics, external sync), you can enable fire-and-forget mode to move plugin execution to a background queue.

Configuration

{
  "Id": "background-audit",
  "ObjectType": "Microsoft.SharePoint.*",
  "EventType": "After",
  "EndpointUrl": "http://audit-service:5000/api/audit",
  "FireAndForget": true,
  "MaxRetries": 3,
  "RetryDelayMs": 1000,
  "Enabled": true
}

How It Works

1. When a storage operation completes, the After event is enqueued to a bounded Channel-based background queue (capacity: 1,000 items)
2. A PluginBackgroundService processes items from the queue asynchronously
3. Failed calls are retried with exponential backoff: RetryDelayMs × 2^attempt (e.g., 1s, 2s, 4s)
4. After MaxRetries failures, the item is dead-lettered (logged as a warning and discarded)

Properties

Property	Type	Default	Description
FireAndForget	bool	false	Enable background processing. Only valid for After hooks — Before hooks must always be synchronous to support abort.
MaxRetries	int	3	Maximum retry attempts on failure
RetryDelayMs	int	1000	Base delay between retries (exponential backoff applied)

When to Use

• Audit logging — recording changes doesn't need to block the client
• External sync — replicating to another system can happen asynchronously
• Analytics — usage tracking shouldn't impact response time
• Notifications — sending emails/webhooks after operations

Performance Comparison

Synchronous (default):
  Client → Server → [BEFORE Plugin] → Storage → [AFTER Plugin ⏳] → Response
  Total: Storage time + Plugin time

Fire-and-Forget:
  Client → Server → [BEFORE Plugin] → Storage → Response (immediate)
                                                    └→ [AFTER Plugin] (background)
  Total: Storage time only

---

Priority Ordering

Plugins execute in ascending Priority order. Lower values execute first. Ties are broken alphabetically by Id.

Configuration

{
  "Plugins": [
    {
      "Id": "audit-logger",
      "ObjectType": "Microsoft.SharePoint.SPList",
      "EventType": "Before",
      "EndpointUrl": "http://localhost:5001/audit",
      "Priority": 200
    },
    {
      "Id": "name-validator",
      "ObjectType": "Microsoft.SharePoint.SPList",
      "EventType": "Before",
      "EndpointUrl": "http://localhost:5002/validate",
      "Priority": 10
    }
  ]
}

Execution order: name-validator (priority 10) → audit-logger (priority 200)

Use Cases

• Ensure validation runs before audit (validator at priority 10, audit at 200)
• Ensure data enrichment runs before sync (enrichment at 50, sync at 150)
• Default priority is 100 — custom plugins slot in relative to this

---

Plugin Metrics & Diagnostics

Monitor plugin health and performance with the built-in metrics endpoint.

Endpoint

GET /_api/diagnostics/plugins

No authentication required.

Response Format

{
  "plugins": {
    "list-audit": {
      "totalInvocations": 142,
      "successCount": 140,
      "failureCount": 2,
      "averageResponseMs": 23.5,
      "lastInvoked": "2026-03-21T14:30:00.0000000Z",
      "lastError": "Connection refused"
    },
    "file-validator": {
      "totalInvocations": 58,
      "successCount": 58,
      "failureCount": 0,
      "averageResponseMs": 12.1,
      "lastInvoked": "2026-03-21T14:28:00.0000000Z",
      "lastError": null
    }
  },
  "queue": {
    "depth": 0,
    "totalEnqueued": 95,
    "totalProcessed": 95,
    "totalDeadLettered": 0
  }
}

Fields

Per-Plugin Metrics:

Field	Description
totalInvocations	Total number of times this plugin was called
successCount	Number of successful calls
failureCount	Number of failed calls
averageResponseMs	Average response time in milliseconds
lastInvoked	ISO 8601 timestamp of last invocation
lastError	Last error message (null if no errors)

Queue Metrics (fire-and-forget):

Field	Description
depth	Current number of items waiting in the background queue
totalEnqueued	Total items ever enqueued
totalProcessed	Total items successfully processed
totalDeadLettered	Total items that failed after all retries

Use Cases

• Monitoring dashboards — track plugin health at a glance
• Alerting — detect when failureCount increases or depth grows
• Performance tuning — identify slow plugins via averageResponseMs
• Troubleshooting — check lastError for failure details

---

Runtime Plugin Management API

Manage plugins at runtime without restarting the server. All endpoints are under /_admin/plugins.

List All Plugins

GET /_admin/plugins

Response:

{
  "value": [
    {
      "id": "list-audit",
      "objectType": "Microsoft.SharePoint.SPList",
      "eventType": "After",
      "endpointUrl": "http://localhost:5000/api/hooks/list",
      "headers": { "X-API-Key": "***" },
      "timeoutMs": 5000,
      "continueOnError": true,
      "enabled": true,
      "priority": 100,
      "configSource": "appsettings"
    }
  ],
  "count": 1
}

Get Specific Plugin

GET /_admin/plugins/{id}

Returns plugin configuration or 404 if not found.

Register New Plugin

POST /_admin/plugins
Content-Type: application/json

{
  "Id": "runtime-sync",
  "ObjectType": "Microsoft.SharePoint.SPListItem",
  "EventType": "After",
  "EndpointUrl": "http://sync-service:5000/items",
  "Priority": 150,
  "FireAndForget": true,
  "Enabled": true
}

Response: 201 Created with plugin configuration. ConfigSource is automatically set to "api".

Returns 409 Conflict if a plugin with the same ID already exists.

Update Plugin

PUT /_admin/plugins/{id}
Content-Type: application/json

{
  "ObjectType": "Microsoft.SharePoint.SPListItem",
  "EventType": "After",
  "EndpointUrl": "http://new-sync-service:5000/items",
  "Priority": 200,
  "Enabled": true
}

Response: 200 OK with updated configuration. ConfigSource is preserved from the original plugin.

Delete Plugin

DELETE /_admin/plugins/{id}

Response: - 200 OK — plugin removed - 403 Forbidden — plugin is configured via appsettings.json and cannot be deleted (use PUT to modify instead) - 404 Not Found — plugin does not exist

Test Plugin Connectivity

POST /_admin/plugins/{id}/test

Response:

{
  "pluginId": "list-audit",
  "success": true,
  "latencyMs": 23,
  "error": null,
  "timestamp": "2026-03-21T14:30:00.0000000Z"
}

Tests connectivity by sending an HTTP HEAD request to the plugin's endpoint URL. Useful for verifying plugin availability after deployment.

ConfigSource

The ConfigSource property indicates where a plugin was registered:

Value	Meaning	Can DELETE?	Can PUT?
"appsettings"	Loaded from appsettings.json	No (403)	Yes
"api"	Registered via POST /_admin/plugins	Yes	Yes

Config-based plugins are protected from accidental deletion. Use PUT to modify them, or edit appsettings.json directly.

---

Hot-Reload

Cesivi automatically detects changes to appsettings.json and reloads plugin configuration without requiring a server restart.

How It Works

1. The configuration system uses reloadOnChange: true for JSON config files
2. A ChangeToken monitors the Cesivi:Plugins configuration section
3. When changes are detected, config-based plugins are reloaded automatically
4. Runtime-added plugins (registered via the admin API) are preserved during reload — only appsettings-sourced plugins are refreshed

What Triggers Reload

• Editing appsettings.json and saving
• Editing appsettings.{Environment}.json and saving
• Editing the custom config file (if configured)

What Happens During Reload

1. All plugins with ConfigSource: "appsettings" are removed
2. New plugin configurations from the config file are loaded
3. Plugins with ConfigSource: "api" are left unchanged
4. A log entry is written: Plugin configuration reloaded: X config plugins, Y runtime plugins

Example

# 1. Server is running with 2 config plugins
# 2. Edit appsettings.json — add a third plugin
# 3. Save the file
# 4. Check logs:
#    [INFO] Plugin configuration reloaded: 3 config plugins, 0 runtime plugins
# 5. New plugin is immediately active — no restart needed

---

For Developers

Contributing to the plugin system? See project knowledge:

• ../_project/areas/plugins/ARCHITECTURE.md - Complete architecture with diagrams, component breakdown, performance characteristics
• ../_project/areas/plugins/HOOK_COVERAGE_AUDIT.md - Detailed audit of all 208 IStorageService methods
• ../_project/areas/plugins/INDEX.md - Plugin system development knowledge index
• ../_project/plans/PLUGIN_SYSTEM_IMPROVEMENTS.md - 5-phase improvement plan (19-33 hours)

Implementation Files: - Cesivi.Server/Services/PluginService.cs (~370 lines) - Plugin execution engine - Cesivi.Server/Services/StorageServiceWithPlugins.cs (~903 lines) - Decorator with hooks - Cesivi.Models/Models/PluginConfiguration.cs (~140 lines) - Configuration models - examples/plugins/ - Three production-ready example plugins

---

See Also

• PLUGIN_QUICK_START.md - 5-minute quickstart tutorial
• PLUGIN_TESTING_GUIDE.md - How to test plugins
• ARCHITECTURE.md - Overall Cesivi system architecture
• STORAGE_PROVIDERS.md - Storage backend options
• DEVELOPMENT-METHODS.md - Development patterns