Skip to content

Cesivi — Testing Guide

This guide covers how to build, run, and understand the Cesivi test suites. For known test limitations, see KNOWN_LIMITATIONS.md.

Test Projects Overview

Cesivi has 10 active test projects spanning ~8,470+ tests across multiple frameworks:

Project Framework Tests Purpose
Cesivi.Server.Tests .NET 10.0 5,072+ Core server unit & controller integration tests
Cesivi.Tests.Storage .NET 10.0 216 Storage abstraction layer tests (all 8 providers)
Cesivi.Tests.RestSoap .NET Framework 4.8.1 487 REST/SOAP API integration tests
Cesivi.Tests.Csom .NET Framework 4.8.1 ~518 CSOM protocol integration tests
Cesivi.Tests.PnP .NET Framework 4.8.1 ~77 PnP PowerShell compatibility tests
Cesivi.Tests.WebUI .NET 10.0 800+ Playwright E2E browser tests
Cesivi.Tests.PluginFramework .NET 10.0 70 Plugin system tests
Cesivi.Tests.Performance .NET 10.0 ~10 Performance benchmarks
Cesivi.Tests.PnP.Modern .NET 10.0 ~6 Modern PnP Core SDK tests
Cesivi.Tests.AppDomainIsolation .NET Framework 4.8.1 ~12 AppDomain isolation tests

Why Two Frameworks?

  • .NET 10.0 projects (Server.Tests, Storage, WebUI, PluginFramework) run in-process using WebApplicationFactory or Playwright.
  • .NET Framework 4.8.1 projects (Csom, PnP, RestSoap) must use .NET Framework because the SharePoint CSOM and PnP PowerShell client libraries require it. These tests start an external Cesivi.Server process and connect over HTTP.

Prerequisites

  1. .NET 10.0 SDK — for building the server and .NET 10.0 test projects
  2. .NET Framework 4.8.1 Developer Pack — for CSOM/PnP/RestSoap test projects
  3. PowerShell 7+ — for PnP PowerShell tests
  4. Playwright browsers — installed automatically on first E2E run

Building

# Kill any orphaned processes first (prevents file-lock hangs)
bash Scripts/kill-orphans.sh --quiet

# Build the entire solution
export DOTNET_CLI_UI_LANGUAGE=en
dotnet build --verbosity quiet
# Expected: 0 Error(s), 0 Warning(s)

Running Tests

The test watchdog (Scripts/test-watchdog.sh) is a sliding-timeout test runner that kills tests only when no output has been produced for N seconds. This is the recommended way to run all test suites:

# Server Tests (in-process, fast)
bash Scripts/test-watchdog.sh 120 Cesivi.Server.Tests/Cesivi.Server.Tests.csproj

# Storage Tests
bash Scripts/test-watchdog.sh 120 Cesivi.Tests.Storage/Cesivi.Tests.Storage.csproj

# REST/SOAP Integration Tests (external process)
bash Scripts/test-watchdog.sh 180 Cesivi.Tests.RestSoap/Cesivi.Tests.RestSoap.csproj

# CSOM Integration Tests (external process)
bash Scripts/test-watchdog.sh 180 Cesivi.Tests.Csom/Cesivi.Tests.Csom.csproj

# PnP PowerShell Tests (external process)
bash Scripts/test-watchdog.sh 180 Cesivi.Tests.PnP/Cesivi.Tests.PnP.csproj

# Plugin Framework Tests
bash Scripts/test-watchdog.sh 120 Cesivi.Tests.PluginFramework/Cesivi.Tests.PluginFramework.csproj

Warning: Never use --blame-hang-timeout — it spawns a dump collector that can hang worse than the test itself.

Running Individual Tests

# Run a specific test class
dotnet test Cesivi.Server.Tests --filter "FullyQualifiedName~ClassName" --verbosity quiet

# Run a specific test method
dotnet test Cesivi.Server.Tests --filter "FullyQualifiedName~ClassName.MethodName" --verbosity quiet

E2E Browser Tests

E2E tests use Playwright to automate a Chromium browser against a running Cesivi Server + WebUI:

# Run all E2E tests (may take a while)
bash Scripts/test-watchdog.sh 180 Cesivi.Tests.WebUI/Cesivi.Tests.WebUI.csproj

# Run a specific E2E test class
dotnet test Cesivi.Tests.WebUI --filter "FullyQualifiedName~BrowserTests.CrudFlowTests" --verbosity quiet

# Run E2E tests in batches (recommended for large suites)
bash Scripts/run-e2e-batch.sh 1 10

E2E tests automatically start the server and WebUI processes. Default ports: - Server: http://localhost:5010 - WebUI: http://localhost:5510

Test Architecture

In-Process Tests (Server.Tests, Storage)

Test → WebApplicationFactory → In-process HTTP pipeline → Assertions

These tests create an in-memory test host with the full Cesivi middleware pipeline. Each test gets an isolated service scope. No external process needed.

External Process Tests (CSOM, PnP, RestSoap)

Test → ServerProcessFixture (starts Cesivi.exe) → HTTP requests → Assertions

These tests start a real Cesivi.Server process and communicate over HTTP. The ServerProcessFixture handles process lifecycle, and CsomStateReset clears accumulated CSOM client state between tests via reflection.

E2E Browser Tests (WebUI)

Test → ServerManager (starts Server + WebUI) → Playwright browser → Page assertions

Playwright launches a headless Chromium browser and interacts with the WebUI pages. The WebUiTestBase base class handles browser setup/teardown.

Test Isolation

All tests run sequentially (no parallelization) to prevent shared-state conflicts:

  • xunit.runner.json: parallelizeTestCollections: false, maxParallelThreads: 1
  • External tests use ServerProcessFixture (one server per test assembly)
  • CSOM tests call CsomStateReset between tests to clear static client state
  • Server.Tests use isolated DI scopes per test

Test Variance

Some test counts vary by ±5-15 between runs. This is expected:

  • HTTP timing: External tests depend on network round-trips
  • Async operations: Race conditions in event-driven tests
  • CSOM state accumulation: Client-side state leaks between tests in suite runs

Known Test Limitations

These are client-side limitations that cannot be fixed server-side:

Area Tests Affected Cause
PnP 3.x client ~39 tests Missing cmdlets, parameter validation, SDK incompatibilities
CSOM suite hang CsomListItemTests Static state accumulation in suite runs (pass individually)
LoadQueryWhere 3 tests Client-side BaseType enum limitation

For the complete list, see KNOWN_LIMITATIONS.md.

Useful Scripts

Script Purpose
Scripts/test-watchdog.sh Sliding-timeout test runner (recommended)
Scripts/kill-orphans.sh Kill orphaned Cesivi.exe and MSBuild processes
Scripts/run-e2e-batch.sh Run E2E test classes by index range
Scripts/run-e2e-baseline.sh Full E2E baseline run
Scripts/run-e2e-audit.sh E2E test audit with detailed reporting

Last Updated: 2026-03-21