Tutorial D: Cesivi + Accept All + In-Memory + Lucene on Windows¶

Overview¶

This tutorial guides you through setting up Cesivi Server with: - Authentication: Accept All (no authentication required) - Storage: In-Memory (fast, ephemeral) - Search: Lucene full-text search engine - Platform: Windows

Use Case: High-performance development and testing environment with full-text search capabilities. Perfect for search functionality testing, performance benchmarks, and rapid prototyping.

Time Required: ~10 minutes

Prerequisites¶

Windows 10/11 or Windows Server 2019+
.NET 10.0 SDK installed
PowerShell 5.1 or later

Architecture¶

┌─────────────────────────────────────────────────────────────┐
│                      Windows Host                            │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐   │
│  │              Cesivi Server                   │   │
│  │                    :5000                              │   │
│  │  ┌──────────────┐  ┌──────────────┐  ┌────────────┐  │   │
│  │  │ Accept All   │  │  In-Memory   │  │  Lucene    │  │   │
│  │  │ Auth         │  │   Storage    │  │  Search    │  │   │
│  │  │ (No Auth)    │  │  (RAM)       │  │  Engine    │  │   │
│  │  └──────────────┘  └──────────────┘  └────────────┘  │   │
│  └──────────────────────────────────────────────────────┘   │
│                              │                               │
│                         RAM (Fast!)                          │
│                    No persistence on restart                 │
└─────────────────────────────────────────────────────────────┘

Key Characteristics: - Ultra-fast: No disk I/O, everything in RAM - No auth overhead: Requests processed immediately - Full-text search: Lucene provides powerful search capabilities - Ephemeral: Data lost on restart (perfect for testing)

Step 1: Create Configuration¶

Navigate to the Cesivi.Server directory:

cd C:\Source\_AI\Cesivi2\Cesivi.Server

Create appsettings.TutorialD.json:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Cesivi": "Debug",
      "Cesivi.Services.Search": "Debug"
    }
  },
  "Cesivi": {
    "DataRootPath": "",
    "LogPath": "C:/CesiviData/TutorialD/Logs",
    "HostName": "localhost",
    "UseHttps": false,
    "HttpPort": 5000,

    "StorageProvider": "InMemory",

    "SearchEngine": "Lucene",
    "LuceneIndexPath": "C:/CesiviData/TutorialD/LuceneIndex",

    "Identity": {
      "Providers": {
        "AcceptAll": {
          "Enabled": true,
          "Priority": 1,
          "DefaultUsername": "SHAREPOINT\\developer",
          "DefaultEmail": "developer@sharepoint.local",
          "DefaultDisplayName": "Developer"
        },
        "NTLM": {
          "Enabled": false
        },
        "OAuth2": {
          "Enabled": false
        }
      }
    },

    "Authentication": {
      "AcceptAllCredentials": true,
      "AllowAnonymous": true,
      "EnableNTLM": false,
      "EnableJWT": false,
      "EnableBasic": false
    },

    "Search": {
      "Lucene": {
        "MaxResults": 1000,
        "DefaultResultCount": 50,
        "EnableFuzzySearch": true,
        "FuzzyPrefixLength": 2,
        "EnableWildcardSearch": true,
        "BoostTitle": 2.0,
        "BoostContent": 1.0,
        "IndexBatchSize": 100
      }
    }
  }
}

Step 2: Create Lucene Index Directory¶

# Create directories
New-Item -ItemType Directory -Path "C:\CesiviData\TutorialD\LuceneIndex" -Force
New-Item -ItemType Directory -Path "C:\CesiviData\TutorialD\Logs" -Force

Write-Host "Created Lucene index directory"

Step 3: Start Cesivi Server¶

# Set environment
$env:ASPNETCORE_ENVIRONMENT = "TutorialD"

# Start the server
dotnet run

You should see:

info: Cesivi[0]
      Cesivi Server started
      Listening on: http://localhost:5000
      Storage Provider: InMemory
      Search Engine: Lucene
      Identity Providers: AcceptAll (1)

Step 4: Test Basic Access (No Auth Required!)¶

Open a new PowerShell terminal:

# Simple GET request - no credentials needed!
$web = Invoke-RestMethod -Uri "http://localhost:5000/_api/web" `
    -Headers @{ "Accept" = "application/json;odata=verbose" }

Write-Host "Web Title: $($web.d.Title)"
Write-Host "Web URL: $($web.d.Url)"

Or use curl:¶

curl http://localhost:5000/_api/web -H "Accept: application/json;odata=verbose"

Step 5: Create Test Data for Search¶

Let's create a document library with searchable content:

$headers = @{
    "Accept" = "application/json;odata=verbose"
    "Content-Type" = "application/json;odata=verbose"
}

# Create a document library
$libraryBody = @{
    "__metadata" = @{ "type" = "SP.List" }
    "Title" = "Technical Documents"
    "BaseTemplate" = 101  # Document Library
    "Description" = "Library for technical documentation and guides"
} | ConvertTo-Json

$library = Invoke-RestMethod -Uri "http://localhost:5000/_api/web/lists" `
    -Method POST `
    -Headers $headers `
    -Body $libraryBody

Write-Host "Created library: $($library.d.Title)"

Add Searchable Documents¶

# Add multiple documents with different content
$documents = @(
    @{
        Title = "SharePoint Development Guide"
        Content = "This guide covers SharePoint development including CSOM, REST API, and PnP PowerShell. Learn about lists, libraries, content types, and workflows."
        Author = "John Developer"
        Category = "Development"
    },
    @{
        Title = "Azure Integration Best Practices"
        Content = "Best practices for integrating SharePoint with Azure services including Azure Functions, Logic Apps, and Azure Storage."
        Author = "Jane Cloud"
        Category = "Cloud"
    },
    @{
        Title = "Migration from On-Premises"
        Content = "Step-by-step guide for migrating SharePoint on-premises to SharePoint Online. Covers assessment, planning, and execution phases."
        Author = "Mike Migration"
        Category = "Migration"
    },
    @{
        Title = "Security and Compliance Guide"
        Content = "Comprehensive security guide covering permissions, authentication, data loss prevention, and compliance features in SharePoint."
        Author = "Sarah Security"
        Category = "Security"
    },
    @{
        Title = "Performance Optimization Tips"
        Content = "Tips for optimizing SharePoint performance including indexing, caching, CDN usage, and query optimization techniques."
        Author = "Pete Performance"
        Category = "Performance"
    }
)

foreach ($doc in $documents) {
    $itemBody = @{
        "__metadata" = @{ "type" = "SP.Data.Technical_x0020_DocumentsItem" }
        "Title" = $doc.Title
        "FileLeafRef" = "$($doc.Title -replace ' ', '_').docx"
    } | ConvertTo-Json

    try {
        $item = Invoke-RestMethod -Uri "http://localhost:5000/_api/web/lists/getbytitle('Technical Documents')/items" `
            -Method POST `
            -Headers $headers `
            -Body $itemBody

        Write-Host "Added: $($doc.Title)"
    } catch {
        Write-Host "Error adding $($doc.Title): $_"
    }
}

Write-Host "`nCreated 5 searchable documents"

Step 6: Test Lucene Search¶

Simple Keyword Search¶

# Search for "SharePoint"
$searchQuery = "SharePoint"
$searchUrl = "http://localhost:5000/_api/search/query?querytext='$searchQuery'"

$results = Invoke-RestMethod -Uri $searchUrl `
    -Headers @{ "Accept" = "application/json;odata=verbose" }

Write-Host "Search results for '$searchQuery':"
$results.d.query.PrimaryQueryResult.RelevantResults.Table.Rows.results | ForEach-Object {
    $title = ($_.Cells.results | Where-Object { $_.Key -eq "Title" }).Value
    Write-Host "  - $title"
}

Phrase Search¶

# Search for exact phrase
$phraseQuery = '"best practices"'
$phraseUrl = "http://localhost:5000/_api/search/query?querytext='$phraseQuery'"

$phraseResults = Invoke-RestMethod -Uri $phraseUrl `
    -Headers @{ "Accept" = "application/json;odata=verbose" }

Write-Host "`nPhrase search results for $phraseQuery`:"

Wildcard Search¶

# Wildcard search
$wildcardQuery = "migrat*"
$wildcardUrl = "http://localhost:5000/_api/search/query?querytext='$wildcardQuery'"

$wildcardResults = Invoke-RestMethod -Uri $wildcardUrl `
    -Headers @{ "Accept" = "application/json;odata=verbose" }

Write-Host "`nWildcard search results for '$wildcardQuery':"

Boolean Search¶

# Boolean AND search
$boolQuery = "SharePoint AND security"
$boolUrl = "http://localhost:5000/_api/search/query?querytext='$boolQuery'"

$boolResults = Invoke-RestMethod -Uri $boolUrl `
    -Headers @{ "Accept" = "application/json;odata=verbose" }

Write-Host "`nBoolean search (SharePoint AND security):"

Step 7: Advanced Search Queries¶

Search with Row Limit¶

# Limit results to 3
$limitedUrl = "http://localhost:5000/_api/search/query?querytext='SharePoint'&rowlimit=3"

$limitedResults = Invoke-RestMethod -Uri $limitedUrl `
    -Headers @{ "Accept" = "application/json;odata=verbose" }

Write-Host "Top 3 results:"

Search with Start Row (Paging)¶

# Get results starting from row 2
$pagedUrl = "http://localhost:5000/_api/search/query?querytext='*'&startrow=2&rowlimit=3"

$pagedResults = Invoke-RestMethod -Uri $pagedUrl `
    -Headers @{ "Accept" = "application/json;odata=verbose" }

Write-Host "Paged results (starting from row 2):"

Fuzzy Search (Typo-Tolerant)¶

# Fuzzy search - finds "SharePoint" even with typo
$fuzzyQuery = "SharePont~"  # Note the typo
$fuzzyUrl = "http://localhost:5000/_api/search/query?querytext='$fuzzyQuery'"

$fuzzyResults = Invoke-RestMethod -Uri $fuzzyUrl `
    -Headers @{ "Accept" = "application/json;odata=verbose" }

Write-Host "Fuzzy search results (typo-tolerant):"

Step 8: Benchmark Performance¶

Since we're using in-memory storage with AcceptAll auth, performance should be excellent:

# Simple performance test
$iterations = 100
$stopwatch = [System.Diagnostics.Stopwatch]::StartNew()

for ($i = 0; $i -lt $iterations; $i++) {
    $null = Invoke-RestMethod -Uri "http://localhost:5000/_api/web" `
        -Headers @{ "Accept" = "application/json;odata=verbose" }
}

$stopwatch.Stop()
$avgMs = $stopwatch.ElapsedMilliseconds / $iterations

Write-Host "Performance Test Results:"
Write-Host "  Total iterations: $iterations"
Write-Host "  Total time: $($stopwatch.ElapsedMilliseconds) ms"
Write-Host "  Average per request: $avgMs ms"
Write-Host "  Requests per second: $([math]::Round(1000 / $avgMs, 2))"

Step 9: Test with CSOM (No Credentials!)¶

using Microsoft.SharePoint.Client;

class Program
{
    static void Main()
    {
        // No credentials needed with AcceptAll!
        var context = new ClientContext("http://localhost:5000");

        // Load and execute
        var web = context.Web;
        context.Load(web, w => w.Title, w => w.Url);
        context.ExecuteQuery();

        Console.WriteLine($"Web Title: {web.Title}");
        Console.WriteLine($"Web URL: {web.Url}");

        // Search
        var keywordQuery = new KeywordQuery(context);
        keywordQuery.QueryText = "SharePoint";

        var searchExecutor = new SearchExecutor(context);
        var results = searchExecutor.ExecuteQuery(keywordQuery);
        context.ExecuteQuery();

        Console.WriteLine($"Search found {results.Value[0].TotalRows} results");
    }
}

Step 10: Verify In-Memory Behavior¶

Data Exists Only in RAM¶

# Create some data
$listBody = @{
    "__metadata" = @{ "type" = "SP.List" }
    "Title" = "Temporary List $(Get-Date -Format 'HHmmss')"
    "BaseTemplate" = 100
} | ConvertTo-Json

$newList = Invoke-RestMethod -Uri "http://localhost:5000/_api/web/lists" `
    -Method POST `
    -Headers @{
        "Accept" = "application/json;odata=verbose"
        "Content-Type" = "application/json;odata=verbose"
    } `
    -Body $listBody

Write-Host "Created list: $($newList.d.Title)"
Write-Host ""
Write-Host "Note: This list exists only in memory!"
Write-Host "If you restart the server, it will be gone."

Restart to Clear¶

# Restart the server (Ctrl+C, then dotnet run)
# All data will be cleared!

Troubleshooting¶

"Lucene index is locked"¶

Cause: Previous process didn't close cleanly

Solution:

# Delete lock files
Remove-Item "C:\CesiviData\TutorialD\LuceneIndex\write.lock" -ErrorAction SilentlyContinue

# Restart server

"Search returns no results"¶

Cause: Index not yet built or search query syntax error

Solution: 1. Wait a few seconds after adding content (indexing is async) 2. Check query syntax (use simple keywords first) 3. Check logs for indexing errors

"Out of memory"¶

Cause: Too much data in memory

Solution: - Restart server to clear in-memory data - Use FileSystem or database storage for large datasets - Increase available RAM

When to Use This Configuration¶

Good For: - Unit testing - Integration testing - Performance benchmarks - Rapid prototyping - Search functionality development - Demo environments

Not Good For: - Persistent data needs - Production use - Large datasets - Multi-user testing

Clean Up¶

# Stop the server (Ctrl+C)

# Remove Lucene index
Remove-Item -Recurse -Force "C:\CesiviData\TutorialD" -ErrorAction SilentlyContinue

# Remove config file
Remove-Item "appsettings.TutorialD.json" -ErrorAction SilentlyContinue

Summary¶

You have successfully set up: - Accept All authentication for frictionless development - In-Memory storage for maximum performance - Lucene search engine for full-text search

Key Characteristics¶

Feature	Value
Authentication	None (Accept All)
Storage	In-Memory (RAM)
Search Engine	Lucene
Data Persistence	None (lost on restart)
Performance	Excellent

Lucene Search Features¶

Full-text search
Phrase search ("exact phrase")
Wildcard search (prefix, suffix, mid*dle)
Boolean operators (AND, OR, NOT)
Fuzzy search (typo~)
Field-specific search (title:keyword)

Next Steps¶

Try Tutorial E for TfIdf search in Docker
Try Tutorial C for persistent storage
Read Search API Reference for advanced queries