Cesivi Server - Migration Tutorial

Version: 1.1 Last Updated: 2026-03-28

---

Table of Contents

1. Overview
2. Migration Planning
3. Step 1: Assess Source SharePoint Site
4. Step 2: Configure Cesivi Server
5. Step 3: Run Migration Discovery
6. Step 4: Execute Full Migration
7. Step 5: Validate Migrated Data
8. Step 6: Handle Migration Errors
9. Common Migration Scenarios
10. Best Practices

---

Overview

This tutorial provides a step-by-step guide for migrating SharePoint data to Cesivi Server. Whether you're migrating for testing, development, or demo purposes, this guide covers the complete process from assessment to validation.

What You'll Learn

• How to assess your SharePoint site for migration
• How to configure Cesivi Server for migration
• How to use the Cesivi.MigrationTool
• How to validate migrated data
• How to handle common migration issues

Prerequisites

• Access to source SharePoint site (SharePoint 2016/2019/SE or Online)
• Site Collection Administrator permissions
• Cesivi Server installed and running
• PowerShell 5.1 or later (PowerShell 7 recommended)
• PnP PowerShell module (optional but recommended)

Time Estimate

Site Size	Lists	Items	Estimated Time
Small	1-10	<1,000	15-30 minutes
Medium	10-50	1,000-10,000	1-2 hours
Large	50+	10,000+	2-6 hours

---

Migration Planning

What Gets Migrated

Component	Supported	Notes
Site Metadata	✅ Full	Title, URL, language, template
Lists & Libraries	✅ Full	All list templates supported
List Items	✅ Full	All field types supported
Documents	✅ Full	Content + metadata
Users	✅ Full	Site users with properties
Groups	✅ Full	Groups + memberships
Permissions	✅ Full	Role assignments
Content Types	✅ Full	Definitions + inheritance
Site Columns	✅ Full	All field types
Taxonomy	✅ Full	Term stores, sets, terms
Views	⚠️ Partial	Basic views only
Workflows	❌ None	Not supported

Migration Checklist

Before starting, verify:

• [ ] Source site URL is accessible
• [ ] Credentials have Site Collection Admin rights
• [ ] Target Cesivi Server is running
• [ ] Sufficient disk space for MockData
• [ ] Network connectivity is stable
• [ ] Migration window scheduled (for large sites)

---

Step 1: Assess Source SharePoint Site

1.1 Connect to Source Site

Using PnP PowerShell:

# Install PnP PowerShell if needed
Install-Module PnP.PowerShell -Force

# Connect to source site
$sourceUrl = "https://your-sharepoint-site.com/sites/TeamSite"
$credential = Get-Credential -Message "Enter SharePoint credentials"
Connect-PnPOnline -Url $sourceUrl -Credentials $credential

# Or for SharePoint Online with modern auth
Connect-PnPOnline -Url $sourceUrl -Interactive

1.2 Discover Site Structure

# Get site information
$web = Get-PnPWeb
Write-Host "Site Title: $($web.Title)"
Write-Host "Site URL: $($web.Url)"
Write-Host "Template: $($web.WebTemplate)"

# Get all lists
$lists = Get-PnPList
Write-Host "`nLists Found: $($lists.Count)"

# Create summary report
$summary = @()
foreach ($list in $lists) {
    $summary += [PSCustomObject]@{
        Title = $list.Title
        Template = $list.BaseTemplate
        ItemCount = $list.ItemCount
        Hidden = $list.Hidden
    }
}

$summary | Where-Object { -not $_.Hidden } | Format-Table -AutoSize

Expected Output:

Site Title: Team Site
Site URL: https://sharepoint.example.com/sites/TeamSite
Template: STS

Lists Found: 8

Title            Template ItemCount Hidden
-----            -------- --------- ------
Documents        101      156       False
Project Tasks    107      42        False
Announcements    104      5         False
Calendar         106      12        False
Custom List      100      1250      False

1.3 Estimate Migration Size

# Calculate total items and files
$totalItems = 0
$totalFiles = 0
$totalSize = 0

foreach ($list in $lists | Where-Object { -not $_.Hidden }) {
    $totalItems += $list.ItemCount

    if ($list.BaseTemplate -eq 101) {  # Document Library
        $files = Get-PnPListItem -List $list.Title -PageSize 5000
        $totalFiles += $files.Count

        foreach ($file in $files) {
            $totalSize += $file["File_x0020_Size"]
        }
    }
}

Write-Host "`nMigration Estimate:"
Write-Host "  Total Lists: $($lists.Count)"
Write-Host "  Total Items: $totalItems"
Write-Host "  Total Files: $totalFiles"
Write-Host "  Total Size: $([math]::Round($totalSize / 1MB, 2)) MB"

# Estimate time (based on benchmarks)
$estimatedMinutes = [math]::Ceiling($totalItems / 1000 * 2)  # ~2 min per 1000 items
Write-Host "  Estimated Time: $estimatedMinutes minutes"

1.4 Identify Potential Issues

# Check for large lists
$largeLists = $lists | Where-Object { $_.ItemCount -gt 5000 }
if ($largeLists) {
    Write-Warning "Large lists detected (>5000 items):"
    $largeLists | ForEach-Object { Write-Warning "  - $($_.Title): $($_.ItemCount) items" }
}

# Check for complex content types
$contentTypes = Get-PnPContentType
Write-Host "`nContent Types: $($contentTypes.Count)"

# Check for custom fields
$customFields = Get-PnPField | Where-Object { $_.Group -eq "Custom Columns" }
Write-Host "Custom Fields: $($customFields.Count)"

---

Step 2: Configure Cesivi Server

2.1 Start Cesivi Server

# Navigate to Cesivi Server
cd C:\Source\_AI\Cesivi\Cesivi.Server

# Start the server
dotnet run

# Verify it's running
Invoke-RestMethod -Uri "http://localhost:5000/_health"
# Expected: @{status=Healthy}

2.2 Configure Target Site Collection

Edit appsettings.json:

{
  "Cesivi": {
    "ServerUrl": "http://localhost:5000",
    "MockDataPath": "C:/Cesivi/MockData",
    "SiteCollections": [
      {
        "Name": "Migrated",
        "RootSite": {
          "Title": "Migrated Team Site",
          "Url": "/Migrated/TeamSite"
        }
      }
    ]
  }
}

2.3 Prepare Target Directory

# Create MockData directory
$mockDataPath = "C:\Cesivi\MockData"
New-Item -ItemType Directory -Path $mockDataPath -Force

# Verify permissions
$acl = Get-Acl $mockDataPath
Write-Host "Permissions: $($acl.Access | Format-Table)"

---

Step 3: Run Migration Discovery

3.1 Using Cesivi.MigrationTool

# Navigate to migration tool
cd C:\Source\_AI\Cesivi\Cesivi.MigrationTool

# Run discovery (dry-run)
dotnet run -- discover `
    --url "https://your-sharepoint-site.com/sites/TeamSite" `
    --username "DOMAIN\username" `
    --password "password" `
    --output "discovery-report.json"

3.2 Review Discovery Report

# Read discovery report
$report = Get-Content "discovery-report.json" | ConvertFrom-Json

Write-Host "Discovery Report Summary"
Write-Host "========================"
Write-Host "Site: $($report.SiteUrl)"
Write-Host "Lists: $($report.Lists.Count)"
Write-Host "Total Items: $($report.TotalItems)"
Write-Host "Total Size: $([math]::Round($report.TotalSizeBytes / 1MB, 2)) MB"

# Show lists to migrate
Write-Host "`nLists to Migrate:"
$report.Lists | ForEach-Object {
    Write-Host "  [$($_.Selected ? 'X' : ' ')] $($_.Title) ($($_.ItemCount) items)"
}

3.3 Select Items to Migrate

# Create migration manifest
$manifest = @{
    SourceUrl = "https://your-sharepoint-site.com/sites/TeamSite"
    TargetPath = "C:\Cesivi\MockData"
    Options = @{
        IncludeLists = @("Documents", "Project Tasks", "Custom List")
        ExcludeLists = @("Style Library", "Site Assets")
        MaxItemsPerList = 0  # 0 = unlimited
        MaxFileSize = 100    # MB
        IncludeVersions = $false
        IncludePermissions = $true
    }
}

$manifest | ConvertTo-Json -Depth 5 | Out-File "migration-manifest.json"

---

Step 4: Execute Full Migration

4.1 Run Migration with MigrationTool

# Full export
dotnet run -- export `
    --url "https://your-sharepoint-site.com/sites/TeamSite" `
    --username "DOMAIN\username" `
    --password "password" `
    --target "C:\Cesivi\MockData" `
    --strategy Full

# Or selective export
dotnet run -- export `
    --url "https://your-sharepoint-site.com/sites/TeamSite" `
    --username "DOMAIN\username" `
    --password "password" `
    --target "C:\Cesivi\MockData" `
    --strategy Selective `
    --lists "Documents,Project Tasks,Custom List"

4.2 Monitor Progress

The tool displays real-time progress:

Cesivi Migration Tool
=====================
Source: https://sharepoint.example.com/sites/TeamSite
Target: C:\Cesivi\MockData

[1/4] Connecting to SharePoint... OK
[2/4] Exporting web metadata... OK
[3/4] Exporting lists...
  - Documents:     [########--] 156/156 items, 45 files (12.3 MB)
  - Project Tasks: [########--] 42/42 items
  - Custom List:   [####------] 500/1250 items
      Processing: Item 500 of 1250 (40%)
      Speed: 65 items/second
      ETA: 12 seconds

[4/4] Exporting users and groups... OK

Migration Complete!
  Duration: 2 minutes 15 seconds
  Items: 1,448
  Files: 45 (12.3 MB)
  Errors: 0

4.3 Alternative: PnP PowerShell Export

For more control, use the PnP PowerShell export script:

# Full PnP PowerShell export script
$sourceUrl = "https://your-sharepoint-site.com/sites/TeamSite"
$targetPath = "C:\Cesivi\MockData"
$credential = Get-Credential

# Connect
Connect-PnPOnline -Url $sourceUrl -Credentials $credential

# Export web
$web = Get-PnPWeb
$webPath = "$targetPath\sites\site1\webs\web1"
New-Item -ItemType Directory -Path $webPath -Force | Out-Null

@{
    Id = $web.Id
    Title = $web.Title
    Description = $web.Description
    Url = $web.Url
    ServerRelativeUrl = $web.ServerRelativeUrl
    Created = $web.Created
    Language = $web.Language
} | ConvertTo-Json | Out-File "$webPath\web.json"

# Export lists
$listsPath = "$webPath\lists"
foreach ($list in Get-PnPList | Where-Object { -not $_.Hidden }) {
    Write-Progress -Activity "Exporting Lists" -Status $list.Title

    $listPath = "$listsPath\$($list.Title)"
    New-Item -ItemType Directory -Path "$listPath\items" -Force | Out-Null

    # Export list metadata
    @{
        Id = $list.Id
        Title = $list.Title
        Description = $list.Description
        BaseTemplate = $list.BaseTemplate
        ItemCount = $list.ItemCount
        Created = $list.Created
    } | ConvertTo-Json | Out-File "$listPath\list.json"

    # Export items
    $items = Get-PnPListItem -List $list.Title -PageSize 500
    foreach ($item in $items) {
        @{
            Id = $item.Id
            Title = $item["Title"]
            Created = $item["Created"]
            Modified = $item["Modified"]
            Fields = $item.FieldValues
        } | ConvertTo-Json -Depth 5 | Out-File "$listPath\items\item_$($item.Id).json"
    }
}

Write-Host "Export complete! Data saved to: $targetPath"
Disconnect-PnPOnline

---

Step 5: Validate Migrated Data

5.1 Start Cesivi with Migrated Data

# Restart Cesivi to load new data
# (Cesivi loads MockData on startup)
cd C:\Source\_AI\Cesivi\Cesivi.Server

# Stop existing instance (Ctrl+C) and restart
dotnet run

5.2 Verify Data via REST API

$baseUrl = "http://localhost:5000"
$cred = Get-Credential

# Check web
$web = Invoke-RestMethod -Uri "$baseUrl/_api/web" -Credential $cred
Write-Host "Site Title: $($web.d.Title)"

# Check lists
$lists = Invoke-RestMethod -Uri "$baseUrl/_api/web/lists" -Credential $cred
Write-Host "Lists Found: $($lists.d.results.Count)"

foreach ($list in $lists.d.results) {
    Write-Host "  - $($list.Title): $($list.ItemCount) items"
}

5.3 Verify Data via PnP PowerShell

# Connect to Cesivi
Connect-PnPOnline -Url "http://localhost:5000/Migrated/TeamSite" -Credentials $cred

# Compare counts
$sourceWeb = Get-PnPWeb
Write-Host "`nValidation Results:"
Write-Host "  Site Title: $($sourceWeb.Title)"

$lists = Get-PnPList | Where-Object { -not $_.Hidden }
foreach ($list in $lists) {
    $items = Get-PnPListItem -List $list.Title -PageSize 1
    Write-Host "  - $($list.Title): $($list.ItemCount) items ✓"
}

Disconnect-PnPOnline

5.4 Validation Checklist

# Create validation report
$validation = @{
    Timestamp = Get-Date
    Results = @()
}

# Check web
$validation.Results += @{
    Component = "Web"
    Check = "Title matches"
    Expected = $sourceWeb.Title
    Actual = $targetWeb.Title
    Passed = ($sourceWeb.Title -eq $targetWeb.Title)
}

# Check each list
foreach ($list in $sourceLists) {
    $targetList = $targetLists | Where-Object { $_.Title -eq $list.Title }

    $validation.Results += @{
        Component = "List: $($list.Title)"
        Check = "Item count matches"
        Expected = $list.ItemCount
        Actual = $targetList.ItemCount
        Passed = ($list.ItemCount -eq $targetList.ItemCount)
    }
}

# Display results
$validation.Results | Format-Table Component, Check, Expected, Actual, Passed
$passed = ($validation.Results | Where-Object { $_.Passed }).Count
$total = $validation.Results.Count
Write-Host "`nValidation: $passed/$total checks passed"

---

Step 6: Handle Migration Errors

Common Errors and Solutions

Error: Access Denied

Error: 403 Forbidden - Access denied to list 'Sensitive Data'

Solution:

# Request higher permissions, or exclude the list
dotnet run -- export ... --exclude-lists "Sensitive Data"

Error: Item Too Large

Error: File 'LargeVideo.mp4' exceeds maximum size (500 MB)

Solution:

# Increase limit or skip large files
dotnet run -- export ... --max-file-size 1000  # 1000 MB

# Or skip all files
dotnet run -- export ... --no-files

Error: Connection Timeout

Error: The operation has timed out after 120000ms

Solution:

# Increase timeout
dotnet run -- export ... --timeout 300000  # 5 minutes

# Or reduce batch size
dotnet run -- export ... --max-items 100

Error: Invalid JSON

Error: JSON parse error at position 1234

Solution:

# Validate JSON files
Get-ChildItem "C:\Cesivi\MockData" -Recurse -Filter "*.json" | ForEach-Object {
    try {
        Get-Content $_.FullName | ConvertFrom-Json | Out-Null
    } catch {
        Write-Warning "Invalid JSON: $($_.FullName)"
    }
}

# Re-export affected items
dotnet run -- export ... --lists "AffectedList" --overwrite

Error Recovery Workflow

# 1. Review error log
Get-Content "migration-errors.log" | Select-Object -Last 50

# 2. Identify failed items
$errors = Import-Csv "migration-errors.csv"
$failedLists = $errors | Group-Object ListTitle | Select-Object Name, Count

# 3. Retry failed items
foreach ($failed in $failedLists) {
    Write-Host "Retrying: $($failed.Name)"
    dotnet run -- export ... `
        --lists $failed.Name `
        --overwrite `
        --retry-failed
}

# 4. Validate after retry
.\Validate-Migration.ps1

---

Common Migration Scenarios

Scenario 1: Document Library Migration

# Export document library with files
dotnet run -- export `
    --url "https://sharepoint.example.com/sites/Docs" `
    --username "admin@example.com" `
    --password "password" `
    --target "C:\Cesivi\MockData" `
    --lists "Documents,Shared Documents" `
    --max-file-size 100

Scenario 2: List Data Only (No Files)

# Export list metadata and items only
dotnet run -- export `
    --url "https://sharepoint.example.com/sites/Data" `
    --username "admin@example.com" `
    --password "password" `
    --target "C:\Cesivi\MockData" `
    --no-files

Scenario 3: Permissions Migration

# Export with full permission details
dotnet run -- export `
    --url "https://sharepoint.example.com/sites/Secure" `
    --username "admin@example.com" `
    --password "password" `
    --target "C:\Cesivi\MockData" `
    --include-permissions `
    --include-groups

Scenario 4: Incremental Migration

# Initial full export
dotnet run -- export --url ... --target "C:\Cesivi\MockData"

# Later: incremental update
dotnet run -- export `
    --url "https://sharepoint.example.com/sites/TeamSite" `
    --username "admin@example.com" `
    --password "password" `
    --target "C:\Cesivi\MockData" `
    --strategy Incremental `
    --since "2026-01-01"

---

Best Practices

1. Start Small

• Test with a single list first
• Validate before proceeding with full site
• Use --max-items 100 for initial tests

2. Plan for Large Sites

# For large sites, migrate in phases
$phases = @(
    @{ Lists = "Documents"; Priority = 1 },
    @{ Lists = "Project Tasks,Calendar"; Priority = 2 },
    @{ Lists = "Archive"; Priority = 3 }
)

foreach ($phase in $phases | Sort-Object Priority) {
    Write-Host "Phase $($phase.Priority): $($phase.Lists)"
    dotnet run -- export ... --lists $phase.Lists
}

3. Use RAM Disk for Performance

# Export to RAM disk for faster testing
dotnet run -- export ... --target "R:\MockData"

4. Document Everything

# Create migration log
$migrationLog = @{
    StartTime = Get-Date
    SourceUrl = $sourceUrl
    TargetPath = $targetPath
    Options = $options
    Results = @()
}

# After migration
$migrationLog.EndTime = Get-Date
$migrationLog.Duration = $migrationLog.EndTime - $migrationLog.StartTime
$migrationLog | ConvertTo-Json -Depth 10 | Out-File "migration-log.json"

5. Validate Thoroughly

# Comprehensive validation
$validationChecks = @(
    { (Get-PnPWeb).Title -eq $expected.Title },
    { (Get-PnPList).Count -eq $expected.ListCount },
    { (Get-PnPUser).Count -ge 1 },
    { (Get-PnPContentType).Count -gt 0 }
)

$results = $validationChecks | ForEach-Object { & $_ }
if ($results -contains $false) {
    Write-Warning "Validation failed! Review logs."
}

---

Quick Reference

Migration Commands

Action	Command
Discovery	dotnet run -- discover --url URL --output report.json
Full Export	dotnet run -- export --url URL --target PATH
Selective Export	dotnet run -- export --url URL --lists "List1,List2"
No Files	dotnet run -- export --url URL --no-files
Incremental	dotnet run -- export --url URL --strategy Incremental

Validation Commands

# Quick health check
curl http://localhost:5000/_health

# List verification
curl http://localhost:5000/_api/web/lists

# Item count verification
curl "http://localhost:5000/_api/web/lists/getbytitle('List')/items?\$top=1"

---

Document Version: 1.1 Last Updated: 2026-03-28 Author: Cesivi Server Team