Cesivi Server - Export/Import Guide¶

Goal 7: Export/Import Mock Storage - Complete Guide for backing up, migrating, and restoring mock data.

Overview¶

The Export/Import feature allows you to: - Export entire mock storage as a single ZIP file - Import mock data from ZIP archives - Backup data before making changes - Migrate data between environments - Share test datasets with team members

Quick Start¶

Export Mock Data¶

# Export entire storage
curl -X POST http://mocksharepoint.local/_api/admin/storage/export \
  -H "Content-Type: application/json" \
  -d '{}'

# Export with options
curl -X POST http://mocksharepoint.local/_api/admin/storage/export \
  -H "Content-Type: application/json" \
  -d '{
    "includeSiteCollections": true,
    "includeLists": true,
    "includeFiles": true,
    "includeUsers": true,
    "includePermissions": true,
    "includeContentTypes": true,
    "includeSiteColumns": true,
    "compressionLevel": 6
  }'

Response:

{
  "success": true,
  "filePath": "@MockData/.exports/mockdata-export-20251007-102030.zip",
  "fileSize": 15728640,
  "siteCollectionCount": 3,
  "listCount": 25,
  "fileCount": 1234,
  "timestamp": "2025-10-07T10:20:30Z"
}

Import Mock Data¶

# Import from uploaded file (Replace mode)
curl -X POST http://mocksharepoint.local/_api/admin/storage/import \
  -F "file=@mockdata-export-20251007-102030.zip" \
  -F "mode=replace" \
  -F "createBackup=true"

# Import from existing export file (Merge mode)
curl -X POST http://mocksharepoint.local/_api/admin/storage/import-from-file \
  -H "Content-Type: application/json" \
  -d '{
    "fileName": "mockdata-export-20251007-102030.zip",
    "options": {
      "mode": "Merge",
      "createBackup": true,
      "overwriteExisting": false,
      "validateBeforeImport": true
    }
  }'

Response:

{
  "success": true,
  "backupFilePath": "@MockData/.exports/mockdata-export-20251007-102035.zip",
  "siteCollectionCount": 3,
  "listCount": 25,
  "fileCount": 1234,
  "validationErrors": [],
  "warnings": ["Skipped existing file: site1/web.json"],
  "timestamp": "2025-10-07T10:20:45Z"
}

Export Options¶

ExportOptions Properties¶

Property	Type	Default	Description
`includeSiteCollections`	bool	`true`	Include all site collections
`includeLists`	bool	`true`	Include all lists and libraries
`includeFiles`	bool	`true`	Include all files and attachments
`includeUsers`	bool	`true`	Include users and groups
`includePermissions`	bool	`true`	Include permissions
`includeContentTypes`	bool	`true`	Include content types
`includeSiteColumns`	bool	`true`	Include site columns
`compressionLevel`	int	`6`	Compression level (0-9)

Compression Levels¶

0: No compression (fastest, largest file)
1-5: Fast compression
6: Optimal compression (default)
7-9: Maximum compression (slowest, smallest file)

Import Options¶

ImportOptions Properties¶

Property	Type	Default	Description
`mode`	ImportMode	`Merge`	Import mode (Replace or Merge)
`createBackup`	bool	`true`	Create backup before import
`overwriteExisting`	bool	`false`	Overwrite existing files in Merge mode
`validateBeforeImport`	bool	`true`	Validate ZIP structure before import

Import Modes¶

Replace Mode¶

Deletes ALL existing mock data (except .exports folder)
Extracts all files from ZIP
Use for: Complete data replacement, fresh start

{
  "mode": "Replace",
  "createBackup": true
}

Merge Mode¶

Keeps existing mock data
Adds new files from ZIP
Optionally overwrites existing files
Use for: Adding new data, updating specific items

{
  "mode": "Merge",
  "createBackup": true,
  "overwriteExisting": false
}

REST API Endpoints¶

POST /_api/admin/storage/export¶

Export mock storage to ZIP file.

Request Body:

{
  "includeSiteCollections": true,
  "includeLists": true,
  "includeFiles": true,
  "compressionLevel": 6
}

Response: ExportResult

POST /_api/admin/storage/import¶

Import mock storage from uploaded ZIP file.

Form Data: - file: ZIP file (multipart/form-data) - mode: "replace" or "merge" (optional) - createBackup: true/false (optional) - overwriteExisting: true/false (optional) - validateBeforeImport: true/false (optional)

Response: ImportResult

Upload Limit: 500 MB

POST /_api/admin/storage/import-from-file¶

Import from existing export file in .exports folder.

Request Body:

{
  "fileName": "mockdata-export-20251007-102030.zip",
  "options": {
    "mode": "Merge",
    "createBackup": true
  }
}

Response: ImportResult

GET /_api/admin/storage/exports¶

List all available export files.

Response:

[
  {
    "fileName": "mockdata-export-20251007-102030.zip",
    "filePath": "C:\\Source\\_AI\\Cesivi\\@MockData\\.exports\\mockdata-export-20251007-102030.zip",
    "fileSize": 15728640,
    "created": "2025-10-07T10:20:30Z"
  }
]

GET /_api/admin/storage/exports/{fileName}¶

Download an export file.

Response: ZIP file download

DELETE /_api/admin/storage/exports/{fileName}¶

Delete an export file.

Response:

{
  "message": "Export file deleted: mockdata-export-20251007-102030.zip"
}

GET /_api/admin/storage/statistics¶

Get current storage statistics.

Response:

{
  "siteCollectionCount": 3,
  "listCount": 25,
  "fileCount": 1234,
  "totalSizeBytes": 52428800,
  "totalSize": "50 MB"
}

Use Cases¶

1. Backup Before Major Changes¶

# Create backup
BACKUP=$(curl -s -X POST http://mocksharepoint.local/_api/admin/storage/export \
  -H "Content-Type: application/json" -d '{}' | jq -r '.filePath')
echo "Backup created: $BACKUP"

# Make your changes...

# If needed, restore from backup
curl -X POST http://mocksharepoint.local/_api/admin/storage/import-from-file \
  -H "Content-Type: application/json" \
  -d "{\"fileName\": \"$(basename $BACKUP)\", \"options\": {\"mode\": \"Replace\"}}"

# Developer A: Export data
curl -X POST http://mocksharepoint.local/_api/admin/storage/export \
  -H "Content-Type: application/json" -d '{}' > export-result.json

# Copy ZIP file to shared location
FILE=$(jq -r '.filePath' export-result.json)
cp "$FILE" /shared/team-datasets/

# Developer B: Import data
curl -X POST http://mocksharepoint.local/_api/admin/storage/import \
  -F "file=@/shared/team-datasets/mockdata-export-20251007-102030.zip" \
  -F "mode=replace"

3. Environment Migration¶

# Export from DEV
curl -X POST http://dev.mocksharepoint.local/_api/admin/storage/export \
  -H "Content-Type: application/json" -d '{}' | jq -r '.filePath'

# Download export
wget http://dev.mocksharepoint.local/_api/admin/storage/exports/mockdata-export-20251007-102030.zip

# Import to STAGING
curl -X POST http://staging.mocksharepoint.local/_api/admin/storage/import \
  -F "file=@mockdata-export-20251007-102030.zip" \
  -F "mode=replace" \
  -F "createBackup=true"

4. Incremental Updates¶

# Import new data without losing existing data
curl -X POST http://mocksharepoint.local/_api/admin/storage/import \
  -F "file=@new-sites.zip" \
  -F "mode=merge" \
  -F "overwriteExisting=false"

5. Docker Volume Persistence¶

# Export from container
docker exec cesivi curl -X POST http://localhost/_api/admin/storage/export \
  -H "Content-Type: application/json" -d '{}'

# Download from container
docker cp cesivi:/app/@MockData/.exports/mockdata-export-20251007-102030.zip ./backup.zip

PowerShell Examples¶

Export¶

$exportOptions = @{
    includeSiteCollections = $true
    includeLists = $true
    includeFiles = $true
    compressionLevel = 9
} | ConvertTo-Json

$result = Invoke-RestMethod -Uri "http://mocksharepoint.local/_api/admin/storage/export" `
    -Method Post `
    -Body $exportOptions `
    -ContentType "application/json"

Write-Host "Export created: $($result.filePath) ($($result.fileSize) bytes)"

Import¶

$importFile = "C:\backups\mockdata-export-20251007-102030.zip"
$boundary = [System.Guid]::NewGuid().ToString()

$bodyLines = @(
    "--$boundary",
    "Content-Disposition: form-data; name=`"file`"; filename=`"$(Split-Path $importFile -Leaf)`"",
    "Content-Type: application/zip",
    "",
    [System.IO.File]::ReadAllText($importFile),
    "--$boundary",
    "Content-Disposition: form-data; name=`"mode`"",
    "",
    "replace",
    "--$boundary--"
)

$body = $bodyLines -join "`r`n"

$result = Invoke-RestMethod -Uri "http://mocksharepoint.local/_api/admin/storage/import" `
    -Method Post `
    -Body $body `
    -ContentType "multipart/form-data; boundary=$boundary"

Write-Host "Import completed: $($result.fileCount) files"

List Exports¶

$exports = Invoke-RestMethod -Uri "http://mocksharepoint.local/_api/admin/storage/exports" `
    -Method Get

$exports | Format-Table FileName, @{Label="Size";Expression={"{0:N2} MB" -f ($_.fileSize / 1MB)}}, Created

File Structure¶

Export ZIP Structure¶

mockdata-export-20251007-102030.zip
├── site1/
│   ├── web.json
│   ├── Lists/
│   │   ├── Documents/
│   │   │   ├── list.json
│   │   │   ├── items.json
│   │   │   └── files/
│   │   └── Tasks/
│   └── users.json
├── site2/
└── site3/

.exports Directory¶

Located in @MockData/.exports/: - Stores all export files - Not deleted during Replace imports - Managed via REST API or manual file operations

Security Considerations¶

Path Traversal Protection¶

The import service validates ZIP files for: - Path traversal attempts (../) - Absolute paths - Suspicious file patterns

File Size Limits¶

Upload limit: 500 MB
Configure in StorageManagementController: [RequestSizeLimit(500_000_000)]

Backup Safety¶

Backups are created before Replace imports
Backup files are stored in .exports folder
Manual restoration possible if import fails

Error Handling¶

Export Errors¶

{
  "success": false,
  "error": "Mock data directory not found: @MockData",
  "fileSize": 0,
  "siteCollectionCount": 0
}

Import Errors¶

{
  "success": false,
  "error": "Import file validation failed",
  "validationErrors": [
    "Suspicious path detected: ../../etc/passwd",
    "ZIP file is empty"
  ],
  "warnings": []
}

Common Issues¶

Issue	Cause	Solution
"File not found"	Wrong file path	Check file path and permissions
"ZIP validation failed"	Corrupted ZIP	Re-create export with validation
"Upload size exceeded"	File > 500 MB	Increase `RequestSizeLimit`
"Backup creation failed"	Disk space	Free up space or disable backup

Performance¶

Export Performance¶

Small dataset (<10 MB): ~1 second
Medium dataset (10-100 MB): ~5-10 seconds
Large dataset (>100 MB): ~30-60 seconds

Compression level affects speed: - Level 0 (no compression): Fastest - Level 6 (optimal): Balanced - Level 9 (maximum): Slowest

Import Performance¶

Small dataset: ~1-2 seconds
Medium dataset: ~10-20 seconds
Large dataset: ~1-2 minutes

Replace mode is faster than Merge mode (no existence checks).

Docker Integration¶

Export from Container¶

# Export inside container
docker exec cesivi curl -X POST http://localhost/_api/admin/storage/export \
  -H "Content-Type: application/json" -d '{}'

# Copy to host
docker cp cesivi:/app/@MockData/.exports/mockdata-export-20251007-102030.zip ./

Import to Container¶

# Copy ZIP to container
docker cp ./mockdata-export.zip cesivi:/tmp/

# Import from inside container
docker exec cesivi curl -X POST http://localhost/_api/admin/storage/import \
  -F "file=@/tmp/mockdata-export.zip" \
  -F "mode=replace"

Volume Mapping¶

# docker-compose.yml
services:
  cesivi:
    volumes:
      - ./MockData:/app/@MockData
      - ./exports:/app/@MockData/.exports  # Persistent exports

Troubleshooting¶

Export Issues¶

Problem: Export file is empty or corrupted

# Check storage statistics first
curl http://mocksharepoint.local/_api/admin/storage/statistics

# Try with no compression
curl -X POST http://mocksharepoint.local/_api/admin/storage/export \
  -H "Content-Type: application/json" \
  -d '{"compressionLevel": 0}'

Problem: Export takes too long

# Use lower compression level
curl -X POST http://mocksharepoint.local/_api/admin/storage/export \
  -H "Content-Type: application/json" \
  -d '{"compressionLevel": 3}'

Import Issues¶

Problem: Import fails with validation errors

# Disable validation (use with caution)
curl -X POST http://mocksharepoint.local/_api/admin/storage/import \
  -F "file=@data.zip" \
  -F "validateBeforeImport=false"

Problem: Existing files not overwritten in Merge mode

# Enable overwrite
curl -X POST http://mocksharepoint.local/_api/admin/storage/import \
  -F "file=@data.zip" \
  -F "mode=merge" \
  -F "overwriteExisting=true"

Best Practices¶

Always create backups before Replace imports
Validate exports before sharing with team
Use Merge mode for incremental updates
Use Replace mode for complete environment refresh
Test imports in non-production environments first
Monitor file sizes to avoid hitting upload limits
Clean up old exports regularly to save disk space
Document export contents with naming conventions

Future Enhancements¶

Planned for future releases: - Scheduled automatic backups - Differential exports (only changed files) - Encryption support for exports - Import progress tracking - Batch import/export operations - CLI tool for offline operations

DOCKER_GUIDE.md - Docker deployment
DEPLOYMENT_GUIDE.md - Production deployment
ARCHITECTURE.md - Storage architecture
MIGRATION_GUIDE.md - Data migration from real SharePoint

Cesivi Server - Export/Import Guide¶

Overview¶

Quick Start¶

Export Mock Data¶

Import Mock Data¶

Export Options¶

ExportOptions Properties¶

Compression Levels¶

Import Options¶

ImportOptions Properties¶

Import Modes¶

Replace Mode¶

Merge Mode¶

REST API Endpoints¶

POST /_api/admin/storage/export¶

POST /_api/admin/storage/import¶

POST /_api/admin/storage/import-from-file¶

GET /_api/admin/storage/exports¶

GET /_api/admin/storage/exports/{fileName}¶

DELETE /_api/admin/storage/exports/{fileName}¶

GET /_api/admin/storage/statistics¶

Use Cases¶

1. Backup Before Major Changes¶

2. Share Test Dataset¶

3. Environment Migration¶

4. Incremental Updates¶

5. Docker Volume Persistence¶

PowerShell Examples¶

Export¶

Import¶

List Exports¶

File Structure¶

Export ZIP Structure¶

.exports Directory¶

Security Considerations¶

Path Traversal Protection¶

File Size Limits¶

Backup Safety¶

Error Handling¶

Export Errors¶

Import Errors¶

Common Issues¶

Performance¶

Export Performance¶

Import Performance¶

Docker Integration¶

Export from Container¶

Import to Container¶

Volume Mapping¶

Troubleshooting¶

Export Issues¶

Import Issues¶

Best Practices¶

Future Enhancements¶

Related Documentation¶