Storage Converter Tool¶

Home → Documentation → Usage → Storage Converter

Overview¶

The Storage Converter (CesiviStorageConverter) is a CLI tool for migrating Cesivi data between different storage providers. Use it to:

Migrate to SQLite for better query performance
Backup data from SQLite to file system
Move data between environments
Convert legacy data to newer storage formats

Supported Storage Providers¶

Provider	Description	Best For
`filesystem`	JSON files in MockData/ folder	Development, debugging, Git version control
`sqlite`	SQLite database file	Production, high performance, transactions
`litedb`	LiteDB embedded NoSQL database	Single-file deployment, MongoDB-like API
`inmemory`	Runtime memory only	Unit tests, temporary data (NOT persistent)

Note: The inmemory provider does NOT persist data to disk. Data is lost on restart. Use it only for testing/validation, not as a migration target.

Installation¶

The converter is built as part of the Cesivi solution:

cd Cesivi.StorageConverter
dotnet build

The executable is located at: - Windows: bin/Debug/net10.0/CesiviStorageConverter.exe - Linux/Mac: bin/Debug/net10.0/CesiviStorageConverter

Quick Start¶

FileSystem to SQLite (Most Common)¶

CesiviStorageConverter convert \
  -s filesystem \
  -t sqlite \
  --source-path ./MockData \
  --target-path ./Data/sharepoint.db

SQLite to FileSystem (Backup)¶

CesiviStorageConverter convert \
  -s sqlite \
  -t filesystem \
  --source-path ./Data/sharepoint.db \
  --target-path ./BackupData

Command Reference¶

`convert` Command¶

Converts data from one storage provider to another.

CesiviStorageConverter convert [OPTIONS]

Required Options¶

Option	Short	Description
`--source <TYPE>`	`-s`	Source storage type: `filesystem`, `sqlite`, `litedb`, `inmemory`
`--target <TYPE>`	`-t`	Target storage type: `filesystem`, `sqlite`, `litedb`, `inmemory`

Path Options¶

Option	Description	Default
`--source-path <PATH>`	Source path (filesystem: folder, sqlite/litedb: database file)	`./MockData`, `./Data/sharepoint.db`, or `./Data/sharepoint.litedb`
`--target-path <PATH>`	Target path (filesystem: folder, sqlite/litedb: database file)	`./MockData`, `./Data/sharepoint.db`, or `./Data/sharepoint.litedb`

Option	Description	Example
`--webapp <NAME>`	Only convert specific web application(s)	`--webapp Cesivi`
`--site <NAME>`	Only convert specific site collection(s)	`--site teamsite`

Document Options¶

Option	Description	Default
`--no-documents`	Skip binary file content (metadata only)	Include documents
`--max-doc-size <MB>`	Maximum document size to convert in MB	0 (unlimited)

Behavior Options¶

Option	Description	Default
`--no-clear`	Don't clear target storage before conversion	Clear target
`--stop-on-error`	Stop on first error (strict mode)	Continue on errors

Usage Examples¶

Example 1: Full Migration to SQLite¶

Convert all data from file system to SQLite database:

CesiviStorageConverter convert \
  -s filesystem \
  -t sqlite \
  --source-path /var/cesivi/MockData \
  --target-path /var/cesivi/Data/sharepoint.db

Example 2: Selective Migration¶

Convert only a specific web application:

CesiviStorageConverter convert \
  -s filesystem \
  -t sqlite \
  --webapp "Intranet"

Example 3: Metadata-Only Backup¶

Backup structure without large files:

CesiviStorageConverter convert \
  -s sqlite \
  -t filesystem \
  --no-documents \
  --target-path ./MetadataBackup

Example 4: Skip Large Files¶

Convert but skip files larger than 50MB:

CesiviStorageConverter convert \
  -s filesystem \
  -t sqlite \
  --max-doc-size 50

Example 5: Merge into Existing Data¶

Add data to existing storage without clearing:

CesiviStorageConverter convert \
  -s filesystem \
  -t sqlite \
  --source-path ./NewData \
  --no-clear

Example 6: Strict Mode¶

Stop immediately on any error:

CesiviStorageConverter convert \
  -s filesystem \
  -t sqlite \
  --stop-on-error

Example 7: FileSystem to LiteDB¶

Convert to LiteDB embedded database:

CesiviStorageConverter convert \
  -s filesystem \
  -t litedb \
  --source-path ./MockData \
  --target-path ./Data/sharepoint.litedb

Example 8: SQLite to LiteDB Migration¶

Migrate between database providers:

CesiviStorageConverter convert \
  -s sqlite \
  -t litedb \
  --source-path ./Data/sharepoint.db \
  --target-path ./Data/sharepoint.litedb

Output¶

The converter displays real-time progress and a summary table:

╭─────────────────────────────────────────────────────────╮
│                SP Storage Converter                      │
╰─────────────────────────────────────────────────────────╯

Source: filesystem (./MockData)
Target: sqlite (./Data/sharepoint.db)

Storage validation successful!

Converting storage... ━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% ⠿

┌─────────────────┬───────┐
│ Entity Type     │ Count │
├─────────────────┼───────┤
│ Web Applications│     1 │
│ Site Collections│     3 │
│ Webs            │     7 │
│ Lists           │    15 │
│ List Items      │   234 │
│ Documents       │    45 │
│ Users           │    12 │
│ Groups          │     5 │
│ Content Types   │    28 │
│ Fields          │   156 │
│ Total           │   506 │
└─────────────────┴───────┘

Duration: 00:00:12.345
Status: Success

Entities Converted¶

The converter migrates all Cesivi entities:

Category	Entities
Structure	Web Applications, Site Collections, Webs (Sites)
Content	Lists, List Items, Documents (binary files)
Security	Users, Groups
Schema	Content Types, Fields

Error Handling¶

Default Behavior (Continue on Errors)¶

By default, the converter continues when individual items fail: - Failed items are logged - Summary shows error count - Exit code is 0 if most items succeeded

Strict Mode (`--stop-on-error`)¶

In strict mode: - Conversion stops on first error - Exit code is 1 - Partial data may be in target

Common Errors¶

Error	Cause	Solution
"Could not validate source storage"	Source path doesn't exist	Check `--source-path`
"Could not validate target storage"	Can't write to target	Check permissions
"Storage type not supported"	Unknown provider name	Use: filesystem, sqlite, litedb, inmemory
"Skipping large file"	File exceeds `--max-doc-size`	Warning only, not error

Performance Tips¶

Use SQLite for production - 10x faster queries than file system
Skip documents for testing - Use --no-documents for quick schema migration
Filter by webapp/site - Convert only what you need
Use SSD storage - Especially for SQLite databases

Integration with Cesivi Server¶

After converting to SQLite or LiteDB, configure the server to use it:

SQLite:

{
  "Cesivi": {
    "StorageProvider": "Sqlite",
    "SqlitePath": "./Data/sharepoint.db"
  }
}

Or via environment variable:

export CESIVI_SQLITE_PATH=/var/data/sharepoint.db

LiteDB:

{
  "Cesivi": {
    "StorageProvider": "LiteDb",
    "LiteDbPath": "./Data/sharepoint.litedb"
  }
}