Storage and Capacity Management
Manage tenant data, container images, and deployment backups separately. A software upgrade does not enlarge cloud disks, LUKS volumes, or filesystems. Do not run a software upgrade during storage expansion or migration.
Storage Planning
| Location | Purpose | Management |
|---|---|---|
| System disk and deployment directory | Host tools, configuration, upgrade state, encrypted database backups | At least 60 GiB system disk recommended; reserve backup and temporary space |
| Docker storage | Old and new images, writable container layers, logs | Check the actual storage paths configured for the daemon |
/srv/soln/data | PostgreSQL and MinIO persistent data, including workflow databases and attachments | Prefer a separate encrypted data disk for enterprise deployments; size for business growth |
Keep SOLN_DATA_ROOT at the same mount path when expanding or migrating storage. The 8 GiB encrypted file container is a small-capacity example, not a default enterprise data capacity. Enlarging its system disk does not enlarge the LUKS file container.
Inspect Capacity
Run in a root shell on the deployment server:
sl storage status --dir /opt/soln-tenant-runtime --json
The command reports deployment and tenant data filesystems, mount source, available capacity, and Docker storage. Diagnostic failures appear in errors; missing capacity information is not a passing check. On Linux, server tenant data must be an actual mount. A directory with mode 700 alone does not establish that storage is mounted.
dataMount.encryption=luks identifies a Linux encrypted mapping. unverified means the encryption method was not verified; it proves neither encrypted nor unencrypted storage. Verify cloud-provider encryption in the cloud console. A configuration declaration cannot enable encryption.
For installed deployments, a short-lived, read-only container using the current Runtime image probes the Docker host’s capacity. It does not read tenant file contents and is removed after completion. This avoids treating the upgrade helper’s own disk as host storage.
Docker Containerd Image Store
Docker’s containerd image store may reside outside DockerRootDir. Inspect:
docker info --format '{{.DockerRootDir}}'
docker info --format '{{json .DriverStatus}}'
If io.containerd.snapshotter.v1 appears, verify the actual containerd storage path and set SOLN_DOCKER_IMAGE_STORE_PATH in the deployment .env. /var/lib/containerd is common for default installations; use the configured path when customized.
SOLN_DOCKER_IMAGE_STORE_PATH=/REPLACE_WITH_ACTUAL_IMAGE_STORAGE_PATH
Before the first installation, set it in .env.example if .env does not exist. This field controls capacity checks; it does not migrate images or configure Docker. An upgrade stops if containerd image storage is enabled but its path is unspecified.
Upgrade Capacity Checks
sl upgrade --dir /opt/soln-tenant-runtime --check --json
When an update is available, the CLI checks:
- Image storage: four times the compressed layers of new product images not yet downloaded, allowing for downloading and unpacking.
- Deployment directory: twice the current tenant database size for encrypted backups.
- Tenant data volume: twice the current tenant database size for migration writes.
- Each filesystem: another 1 GiB reserve. Allocations on the same filesystem are combined.
These are conservative budgets, not exact migration sizes, and do not include future business growth. No new-release budget is computed when no update is available. doctor also requires at least 1 GiB free on a single-machine tenant data filesystem.
Insufficient capacity stops the upgrade before services are stopped or database migrations run. The tenant’s software upgrade uses the same checks and reports the affected storage type and capacity requirements. Backups reside in <deployment>/.soln/backups; they are not moved to the tenant data disk automatically. Database backups exclude MinIO attachments.
Upgrade Artifact Cleanup
After a successful upgrade, the CLI confirms that Runtime, Web, and Worker use the current images and are ready before cleaning retired artifacts. Tenant-page upgrades use the same rule. It retains the current release, the previous release's images and signature files, and the encrypted database backup required for that rollback.
Older CLI upgrade backups, release Compose files, signature directories, and recorded product image references are eligible for removal. Shared images remain protected. Only successfully exited Migrator and sticker-seeder containers from this deployment may be removed to release retired images. Images used by other containers are never forcibly removed.
Preview cleanup without deleting artifacts:
sl storage cleanup --dir /opt/soln-tenant-runtime --json
Apply cleanup after confirming the current release:
sl storage cleanup --dir /opt/soln-tenant-runtime --yes --json
Cleanup does not prune all Docker images or delete data volumes, tenant data, secrets, or manual backups. Product images are tracked in .soln/cleanup-inventory.json; unrecorded historical images are not automatically removed. Upgrade backups support one rollback target and do not replace independent business backups.
Readiness failures defer cleanup. Cleanup warnings appear in the upgrade's cleanup result without failing a successful software upgrade; resolve the warning and retry cleanup. Upgrade, rollback, and cleanup share .soln/lifecycle.lock. After an interrupted process, verify that no lifecycle operation is running before manually removing a stale lock directory.
Log Cleanup by Time
A space administrator can clean tenant logs through the Runtime command registry. Sign in with sl login using the tenant's space administrator account first; deployment login is a separate session. Log cleanup is separate from upgrade artifact cleanup and does not run during upgrades.
sl system log category list --json
# Preview; the cutoff is exclusive
sl system log cleanup --category all \
--before '2026-07-01T00:00:00+08:00' --limit 500 --json
# Apply the same batch
sl system log cleanup --category all \
--before '2026-07-01T00:00:00+08:00' --limit 500 --yes --json
Replace all with one category key to clean that category. --before requires a past ISO timestamp with a timezone. --limit bounds source records per category to 1–1000, default 500. Repeat with the same cutoff when hasMore is true. selected counts selected source records, affected counts deleted or redacted source records, and detailRows counts diagnostic child rows changed. Preview does not reserve records.
| Category | Key | Behavior |
|---|---|---|
| Login records | auth | Delete old login audit rows; retain active sessions |
| System audit | audit | Delete old operation audit rows and their search copies |
| Automation | automation | Clear finished execution error details and node input diagnostics; retain summaries, outputs and human-task evidence |
| Scripts | script | Delete finished Studio execution logs; retain scripts, bindings and versions |
| OpenLink | open-link | Clear finished diagnostics and successful request inputs; retain failed-item retry inputs, results and request identities |
| Model calls | model-usage | Clear raw usage payloads, diagnostic metadata and error text; retain Token counts, costs, pricing facts and quota evidence |
| Data-processing and button jobs | job | Clear finished task error text; retain results, batch items and notification state |
| Background log searches | search | Delete finished search task rows; retain source logs and exports |
| Approval and human tasks | approval, workflow | Protected: retain instances, decisions, comments, discussions and human tasks |
| AI conversations | luna | Protected: retain runs, events, steps and tool results |
| Execution receipts | connector, print-design, runtime-operation | Protected: retain identities and receipts that prevent duplicate execution |
| Messages and quotas | message, model-control | Protected: retain inbox messages, delivery receipts, quota events, counters and reservations |
Active, waiting or retry-scheduled work is excluded. Each batch is atomic, tenant-scoped and mutually exclusive with another log cleanup. Successful changes create a cleanup audit record. Historical diagnostic details become unavailable after cleanup. PostgreSQL normally reuses freed space internally; data files may not shrink immediately. Cleanup does not run VACUUM FULL or remove attachments.
Server Logs
| Log type | Management |
|---|---|
| systemd journal and services writing to it | Preview and clean archived files with the commands below |
| Host Nginx access/error files | Use the server's Nginx logrotate policy for rotation, compression and archive retention |
| Docker container output | Configure log-driver rotation limits; Docker file logs do not support deleting individual entries before a timestamp |
| PostgreSQL WAL and database files | Managed by PostgreSQL; do not delete them as ordinary logs |
On Linux, preview journal cleanup. Add --yes as root to rotate and vacuum journal archives through systemd:
sl storage log-cleanup --category journal \
--before '2026-07-01T00:00:00+08:00' --json
sl storage log-cleanup --category journal \
--before '2026-07-01T00:00:00+08:00' --yes --json
The cutoff is converted to a retention duration. systemd removes whole eligible archives and retains files containing newer entries. Preview reports total journal usage, not the exact bytes vacuum will remove. Do not truncate active Nginx or Docker files to simulate time-based cleanup.
Expand or Migrate Storage
Set capacity alerts based on usage and growth, for example a warning at 70% and planned expansion at 80%. Track absolute free space too. Monitor tenant data, backups, and image storage; CLI diagnostics do not provide continuous monitoring.
| Storage type | Sequence |
|---|---|
| Cloud-provider encrypted disk | Expand in the cloud console, detect the new capacity, expand partitions and filesystem as required |
| LUKS block device | Expand the underlying disk, then partitions, encrypted mapping, and filesystem as required |
| LUKS file container | Check its containing disk, enlarge the file, update loop-device capacity, mapping, and filesystem |
| Migration to a new disk | Back up, stop business/database/object-storage writes, copy and verify data, mount the new disk at the original path, restart and verify |
Expansion commands depend on devices, partitions, LVM, and filesystem type. Inspect the layout before following distribution and provider instructions. Never expand an existing volume by running luksFormat or mkfs, or deleting database directories or Docker volumes.
Before migration, preserve database and object data, deployment configuration, and recovery keys in an independent failure domain and verify recovery. Copying a live PostgreSQL data directory is not a valid migration. After migration, verify permissions, database, attachments, and public health. Retain the original volume until verification succeeds.
Restart Recovery and Key Management
Choose a recovery scheme before enterprise handoff:
| Scheme | Startup behavior |
|---|---|
| Provider-encrypted data disk | The provider handles encryption; the OS mounts by filesystem UUID. Verify account/KMS permissions and disk-type support |
| LUKS with an external key service | The host uses a controlled identity to obtain the key, unlock, and mount; data services stay stopped if keys are unavailable |
| LUKS with a manual passphrase | An administrator unlocks and mounts before starting Docker |
A local key file can automate unlocking, but when keys and data share the server, a complete system disk or snapshot can expose the key. This does not provide independent key isolation.
On a dedicated single-machine host, configure Docker with RequiresMountsFor=/srv/soln/data and retain AssertPathIsMountPoint=/srv/soln/data. Mount dependencies do not supply LUKS keys; configure crypttab, fstab, or the key service first. Shared Docker hosts require administrator-designed service dependencies instead of a daemon-wide restriction.
The CLI does not create or format disks, configure automatic unlocking, or store unlock keys. See Single-Machine Installation for restart recovery.