Deployment diagnostics
Diagnostic commands query the current tenant's Soln deployment. Other software on the same server is outside the query scope. Space administrators and authenticated platform operators have access. Other members need Enter Page under Integration → Environment Monitoring in their role's feature permissions; querying logs requires Service Logs, while restarting a service requires the separate Restart Services permission.
List services
Runtime command: system.environment.services.list.
sl system environment services list --json
Returns exact Compose service names, health, start times and restart counts, plus the domain certificate renewal row. Use Compose service names when querying container logs; the host renewal row is not a Docker log source. available: false means the deployment could not be inspected.
The final service row is Domain certificate renewal (domain-certificate-renewal). This is the host task proven to manage the configured domain’s Nginx certificate, not a Docker container. The observer matches the Nginx domain and certificate files, certificate hostname coverage, Certbot renewal lineage and the service command triggered by each installed timer. Timer names are not fixed: certbot.timer, soln-certbot.timer and custom Certbot timer names can be matched. Unrelated certificates/tasks are excluded; two active owners, unsupported commands/managers, or unproven ownership remain unknown. certificateRenewal includes the matching status and selected manager’s timer, service and certificate names for AI inspection. A host observer publishes its enablement/activity and the latest Certbot execution result every minute through the deployment state mount. scheduled means automatic checks are enabled, checking means a check is running, disabled means the timer is not enabled/active, and unhealthy means the latest check failed. Idle one-shot services are normal between checks. Missing, malformed, mismatched or older-than-three-minute evidence is unknown; a currently valid certificate alone does not prove automatic renewal. This row is included in both the overview and service-list commands; Docker log queries do not read its systemd journal. Existing hosts need the updated installer host observer before this status can be collected.
Restart a service
Runtime command: system.environment.service.restart.
sl system environment service restart --service postgres --yes
Use an exact name from system.environment.services.list. This write requires the separate Restart Services permission and explicit --yes confirmation. It targets one service in the current Soln Compose project; the Runtime management service, unrelated projects, ambiguous replicas, and containers without a restart policy are rejected. The environment-monitor AI assistant does not expose this write command.
Query service logs
Runtime command: system.environment.logs.query.
sl system environment logs query --service tenant-runtime-service --since 30m --tail 200 --level error --json
sl system environment logs query --since 1h --keyword 'request-id-from-the-error' --json
| Option | Meaning |
|---|---|
--service | Exact name from the service list, or all (default). |
--since | Relative lookback from 1 minute to 7 days, such as 30m, 1h or 7d. Defaults to 1h. |
--tail | Recent source lines per service and maximum merged results, 1–1000. Defaults to 200. |
--level | all, error, warn, info or debug. Defaults to all. |
--keyword | Case-insensitive literal substring, at most 200 characters. Use a request ID or error code to correlate events. |
Filters apply to the bounded recent source lines, so fewer matches can be returned. The result includes collection time, source service, timestamp, stdout/stderr, detected level and redacted message. Secrets are redacted before results leave Runtime. Queries read existing Docker logs on demand; they do not create a separate log archive in the tenant database or execute shell commands.
Check available, truncated and unavailableServices. An unavailable or partial result does not prove that services are healthy. A missing service returns an error; discover its actual name first.
AI diagnostics
Open Integration → Environment monitoring → Service logs and expand the AI panel, or press Alt+L. The page has its own conversation history. It can query these service diagnostics and use existing permitted read commands to correlate business records, approval instances and execution receipts. The panel is restricted to read commands; it does not operate the interactive terminal. Service-log collection requires an installed Docker deployment.
Example: “Check errors from the last 30 minutes, then find the approval execution associated with request ID …”.
Analyze the complete monitoring page
sl system environment overview get --json
Runtime command: system.environment.overview.get; strict input {}. It requires Environment monitoring and Service Logs access, follows the workspace Luna command allowlist, and is read-only. The page's AI conversation keyboard shortcut and the service-log conversation use the same command path: CLI → Runtime command registry → tenant repositories. No MCP or shell tool is needed.
The response covers all page sections: CPU/cores and memory, collected CPU/memory and disk/network IO history, storage capacity and directory distribution, deployment version/uptime, service health/restarts, database pools and waiting requests, saved/applied workflow dispatch and execution concurrency, Worker heartbeat and performance groups/trends. resources, databasePools, workflowCapacity, performance and certificate each return ready, partial or unavailable. Overall collectedAt is assembly time; use each section's own timestamps. Missing data never means healthy. Collection errors return a safe code without raw database/host details. Logs are separate bounded queries through system.environment.logs.query; this command does not open a terminal or read manual shell output.
For advice, AI must cite observations and coverage, distinguish startup delay/connection acquisition from human and callback waits, and compare saved with actual applied values. Recommendations should state current → proposed values, reasons and how to verify under similar load using waiters, P95 and errors; insufficient evidence means retain current values and collect more evidence. It cannot derive supported user counts, database max_connections, replica count or slow SQL from these readings. Execution bulk + control + reservedConnections must fit databasePoolMax. Values apply per executor process. The command does not change settings; confirm and apply any recommendation in the page's configuration controls. This capability does not authorize writes, cleanup or restarts.
The top metric cards include Domain certificate validity. The overview response adds certificate: hostname, port, issuer, validFrom, validTo, remaining full days (remainingDays), trusted, hostnameMatches, status, reason and collectedAt. It inspects only the workspace URL configured by RUNTIME_ORIGIN, including a CDN or external gateway certificate when that is the public endpoint. The check sends no application credentials. Results are cached for five minutes (failures for 30 seconds) and each handshake is bounded to three seconds. HTTP URLs are notApplicable; collection failures are unavailable, not valid. Certificates with 30 days or less remaining are expiring; expired certificates and trust/hostname errors are explicit. Remaining full days are recalculated from the cached certificate, so less than one day displays 0 while still valid. The card shows remaining days, with the domain on the third line and no hover details. 30 days or more is green, 10–29 days is yellow, and fewer than 10 days is red. Expired or missing validity displays 0 days. Displaying 0 for missing data does not mean expiry was confirmed; AI must use the original status and reason.
Runtime collects deployment metrics in the background from server startup, independently of page visits. CPU/memory, disk/network IO and service status run about every 15 seconds; capacity and directory sizes run about every minute, with no overlapping runs for the same collector. Slow directory/log scans do not hold up the other collectors. The page reads the latest in-memory results and loads certificate, database pool/workflow capacity and operation timing sections independently. Refresh reads the latest result; it does not force a host scan. The timing section retains its separate bounded database query/cache. Certificate collection also runs in the background, reusing the five-minute TLS cache and 30-second failure cache. Resource and IO history retains up to one hour of independently timestamped samples and resets on Runtime restart. history.source distinguishes resources from io; use only the matching source for each trend. resources.data.collection reports collecting, ready, partial, stale or unavailable for each collector. Before a first sample, that collector has no timestamp; certificate collectedAt is also null before its first check. A failed refresh retains the previous value and timestamp as stale; samples older than three collection intervals are also stale. Do not interpret assembly time as measurement time or old service status as current health.
Read operation timing
sl system environment performance get --json
Runtime command: system.environment.performance.get; strict input {}. Requires Environment monitoring and Service Logs access. Returns timing groups for the last hour: sample and failure counts, average, P95, maximum, and cumulative main-pool connection-acquisition P95. Timing kinds are separate: operation duration, service latency, or request elapsed up to audit preparation. Acquisition includes pool waiting and connection setup, not SQL execution. Also includes approval/automation workflows and nodes, OpenLink executions/items and node clocks, data-processing jobs, and scripts. Workflow elapsed includes waits; human processing, callbacks, startup waits, nodes and background execution are distinct scopes. Only completed intervals are sampled, at their end timestamps; startup waits use creation-to-start timestamps and can include intentional serialization/retries. In-progress ages are excluded. Each source reads up to 10,000 recently updated rows (scripts by finish time), with a separate cap of 20,000 derived samples; check scannedRecords, timedSamples and truncated. One source record can produce several scope samples. These statistics do not represent total request throughput. Missing timing is excluded. Does not read SQL text or message bodies.
The result includes bucketSeconds: 300, all-operation trends and each group's points. Points contain the UTC bucket start at, count, averageMs and p95Ms. Only populated buckets are returned; the first and last intervals may be partial within from/to. Global percentiles use raw event durations, not averages of operation percentiles.
Confirm recovery after restart
A restart request returning successfully is not proof of service recovery. The Environment Monitoring page checks services with configured health checks after a restart and reports recovery only after a healthy observation. If recovery cannot be confirmed within the check period, inspect the current service state and logs; do not treat the timeout as proof that the restart was never sent.
From the CLI, query system.environment.services.list again and inspect the selected service's health. Use system.environment.logs.query for diagnostics. The page restart button is offered for unhealthy/stopped services with configured health checks and does not restart the Runtime management service itself.