Test Scripts

The scripts/test/ directory contains cross-platform and integration test suites.

All tests (test/all.ts)

Runs typecheck, unit tests (lib, server, web, scripts), Playwright, and server ↔ ariadne contracts:

bun test:all
bun test:all --skip-e2e
bun test:all --skip-contracts
bun test:all --apps          # also Flutter ariadne + keryx + theseus

Theseus scaffold only: bun analyze:theseus and bun test:theseus. Ariadne: bun analyze:ariadne and bun test:ariadne.

Use bun test:all, not bun test. bun test is Bun’s unit-test runner and would pick up Playwright *.spec.ts files.

Contract Tests (test/contracts.ts)

Validates that the server API responses match the Dart models in Ariadne.

bun test:contracts
# or directly: bun scripts/test/contracts.ts

Phase 1 (Bun):

1. Start an in-memory test server
2. Seed navigation data (location, floor, nodes, edges, token)
3. Hit navigation API endpoints
4. Write real JSON responses to ariadne/test/fixtures/*.json

Phase 2 (Dart):

1. Run flutter test test/contract_test.dart in the ariadne directory
2. Dart tests read the fixtures and verify models parse correctly

Binary Smoke Tests (test/binary.sh)

Compiles the server binary for the host architecture, boots it with a minimal config, and verifies key endpoints.

bash scripts/test/binary.sh               # compile + test
bash scripts/test/binary.sh --skip-build  # test existing binary

Steps:

1. Compile binary for host architecture
2. Create temp working directory with minimal config/main.config.json (or legacy root path when applicable)
3. Start the binary on a random port
4. Hit endpoints: /api/v1/version, /, static assets, /api/v1/login, authenticated /api/v1/* routes
5. Report pass/fail counts

This runs automatically as part of the server build pipeline (build/server.sh).

VM Integration Tests (test/vm/)

Full lifecycle test designed to run on a clean Linux VM (Ubuntu/Debian). The test suite follows the same embed pattern as the installer: TypeScript test files in scripts/test/vm/ are bundled at build time into a self-contained shell script (releases/test-vm.sh).

Usage

sudo bash test-vm.sh                      # run all tests
sudo bash test-vm.sh --quick              # skip optional service tests
sudo bash test-vm.sh --suite=server       # run only the server integration suite
sudo bash test-vm.sh --suite=mqtt,server  # run specific suites (comma-separated)
sudo bash test-vm.sh --offline-dir=/path/to/bundle  # bootstrap from local offline artifacts
sudo bash test-vm.sh --offline-dir=/path/to/bundle --prerelease  # build not uploaded yet: never use the update server
sudo bash test-vm.sh --clean              # remove all artifacts and exit

Architecture

scripts/test/vm/*.ts           Source TypeScript test files
scripts/build/templates/vm.sh  Shell template (preflight + installer bootstrap)
scripts/build/generators/vm-tests.ts  Build-time bundler
releases/test-vm.sh            Generated self-contained test script

Build flow:

1. generators/vm-tests.ts bundles all TypeScript files in test/vm/ into a single script via Bun.build()
2. The bundle is embedded into templates/vm.sh via a heredoc placeholder (###vm_tests###)
3. The final test-vm.sh is written to releases/ and uploaded during the server build

Runtime flow:

1. Shell bootstrap: preflight checks (root, curl, systemd), download installer, run Phase 1 + Phase 2 install
2. Extract the embedded TypeScript test bundle to /tmp/safecall-vm-tests.ts
3. Run tests via the installed binary: BUN_BE_BUN=1 ./safecall /tmp/safecall-vm-tests.ts [flags]
4. TypeScript runner handles all post-installation verification, reports results
5. Shell handles final cleanup

Test Suites

Tests execute in the following order. Suites marked with (quick-skip) are skipped in --quick mode.

Suite	File	Description
install	test_install.ts	Verify Phase 1 + Phase 2: files, permissions, config, version, service state
integrity	test_integrity.ts	Verify checksums.json creation, file hash validation, tamper detection
doctor	test_doctor.ts	Doctor command: up-to-date check, 4.x migration detection, update prompt, corruption detection
cli	test_cli.ts	CLI commands: status, help, selftest, config show/set/reset, generate, update, integrations
service	test_service.ts	Service management: re-setup with custom user, service file validation
server	test_server.ts	(quick-skip) Server integration: nginx/apache/caddy config, detection paths, sample-config output
monit	test_monit.ts	(quick-skip) Monit install, config creation, check script, service status
mqtt	test_mqtt.ts	(quick-skip) Install mosquitto, MQTT connection test, pub/sub round-trip
grafana	test_grafana.ts	(quick-skip) Grafana JSON datasource API: health, search, query endpoints
zabbix	test_zabbix.ts	(quick-skip) Bridge LLD endpoints, Zabbix CLI --dry-run validation
migrate	test_migrate.ts	Simulate 4.x layout, run repair --migrate, verify layout migration preserving data
repair	test_repair.ts	Simulate breakage, re-install to repair, verify restore, test uninstall
custom_dir	test_custom_dir.ts	Install to custom directory, verify, one-liner --setup flag test

Framework (framework.ts)

Shared test utilities used by all suites:

• Formatting: section(), pass(), fail(), skip()
• File assertions: assert_file_exists(), assert_file_not_exists(), assert_dir_exists(), assert_executable(), assert_symlink()
• Service assertions: assert_service_active(), assert_service_enabled(), assert_service_inactive()
• Network assertions: assert_port_listening(), assert_http_ok()
• Content assertions: assert_contains()
• Utilities: run() (shell command execution), read_http_port(), full_cleanup()

All system interactions use Bun shell ($) for consistent behaviour on Linux VMs.

Adding a New Test Suite

1. Create a new file scripts/test/vm/test_<name>.ts
2. Export a TestSuite object:

import { section, pass, fail, type TestSuite, type TestContext } from "./framework";

export const suite_<name>: TestSuite = {
  name: "<name>",
  skip_in_quick: true,  // optional: skip in --quick mode
  async run(ctx) {
    section("<NAME>: description");
    // ... test logic using framework assertions ...
  },
};

3. Import and add it to the ALL_SUITES array in scripts/test/vm/main.ts
4. Rebuild to generate the updated test-vm.sh:

cd scripts/build && bun generators/vm-tests.ts

Prerequisites

• Clean Ubuntu/Debian VM (22.04+ recommended)
• Root access
• Internet connection (unless using --offline-dir with extracted bundle artifacts)

For offline bundle preparation and transfer, see Offline Installation.

Running on a local VirtualBox VM (test/vm-virtualbox.ts)

bun test:vm runs the suite on a local VirtualBox VM from the Mac and leaves the VM as it found it:

1. Restore the clean snapshot and start the VM (headless unless --gui)
2. Wait for Guest Additions and for the guest clock to resync with the host
3. Copy the test script (or an offline bundle) into the guest and run it as root
4. Power the VM off and restore the snapshot again, also after a failure or Ctrl+C

bun test:vm                         # full suite against the published release
bun test:vm --bundle=releases/safecall-offline-9.8.7.tar.gz  # a local build, before upload
bun test:vm --quick                 # skip optional service suites
bun test:vm --suite=server,mqtt     # specific suites
bun test:vm --gui                   # watch the VM window
bun test:vm --check                 # start, log in, check sudo, restore (no tests)

Without --bundle, the runner copies releases/test-vm.sh in, which installs the release published on the update server. With --bundle, it extracts the bundle in the guest and runs the bundle's own test-vm.sh --offline-dir=<bundle> --prerelease. Every install, update, repair and doctor check then uses the bundle (SAFECALL_OFFLINE_DIR). The update server is also blocked in the guest's /etc/hosts, so a test can't quietly pass against the previous release. The run fails unless the suite installed the bundle's manifest version.

The suite's exit code decides the result. The output is also written to releases/vm-test.log.

Setup (once): the VM needs Guest Additions, a snapshot to return to, and a user who can sudo. Commands reach the guest through VBoxManage guestcontrol, which logs in as that user. Store the user's password in the macOS Keychain (-w prompts for it):

security add-generic-password -s safecall-vm -a chell -w

Defaults are VM Workhorse, snapshot Ready To Test, user chell. Override them with SAFECALL_VM_NAME, SAFECALL_VM_SNAPSHOT and SAFECALL_VM_USER. SAFECALL_VM_PASSWORD replaces the Keychain item. The runner refuses to start if the VM is already running, so it never restores a snapshot under a session you are using.

bun commit --vm runs the --bundle form on the new build before anything is uploaded (see the server release).

Other Test Commands

Server unit tests and web type checking are run separately:

# Server unit tests
cd server && bun test

# Ariadne tests
cd ariadne && flutter test

# Web type checking
cd web && bun run typecheck