Network¶

Participation Modes¶

SALUT_CLIENT_ONLY=1

SALUT_CLUSTER_ENABLED=0

Native Distributed Inference¶

Salut owns discovery, pairing, an immutable execution plan, process supervision, request routing, failure handling, and cleanup. llama.cpp owns GGUF loading, device execution, and generation.

N is the number of selected compute devices. It is independent from physical host count, API coordinator count, and supervised process count. The active plan requires the same production path at N=1, N=2, N=3, N=4, and N=8.

Protocol version 2 identifies Darwin arm64 Metal as MTL0, Darwin x86_64 Accelerate as BLAS, and Windows AMD64 CUDA as CUDA0, all at one source commit. Local N=1 uses the coordinator device without an RPC worker. A mixed plan uses one or more remote workers and may also select the coordinator. An all-remote plan requires at least two workers. Native execution supports Apple-silicon coordinators and workers, plus Windows AMD64 CUDA workers running from source. Packaging supports Apple silicon. Intel native execution, Windows coordination and Windows packaging remain unimplemented.

Automatic GGUF loading requires the complete verified b11960 Apple-silicon runtime. It selects the local Metal device first, then healthy paired Metal or Windows CUDA devices at the same llama.cpp commit when more capacity is needed. Fresh signed memory reports and the pinned native no-allocation fitter check shared host and device memory, including RPC tensor transfer transients, before the model controller starts. Placement does not yet optimize from measured bandwidth.

During development, a distinct paired operator may also submit one exact execution plan through peer HTTPS when the coordinator explicitly allows its full fingerprint in SALUT_LLAMA_OPERATOR_FINGERPRINTS.

uv run python -m salut.inference.operator deploy \
  --host coordinator.local \
  --coordinator-fingerprint <full-fingerprint> \
  --plan plan.json

The empty operator allowlist denies every remote operator. Client-only and solo nodes reject inbound deployment controls.

Computation Transport¶

Every raw ggml-rpc-server listener binds to literal 127.0.0.1. A deployment-scoped forwarder carries RPC bytes across the LAN through exact-certificate mutual TLS tied to paired Salut identities.

For every local RPC connection:

1. The coordinator opens a fresh encrypted connection.

2. Both forwarders verify the expected certificate fingerprint.

3. The coordinator sends a fresh 32-byte nonce.

4. The worker must echo that nonce before either side relays RPC bytes.

5. Fixed TCP keepalive and a bounded reconnect grace period detect loss.

Port 17258 carries signed peer controls. The encrypted computation forwarder uses an operating-system-assigned incoming port for the deployment lifetime. The firewall must allow incoming connections to Salut for that dynamic port. Raw RPC never listens on the LAN.

Product transport does not depend on SSH. SSH can still be used to operate development machines while testing the application.

Process and Cleanup Ownership¶

Salut stages a verified backend closure in an owner-only content-addressed directory. Parent-liveness watchdogs own the controller and worker process groups. Signed start and stop controls bind deployment, plan, coordinator, and worker identities.

A stopped worker deployment remains in a 15-minute replay tombstone during one daemon lifetime. Coordinator cleanup starts every remote stop concurrently, cancels operations still pending after ten seconds, and joins those cancellations before router shutdown.

After a terminal path, no backend process, forwarder, listener, active request, deployment, temporary model copy, or file lease may remain. Documented model and RPC caches may remain only in managed directories with recorded sizes.

Evidence Status¶

The native Salut path has run physically on Keiko and Scotty through signed, certificate-pinned HTTPS with retained model placement, graph execution, process, socket, and cleanup evidence. A separate matching-version development run combined macOS Metal and Yar’s Windows CUDA device through an SSH tunnel. That standalone run confirms backend portability but does not prove the Salut product transport.

Automatic loading has software tests and pinned native estimation. Physical comparison of predicted and actual allocations and aggregate-only capacity acceptance remain pending. The remaining acceptance work also includes native Windows supervision, a production macOS and Windows N=2 run, a physical three-node aggregate-only run, wired-Ethernet operating limits, explicit worker-loss behavior, and cleanup evidence for every terminal path.

Cluster Tokens¶

Cluster tokens can automate pairing in a controlled lab. Set the same strong secret on each intended peer through Settings or SALUT_CLUSTER_TOKEN. Salut compares fingerprint-bound HMAC-SHA256 values and never sends the token itself over the network.

Treat the token as a password. A matching token authorizes pairing, but it does not authorize remote deployment control. The operator fingerprint allowlist is still required for deploy, status, and stop operations.