API¶

Base URL¶

http://localhost:7258/v1

If an API key is configured, include it in the Authorization header:

Authorization: Bearer your-api-key

The local OpenAI-compatible API uses HTTP. Peer controls and native computation use separate authenticated encrypted connections.

Health¶

GET /health

Returns node identity, peer count, ready model IDs, and service status.

List Models¶

GET /v1/models

Returns ready local deployments and models advertised by healthy paired peers.

curl http://localhost:7258/v1/models

{
  "object": "list",
  "data": [
    {
      "id": "<exact-model-id-from-the-deployment-plan>",
      "object": "model",
      "owned_by": "salut"
    }
  ]
}

Load and Unload Models¶

POST /v1/models/load
POST /v1/models/unload

/v1/models/load is reserved for automatic GGUF admission and currently returns 501 gguf_admission_unavailable. Development deployments are created through the signed operator operation described below.

/v1/models/unload stops the exact active deployment that owns the supplied model ID. It returns 404 when that model is not deployed.

curl http://localhost:7258/v1/models/unload \
  -H "Content-Type: application/json" \
  -d '{"model":"<model-id-from-v1-models>"}'

Chat Completions¶

POST /v1/chat/completions

The request is served by a matching local deployment or proxied to a healthy paired peer that advertises the model. Salut does not start an unadmitted deployment as a side effect of a chat request.

curl http://localhost:7258/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-from-v1-models>",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 128
  }'

Streaming uses text/event-stream and ends with data: [DONE].

curl -N http://localhost:7258/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-from-v1-models>",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

Cluster Information¶

GET /v1/cluster
GET /v1/cluster/peers

These operations report discovery, capability, and health information. A peer appearing here does not prove that it executed part of a model. Accepted distributed evidence also records device allocation and graph work.

Native Deployment Operator¶

The development deployment operation is a strict MessagePack API on peer HTTPS. It is not part of the OpenAI-compatible interface.

• POST /v1/peer/llama-deployment/deploy

• POST /v1/peer/llama-deployment/status

• POST /v1/peer/llama-deployment/stop

Each request is limited to 64 KiB. The remote operator must sign it with a distinct paired Salut identity and must be listed by full fingerprint in SALUT_LLAMA_OPERATOR_FINGERPRINTS on the coordinator. An empty allowlist denies every remote operator. Request IDs are retained for ten minutes and cannot be replayed.

Use the included command instead of constructing signed MessagePack manually:

uv run python -m salut.inference.operator deploy \
  --host coordinator.local \
  --coordinator-fingerprint <full-fingerprint> \
  --plan plan.json

uv run python -m salut.inference.operator status \
  --host coordinator.local \
  --coordinator-fingerprint <full-fingerprint>

uv run python -m salut.inference.operator stop \
  --host coordinator.local \
  --coordinator-fingerprint <full-fingerprint> \
  --deployment-id <deployment-id> \
  --plan-sha256 <plan-sha256>

Responses report idle, starting, ready, stopping, failed, or closed. Native controls return 400 for an invalid plan or identity, 403 for authorization failure, 409 for replay or a conflicting deployment, 413 above the body limit, and 503 when the pinned backend or cleanup operation is unavailable.

Client Example¶

from openai import OpenAI

client = OpenAI(base_url="http://localhost:7258/v1", api_key="unused")
response = client.chat.completions.create(
    model="<model-id-from-v1-models>",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)
for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

You can query any API node. If another paired node already owns the exact model, Salut can proxy the request. Automatic creation of a distributed GGUF deployment from an arbitrary API node remains in progress.