Slot lifecycle reference

Every slot is a small state machine. The SlotManager drives transitions, the dashboard SSE stream renders them live, and each transition is persisted atomically to state.json. The dashboard shows real transitions — not systemd snapshots.

States

SlotState has nine values. Each value is also its JSON / SSE wire string.

State	Wire value	Meaning
OFFLINE	offline	Slot is not running. No systemd unit active.
PULLING	pulling	Model files are downloading or being verified. Unit not yet started.
STARTING	starting	systemd unit started; waiting for the container to come up.
WARMING	warming	Container up; health probe returns non-ready while the model loads into VRAM / GTT.
READY	ready	Passed the full health probe (non-empty /v1/models + sentinel completion). Ready to serve.
SERVING	serving	An inference request is actively in-flight.
IDLE	idle	Container up but cannot fulfil requests right now (see below).
UNLOADING	unloading	Graceful shutdown in progress; waiting for the container to exit.
ERROR	error	Slot has failed. Details in state.json and journald.

Note

A health-probe timeout during warming resolves to WARMING, never a false READY — a slot is only ever advertised dispatchable once it has actually answered a health probe. The fail-watcher still polls the underlying systemd unit while a slot sits in WARMING, so a container that dies mid-load is caught even though /health alone can’t distinguish “still loading” from “hung.”

The two faces of idle

idle is reached two ways; both render identically on the wire but the state machine arrives via different edges:

1. Process up, no model (warming → idle): the container is reachable but /v1/models is empty (e.g. launched with --model "" or the model file is missing). Routers must treat this distinctly from ready — never route an inference request to a slot that can’t fulfil it.
2. Warm but quiet (ready → idle): a previously-ready slot received no request for longer than its idle timeout. A candidate for unloading.

The slot detail panel in the dashboard, reflecting the current SlotState in real time.

Legal transitions

Transitions are enforced by the SlotManager; an illegal attempt raises IllegalSlotTransition (HTTP 409).

From	Allowed next states
OFFLINE	PULLING, STARTING
PULLING	STARTING, ERROR, OFFLINE
STARTING	WARMING, ERROR, OFFLINE
WARMING	READY, IDLE, ERROR, OFFLINE
READY	SERVING, IDLE, UNLOADING, ERROR
SERVING	READY, IDLE, ERROR
IDLE	SERVING, UNLOADING, READY
UNLOADING	OFFLINE, ERROR
ERROR	OFFLINE, PULLING, STARTING

The happy path is offline → pulling → starting → warming → ready, then ready ↔ serving under load, drifting to idle when quiet, and unloading → offline on teardown. error can recover back into offline, pulling, or starting.

Failure detection — the fail-watcher

While a slot sits in warming, ready, serving, or idle, a background per-slot task polls the container unit’s systemctl is-active every 2 seconds:

• warming is judged purely on unit liveness — a big model load can legitimately hold /health down for minutes, so /health is never consulted here. Three consecutive inactive polls flip the slot to error with message "container unit died while warming".
• ready / serving / idle are judged on unit liveness and /health: an active-but-unhealthy unit (two consecutive failed /health probes) flips to error with message "model server failed /health probe".
• A unit that goes inactive cleanly — a GPU-arbiter handoff, a plain systemctl stop, an OOM-kill pending restart — is reflected as offline (message "container stopped (auto-reloads on next request)"), not error. The red error dot is reserved for genuine spawn/health/load failures; a clean stop is a normal condition the dispatcher lazily reloads on the next request.

Detection latency is therefore up to a few poll intervals (roughly 4-6 seconds), never instant.

Reaching ready without an explicit model

Note

Some providers are self-managed and need no explicit model_id to reach ready: kokoro, qwen3tts, moonshine, and vibevoice. Other providers require a loaded model before they can serve.

An NPU FLM slot of type = "transcription" or type = "embedding" is a different case: a trio shadow. The NPU runs a single FLM process — the device = "npu", type = "llm" anchor — which also serves transcription/ embedding when the anchor’s [npu] toggles are on. A shadow slot is not independently loadable (a standalone load call on it 500s the single-tenant NPU), so load() skips both the spawn and the readiness probe for it and transitions it straight to READY with message "served by NPU FLM anchor (trio shadow)". Its live state is otherwise derived from the anchor, and inference requests are dispatched to the anchor’s FLM process, never to the shadow slot’s own (non-existent) port.

Idle eviction and wake-on-request

Beyond the idle-timeout demotion to IDLE, a slot can be unloaded (ready/idle → unloading → offline) two ways:

• Hard TTL. A slot idle past its resolved idle_timeout_s (per-slot TOML override, else a 300-second default) is unloaded outright, freeing host RAM. An explicit idle_timeout_s = 0 pins a slot against this.
• Host-memory-pressure LRU eviction. When host MemAvailable drops below a configured floor, hal0 evicts idle, lru = true-flagged slots oldest-last_used-first until free RAM recovers (or no eligible slots remain). A slot actively serving a request is never evicted, and agent, utility, and npu are never evicted by pressure regardless of their lru flag — they’re the anchors the dispatch fallback chains depend on.

Either kind of eviction leaves the slot in OFFLINE, not ERROR — it’s a resource-management transition, not a failure. A request that lands on an idle-evicted slot triggers a lazy reload rather than a 404: the dispatcher kicks off SlotManager.load() for the target slot and returns a structured slot.loading 503 with Retry-After, so the client’s retry lands once the slot is back up.

Fallback on load

If a slot’s configured model.default isn’t locally servable — deleted, never pulled, or pulled-away — load() doesn’t just fail. It searches the local registry for the best match by the slot’s capability (chat, embed, rerank, asr, tts) and loads that instead, preferring name-similarity to the configured id. Diffusion / image / video artifacts are always excluded from this fallback search for a non-image slot, so a text slot can never silently come up serving an image-gen model it can’t actually run.

Persistence — state.json

Each slot’s current state is stored as JSON at /var/lib/hal0/slots/<name>/state.json. The record is intentionally flat for human readability and written through an atomic same-directory tempfile + os.replace, so an SSE reader or the dashboard never observes a half-written transition.

Field	Type	Notes
name	str	Slot name.
state	str	One of the wire values above.
model_id	str | null	Currently-assigned model id.
port	int	Host port.
updated_at	float	Epoch seconds of the last transition.
message	str	Human-readable detail (errors, notes).
extra	object	Arbitrary extra metadata.

A malformed state.json (missing/invalid state, unreadable file, bad JSON) raises SlotConfigError rather than being silently swallowed.

Typed slot errors

Slot operations raise typed errors so the API renders a structured envelope with a stable slot.* code:

Error	Code	HTTP
SlotNotFound	slot.not_found	404
IllegalSlotTransition	slot.illegal_transition	409
SlotNotReady	slot.not_ready	503
SlotSpawnFailed	slot.spawn_failed	500
SlotHealthFailed	slot.health_failed	503
SlotConfigError	slot.config_error	400
NpuExclusivityViolation	slot.npu_exclusivity_violation	409

NpuExclusivityViolation guards the AMD XDNA hardware context, which admits exactly one NPU LLM at a time: two enabled device=npu, type=llm slots cannot coexist.

Related references

• hal0 CLI — hal0 slot list reports these states.
• Config schema — the per-slot TOML that defines a slot.
• Slots — the concept and dispatch model.