post-mortem · oracle_of_delphi · Rust, TypeScript · 4 pages
It Was Never the Model
four failures in a local voice assistant, at four different layers. two of them presented as the language model being inadequate. none of them were.
abstract
Getting a five-process local assistant working was almost entirely a debugging project. These four are chosen because each lived at a different layer of the stack and each presented as something other than what it was. The through line is that none of them were in the language model, including the one where it started answering in Thai.
the blank interface
The orchestrator closed its socket while the client still had unread request bytes in flight. On Windows that sends a TCP reset rather than a graceful shutdown, and a reset truncates whatever the peer had not read yet. The 500KB bundle was arriving cut off, so nothing executed and nothing rendered. Invisible from the browser, which saw a connection that ended normally.
the daemon killed by its own logger
It worked when launched by hand and failed when auto-launched, which is the shape of an environment difference rather than a logic error. The daemon opened its audit log first thing on startup, fell back to the current working directory, and that directory was writable from my shell but not from the shortcut. Opening the log threw, and the error propagated out of startup and killed the process before it ever bound its pipe. A logging failure took down the whole service.
the tools that never worked
Two layers, both mine, both presenting as model quality. The model server was missing the flag that applies the chat template, which is what teaches the format, so the model had never been shown how to call a tool. Once that was fixed, the model streamed calls in fragments, name first and arguments dribbling in across messages, and my parser treated each fragment as a separate nameless call.
the assistant that spoke Thai
Replies would occasionally come back as fluent, correct Thai. Nothing deterministic produces that by accident, so it looked like the clearest possible model failure. The cause was temperature at 0.7: in the low-confidence moment before a tool call, the sampler had room to pick a plausible-but-unusual continuation, and a language switch is exactly that. Dropping to 0.3 stopped it. One line in a config file.