I Built an AI Layer That Never Answers. Here Is What a Good Question Actually Takes

Type "I am not feeling well and want to know what to do" into a service I built, and it will not tell you what to do. It asks what symptoms you have. Then it asks when they started. Then it stops asking, because those are the only two fields its pack marks as required, and hands your application a typed, versioned, auditable JSON document called a Context Bundle. Total engine time across the whole conversation: well under a millisecond. Number of answers given: zero.

That refusal to answer is the entire product, and it came from two ideas that had been sitting with me for a while. The first I owe to Marc Brooker, who argues that a specification is not something you hand down before the work; it is something you arrive at by iterating, and that a good spec is increasingly what lets an AI do the work at all (Spec Driven Development isn't Waterfall). Requirements do not show up clarified. We clarify them by asking, which is to say, by having a conversation. The second I owe to Rafael Peixoto, who convinced me a year or two ago that the future of AI was neurosymbolic, flexible neural models wrapped in symbolic structure you can actually check, back when the whole field was betting the other way. OrbitSpec is what those two ideas look like running as a service: a short conversation that turns a vague request into a checkable spec, driven by a deterministic engine that treats the language model as a guest, never the host.

This is also the third field report in a series that turned out to be one system seen from three sides. Smithproof governs what agents do. plan-to-pr structures what we delegate to them. OrbitSpec, chronologically the first thing I built and the last one written up, sits at the very beginning of the chain, where a human types something vague and every downstream cost begins. Its chassis started from tiangolo's full-stack template, credited where credit belongs; the engine on top is the story here.

What it is

OrbitSpec is clarification as a service: a B2B API that sits in front of an AI product. The customer's app creates a session, submits the user's draft, and gets back one of two outcomes. Either needs_clarification, carrying exactly one question to relay to the user, or ready, carrying the Context Bundle for whatever model the customer runs downstream.

The whole loop from a terminal: a vague draft, two questions, a bundle. The engine asked two questions because the pack requires two of its seven fields, and it never asks more than the pack demands.

The behavior lives in data, not code. Versioned YAML Intent Packs define the fields a domain needs, which ones are required, what question to ask for each, and a clarification priority. A session pins its pack version at creation, so a pack upgrade never changes the rules of a conversation mid-flight, and the pinned version inside every bundle makes the bundle reproducible: same pack, same answers, same output. There are packs for health triage, legal intake, insurance claims, research, and a generic fallback.

Three properties do the heavy lifting, and all three are old software virtues rather than new AI ones. The loop is deterministic: the engine that merges scope, validates fields, and picks the next question is plain code with documented precedence. The question is singular: one clarifier per turn, chosen as the smallest useful question, because users answer one question and abandon questionnaires. And everything is audited: every step of every turn lands in an append-only event log with its latency.

The cheapest reliability win happens before the model runs

The industry works hard on making models answer better. Retrieval, reranking, guardrails, evals: all of it operates on the answer side of the transaction. But a large share of bad AI outputs are not badly answered questions; they are correctly answered bad questions. "I need help with a health issue" has no good answer, and everything a model generates from it is expensive guesswork wearing a confident tone.

The unreasonable effectiveness of OrbitSpec's design is that fixing this costs almost nothing. Here is the engine's own audit ledger for the session above:

Every step of every turn, timestamped with latency. The whole engine spends microseconds per turn.

Microseconds per turn, and each row is attributable: which step ran, when, inside which session, under which trace id. When something goes wrong downstream, the question that produced the bundle is not a memory in a chat log; it is a queryable record. In the smithproof post I wrote that spend you can read is spend a control plane can cap. The same holds here: intent you can read is intent you can audit.

One node in someone else's agent

OrbitSpec is not an app, and the fastest way to misjudge it is to evaluate it like one. The repository ships four demo chatbots built with LangGraph, each consuming the service through its generated Python SDK, and their architecture is the point: OrbitSpec is one node in the customer's agent graph. User input flows into the clarification node; the node returns either a question, which the chat relays, or a bundle, which the rest of the graph consumes.

The health-triage demo. The chat UI belongs to the customer; the questions come from the pack; the assistant never answers, it clarifies.

The SDK itself is part of the reliability story. It is generated from the OpenAPI spec, and CI regenerates it on every backend change and fails on drift, so the client the demos consume cannot quietly diverge from the API that serves them. Integration contracts rot in the seams; this one is checked on every pull request.

The 138-second lesson

The deep technical decision in this build is about where intelligence belongs, and it was learned the expensive way.

The obvious design uses an LLM to extract fields from the draft: the user says "I've had a headache and some dizziness since yesterday afternoon", and the model fills symptoms and duration without asking. The first implementation ran that extraction on a local model through Ollama on CPU, and the repository preserves the post-mortem: an average of 138 seconds per field-detection request, against a 60-second timeout. The extraction that was supposed to save the user one question was taking two minutes to not answer it.

The fix had three layers, and the ordering matters more than any of them individually.

Rules are the floor. The clarification loop is fully functional with no LLM at all: fields fill through answers, the engine stays in microsecond territory, and the product works. Intelligence is an upgrade, never a dependency. That single decision is why a two-minute model could not take the product down with it.

Inference is a swap, not a rewrite. Extraction moved to a hosted router running a small Llama model, seconds instead of minutes, and local Ollama stayed available behind a compose profile for anyone who wants inference on their own metal. Because extraction sits behind an interface with a rules fallback, the provider is a configuration choice.

Nothing the model says gets in without grounding. Extracted values pass a validation layer before they are accepted: hallucination-prone fields must have lexical grounding in the user's own words, values that are mostly an echo of the whole input are rejected as copies, and date-like fields must pass a plausibility check. A small model that is wrong quickly is only useful if wrongness cannot enter the bundle.

The smithproof post argued that the model you can afford to run inline is the model that governs. This is the same law from the other side: the model you cannot afford to wait for is the model that cannot help you, and the deterministic engine you barely notice is the one that always can.

Boring on purpose

Two smaller pieces deserve their line, both of them the kind of engineering that never demos well. Session state lives in Redis hashes guarded by a Lua-scripted optimistic lock: a single atomic script checks a version field before writing, returns distinct error codes for a missing session versus a version conflict, and refreshes the TTL, so two concurrent answers to the same session cannot silently overwrite each other. And the bundle's trace block carries the full identity chain, trace id, session id, workspace id, generated-at, so any bundle can be walked back to every step that produced it.

None of this is novel. All of it is what makes the difference between a demo of clarification and a service another company could build on.

Where it stands

The Context Bundle today carries intent, the original request, typed fields, output constraints, and trace identity. The roadmap fields are visible in the schema: a restatement of the request in the engine's words, assumptions, unknowns, and acceptance criteria all ship as empty placeholders, waiting for the layer that fills them. Scope Profiles, remembered answers that follow a user across sessions, have their slot in the merge precedence and their flag in the API, with persistence to come. Clarifiers are free-text today; choice and date types are drawn in the type system. The seams are cut where the next features slot in rather than bolt on.

I built OrbitSpec before either of the other two systems, and shelved it to go build them. Coming back to it with the trilogy complete, the shape of the whole is clearer than it was from inside any one part: specify the question, structure the delegation, govern the agent. Three layers, one lesson each, and the same conclusion every time. The parts of an AI system you can trust are the parts that were never intelligent, and their job is to make the intelligent parts safe to use.