> ## Documentation Index
> Fetch the complete documentation index at: https://docs.open.cx/llms.txt
> Use this file to discover all available pages before exploring further.

# 2026 08 26 luna voice search hardening design

# Luna Voice Search Hardening Design

## Goal

Make Luna the only production voice-eval target and make voice knowledge/instruction answers search-first, multilingual, grounded, and low-latency without preloading an organization's corpus into the worker.

## LiveKit research decision

LiveKit Agents 1.5.1 enables preemptive generation by default. The SDK begins the LLM request when it receives a final transcript segment before end-of-turn confirmation, but it waits for the speech handle to be scheduled and authorized before executing function tools. LiveKit documents `Agent.llmNode` as the customization point for observing LLM chunks and tool calls.

Primary documentation:

* [Preemptive generation and turn completion](https://docs.livekit.io/agents/multimodality/audio/)
* [Turn-taking tuning and preemptive limits](https://docs.livekit.io/agents/logic/turns/tuning/)
* [Pipeline node hooks, including `llmNode`](https://docs.livekit.io/agents/logic/nodes/)
* [`user_input_transcribed` interim/final events](https://docs.livekit.io/reference/agents/events/)
* [Turn-handling options](https://docs.livekit.io/reference/agents/turn-handling-options/)

The documented RAG hook, `on_user_turn_completed`, runs after the turn; it cannot hide retrieval under a long caller utterance. The exact installed SDK source also confirms that preemptive generation starts LLM inference early but defers normal function-tool execution until the speech handle is authorized. This is why the design observes completed search arguments in `llmNode` instead of speculatively executing arbitrary LiveKit tools.

LiveKit documents interim transcript events, but does not document executing
tools from unstable interim text or reconciling those effects when a transcript
is revised. The design therefore does not invent an interim-transcript tool
runner. It uses LiveKit's supported preemptive generation window: a final STT
segment can start the LLM while the caller is still speaking or while end-of-turn
confirmation is pending, and only completed read-only search arguments are
executed early.

LiveKit also documents that preemptive generation is skipped once speech exceeds
`maxSpeechDuration`, whose default is 10 seconds. Production raises that bound to
60 seconds so ordinary long support explanations can still overlap LLM and
search work with the caller's speech. Preemptive TTS remains disabled, retries
remain capped at three, and turns longer than one minute fall back to normal
post-turn generation to bound discarded model/search work.

The worker will therefore observe completed `search_knowledge` tool-call arguments in `llmNode`. During a preemptive generation it may start that read-only search immediately and cache the in-flight promise. When LiveKit later authorizes the tool after end-of-turn, the normal tool execution reuses the exact promise. Speculative transfer, workflow, action, survey-submission, and hangup tools remain prohibited because they have side effects and a preemptive generation can be discarded.

## Search contract

`search_knowledge` accepts one to four standalone queries. Luna supplies, in order when applicable:

1. A concise query in the caller's language.
2. An English equivalent.
3. English help-center title keywords.
4. A positive-only query that omits explicitly excluded products or paths.

The model-generated query array remains the semantic retrieval input. It is not
trusted as the sole workflow-state representation: Luna can occasionally write
a query for the generic step it expects to take next instead of the branch the
caller is actually in. Before preemptive generation begins, the worker therefore
captures a separate bounded `turnContext` from the authorized chat context. It
contains the latest caller utterance, the most recent structured-value length,
and allowlisted call metadata such as caller country. Long numeric values are
replaced with their digit count, and unrelated session metadata is excluded.
`turnContext` stays outside the embedding batch and is used only to focus the
canonical instruction excerpt on the active branch.

The same redaction is enforced on the model-generated query array before the
backend or embedding provider receives it. Prompting Luna to omit a raw
reference is useful, but it is not treated as a privacy boundary: any run of
four or more structured digits becomes a count marker such as
`[10-digit structured value]` while product and branch terms remain intact.

The backend embeds the array in one request, restores provider response order by
embedding index, and searches instruction and knowledge candidates independently
for every embedding. At most two query variants search concurrently to protect
the database pool; all variants are still evaluated. The merge combines
reciprocal rank with cross-query consensus, preserves represented source
categories, deduplicates candidates, and applies a deterministic final rank.
Instruction and knowledge allowlists remain independent and strict. No
selection means all content in that category is searchable.

## Grounding and context

The final rank reserves space for both source categories when both have relevant candidates. A calibrated relevance rule may return `no_match` even when vector search returned nearby documents. Results expose no internal score to the model; score and candidate diagnostics go to telemetry.

The worker keeps only a small per-call cache of read-only searches keyed by normalized query arrays. It does not preload source content. A cached request is single-use and can be consumed only by a tool execution with exactly matching arguments. At most three speculative requests may remain in flight; pending requests are never evicted merely to start more backend work. Discarded preemptive generations cannot update call state. A follow-up query may include the IDs returned by the immediately preceding successfully executed search so the backend can pin those authorized sources into the candidate set. The backend revalidates every pinned ID against the current instruction/knowledge scope and contact segments.

Instruction results are rendered from the current canonical instruction, never
from a stale vector chunk. Secondary instructions contain only the section that
matches the current branch. The primary instruction also contains the nearest
numbered workflow prerequisite: Step 1 may include Step 2, while a later step
includes its immediately preceding step. This preserves required lookup calls
without exposing an unrelated later troubleshooting branch early. A streamed
question is a hard runtime boundary: once the model asks the caller a question,
dependent tool calls later in that generation are suppressed and must wait for
the caller's next turn.

Structured-value ranking recognizes both numeric and spoken-word count phrases
such as “exactly 8 digits” and “eight-digit confirmation.” An exact-count branch
is promoted, while a title that explicitly describes fewer, more, invalid, or
wrong values at that same count is penalized. Result order remains a relevance
hint rather than an authorization boundary: an exact trigger in a later result
overrides an unrelated earlier result, but never permits skipping a prerequisite.

The backend gives embeddings a four-second tail budget inside a 5.5-second
absolute search deadline; the worker aborts at six seconds, leaving transport
headroom. These are ceilings, not delays. Successful K=10 TicketSwap tool calls
remained 721 ms p50 and 1,567 ms p95. The OpenAI Responses adapter also converts
LiveKit's exact function-choice shape to the provider's expected shape so the
forced search contract works on both supported OpenAI API modes.

The multi-query wire change is an intentional backend/worker contract break, so
the voice contract version and deployment guard are bumped together. This keeps
an old worker expecting singular `query` from deploying against a backend that
requires `queries`, and vice versa.

## Search-first enforcement

Before the initial LLM generation for a substantive caller turn, the worker forces `search_knowledge` when the agent has the reserved search tool and no search result follows the latest caller message. Greetings, short acknowledgements, single survey/keypad answers, and farewells are exempt. Long structured values such as reference numbers are substantive. After the search result, normal automatic tool selection resumes.

## Workflow reliability

Search hardening ships first and the focused Luna evals are rerun before workflow changes. Remaining authentication, troubleshooting, and survey failures must be reproduced by focused tests. Reusable state belongs in worker/backend orchestration, not org-specific prompt branches. Tool results should expose a typed next action, required input, and attempts remaining where the underlying workflow contract can support them. A survey controller owns the active question, answer binding, handoff lock, and exactly-once submission.

## Output and harness reliability

Spoken digit sequences, telephone numbers, and initialisms use deterministic formatting before TTS. Terminal tool-only turns are recorded as valid tool outcomes by the eval harness; latency evaluation does not require a streamed sentence when the turn is intentionally terminal.

The production-derived MoneyGram scenarios exposed three harness defects that
were not model failures: caller-role wording was too narrow, confirmation was
checked after field-entry prompts, and the C V V evaluator rejected the natural
compact spelling. Those parsers now accept production-equivalent wording. The
remaining ten-digit failures were a real retrieval/context gap: the correct
instruction reached the model, but a model-generated query and a broad secondary
instruction could advance to sender/receiver troubleshooting before the exact
money-order/money-transfer branch completed. Separate turn context plus
section-scoped instruction output raised that focused Luna gate from 7/10 to
10/10 (`voice-search-moneygram-ten-digit-luna-k10-v6-20260826`).

## Verification

Every production change follows red-green-refactor. Unit and focused integration tests run without starting, stopping, or killing external workers. Eval order is focused Luna K=1, full Luna K=1, focused Luna K=10, then the Luna-only K=10 gate. All gating scenarios must pass 100%.