> ## Documentation Index
> Fetch the complete documentation index at: https://docs.open.cx/llms.txt
> Use this file to discover all available pages before exploring further.

# 2026 08 24 voice knowledge search design

# Voice Knowledge Search Design

## Goal

Replace voice-agent knowledge preloading with search-first retrieval while preserving strict, independently configurable access boundaries for work instructions and knowledge-base content. The voice worker remains network-isolated: it receives scope metadata at bootstrap and performs retrieval only through the internal voice HTTP API.

## Product semantics

Instructions and knowledge are separate optional allowlists. An empty selection means search everything in that category; one or more selections restrict only that category.

| Instruction selection | Knowledge selection | Searchable corpus                                    |
| --------------------- | ------------------- | ---------------------------------------------------- |
| None                  | None                | All eligible instructions and all eligible knowledge |
| Some                  | None                | Selected instructions and all eligible knowledge     |
| None                  | Some                | All eligible instructions and selected knowledge     |
| Some                  | Some                | Selected instructions and selected knowledge         |

No instruction or knowledge content is preloaded in this version. A later optimization may replace search with preload when an allowed corpus is proven small, but that optimization must preserve the same scope semantics and eval contracts.

Per-call dynamic instructions remain a separate explicit mechanism. They continue to be appended to the call prompt when requested and do not alter either search allowlist.

## Scope model and persistence

Instruction scope reuses the existing explicit selections:

* `phone_agent_instructions` for individual instructions.
* `phone_agent_selected_directories` for an instruction directory and all current or future descendants.
* No rows in either table means `all`; any row means `selected`.

Knowledge scope gains two equivalent selection forms:

* Individual knowledge-base item selections, with foreign keys to the agent and item.
* Virtual knowledge-folder selections, using the canonical folder identity already used by the knowledge browser. A folder selection covers current and future items classified into that folder.
* No knowledge item or folder rows means `all`; any row means `selected`.

The canonical folder-key utility is shared by the knowledge browser, dashboard selector, validation, and retrieval filter so a displayed folder cannot resolve differently at search time. New selection tables use cascading foreign keys where a physical record exists, tenant-scoped validation on every write, unique constraints, and a migration with the required `lock_timeout`.

Existing agents need no data migration. Existing instruction selections become a selected instruction scope; absence of knowledge selections means all eligible knowledge is searchable. The legacy hidden `use_org_knowledgebase` control does not define the new scope.

## Bootstrap and runtime contract

Bootstrap stops assembling selected instruction content. Its prompt contains only the agent's base instructions, explicitly requested dynamic instructions, and existing per-call additions.

Bootstrap returns and persists a scope snapshot shaped around two independent categories:

* instruction mode plus explicit instruction and directory identifiers;
* knowledge mode plus explicit item and folder identifiers;
* capture timestamp.

The snapshot is for audit and eval assertions. Authorization is re-evaluated on every search so a mid-call configuration change fails closed instead of relying on stale bootstrap state.

The worker always registers `search_knowledge` for a resolved phone agent. The tool calls a typed endpoint under `/internal/voice/v1/sessions/:sessionId/tools/knowledge-search` with the organization, agent, and a short standalone query. The backend verifies service authentication, session ownership, agent ownership, and that the session belongs to the agent before resolving contact-aware scope.

This is a breaking internal wire change, so the voice runtime contract version, generated worker client, deploy preflight pins, and contract guard tests change together.

## Retrieval pipeline

The backend owns the complete retrieval path:

1. Resolve the current instruction and knowledge modes independently.
2. Expand selected instruction directories and selected knowledge folders.
3. Apply organization, public visibility, phone-channel, contact segment, draft/deletion, and active-time restrictions before candidate limiting.
4. Create one embedding for the tool query.
5. Search instruction mirrors and knowledge chunks concurrently, retaining comparable vector distances.
6. Merge and rank both result types into one precision-oriented top-k response.

Instruction matches are authoritatively joined back to published instruction rows before return; stale mirrors cannot expose draft, expired, wrong-channel, wrong-segment, or foreign instructions. Knowledge allowlist predicates are pushed into the database query before the nearest-neighbor limit so a small selected folder is not crowded out by forbidden high-scoring candidates.

Each result contains a stable type, source identifier, chunk index where applicable, title, bounded content, and optional source URL. The model does not need raw vector distances, but backend telemetry records the query, both scope modes, candidate counts, returned IDs, latency, and result status.

There is no query-generation model or reranker in the first version. The voice model receives the call history and is instructed to submit a standalone search query containing the relevant prior-turn context. Tool outputs remain in the agent session context. Results are deliberately few and content is truncated to bound both time-to-first-response and context growth.

The backend owns a deadline shorter than the worker's request deadline. A genuine miss returns `no_match`; backend or network failure returns an unavailable outcome from the worker wrapper. Both outcomes tell the agent not to invent facts and to use its existing instructions or offer the configured fallback.

## Dashboard configuration

The existing phone-agent Knowledge Sources section becomes a search-scope selector with two independently expandable root groups:

* **Instructions** — the current instruction directory tree and individual instructions.
* **Knowledge** — knowledge-source folders with lazy, paginated item loading for large corpora.

Each root displays `All searchable` when it has no selection and `Restricted` with a selected count otherwise. Selecting the first child visibly changes that category to restricted and shows concise help text explaining that only checked content will be searched. Clearing the last selection restores `All searchable`.

Folder selections include current and future descendants/items. Individual selections outside a selected folder remain explicit. The API returns explicit selections separately from effective inherited state so saving cannot accidentally make descendants permanent.

The existing preload token meter and memory-cap warnings are removed because selected content no longer enters the initial prompt. The selector uses native checkboxes, semantic group labels, keyboard-operable disclosure controls, visible focus states, and text in addition to color for `All searchable`, `Restricted`, inherited, unsaved, loading, and failure states. Knowledge items load only when their folder opens; loading and save failures remain recoverable without losing local selection state.

## Eval and test coverage

### Deterministic contracts

* All four independent scope combinations in the product-semantics table.
* Directory/folder descendant inheritance, future-item inclusion, individual selection, and clearing the final selection back to `all`.
* Cross-organization rejection on dashboard writes and runtime searches.
* Same-query contact pair proving segment-member inclusion and non-member exclusion by exact result ID.
* Draft, deleted, expired/not-yet-active, private, and wrong-channel exclusions.
* Unified top-k ordering across near-duplicate instruction and knowledge results.
* No bootstrap preloading in either scope mode.
* Search timeout, upstream failure, empty embedding, and no-match behavior.
* Worker tool serialization, typed HTTP calls, and contract-version/deploy guards.
* Dashboard selection utilities, lazy folder loading, independent dirty-state tracking, keyboard controls, and accessible names/status text.

### Voice behavior evals

Port the existing Foodics knowledge scenarios onto the production-shaped HTTP bootstrap and search boundary used by the MoneyGram voice bench. Every scenario asserts both end behavior and exact retrieval/tool evidence:

* Production-grounded English how-to retrieval.
* Near-duplicate product discrimination.
* Arabic query retrieving English source material and answering in Arabic.
* Honest handling of a corpus miss.
* Multi-turn follow-up whose second query must carry context from the first turn or correctly reuse the retained result.
* Strict instruction allowlist plus unrestricted knowledge.
* Strict knowledge allowlist plus unrestricted instructions.
* Segment-member/non-member pair with a hard no-leak assertion.
* Conflicting or stale synthetic documents where production data cannot provide a stable fixture.

Run new scenarios at K=1 while developing, then K=10 for hard assertions before certification. Retrieval IDs, tool inputs/outputs, scope snapshots, latency, and priced transcript artifacts are saved for diagnosis.

A real participant-audio knowledge scenario is added to the opt-in provider-roundtrip suite when the local voice stack is available. It proves audio to transcription to tool call to spoken answer, while the merge gate keeps the less flaky text-turn semantic scenarios that still use the real voice model and real HTTP retrieval boundary.

## Rollout and observability

The stacked PR targets the network-isolation branch until that branch merges, then retargets to `main`. Deployment requires the backend contract first and the existing readiness preflight before workers roll.

Operational telemetry distinguishes `ok`, `no_match`, `restricted_empty`, `timeout`, and `unavailable`, and records instruction/knowledge scope modes plus returned identifiers. It must never record full private document bodies in span attributes.

## Non-goals

* No automatic preload threshold in this PR.
* No query-generation model, semantic reranker, or persistent server-side conversation memory.
* No dashboard control for making a category search nothing; an empty selection intentionally means all.
* No direct database, embedding-provider, or backend module import from the voice worker.
* No silent fallback from a selected category to the organization's unrestricted corpus.