Skip to main content

Voice Knowledge Search Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.
Goal: Give network-isolated voice agents search-first access to independently allowlisted instructions and knowledge, expose both scopes in the dashboard, and certify end-to-end behavior with production-grounded evals. Architecture: The backend resolves and enforces both allowlists, embeds each query once, searches instruction mirrors and knowledge chunks concurrently, and returns one ranked top-k result through the internal voice HTTP contract. The worker only shapes the search_knowledge tool and forwards calls. The dashboard stores explicit instruction and knowledge selections while empty selections mean all independently. Tech Stack: TypeScript, Fastify, Zod/OpenAPI, Kysely/PostgreSQL/pgvector, LiveKit Agents, React/TanStack Router, Vitest, OpenCX agent-eval harness. Spec: docs/superpowers/specs/2026-08-24-voice-knowledge-search-design.md

Global Constraints

  • The voice worker imports no backend modules and reaches platform state only through /internal/voice/v1.
  • No instruction or knowledge content is preloaded; empty selection means all independently for each category.
  • Selected scopes fail closed and never fall back to unrestricted organization content.
  • Every search enforces organization, public visibility, phone channel, contact segment, draft/deletion, and active-time boundaries before candidate limiting.
  • Retrieval uses one embedding with no query-generation model or reranker.
  • Every Kysely migration up() and down() begins with SET LOCAL lock_timeout = '10s'.
  • Handwritten code remains strictly typed: no any, no unknown in MCP code, and no inline casts other than as const.
  • Any change under backend/voice-agent or backend/src/voice-runtime requires cd backend && pnpm voice:test before completion.
  • Public copy describes outcomes and never names internal models, providers, infrastructure vendors, or libraries.

Task 1: Persist explicit knowledge selections

Files:
  • Create: backend/src/db/kysely-migrations/20260824160000_add_phone_agent_knowledge_scopes.ts
  • Create: backend/src/phone/phone.knowledge-scope-persistence.spec.ts
  • Modify (generated): backend/src/db/opencx.ts
Interfaces:
  • Produces phone_agent_knowledgebase_items(id, phone_agent_id, knowledgebase_item_id).
  • Produces phone_agent_knowledgebase_folders(id, phone_agent_id, folder_key).
  • Agent/item foreign keys cascade and both agent-selection pairs are unique.
  • Step 1: Write the failing schema contract test
  • Step 2: Run RED
Run: cd backend && pnpm vitest run src/phone/phone.knowledge-scope-persistence.spec.ts Expected: type/schema failure because both tables are absent.
  • Step 3: Add the migration and regenerate types
Implement up()/down() with lock timeout, UUID keys, cascading FKs, and unique constraints. Run cd backend && NODE_ENV=test pnpm dev:prepare.
  • Step 4: Run GREEN and commit
Run: cd backend && pnpm vitest run src/phone/phone.knowledge-scope-persistence.spec.ts Commit: feat(backend, voice): persist knowledge scopes with only the migration, generated DB type, and focused test staged.

Task 2: Canonicalize knowledge-folder identities

Files:
  • Create: backend/src/rag/knowledgebase/knowledgebase-folder.utils.ts
  • Create: backend/src/rag/knowledgebase/knowledgebase-folder.utils.spec.ts
  • Modify: backend/src/airbyte/knowledgebase-items.service.ts
  • Modify: backend/src/airbyte/knowledgebase-items.service.spec.ts
Interfaces:
  • Produces KnowledgebaseFolderIdentity = { dataSource: KnowledgebaseDataSourceEnum; stream: string | null }.
  • Produces static KnowledgebaseFolderUtils.toKey, fromKey, and fromItem methods.
  • Step 1: Write table-driven RED tests with literal expectations
Run: cd backend && pnpm vitest run src/rag/knowledgebase/knowledgebase-folder.utils.spec.ts Expected: missing module/export.
  • Step 2: Implement and reuse the static utility
Use URL encoding for stream delimiters, return null for malformed keys, and replace the browser service’s duplicate website/workflow/integration classification.
  • Step 3: Run GREEN and commit
Run: cd backend && pnpm vitest run src/rag/knowledgebase/knowledgebase-folder.utils.spec.ts src/airbyte/knowledgebase-items.service.spec.ts src/airbyte/knowledgebase-items.workflow-folders.spec.ts Commit: refactor(backend, knowledge): share folder identity.

Task 3: Extend phone-agent scope configuration APIs

Files:
  • Modify: backend/src/phone/dto/knowledge-sources.dto.ts
  • Modify: backend/src/phone/phone.service.ts
  • Create: backend/src/phone/phone.knowledge-scope-selection.spec.ts
  • Modify: backend/src/phone/phone.controller.ts
  • Modify: backend/src/public-api/controllers/phone.public.controller.ts
  • Modify: backend/src/public-api/controllers/phone.public.controller.spec.ts
  • Modify (generated): backend/tmp/public-openapi.json
  • Modify (generated): backend/mcp-server/src/schema.ts
  • Modify: backend/mcp-server/src/tools/phone.ts
  • Modify: backend/mcp-server/src/tools/phone.spec.ts
Interfaces:
  • Extends update input with optional knowledgeItemIds/knowledgeFolderKeys; omission leaves knowledge rows unchanged for older clients.
  • Adds instruction and knowledge modes plus explicit IDs/folder summaries to KnowledgeSourceTreeDto while retaining existing instruction fields.
  • Adds paginated knowledge-item listing by canonical folder key.
  • Changes PhoneService.updateAgentKnowledgeSources to one object input and one transaction.
  • Step 1: Write RED service/controller tests
Use a literal four-row matrix for all/all, selected/all, all/selected, and selected/selected. Also assert foreign item/folder selection is rejected and a legacy two-array request does not clear knowledge rows. Run: cd backend && pnpm vitest run src/phone/phone.knowledge-scope-selection.spec.ts src/public-api/controllers/phone.public.controller.spec.ts Expected: new fields/modes and signature are absent.
  • Step 2: Implement transactional DTO/service/controller behavior
Validate agent, directory, instruction, item, and folder ownership before writes. Lazy item listing returns IDs/titles only and filters public, non-deleted, voice-eligible rows.
  • Step 3: Run GREEN
Run: cd backend && pnpm vitest run src/phone/phone.knowledge-scope-selection.spec.ts src/phone/phone.knowledge-sources.nested-directories.spec.ts src/public-api/controllers/phone.public.controller.spec.ts
  • Step 4: Regenerate public types and update MCP
Run cd backend && pnpm gen:spec, then cd mcp-server && pnpm gen:types:offline. Extend the MCP tool with optional knowledge_item_ids/knowledge_folder_keys, forwarding only supplied arrays. Run cd backend && pnpm --filter @opencx/mcp build && pnpm --filter @opencx/mcp test.
  • Step 5: Commit
Commit: feat(backend, voice): configure search scopes with source, generated schemas, and tests staged explicitly.

Task 4: Replace bootstrap preloading with scope snapshots

Files:
  • Modify: backend/src/voice-channel/voice-agent-routing.service.ts
  • Modify: backend/src/voice-channel/voice-agent-routing.service.spec.ts
  • Modify: backend/src/voice-runtime/dtos/bootstrap.dto.ts
  • Modify: backend/src/voice-runtime/dtos/events.dto.ts
  • Modify: backend/src/voice-channel/voice-call-session.service.ts
  • Modify: backend/src/chat-session/dtos/chat-session.dto.ts
  • Modify: backend/src/voice-runtime/services/voice-bootstrap.service.ts
  • Modify: backend/src/voice-runtime/__tests__/voice-bootstrap.e2e.spec.ts
Interfaces:
  • Produces VoiceAgentRoutingService.getKnowledgeScope({ agentId, contactId }).
  • Snapshot: { instruction_scope: { mode, instruction_ids, directory_ids }, knowledge_scope: { mode, item_ids, folder_keys }, captured_at }.
  • Bootstrap excludes selected content while explicit dynamic instructions remain appended.
  • Step 1: Write RED routing/bootstrap tests
Add knowledge-selected/instructions-all and dynamic-instruction cases. Run cd backend && pnpm vitest run src/voice-channel/voice-agent-routing.service.spec.ts src/voice-runtime/__tests__/voice-bootstrap.e2e.spec.ts. Expected: selected content is still preloaded and new fields are missing.
  • Step 2: Implement no-preload scope resolution
Determine mode from raw explicit rows, keep explicit directory/folder selectors for audit, and resolve contact-eligible selected instruction/item IDs separately. Preserve dynamic instruction assembly.
  • Step 3: Run GREEN and commit
Run: cd backend && pnpm vitest run src/voice-channel/voice-agent-routing.service.spec.ts src/voice-runtime/__tests__/voice-bootstrap.e2e.spec.ts Commit: feat(backend, voice): bootstrap search scopes.

Task 5: Build scoped, ranked backend retrieval

Files:
  • Modify: backend/src/rag/knowledgebase/knowledgebase.ts
  • Create: backend/src/rag/knowledgebase/knowledgebase.allowlist.spec.ts
  • Create: backend/src/voice-runtime/services/voice-knowledge-search.service.ts
  • Create: backend/src/voice-runtime/services/voice-knowledge-search.service.spec.ts
Interfaces:
  • Extends KnowledgebaseService.search with strict sourceRecordIds, allowedItemIds, and allowedFolderKeys; undefined is unrestricted and [] returns no results.
  • Produces VoiceKnowledgeSearchService.search({ orgId, agentId, contactId, query }).
  • Results are { type, id, chunkIndex, title, content, sourceUrl }, ranked together by distance and capped at five.
  • Step 1: Write RED access-filter tests
Seed a closer forbidden vector and farther allowed vector, then assert the allowed one survives limit: 1. Cover empty scope, segment member/non-member, draft, deleted, scheduled, private, wrong-channel, and foreign rows with literal IDs. Run: cd backend && pnpm vitest run src/rag/knowledgebase/knowledgebase.allowlist.spec.ts Expected: new search parameters are absent or the forbidden row wins.
  • Step 2: Implement SQL predicates before ANN limiting
Apply filters to embedding and include-by-ID queries; immediately return [] for explicitly empty scopes. Run the allowlist test plus existing segment/channel/schedule/deletion specs.
  • Step 3: Write RED service tests
Use real DB vectors and mock only the external embedding response with its complete shape. Test independent scopes, authoritative instruction status filtering, distance merge across result types, truncation, no match, and telemetry-safe IDs. Run: cd backend && pnpm vitest run src/voice-runtime/services/voice-knowledge-search.service.spec.ts Expected: module missing.
  • Step 4: Implement one-embedding concurrent retrieval
Query instruction mirrors and normal knowledge concurrently; re-read matched instruction rows with TrainingService.getAiInstructionsByIds, merge comparable distances, cap to five, and never put content in span attributes.
  • Step 5: Run GREEN and commit
Run: cd backend && pnpm vitest run src/voice-runtime/services/voice-knowledge-search.service.spec.ts src/rag/knowledgebase/knowledgebase.allowlist.spec.ts src/rag/knowledgebase/knowledgebase.segments.spec.ts src/rag/knowledgebase/knowledgebase.channel-schedule.spec.ts src/rag/knowledgebase/knowledgebase.soft-deleted-excluded.spec.ts Commit: feat(backend, voice): search scoped knowledge.

Task 6: Expose the internal voice search endpoint

Files:
  • Create: backend/src/voice-runtime/dtos/knowledge-search.dto.ts
  • Modify: backend/src/voice-runtime/voice-call-ops.controller.ts
  • Create: backend/src/voice-runtime/__tests__/voice-knowledge-search.e2e.spec.ts
  • Modify: backend/src/voice-runtime/contract-version.ts
  • Modify: backend/voice-agent/src/__tests__/deploy-workflow.guard.spec.ts
  • Modify: .github/workflows/voice-agent-deploy.yml
Interfaces:
  • Adds voiceSearchKnowledge at POST /internal/voice/v1/sessions/{sessionId}/tools/knowledge-search.
  • Request: { orgId, agentId, query }, trimmed to 1–500 characters.
  • Response status: ok | no_match | restricted_empty, plus both modes and typed results.
  • Step 1: Write RED real-HTTP tests
Assert exact seeded IDs for a valid session, 404 for foreign org/agent/session, 400 for blank query, and a bounded backend deadline. Run: cd backend && pnpm vitest run src/voice-runtime/__tests__/voice-knowledge-search.e2e.spec.ts Expected: route 404.
  • Step 2: Implement guarded route and deadline
Use requireSession and requireAgent, require the session’s phone agent to equal agentId, and pass the session contact ID to search.
  • Step 3: Bump/pin contract and run GREEN
Use one new version literal in backend, deploy guard, and both deploy readiness checks. Run runtime/contract/deploy-guard tests.
  • Step 4: Commit
Commit: feat(backend, voice): expose knowledge search.

Task 7: Register the isolated worker search tool

Files:
  • Modify (generated): backend/tmp/voice-openapi.json
  • Modify (generated): backend/voice-agent/src/api/schema.ts
  • Modify: backend/voice-agent/src/api/voice-backend-api.ts
  • Modify: backend/voice-agent/src/api/voice-backend-api.spec.ts
  • Create: backend/voice-agent/src/tools/knowledge.tools.ts
  • Create: backend/voice-agent/src/tools/knowledge.tools.spec.ts
  • Modify: backend/voice-agent/src/voice-agent-dynamic-imports.ts
  • Modify: backend/voice-agent/src/voice-agent-dynamic-imports.spec.ts
  • Modify: backend/voice-agent/src/index.ts
  • Modify: backend/voice-agent/src/testing.ts
Interfaces:
  • Produces VoiceBackendApi.searchKnowledge with a four-second client deadline.
  • Produces getKnowledgeTools({ orgId, agentId, sessionId }) and search_knowledge.
  • Tool preserves exact IDs; corpus miss and unavailability have distinct honest-answer guidance.
  • Step 1: Write RED API/tool tests
Test success serialization, no_match guidance containing do not guess, unavailable guidance on rejected HTTP, and the exact timeout passed by the façade. Run: cd backend && pnpm --filter @opencx/voice-agent test src/api/voice-backend-api.spec.ts src/tools/knowledge.tools.spec.ts Expected: API method/tool missing.
  • Step 2: Regenerate and implement
Dump the internal spec, run cd backend/voice-agent && pnpm gen:client:offline, implement the API/tool, add a literal dynamic-import thunk to the prewarm list, and merge the tool for every resolved agent.
  • Step 3: Run GREEN, isolation, type, and build checks
Run: cd backend && pnpm --filter @opencx/voice-agent test && pnpm --filter @opencx/voice-agent test:isolation && pnpm --filter @opencx/voice-agent tsgo && pnpm --filter @opencx/voice-agent build
  • Step 4: Commit
Commit: feat(voice-agent): add knowledge search tool.

Task 8: Build the two-folder dashboard selector

Files:
  • Modify: dashboard/apps/dashboard/app/(authenticated)/(dashboard)/(channels)/channels/phone/_components/KnowledgeSourcesSelector/selection.utils.ts
  • Modify: dashboard/apps/dashboard/app/(authenticated)/(dashboard)/(channels)/channels/phone/_components/KnowledgeSourcesSelector/selection.utils.test.ts
  • Modify: dashboard/apps/dashboard/app/(authenticated)/(dashboard)/(channels)/channels/phone/_components/KnowledgeSourcesSelector/index.tsx
  • Create: dashboard/apps/dashboard/app/(authenticated)/(dashboard)/(channels)/channels/phone/_components/KnowledgeSourcesSelector/index.test.tsx
  • Modify: dashboard/apps/dashboard/app/(authenticated)/(dashboard)/(channels)/channels/phone/_components/ModifyAIPhoneAgent/ModifyAIPhoneAgent__Form.tsx
  • Modify (generated): dashboard/packages/sdk/src/schema.ts
Interfaces:
  • Selection utilities track instruction IDs/directories separately from knowledge item IDs/folder keys.
  • Component ref remains save, hasChanges, fetchTree.
  • Roots expose All searchable or Restricted · N selected as visible text.
  • Step 1: Write RED utility tests
Assert zero means all, first checked child restricts only its category, instruction changes do not dirty knowledge, inherited rows stay implicit, and clearing the final selection restores all. Run: cd dashboard && pnpm --filter @open/dashboard test -- KnowledgeSourcesSelector/selection.utils.test.ts Expected: knowledge selection API absent.
  • Step 2: Implement pure state utilities and run GREEN
Keep set arithmetic outside React. Rerun the focused utility test.
  • Step 3: Write RED component tests
Render the real component with API handlers. Assert named keyboard-operable root disclosures, both statuses, lazy folder item loading, independent status change, and retained local checks after a save failure. Run: cd dashboard && pnpm --filter @open/dashboard test -- KnowledgeSourcesSelector/index.test.tsx Expected: roots/lazy UI absent.
  • Step 4: Regenerate SDK and implement the component
Run pnpm gensdk. Replace token warnings with scope summaries; use native checkboxes, existing Collapsible primitives, visible focus, text states, and folder-level lazy pagination.
  • Step 5: Run GREEN, typecheck, lint, and commit
Run focused selector tests, cd dashboard && pnpm --filter @open/dashboard type-check, and focused lint. Commit: feat(dashboard, voice): configure search scopes.

Task 9: Port and expand production-grounded evals

Files:
  • Create: backend/src/agent-evals/benches/foodics-voice-knowledge.bench.spec.ts
  • Modify: backend/src/agent-evals/benches/foodics-voice.bench.spec.ts
  • Modify: backend/src/agent-evals/gate-runner-config.ts
  • Modify: backend/src/agent-evals/gate-runner-config.spec.ts
  • Create: backend/src/agent-evals/research/foodics-voice-knowledge-search.md
  • Modify: backend/src/voice-e2e/provider-roundtrip.e2e.spec.ts
  • Modify: backend/scripts/voice-smoke/provider-roundtrip-helpers.ts
Interfaces:
  • FoodicsKnowledgeCall.start uses RealBackendTestServer, VoiceBackendApi.bootstrap, and getKnowledgeTools.
  • Artifacts include scope snapshot, tool input/output, exact IDs, latency, and priced usage.
  • Both voice gate targets include moneygram-voice and foodics-voice-knowledge.
  • Step 1: Write/run RED gate-config test
Expect both target bench arrays to equal ['moneygram-voice', 'foodics-voice-knowledge']. Run the focused config spec and observe failure.
  • Step 2: Port production-grounded cases at K=1
Port Foodics Waiter code, KDS near-duplicate, Arabic-to-English cashier PIN, and drone-delivery corpus miss through real HTTP bootstrap/search. Assert exact retrieval IDs before judging the spoken answer.
  • Step 3: Add K=1 gap cases
Add multi-turn follow-up, selected instructions plus all knowledge, selected knowledge plus all instructions, segment member/non-member no-leak pair, and stable synthetic conflict/staleness when production data lacks the edge.
  • Step 4: Add deterministic no-preload and latency contracts
Against the restored 3,143-item corpus, assert zero bootstrap preload and each repeated search completes under three seconds with expected IDs.
  • Step 5: Run K=1 and repair only evidence-backed failures
Run cd backend && AGENT_EVALS=1 EVAL_K=1 pnpm vitest run src/agent-evals/benches/foodics-voice-knowledge.bench.spec.ts and require an artifact per test.
  • Step 6: Add opt-in audio knowledge roundtrip
Reuse participant-audio publication and provider attribution; assert user transcript, search tool evidence, grounded assistant row, and post-turn audio under existing provider prerequisites.
  • Step 7: Run both voice targets at K=10 and commit
Run the gate runner for both target models, record cost/latency, and classify failures from IDs/traces. Commit: test(backend, voice): gate knowledge search.

Task 10: Verify, review, repair, and open the draft PR

Files:
  • Review every file changed from base 5f10cfcf674eca5e771be8b6c2663397e938cdc2.
  • Modify only files required by failing verification or concrete review findings.
Interfaces:
  • Pushes m-tabaza/feature/voice-knowledge-search-v2.
  • Opens a draft PR based on m-tabaza/feat/voice-agent-network-isolation until #2260 merges.
  • Step 1: Run focused verification
Run backend pnpm tsgo, MCP build/tests, worker type/unit/isolation/build, dashboard type/focused tests, and every touched backend spec.
  • Step 2: Run mandatory full voice verification
Run: cd backend && pnpm voice:test Expected: PASS across worker unit/isolation and backend/runtime/worker-spawning voice suites.
  • Step 3: Run formatting/lint/diff checks
Run pnpm fmt:check, pnpm lint, git diff --check, and inspect generated-file drift.
  • Step 4: Review for production readiness
Inspect the full base diff for tenant leaks, selected-to-all fallback, wrong empty-array semantics, stale scope use, unbounded response/context, timeout mismatch, direct worker backend imports, permission gaps, public compatibility, migration lock timeout, contract drift, inaccessible UI state, and eval assertions that test mocks.
  • Step 5: Fix each concrete issue through RED/GREEN
For every finding, first add a focused failing regression test, observe the expected failure, make the smallest fix, then rerun the affected suite/type/lint checks. Commit independent fixes atomically.
  • Step 6: Push and open the draft PR
Push the branch, then run gh pr create --draft --repo opencx-labs/opencx --base m-tabaza/feat/voice-agent-network-isolation --head m-tabaza/feature/voice-knowledge-search-v2 --title "feat: add voice knowledge search" --body-file /tmp/voice-knowledge-search-pr.md. The PR body includes independent scope semantics, no-preload behavior, network boundary, eval results, exact verification commands, stacked dependency on #2260, and remaining non-gating risks.