Back to releases
v1.4.0

Retrieval telemetry and the knowledge health loop — maintaining a knowledge base by numbers (v1.4.0)

Every search's citation lineage is persisted asynchronously (never slowing retrieval), query text is opt-in, and a new "Health" tab shows empty-recall rate, zero-recall queries, sleeping content and a unified content activity feed — plus a pre-deletion citation warning, a force-recrawl entry point, and freshness metadata on search results. "Is my knowledge base still healthy?" goes from guesswork to a verifiable loop.

TelemetryKnowledge operationsWeb ConsoleREST

The biggest enemy of an enterprise knowledge base was never the setup — it’s maintenance: content silently goes stale, and nobody knows what to update or delete. v1.4.0 invents no new detection: content fingerprints for web sources, incremental cursors for Feishu, revision history for local files — those facts were already persisted. What it adds is the other half of the loop: recording how content gets used, and aggregating that into a panel administrators can read, act on, and verify.

Record: what each search returned is no longer thrown away

Every evidence row returned to a caller (down to the chunk-revision level, with its rank) is now recorded asynchronously. Writes go through a bounded in-process buffer with batched inserts: when the buffer is full, batches are dropped and counted (exported as a metric), and retrieval is never slowed down — fault-injection tests prove the search response is byte-identical even when the whole telemetry path fails.

Query text is opt-in: off by default; only after a knowledge-base admin explicitly enables it are queries recorded (30-day retention, admin-only visibility, revocable anytime). SearchRun metadata stays query-free — the privacy boundary established in v0.9.0 remains untouched.

Insight: the Health tab answers four questions

A sixth tab joins the workbench — four cards and three panels:

  • Empty-recall rate (the north-star metric): what fraction of searches come back empty, and whether that’s improving — with an explicit scope: console test traffic excluded, failures excluded from the denominator;
  • Top zero-recall queries: what users keep asking that the base can’t answer, ranked by frequency; with recording enabled you see the exact phrasing and jump straight into FAQ creation with the question pre-filled;
  • Sleeping content: documents indexed 30+ days ago that never appeared in any search result — an archive shortlist. If the window has fewer than 10 completed searches, the list is withheld (a zero-traffic base shouldn’t have all its content mislabeled as sleeping);
  • Content activity: source syncs, new content versions, manual chunk edits and deletions — one timeline across web, Feishu, local files and FAQ sources.

Action and guardrails: every signal leads somewhere, and deletion checks citations first

  • Zero recall → create an FAQ (question pre-filled with the query text);
  • Sleeping content → inspect; deletion always goes through the existing flow, but the confirmation dialog now tells you “this content was returned in N searches in the last 30 days” — relied-upon content no longer disappears silently;
  • Web sources gain a “force recrawl” entry (ignores change detection, full re-fetch, with a cost confirmation);
  • Search results (REST and MCP) expose document-updated and page-fetched timestamps, letting downstream agents judge evidence freshness themselves.

Adversarial review of ourselves

This release itself went through a full adversarial code review: on code that was complete with all tests green, the review caught a fatal wiring defect (the cleanup loop accepted a parameter but never called it — query text would have been retained forever), plus a batch of correctness and performance issues, all fixed before tagging. No known leftovers.

Upgrade and you’re done: standard migrations, no breaking protocol changes; query-text recording ships off by default, so no configuration is required.