Make an LLM talk about the real world without making things up

---
name: grounded-generation
description: Make an LLM talk about the real world without making things up — data layer asserts facts with provenance, the model only phrases them, unsourced/stale claims are stripped, and it degrades to typed silence instead of inventing. Use when an LLM speaks about facts with a checkable ground truth (weather, place, prices, live data).
---

# Implement: grounded generation (no hallucinated facts)

You are stopping a developer's LLM feature from inventing real-world facts.
Implement the discipline below in their codebase. The core inversion: the model is
a PHRASER over facts a data layer asserts — it never originates a fact. Adapt to
their language, their data sources, and their model.

## Detect their setup

Where real facts enter (which APIs / data sources), where the model call is, and
whether facts currently reach the model as structured values or already baked into
a prose prompt (if baked-in, that's the first thing to fix — the model can't be
gated on provenance it can't see).

## Build this

1. **Structured facts with provenance.** Make the data layer return each fact as
   `{ value, source, observedAt, confidence }`, not prose. A fact missing any of
   source/observedAt/confidence does not pass to the model.

2. **The grounding gate** — classify every fact GROUNDED / SILENT / VIOLATION and
   block violations before the model. Includes the cardinal-sin check (high
   confidence + no source) and staleness (past its TTL). See
   `reference/grounding-gate.ts`.

3. **Typed silence.** Absence is `{ silent: true, reason }`, handled deliberately
   by the rendering layer — never invent to fill it, never error.

4. **Freshness.** Each fact declares a TTL (producer owns it); strip stale facts
   before the model sees them; conservative default TTL for unknown sources.

5. **Model phrases only.** Prompt the model to reword the given facts and add
   nothing. Then run cheap deterministic gates on its OUTPUT to catch leaks: raw
   null/placeholder strings, internal vocabulary, generic template fallbacks, and
   internal inconsistency (one field contradicting another). Score it.

6. **Degrade to silence.** Timeout / thin data / source down → silence-with-reason
   for that axis. Wrap flaky sources in a circuit breaker (open after N failures,
   cool down, probe).

7. **The replay loop.** Snapshot the raw facts behind every generation. When
   accuracy is questioned, replay the snapshot and grade it against current rules
   across the population — never hand-patch from one anecdote.

8. **A grounding metric.** `(grounded + silent) / (grounded + silent + violations)`
   — silence counts as integrity.

## Guardrails to enforce

- No fact reaches the model without provenance; no high-confidence claim ships
  without a source.
- Silence is a designed response, never an error or a filled-in default.
- Accuracy fixes are evidence-driven (replay + grade), never anecdote-driven.
- When a claim class stops correlating with reality, demote it to silence — don't
  tune the threshold.

## Verify before done

Force a source offline and confirm the feature goes silent-with-reason (not stale,
not invented). Feed a fact with no source and confirm it's blocked. Confirm the
output gates catch an injected fabrication. Confirm a stale fact is stripped.

## Read alongside

`RECIPE.md` (the why + receipts) and `reference/grounding-gate.ts`.

// Reference implementation — the grounding gate (steps 2, 4, 8) + the grade
// primitive (step 7). Provider-neutral. Adapt to your fact shape; don't copy blind.
//
// Core idea: classify every fact as GROUNDED, SILENT, or VIOLATION. Only GROUNDED
// facts reach the model. SILENT is honest absence. VIOLATION never ships and is
// what you measure against.

export interface Fact {
  value: unknown;
  source?: string;
  observedAt?: number;   // epoch ms
  confidence?: number;   // 0..1
  ttlMinutes?: number;   // producer-declared freshness; falls back to default
  silentReason?: string; // set => a typed absence, not a claim
}

export type Verdict =
  | { kind: 'GROUNDED'; fact: Fact }
  | { kind: 'SILENT'; reason: string }
  | { kind: 'VIOLATION'; reason: string; cardinal: boolean };

const CARDINAL_CONFIDENCE = 0.7;   // confidence >= this with no source = impossible
const DEFAULT_TTL_MINUTES = 1440;  // conservative: unknowns are NOT spoken as fresh

export function classify(fact: Fact, now = Date.now()): Verdict {
  // Typed absence is first-class integrity, not a failure.
  if (fact.silentReason || fact.value == null) {
    return { kind: 'SILENT', reason: fact.silentReason ?? 'no-signal' };
  }
  const hasSource = !!fact.source;

  // The cardinal sin: confident claim with no source.
  if (fact.confidence != null && fact.confidence >= CARDINAL_CONFIDENCE && !hasSource) {
    return { kind: 'VIOLATION', reason: 'confident claim with no source', cardinal: true };
  }
  // Provenance is mandatory for any real claim.
  if (!hasSource || fact.observedAt == null) {
    return { kind: 'VIOLATION', reason: 'missing provenance (source/observedAt)', cardinal: false };
  }
  // Staleness: past its freshness window => strip, don't speak as current.
  const ttl = (fact.ttlMinutes ?? DEFAULT_TTL_MINUTES) * 60_000;
  if (now - fact.observedAt > ttl) {
    return { kind: 'VIOLATION', reason: 'stale beyond TTL', cardinal: false };
  }
  return { kind: 'GROUNDED', fact };
}

// Only GROUNDED facts are handed to the model. VIOLATIONs are dropped;
// SILENTs are passed to the render layer as typed absence to handle deliberately.
export function forModel(facts: Fact[], now = Date.now()): { grounded: Fact[]; silent: Verdict[] } {
  const grounded: Fact[] = [];
  const silent: Verdict[] = [];
  for (const f of facts) {
    const v = classify(f, now);
    if (v.kind === 'GROUNDED') grounded.push(v.fact);
    else if (v.kind === 'SILENT') silent.push(v);
    // VIOLATION: intentionally dropped — never reaches the model or the user.
  }
  return { grounded, silent };
}

// The grounding metric — silence counts as integrity, only violations count against.
export function groundingRate(facts: Fact[], now = Date.now()): number {
  let ok = 0, violation = 0;
  for (const f of facts) {
    const v = classify(f, now);
    if (v.kind === 'VIOLATION') violation++;
    else ok++; // GROUNDED and SILENT both count as integrity
  }
  const total = ok + violation;
  return total === 0 ? 1 : ok / total;
}

// The grade primitive (step 7): re-run today's rules over a historical snapshot,
// so accuracy fixes are evidence-driven across the population, not anecdote-driven.
export function grade(snapshot: Fact[], now = Date.now()): { rate: number; violations: Verdict[] } {
  const violations = snapshot
    .map(f => classify(f, now))
    .filter((v): v is Extract<Verdict, { kind: 'VIOLATION' }> => v.kind === 'VIOLATION');
  return { rate: groundingRate(snapshot, now), violations };
}