← WI-066 Cost governance (hierarchical budgets, reserve->authorize->reconcile handshake, cost ledger, over-budget engine + quality floor, approval, local accounting, pricing catalog) ★ Live Board · Overview WI-068 Platform Foundations — floor (id/idempotency/correlation convention, keyless CI floor, legacy-quarantine registry) →

WI-067 · Platform & Contracts · learning-loop-replay

Learning loop & replay (qualification snapshots -> P2, drift detection, circuit breakers, validation/ranking/provider replay, retrospectives, policy comparison)

queued P2 medium risk   Owner: Paul   Reviewer: Mark Hilleary   0% · 0/1 tasks complete

Gates: P8 Requirements: — Updated: 2026-06-22

At a glance

CodeWI-067
PhasePlatform & Contracts
Order28 of 93
Story points8
Primary surfaceQualification snapshots + replay + retrospective tooling
Retires—
Depends onObservability core (operation envelope, OTel trace tree + attributes/events, durable provider-attempt records, P3/P4/P5/P7 trace linkage, redaction, sampling, dashboards), Cost governance (hierarchical budgets, reserve->authorize->reconcile handshake, cost ledger, over-budget engine + quality floor, approval, local accounting, pricing catalog)
Blocks—
Evidence
queued (P8 learning)

Goals

Turn observability + cost data into safe improvement: versioned provider-qualification snapshots that feed P2 routing, drift detection + circuit breakers, replay (validation/ranking offline, provider as a new linked op), and retrospectives + historical policy comparison — never a self-modifying production ranker/router.

Spec document

P8-observability-cost → · source ↗

Implementation — what to build

Build (learning)

  • Qualification snapshot schema + P2 feed; drift detection (baseline windows, drift_score); circuit-breaker contract; replay bundle + validation/ranking/provider replay; retrospective records; historical policy-comparison tooling.

Test

  • Replay determinism (same receipt/hash, no provider call); provider replay = new linked op; policy comparison flags quality-floor violations; drift alert doesn't change routes; circuit-breaker state auditable; no auto-promotion.

Checklist

  • ○ P8 Observability and cost-control policy: scan trace structure, per-scan-type token/tier budgets, and quality/cost metrics are defined.

Gates & testing

  • ○ P8 Observability and cost-control policy: scan trace structure, per-scan-type token/tier budgets, and quality/cost metrics are defined.
    CODE TEST

Requirements

No linked requirements.

Verify (done when)

Versioned qualification snapshots (per provider/model/scan/version) consumed by P2 routing (no ad-hoc raw-table queries); drift detection against an explicit baseline that alerts but never auto-changes routes; circuit breakers (provider/model/scan/tenant) with auditable state; validation + ranking replay reproduce receipt/hash without provider calls; provider replay creates a new linked operation (never overwrites); retrospectives + historical policy comparison; no production policy promoted on lower projected cost alone. Per spec section 34-36, 40-42, 56 (learning rows).

Cost triggers

No paid API / infra spend triggered by this item.

Status & editing

queued   Sprint: P&C Wave 3: Vision Pipeline & Cost

Edit status / sprint on the ★ Live Board → — changes are logged live with who / when / why.

Human-in-the-loop

No human tasks linked.

Findings

None recorded yet.

Concerns

None recorded yet.

Risks

None recorded yet.

Tech debt

None recorded yet.

Tools used

No tool calls recorded.

Files & artifacts

No files / artifacts recorded.

Update log

No log entries yet.