At a glance
| Code | WI-017 |
|---|
| Phase | Platform & Contracts |
|---|
| Order | 17 of 93 |
|---|
| Story points | 13 |
|---|
| Primary surface | OpenTelemetry instrumentation + operation envelope + dashboards |
|---|
| Retires | — |
|---|
| Depends on | Provider gateway + adapter boundary — core (registry, capability, schema+prompt compiler, image prep, ProviderAttempt) |
|---|
| Blocks | Cost governance (hierarchical budgets, reserve->authorize->reconcile handshake, cost ledger, over-budget engine + quality floor, approval, local accounting, pricing catalog), Learning loop & replay (qualification snapshots -> P2, drift detection, circuit breakers, validation/ranking/provider replay, retrospectives, policy comparison) |
|---|
- Evidence
- docs/CC-Analysis/trovesnap-vision-scanning-spec.md (23, 25)
Goals
Bound scan spend and surface quality: every scan emits a trace and respects per-scan_type token/tier budgets (§23, 25).
Implementation — what to build
Spec (docs/specs/P8)
- A scan is a trace TREE, not a flat row; trace/audit/cost-ledger/metrics are distinct; no silent action/retry/fallback.
Build (observability)
- Operation envelope + types; OTel instrumentation + distributed propagation; scan trace tree + trovesnap.* attributes + span events; durable provider_attempt records (never overwritten); P3/P4/P5/P7 trace linkage; redaction + cardinality guardrails; versioned trace sampling; dashboards (scan ops/cost/quality/funnel/provider/retro) + Observatory timeline.
Test (mock providers)
- Expected trace tree per outcome; no-hidden-retries (every attempt has a span+record); trace/ledger consistency (orphan detection); redaction injection; sampling retention rules.
Checklist
- ○ Emit the scan trace tree: profile.resolve → schema.compile → inference → output.validate → result.normalize → candidate.rank (§23).
- ○ Per-scan_type budgets: max_output_tokens, max_candidates, provider tier (economy for discovery, strong only post-isolation) (§25).
- ○ Track quality metrics: candidate precision/recall, retake rate, bbox correction rate, provider disagreement, cost/latency per scan type (§23).
- ○ Enforce cost ceilings + external-query caps (appraisal_value ≤ 3 external comps).
Gates & testing
- ○ P8 Observability and cost-control policy: scan trace structure, per-scan-type token/tier budgets, and quality/cost metrics are defined.
CODE TEST
Requirements
REQ-PLAT-004 accepted Observability and cost controls must bound scan spend and surface quality. Done when: Scan trace structure, per-scan_type token/tier budgets, and quality/cost metrics (precision/recall, retake rate, cost per scan type) are defined.
REQ-SEC-004 accepted Vision image access must be scoped and minimized: scoped to sale/user/org, accessed via expiring signed URLs, sending providers only required images/context, with configurable provider retention. Done when: Signed URLs expire; watchlists stay tenant-scoped; raw model output is treated as untrusted until validated; external appraisal queries cannot leak private client info.
Verify (done when)
Every operation emits one durable operation record + one primary root trace with explicit per-attempt spans (no hidden retries); standard trovesnap.* + gen_ai.* attributes; span-event taxonomy; durable provider-attempt records; trace propagation + linkage across P2/P3/P4/P5/P7; telemetry redaction (no raw images/credentials/PII); cardinality guardrails; versioned sampling (durable facts never sampled away); the operations/cost/quality/funnel/provider/retrospective dashboards. Per spec section 58/59 (observability rows). No action/scan may occur silently.
Cost triggers
No paid API / infra spend triggered by this item.
Status & editing
queued Sprint: P&C Wave 3: Vision Pipeline & Cost
Edit status / sprint on the ★ Live Board → — changes are logged live with who / when / why.
Human-in-the-loop
No human tasks linked.
Findings
None recorded yet.
Concerns
None recorded yet.
Tech debt
None recorded yet.
Tools used
No tool calls recorded.
Files & artifacts
No files / artifacts recorded.
Update log
No log entries yet.