TroveSnap Vision Scanning & Item Discovery Specification

Status: Draft implementation specification
Scope: Image-driven discovery, tagging, watchlist matching, visual overlays, item identification, condition analysis, and appraisal routing.

1. Purpose

TroveSnap Vision is the image-intelligence layer for estate-sale and garage-sale imagery. It progressively allocates inference cost:

Wide scene image(s)
  → low-cost discovery scan
  → candidate objects with bounding boxes
  → centered item scan
  → mark/condition evidence scans
  → appraisal readiness
  → valuation using TroveSnap sales data and external comps

The initial scan does not appraise every object. It identifies which objects deserve additional attention.

2. Architectural Boundary

Vision-model output

The image model may return:

candidate objects and bounding boxes
category and likely item label
observable attributes and visible condition
OCR text, labels, signatures, brands and models
confidence and certainty
matches against supplied watchlists
appraisal-candidate flags and reason codes
requested follow-up photos
room/table counts and approximate cross-image deduplication
appraisal evidence completeness
valuation synthesis only when supplied comparable data

Added later by TroveSnap

The image model must not own or invent:

inventory workflow state
listing publication state
platform synchronization status
POS status or sold price
markdown stage
payment or shipping status
buyer-demand counts not supplied in the request
post-sale disposition

Those come from TroveSnap application logic, APIs, MCP tools, platform integrations, POS, and post-sale workflows.

3. Design Principles

UI-triggered contracts: UI actions resolve to scan_type and scan_version.
One logical contract: OpenAI, Gemini, Claude and local VLMs must normalize into the same result schema.
Strict structure, probabilistic content: Schemas enforce shape; evidence, confidence and unknown manage visual uncertainty.
Progressive cost: Cheap scene discovery, richer item identification, expensive appraisal only when justified.
Separate configuration from output: Human-authored YAML is compiled into provider-specific JSON Schema or grammar-constrained JSON.

4. Scan Types

scan_types:
  table_hunt:
    input: single_scene_image
    purpose: find objects worth scanning separately

  room_scan:
    input: multiple_images_of_one_zone
    purpose: categorize, count, deduplicate and identify candidates

  item_scan:
    input: single_centered_item_image
    purpose: identify and summarize the primary object

  mark_scan:
    input: single_mark_or_label_closeup
    purpose: extract maker, brand, model, serial, signature, date or mark

  condition_scan:
    input: one_or_more_images_of_same_item
    purpose: identify visible defects and missing parts

  appraisal_prepare:
    input: item_evidence_and_known_facts
    purpose: determine readiness and create a search fingerprint

  appraisal_value:
    input: normalized_item_evidence_plus_comparables
    purpose: estimate value ranges and explain adjustments

5. Versioning

Every request includes:

scan:
  type: table_hunt
  version: 2

Every response records:

scan:
  type: table_hunt
  version: 2
  result_schema: table_hunt_result
  result_version: 2

scan_version controls behavior, ranking, routing, prompt rules and budgets. result_version controls field names, enums and serialization.

Stored provenance should include:

provenance:
  taxonomy_version: 12
  watchlist_versions:
    paul_personal: 17
    regional_demand: 6
  provider: gemini
  provider_model: resolved-model-id
  prompt_hash: sha256:...
  schema_hash: sha256:...

6. Canonical Request Envelope

request:
  id: req_8f29

  scan:
    type: table_hunt
    version: 2

  input:
    images:
      - id: img_204
        role: scene

  context:
    sale_id: sale_221
    zone_id: garage_table_1
    zone_type: garage
    location: 95682

  profiles:
    watchlists:
      - id: paul_personal
        version: 17
      - id: regional_demand
        version: 6

  options:
    max_candidates: 6

7. Coordinate Contract

Scene scans must return coordinates for every candidate.

coordinates:
  format: xyxy
  origin: top_left
  range: [0, 1000]
  relative_to: orientation_corrected_image

bbox: [x_min, y_min, x_max, y_max]

Example:

image:
  id: img_table_01
  width_px: 3024
  height_px: 4032
  orientation_applied: true
  coordinate_space: normalized_1000

region:
  bbox: [65, 310, 280, 690]
  center: [173, 500]

Bounding boxes are mandatory in the MVP. Polygons and masks are future optional extensions.

8. Watchlist Contract

Watchlists are separate versioned inputs.

watchlist:
  id: paul_personal
  version: 17

  entries:
    - id: vintage_pyrex
      query: vintage patterned Pyrex bowls
      aliases: [Pyrex nesting bowls, vintage milk-glass bowls]
      clues: [printed_pattern, nesting_set, milk_glass, bottom_maker_mark]
      priority: high

    - id: vintage_audio
      query: vintage stereo receivers
      makers: [Pioneer, Marantz, Sansui, McIntosh]
      clues: [analog_tuning_dial, silver_faceplate, wood_case]
      priority: high

Allowed match types:

watch_match_type:
  - exact
  - brand_model
  - maker
  - visual_pattern
  - attribute
  - semantic
  - category

The result schema must restrict watch_id to IDs supplied in the request.

9. Shared Enums

identity_certainty: [exact, probable, possible, unknown]
visibility: [clear, partial, occluded, distant, blurred, unknown]
condition: [sealed, like_new, excellent, good, fair, poor, parts_only, unknown]
priority: [low, medium, high, critical]
appraisal_state: [no, candidate, priority, blocked]
appraisal_readiness:
  - not_recommended
  - candidate
  - needs_item_photo
  - needs_mark_photo
  - needs_condition_details
  - ready
  - ready_with_limitations

Appraisal reason codes:

appraisal_reason:
  - visible_maker_mark
  - visible_model
  - visible_signature
  - possible_antique
  - collectible_category
  - premium_material
  - scarce_item
  - unusual_design
  - complete_set
  - provenance_present
  - high_value_variance
  - possible_high_value
  - watchlist_interest
  - insufficient_identification

Next actions:

next_action:
  - none
  - isolate_item
  - centered_item
  - crop_item
  - photograph_front
  - photograph_back
  - photograph_top
  - photograph_sides
  - photograph_underside
  - photograph_maker_mark
  - photograph_brand_model
  - photograph_signature
  - photograph_damage
  - photograph_serial
  - enter_dimensions
  - verify_quantity
  - verify_working
  - request_appraisal

Overlay states:

overlay_state:
  - watchlist_match
  - appraisal_candidate
  - high_demand
  - unusual_item
  - uncertain_candidate
  - scan_recommended

The UI owns colors, icons, animation and callout placement.

10. Category Taxonomy

Root categories:

category_root:
  - furniture
  - art
  - decor
  - kitchen
  - appliances
  - electronics
  - tools
  - garden
  - clothing
  - jewelry
  - collectibles
  - books_media
  - toys_games
  - sports_outdoor
  - musical
  - automotive
  - office
  - miscellaneous

Hierarchical examples:

category_examples:
  - furniture.table.dining
  - furniture.table.side
  - furniture.chair.arm
  - furniture.chair.lounge
  - kitchen.cookware.cast_iron
  - kitchen.glassware.mixing_bowl
  - electronics.audio.receiver
  - collectibles.ceramic
  - art.painting

The profile compiler should send only relevant category subsets where practical.

11. `table_hunt`

UI trigger

Find interesting items

Input

scan:
  type: table_hunt
  version: 2

input:
  images:
    - id: img_table_01
      role: scene

profiles:
  watchlists:
    - id: paul_personal
      version: 17

options:
  max_candidates: 6
  include: [watchlist_match, potential_value, unusual_item]
  appraisal: false

Output

scan:
  type: table_hunt
  version: 2
  result_schema: table_hunt_result
  result_version: 2

image:
  id: img_table_01
  coordinate_space: normalized_1000
  orientation_applied: true

result:
  visible_items_estimate: 24

  candidates:
    - id: c1
      image_id: img_table_01

      region:
        bbox: [65, 310, 280, 690]
        center: [173, 500]

      label: patterned nesting bowl set
      category: kitchen.glassware.mixing_bowl
      confidence: 82
      identity: possible
      visual_tags: [patterned_milk_glass, nesting_set]

      watchlist:
        match: true
        watch_id: vintage_pyrex
        score: 86
        match_type: visual_pattern

      appraisal:
        state: candidate
        reasons: [collectible_category, insufficient_identification]

      priority: high

      ui:
        overlay_state: watchlist_match
        overlay_label: "1"
        callout: Scan bottom mark
        selectable: true

      next_scan:
        type: item_scan
        version: 2
        requested_photos: [centered_item, photograph_maker_mark]

Rules:

Do not appraise.
Do not generate full listing descriptions.
Do not return every object.
Every candidate requires a valid bounding box.
Prefer unknown to unsupported brand/model guesses.
Candidates may be returned for watchlist relevance, unusual design or potential value.

12. `room_scan`

UI trigger

Scan this room

Input

scan:
  type: room_scan
  version: 3

input:
  images:
    - {id: img_room_01, role: overview}
    - {id: img_room_02, role: overview}
    - {id: img_room_03, role: table}
    - {id: img_room_04, role: shelving}

context:
  sale_id: sale_221
  zone_id: living_room
  zone_type: living_room

profiles:
  watchlists:
    - {id: organizer_priority, version: 4}
    - {id: local_demand, version: 8}

options:
  deduplicate_across_images: true
  return_counts: true
  return_candidates: true
  max_candidates: 8

Output

result:
  coverage: partial
  visible_instances: 44
  estimated_distinct: 31

  categories:
    - category: furniture
      count: 8
      tags: [sofa, chair, table, cabinet]

    - category: decor
      count: 11
      tags: [lamp, framed_art, ceramics]

  candidates:
    - id: c7
      label: sculptural lounge chair
      category: furniture.chair.lounge
      confidence: 86
      priority: high

      sightings:
        - image_id: img_room_02
          bbox: [510, 165, 825, 770]
          center: [668, 468]
          visibility: clear
          primary: true

        - image_id: img_room_03
          bbox: [80, 210, 370, 790]
          center: [225, 500]
          visibility: partial
          primary: false

      reasons: [unusual_design, possible_watchlist_match]

      ui:
        overlay_state: appraisal_candidate
        overlay_label: "A"
        callout: Photograph chair separately

      next_scan:
        type: item_scan
        version: 2

  missing_coverage: [cabinet_interior, rear_shelf]

Rules:

Return both visible appearances and estimated distinct objects.
Deduplicate only when confidence is sufficient.
Every candidate sighting needs coordinates.
Counts are estimates.
Do not create rich descriptions for all room objects.

13. `item_scan`

UI trigger

Identify this item

Center targeting

center_targeting:
  center_region: [0.25, 0.20, 0.75, 0.80]
  selection_priority:
    - object_containing_image_center
    - largest_object_intersecting_center_region
    - most_complete_object_near_center
  reject_when:
    - no_clear_center_object
    - multiple_equally_prominent_objects
    - center_object_too_occluded
    - center_object_too_small

Input

scan:
  type: item_scan
  version: 2

input:
  images:
    - id: img_item_01
      role: centered_item

target:
  selection: center_primary_object
  ignore_background: true

options:
  summary_max_words: 55
  match_watchlists: true
  flag_appraisal: true

Success output

status: identified

item:
  category: electronics.audio.receiver
  name: Pioneer SX-series stereo receiver

  summary: >
    Silver-face stereo receiver with an analog tuning display,
    wood cabinet, and visible wear along the upper cabinet edge.
    Pioneer branding is visible. The exact model should be
    confirmed from the rear label.

  visual_attributes:
    colors: [silver, brown]
    materials: [metal, wood_or_veneer]
    features: [analog_tuning_dial, wood_case, front_control_knobs]

  visible_condition:
    grade: fair
    issues: [cabinet_edge_wear, surface_dust]

  visible_text:
    brand: Pioneer
    model_candidate: possible_SX-780

  identity:
    certainty: probable
    confidence: 91

  tags: [stereo_receiver, vintage_audio, analog_tuner, silver_face, wood_case]

  appraisal:
    state: candidate
    reasons: [visible_maker_mark, high_value_variance]

  next_photos: [photograph_brand_model, photograph_back, photograph_top]

Rejection output

status: retake_required
reason: multiple_center_objects
guidance: Move closer and center one item.

Rules:

Analyze the primary centered object only.
Distinguish visible facts from possible attributes.
Never infer functionality.
Never claim authenticity, age, material, brand or model without supporting evidence.

14. `mark_scan`

UI trigger

Scan maker mark or label

scan:
  type: mark_scan
  version: 1

input:
  images:
    - id: img_mark_01
      role: mark_closeup

context:
  parent_item_id: candidate_c2

options:
  expected: [brand, model, serial, maker, signature, date]
  preserve_raw_text: true

Output:

status: readable
mark_type: manufacturer_label

raw_text:
  - PIONEER
  - STEREO RECEIVER
  - MODEL SX-780
  - SERIAL NO. ZF3602189

normalized:
  brand: Pioneer
  model: SX-780
  serial: ZF3602189

confidence: 99

Rules:

Preserve literal OCR text.
Return normalized values separately.
Represent uncertain characters explicitly.

15. `condition_scan`

UI trigger

Check condition

scan:
  type: condition_scan
  version: 1

input:
  images:
    - {id: img_condition_front, role: front}
    - {id: img_condition_top, role: top}
    - {id: img_condition_back, role: back}
    - {id: img_condition_damage, role: damage_closeup}

context:
  item_name: Pioneer SX-780 stereo receiver

options:
  visible_only: true
  never_infer_functionality: true

Output:

grade: fair

observations:
  - area: cabinet_top
    issue: finish_wear
    severity: moderate
    image_id: img_condition_top

  - area: cabinet_edge
    issue: veneer_chip
    severity: minor
    image_id: img_condition_damage

apparently_missing_parts: []
functional_status: unknown
confidence: 92

Rules:

Report visible condition only.
Reference the supporting image for every observation.
Never infer working status.

16. `appraisal_prepare`

UI trigger

Check appraisal readiness

scan:
  type: appraisal_prepare
  version: 2

input:
  images:
    - img_item_01
    - img_mark_01
    - img_condition_front
    - img_condition_top
    - img_condition_back

known_facts:
  brand: Pioneer
  model: SX-780
  powers_on: true
  audio_tested: false
  location: 95682

options:
  estimate_value: false
  produce_search_fingerprint: true

Output:

readiness: ready_with_limitations

identification:
  brand: Pioneer
  model: SX-780
  confidence: 98

sufficient:
  - model_identification
  - front_condition
  - cabinet_condition
  - rear_label
  - location

missing:
  - audio_channel_test
  - service_history
  - interior_condition

search_fingerprint:
  query: Pioneer SX-780 stereo receiver powers on cosmetic wear
  category: vintage_audio_receiver
  region: 95682

appraisal_confidence_ceiling: medium

Rules:

Do not estimate value.
Identify missing evidence.
Produce a normalized search fingerprint.

17. `appraisal_value`

UI trigger

Estimate value

scan:
  type: appraisal_value
  version: 2

item:
  brand: Pioneer
  model: SX-780
  visible_condition: fair
  powers_on: true
  audio_tested: false
  location: 95682
  sale_context: estate_sale

comparables:
  internal:
    - id: tc_101
      sold_price: 390
      channel: onsite_estate_sale
      condition: good
      tested: true
      region: northern_california
      sold_date: 2026-03-12

    - id: tc_102
      sold_price: 275
      channel: onsite_estate_sale
      condition: fair
      tested: false
      region: northern_california
      sold_date: 2026-01-18

  external:
    - id: ext_201
      sold_price: 365
      shipping_included: false
      condition: fair
      tested: true

options:
  estimates: [quick_sale, normal_estate_sale, patient_local_sale]
  include_comp_adjustments: true

Output:

currency: USD

estimates:
  quick_sale: [225, 285]
  normal_estate_sale: [285, 375]
  patient_local_sale: [350, 450]

recommended_starting_price: 395
confidence: medium

adjustments:
  - {factor: powers_on, effect: positive}
  - {factor: audio_untested, effect: negative}
  - {factor: cabinet_wear, effect: negative}
  - {factor: recognized_model, effect: positive}

strongest_comps: [tc_102, tc_101, ext_201]
missing_evidence: [audio_channel_test, service_history]

Rules:

TroveSnap’s pricing engine calculates the quantitative baseline.
The LLM assists with comp relevance, condition adjustments, anomalies, missing evidence and explanation.
Unsupported model memory must never be treated as a comparable sale.

18. Provider Adapters

All providers normalize to the same canonical result.

providers:
  openai:
    strict_output: json_schema

  gemini:
    strict_output: json_schema

  claude:
    strict_output: structured_output_or_tool_schema

  local:
    runtime: vllm_or_llama_cpp
    strict_output: [guided_json, json_schema, grammar]

Local models should receive concise field semantics even when grammar constrains syntax:

Detect only objects worth scanning separately.
Use only supplied category and watchlist IDs.
Boxes use normalized 0–1000 xyxy coordinates.
Confidence is 0–100.
Appraisal means candidate screening only, never a price estimate.
Use unknown rather than guessing.

19. Normalization and Validation

Provider response
  → syntax validation
  → schema validation
  → enum normalization
  → coordinate validation
  → business-rule validation
  → canonical TroveSnap result

Retry policy:

retry:
  max_attempts: 2
  first_retry:
    same_model: true
    simplified_prompt: true
  second_retry:
    fallback_provider: true

Validation examples:

validation:
  status: retryable
  reason: invalid_bbox

validation:
  status: accepted_with_warnings
  warnings: [low_confidence_candidate]

20. Candidate Ranking

The model returns evidence and component scores. TroveSnap computes final ranking deterministically.

ranking_weights:
  personal_hunt:
    watchlist_match: 60
    demand_signal: 10
    value_potential: 15
    confidence: 10
    visibility: 5

  estate_organizer:
    watchlist_match: 30
    demand_signal: 25
    value_potential: 25
    confidence: 10
    visibility: 10

21. UI/UX Requirements

Table hunt

draw candidate bounding boxes
show ordinal labels
show semantic icons
allow tap selection
display why the item was flagged
launch requested next scan

Room scan

show overlays per image
show category totals and estimated distinct count
show candidate queue
show missing-coverage checklist
support multiple sightings of one candidate

Item scan

guide user to center one item
request retake for ambiguous scenes
guide requested follow-up angles

Appraisal

Clearly distinguish:

candidate
not ready
ready with limitations
ready
estimated value

22. Storage Model

scan_record:
  id: sc_881
  scan:
    type: table_hunt
    version: 2
    result_schema: table_hunt_result
    result_version: 2
  input:
    image_ids: [img_table_01]
  profiles:
    watchlist_versions:
      paul_personal: 17
  provider:
    name: gemini
    model: resolved-model-id
  result_ref: vision_result_981
  metrics:
    input_tokens: 812
    output_tokens: 294
    latency_ms: 1610
    estimated_cost_usd: 0.0004

Store both raw and normalized results. Scene candidates remain provisional until promoted into an item scan or inventory record.

23. Observability

Trace structure:

trovesnap.scan
  ├─ trovesnap.profile.resolve
  ├─ trovesnap.schema.compile
  ├─ gen_ai.inference
  ├─ trovesnap.output.validate
  ├─ trovesnap.result.normalize
  └─ trovesnap.candidate.rank

Track:

candidate precision and recall
watchlist-match acceptance rate
false-positive rate
retake rate
item-scan conversion rate
appraisal conversion rate
bounding-box correction rate
cross-image duplicate error rate
provider disagreement rate
local-versus-hosted quality delta
tokens, latency and cost per scan type

24. Security and Privacy

Scope image access to the sale, user and organization.
Use expiring signed URLs.
Send providers only required images and context.
Keep watchlists tenant-scoped.
Treat raw model output as untrusted until validated.
Prevent external appraisal searches from exposing private client information.
Make provider retention behavior configurable.

25. Cost Controls

cost_policy:
  table_hunt:
    max_output_tokens: 500
    max_candidates: 6
    provider_tier: economy

  room_scan:
    max_output_tokens: 1200
    max_candidates: 8
    provider_tier: economy_or_standard

  item_scan:
    max_output_tokens: 700
    provider_tier: standard

  mark_scan:
    max_output_tokens: 300
    provider_tier: economy_or_local

  condition_scan:
    max_output_tokens: 700
    provider_tier: standard

  appraisal_prepare:
    max_output_tokens: 600
    provider_tier: economy_or_local

  appraisal_value:
    max_output_tokens: 1200
    provider_tier: strong
    external_queries_max: 3

26. Acceptance Criteria

Table hunt

Every candidate has valid normalized coordinates.
Every candidate includes label, category, confidence, priority and next scan.
Supplied watchlist IDs are never invented.
No prices are estimated.
UI can render and tap every overlay.

Room scan

Accepts multiple images.
Returns visible and estimated distinct counts.
Returns category totals.
Supports multiple sightings per candidate.
Every sighting includes a valid bounding box.
Identifies missing coverage when practical.

Item scan

Analyzes the centered object only.
Returns retake_required when targeting is unreliable.
Produces concise image-derived summary and tags.
Never infers functionality.

Mark scan

Preserves raw OCR text.
Separately normalizes brand, model and serial.
Represents uncertain characters.

Condition scan

Links each observation to an image.
Never claims functionality.

Appraisal preparation

Does not estimate value.
Returns readiness and missing evidence.
Produces search fingerprint.

Appraisal value

Uses supplied internal and external comparables.
Returns multiple sale-context ranges.
Identifies strongest comps and adjustments.
Preserves uncertainty.

27. Implementation Phases

Core discovery: table_hunt, item_scan, strict schemas, overlays, one hosted provider.
Room intelligence: room_scan, multiple sightings, counts, approximate deduplication.
Evidence capture: mark_scan, condition_scan, follow-up-photo guidance.
Appraisal routing: appraisal_prepare, readiness UI, internal comparable retrieval.
Pricing intelligence: appraisal_value, API/MCP comparable ingestion, regional/channel adjustments.
Local inference: local VLM adapter, grammar-constrained output, provider routing and benchmarking.

28. MVP Decisions

Models return schema-constrained JSON; YAML remains the configuration and documentation format.
Use normalized 0–1000 xyxy bounding boxes.
Start with table_hunt, room_scan, and item_scan.
Keep watchlists separate and versioned.
Rank candidates deterministically after model inference.
Require unknown instead of unsupported guesses.
Store raw and normalized outputs.
Keep vision output separate from inventory, POS and lifecycle state.
Use economy models for discovery and stronger models only after item isolation.
Treat TroveSnap’s growing internal sales corpus as the long-term valuation advantage.

29. Product Summary

TroveSnap Vision is a progressive discovery and evidence system that:

scans crowded sale environments
highlights the most relevant objects
personalizes discovery through saved watchlists
returns tappable visual overlays
guides users to better item photos
identifies centered objects
extracts marks and visible condition
determines appraisal readiness
combines evidence with TroveSnap’s proprietary sales corpus
controls inference cost through staged analysis

Scan broadly, highlight selectively, inspect deeply, and appraise only when justified.

TroveSnap Vision Scanning & Item Discovery Specification

1. Purpose

2. Architectural Boundary

Vision-model output

Added later by TroveSnap

3. Design Principles

4. Scan Types

5. Versioning

6. Canonical Request Envelope

7. Coordinate Contract

8. Watchlist Contract

9. Shared Enums

10. Category Taxonomy

11. table_hunt

UI trigger

Input

Output

12. room_scan

UI trigger

Input

Output

13. item_scan

UI trigger

Center targeting

Input

Success output

Rejection output

14. mark_scan

UI trigger

15. condition_scan

UI trigger

16. appraisal_prepare

UI trigger

17. appraisal_value

UI trigger

18. Provider Adapters

19. Normalization and Validation

20. Candidate Ranking

21. UI/UX Requirements

Table hunt

Room scan

Item scan

Appraisal

22. Storage Model

23. Observability

24. Security and Privacy

25. Cost Controls

26. Acceptance Criteria

Table hunt

Room scan

Item scan

Mark scan

Condition scan

Appraisal preparation

Appraisal value

27. Implementation Phases

28. MVP Decisions

29. Product Summary

11. `table_hunt`

12. `room_scan`

13. `item_scan`

14. `mark_scan`

15. `condition_scan`

16. `appraisal_prepare`

17. `appraisal_value`