Technical spec · all specs
Create one traceable, idempotent, seller-controlled path by which external information may become canonical TroveSnap inventory.
source_connections
→ source_import_runs
→ source_import_records
├─ immutable raw payload
├─ normalized payload
└─ source identity and assets
→ inventory_candidates
→ seller review queue
→ promoteInventoryCandidate()
→ estate_sale_items
Every connector, plugin, harness, MCP action, import, synchronization attempt, and promotion decision is auditable through append-only plugin_events and associated import/promotion records.
P5 protects TroveSnap from: connectors writing directly into inventory; repeated imports creating duplicate items; a malformed source overwriting trusted seller data; expiring source-media URLs becoming permanent inventory images; service-role integrations bypassing review; external status values becoming canonical without mapping; buyer submissions treated as seller-approved facts; source credentials leaking into application records; uncertain source matching silently merging separate items; source provenance being lost after promotion.
P5 establishes the logical contract. P6 implements and enforces the physical database schema, RLS, grants, RPCs, indexes, constraints, and service roles.
Bring everything in. Trust nothing automatically. Make review fast. Preserve where every fact came from.
TroveSnap is integration-first — sellers continue using spreadsheets, estate-sale/auction platforms, cloud folders, photo libraries, email alerts, local files, external listing pages, marketplace exports, and MCP tools. TroveSnap reduces their work without forcing replacement, and that flexibility must not weaken canonical integrity. The system separates what an external source said from what TroveSnap normalized from what a seller reviewed and approved from what is currently canonical.
All external information enters through the same logical stages: Connection → Run → Record → Candidate → Review → Promotion. This applies to manual paste/entry; CSV/Excel; Google Drive; OneDrive; Dropbox; Google Photos; iCloud/local photo libraries; EstateSales.net / AuctionNinja / HiBid exports; custom website imports; Gmail alerts/attachments; desktop harness records; MCP tools; future browser-extension/REST/webhook connectors; buyer submissions; vision-discovered candidates; imported status/sold reports; migration tools.
No integration receives a special shortcut into estate_sale_items.
Connector routes, OAuth callbacks, sync jobs, CSV/spreadsheet parsers, MCP tools, harness endpoints, buyer-submission endpoints, email parsers, browser extensions, vision providers, plugin code, and migration helpers operating as source importers may not directly create/update canonical inventory. They write only through the ingestion contract. The authoritative external-to-canonical function is:
promoteInventoryCandidate(
candidateId: CandidateId,
decision: PromotionDecision,
actor: PromotionActor,
reason: PromotionReason,
options?: PromotionOptions
): Promise<PromotionResult>;
External candidates do not become canonical merely because they have high confidence. High-confidence candidates may get a "ready to approve" state, preselected fields, batch grouping, suggested dedup, expedited presentation — but still require an authorized seller or delegated-reviewer decision. Automated scoring may prioritize the queue; it may not approve.
A seller manually creating an item may experience a single save action; internally that may create a manual-entry source record, a candidate, record approval, and call promoteInventoryCandidate(). The seller doesn't see every step, but the invariant remains intact.
After an item is canonical, authorized domain services may edit it through their own controlled contracts. But a connector may not call those edit services directly; an imported change must create a new candidate / proposed update; the seller must review it; the reviewed change applies through promotion.
A source record preserves what arrived. Normalization may be rerun with a new version, but the original raw payload must not be overwritten.
Idempotency prevents the same source object from being ingested repeatedly. Similarity matching may suggest that two candidates (or a candidate and an item) represent the same object. Only a seller review decision may create a new item; merge approved fields into an existing item; link the source to an existing item without changing fields; or reject the candidate.
(1) Source connection contract. (2) Credential-reference contract. (3) Import-run contract. (4) Source import-record contract. (5) Raw/normalized payload separation. (6) Source identity and revision. (7) Imported asset and media-reference behavior. (8) Inventory-candidate contract. (9) Candidate lifecycle. (10) Seller-review contract. (11) Promotion-decision contract. (12) promoteInventoryCandidate() behavior. (13) Source-derived update promotion. (14) Field-level and item-level provenance. (15) Import and promotion idempotency. (16) Candidate duplicate suggestions. (17) Conflict detection. (18) Append-only plugin/ingestion audit events. (19) Connector permissions. (20) Demo vs live execution. (21) Import failure and partial-run behavior. (22) Offline and resumable harness behavior. (23) External listing-link attribution boundary. (24) Current connector integration requirements. (25) Deterministic and live end-to-end tests.
The physical PostgreSQL schema; RLS policies; database grants; secret-manager implementation; OAuth-provider registration; actual provider credentials; Google/Microsoft/Dropbox/Gmail API clients; spreadsheet UI; seller-review UI; vision inference; canonical item edit workflows unrelated to imports; listing publication; marketplace synchronization; payment; shipping; offer logic; raw demand-signal aggregation; post-sale disposition rules. P5 defines the contracts those systems must respect.
External source
▼
Source connection or one-time source descriptor
▼
Import run
▼
Raw source import records
▼
Deterministic normalization
▼
Candidate creation or source-record reconciliation
▼
Duplicate and conflict suggestions
▼
Seller review
├─ approve as new item
├─ merge approved fields into existing item
├─ link source to existing item
├─ request correction
├─ defer
└─ reject
▼
promoteInventoryCandidate()
├─ create or update canonical item
├─ append provenance
├─ attach durable media
├─ finalize candidate status
├─ record promotion decision
└─ emit plugin/audit events
source_type:
- manual_entry
- manual_paste
- csv
- excel
- google_drive
- onedrive
- dropbox
- google_photos
- icloud_photo_library
- local_photo_folder
- estatesales_net_export
- auction_ninja_export
- hibid_export
- custom_website
- gmail_alert
- email_attachment
- desktop_harness
- mcp
- browser_extension
- webhook
- rest_connector
- vision_scan
- buyer_submission
- migration
The source type describes how information entered. It does not determine whether the information is trusted.
A source_connection represents a reusable external source configuration.
source_connection:
id: conn_google_drive_17
tenant_id: tenant_123
owner_actor_id: user_88
type: google_drive
name: Sullivan Estate Drive Folder
status: active
scope:
estate_sale_id: sale_221
source_container_id: drive-folder-id
source_container_path: null
authentication:
mode: server_oauth
credential_ref: cred_google_17
synchronization:
mode: incremental
cursor: opaque-provider-cursor
last_successful_run_id: run_844
next_allowed_sync_at: null
configuration:
include_mime_types: [image/jpeg, image/png, image/webp]
created_by: user_88
created_at: 2026-06-19T18:00:00Z
updated_at: 2026-06-19T18:00:00Z
source_connection_status:
- pending_authorization
- active
- paused
- needs_reauthorization
- error
- revoked
- disconnected
A reusable connection is not required for one uploaded CSV; one pasted block; one manual entry; one buyer submission; one temporary import bundle. These use a one-time source descriptor but still create an import run and import records.
Connection records may store provider IDs, folder/album IDs, paths, sync cursor, selected scopes, polling config, normalization profile, ownership, status. They may not store passwords, OAuth access/refresh tokens, private keys, reusable API secrets, or local filesystem credentials.
credential_refs contains pointers onlycredential_ref:
id: cred_google_17
tenant_id: tenant_123
type: oauth_token_set
location:
provider: server_secret_store
key: oauth/google/tenant_123/conn_google_drive_17
metadata:
provider: google
scopes: [drive.readonly]
expires_at: 2026-06-19T20:00:00Z
created_at: 2026-06-19T18:00:00Z
credential_refs never contains the secret itself.
Stored server-side; encrypted at rest; readable only by authorized service code; inaccessible through anonymous/normal user database roles; never returned to a browser after callback; never copied into plugin events or import records. Where the implementation uses oauth_tokens: RLS stays enabled; no anonymous-user policies expose the table; token access is service-side only; P6 must prevent token selection through normal application roles.
Local harness credentials remain on the seller's machine. TroveSnap may store a reference (type: local_device, device_id, local_key_alias). The server does not receive local site passwords, cloud-drive passwords, browser cookies, or local keychain contents.
TroveSnap will not store a third-party website username/password server-side for scraping. Permitted: official OAuth; official API keys; seller-uploaded exports; public unauthenticated pages where permitted; browser-side/local harness execution with credentials remaining local; email forwarding or OAuth-authorized Gmail ingestion; partner APIs; seller-provided structured files.
A source_import_run represents one bounded execution.
source_import_run:
id: run_844
tenant_id: tenant_123
estate_sale_id: sale_221
source_connection_id: conn_google_drive_17
source_type: google_drive
trigger:
type: user
actor_id: user_88
mode: live
status: completed_with_warnings
correlation_id: ingest_01J...
idempotency_key: conn_google_drive_17:sync:provider-page-token
cursor:
before: previous-cursor
after: next-cursor
counts:
discovered: 17
unchanged: 10
records_created: 7
candidates_created: 6
candidates_updated: 1
rejected: 0
errors: 1
timing:
started_at: 2026-06-19T18:01:00Z
completed_at: 2026-06-19T18:01:04Z
error_summary:
- code: unsupported_file_type
count: 1
source_import_mode: [demo, preview, live, replay, reconciliation]
source_import_run_status: [queued, running, awaiting_input, completed, completed_with_warnings, partially_failed, failed, cancelled]
Demo mode parses real input; may normalize; returns previews; may compute suggested mappings; creates no canonical data, candidates, or durable source records (unless an explicit dev-telemetry mode); must be clearly labeled nonpersistent. Demo mode must not accidentally become live because credentials/env vars are present.
One failed source record does not discard an otherwise valid run. A run may finish completed_with_warnings or partially_failed; each record retains its own status and errors.
A source_import_record represents one source object, file, row, message, media item, submission, or externally observed update.
source_import_record:
id: record_991
tenant_id: tenant_123
estate_sale_id: sale_221
source_connection_id: conn_google_drive_17
source_import_run_id: run_844
source:
type: google_drive
object_type: image
object_id: provider-file-id
object_revision: provider-revision-id
parent_id: drive-folder-id
identity:
source_identity_key: google_drive:conn_google_drive_17:provider-file-id
source_revision_key: provider-revision-id
content_hash: sha256:...
record_fingerprint: sha256:...
raw:
payload_ref: encrypted://source-records/record_991/raw
received_at: 2026-06-19T18:01:01Z
normalized:
schema: photo_source_record
version: 2.0.0
payload:
filename: IMG_2917.jpg
mime_type: image/jpeg
size_bytes: 1932881
captured_at: null
assets:
- source_asset_id: asset_441
role: source_photo
durability: pending_copy
processing:
status: candidate_created
normalizer_id: google-drive-photo-normalizer
normalizer_version: 2.1.0
candidate_ids: [candidate_771]
Immutable; preserves the original representation; may be stored inline only when small/safe, otherwise an encrypted payload reference; may include row values, message metadata, provider metadata, or source JSON; must exclude secrets; must not be rewritten by later normalization.
Uses a versioned schema; may be regenerated; records normalizer + version; must not overwrite the raw payload; may include validation warnings; may contain canonical-looking field names without yet being canonical inventory.
When normalization logic changes: retain the original raw record; create a new normalized revision; record normalizer version; determine whether the candidate needs review again; do not silently mutate a promoted item.
source_asset:
id: asset_441
source_import_record_id: record_991
source:
provider: google_photos
provider_media_id: media_812
temporary_url_ref: short-lived-reference
metadata:
filename: IMG_2917.jpg
mime_type: image/jpeg
size_bytes: 1932881
durability:
status: temporary
expires_at: 2026-06-19T19:01:00Z
owned_copy:
storage_ref: null
content_hash: null
asset_durability: [metadata_only, temporary, copy_pending, durable, unavailable, rejected]
An expiring provider URL must never become the canonical item image URL. Before promotion, selected media must be copied into TroveSnap-controlled storage; content-hashed; tenant-scoped; associated with the source record; associated with the promoted item through provenance. A candidate may be promoted without media when the item contract permits, but temporary links may not be treated as permanent assets.
baseUrl values are temporary and must not be hot-linked. Copy selected media during review, when the candidate is opened for review, or immediately before promotion — before the temporary URL expires.
In the current metadata-only harness: original files remain local; only filename, size, modification time, device identity, and path-derived identity are transmitted; candidate assets are metadata_only; future photo upload must use a separate authorized storage flow; server records must not falsely imply the photo has been uploaded.
An inventory_candidate is a proposed canonical item or proposed source-derived update.
inventory_candidate:
id: candidate_771
tenant_id: tenant_123
estate_sale_id: sale_221
candidate_type: create_item
source:
primary_record_id: record_991
import_run_id: run_844
source_connection_id: conn_google_drive_17
source_type: google_drive
proposed:
title: Vintage stereo receiver
category: electronics.audio.receiver
description: null
quantity: 1
condition: unknown
pricing: null
status: draft
media_asset_ids: [asset_441]
evidence:
source_record_ids: [record_991]
vision_result_refs: []
normalized_field_sources:
title:
source_record_id: record_991
source_path: normalized.payload.suggested_title
confidence:
overall: 620
fields:
title: 700
category: 800
duplicate_review:
state: possible_match
suggested_item_ids: [item_201]
review:
status: needs_review
assigned_to: null
reviewed_by: null
reviewed_at: null
promotion:
status: not_promoted
promoted_item_id: null
version: 3
created_at: 2026-06-19T18:01:02Z
inventory_candidate_type: [create_item, update_item, attach_source, attach_media, status_update]
create_item proposes a new canonical item. update_item proposes approved changes to an existing item. attach_source links the external source to an existing item without changing fields. attach_media proposes attaching durable source media. status_update proposes a source-derived status transition / appends an authorized status event.
inventory_candidate_status: [discovered, normalizing, needs_review, ready_for_review, review_in_progress, changes_requested, approved, rejected, deferred, conflict, promotion_pending, promoted, promotion_failed, superseded, archived]
Candidate fields may be incomplete, conflicting, duplicated, inferred, source-specific, stale, low confidence, or rejected later. No consumer may treat candidate data as canonical inventory unless the interface is explicitly a review or preview surface.
Every external candidate requires an authorized review decision.
candidate_review:
id: review_921
candidate_id: candidate_771
candidate_version: 3
actor:
id: user_88
role: seller
decision: approve_new_item
selected_fields:
title:
value: Pioneer SX-series stereo receiver
source: candidate
edited: true
category:
value: electronics.audio.receiver
source: candidate
edited: false
media_selection:
include: [asset_441]
duplicate_resolution:
action: create_separate
reason:
code: seller_verified_from_photo
note: Exact model still needs rear-label photo.
created_at: 2026-06-19T18:05:00Z
candidate_review_decision: [approve_new_item, merge_into_existing, link_to_existing, approve_status_update, approve_media_attachment, request_changes, defer, reject]
Approval may come from sale owner; authorized seller; authorized organizer; explicitly delegated reviewer; approved internal operations role acting on behalf of the seller. A connector, provider, MCP client, harness device, buyer, or normalization job may not approve its own candidate.
Review and promotion must reference the candidate version reviewed. If the candidate changes after review: promotion fails with a stale-review error; the candidate returns to review; the seller reviews the changed fields.
Permitted when an authorized seller explicitly selects the candidates/run; each candidate has an individual recorded decision; shared defaults are visible; conflicts/duplicate warnings are not hidden; failed candidates don't block successful ones; the operation remains auditable and resumable. Bulk review is not automated approval.
type PromotionDecision =
| { mode: "create_new"; reviewId: ReviewId; expectedCandidateVersion: number; }
| { mode: "merge_into_existing"; reviewId: ReviewId; targetItemId: InventoryItemId; expectedItemVersion: number; expectedCandidateVersion: number; }
| { mode: "link_existing"; reviewId: ReviewId; targetItemId: InventoryItemId; expectedCandidateVersion: number; }
| { mode: "apply_status_update"; reviewId: ReviewId; targetItemId: InventoryItemId; expectedItemVersion: number; expectedCandidateVersion: number; }
| { mode: "attach_media"; reviewId: ReviewId; targetItemId: InventoryItemId; expectedItemVersion: number; expectedCandidateVersion: number; };
interface PromotionActor { actorId: string; actorType: "seller" | "organizer" | "delegate" | "operations"; tenantId: string; }
interface PromotionReason { code: string; note?: string; }
interface PromotionResult {
promotionId: string; candidateId: string; itemId: string;
action: "created" | "merged" | "linked" | "status_event_applied" | "media_attached" | "already_promoted";
itemVersion: number; provenanceId: string; eventIds: string[];
}
promoteInventoryCandidate() Required BehaviorExecutes as one atomic transaction. It must: (1) authenticate the actor; (2) authorize for tenant + sale; (3) lock/serialize candidate access; (4) verify candidate version; (5) verify candidate status; (6) verify an approved review exists; (7) verify the review matches the candidate version; (8) verify the review actor was permitted; (9) load every referenced source record; (10) verify source records belong to the same tenant + permitted sale; (11) validate selected fields against canonical item rules; (12) validate selected media durability; (13) evaluate duplicate resolution; (14) validate target item + version when merging; (15) create/update the canonical item; (16) create item-source provenance links; (17) create field-level provenance where applicable; (18) preserve seller-edited values as seller-reviewed; (19) append canonical item history; (20) apply authorized status events where applicable; (21) mark the candidate promoted; (22) store promoted_item_id; (23) record the promotion decision; (24) emit append-only plugin/audit events; (25) return the canonical item ID; (26) commit atomically.
If any required step fails, the canonical mutation must roll back.
Idempotent by candidate + approved decision. Re-calling with the same candidate ID, candidate version, review decision, promotion mode, and target item must return the original result. A second call must not create another item, duplicate provenance/media/status events, or duplicate plugin events beyond an optional promotion_replayed audit event. A different approved decision requires a new candidate version or explicit superseding workflow.
Four distinct concepts:
18.1 Import-run idempotency — prevents the same sync request creating repeated runs (connection_id + provider_cursor + operation).
18.2 Source-record identity — prevents the same external object creating repeated records: tenant_id + source_connection_id + source_object_type + source_object_id + source_object_revision (Drive file ID+revision; OneDrive item ID+eTag; Photos media ID; Gmail message+attachment ID; auction sale/export row ID; harness device+normalized path).
18.3 Content fingerprint — used when no stable native ID exists (file content hash; normalized row values; sheet+row identity; sender/subject/attachment hash; device/path/size/mtime; stable normalized item fields). Versioned.
18.4 Candidate similarity — may detect two records describing the same item; a candidate matching an existing item; multiple photos of one item; repeated rows; a previously promoted source returning changed. Produces duplicate_review { state: possible_match, suggested_item_ids, confidence, reasons }. Does not automatically merge candidates or canonical items.
Every promoted source-derived item carries provenance back to candidate; review; source import record; import run; connection; source type; external object ID; normalizer version; promotion actor; decision; timestamp.
item_source_provenance:
id: provenance_781
tenant_id: tenant_123
item_id: item_991
candidate_id: candidate_771
candidate_review_id: review_921
promotion_id: promotion_201
source_connection_id: conn_google_drive_17
source_import_run_id: run_844
source_import_record_ids: [record_991]
source:
type: google_drive
object_ids: [provider-file-id]
processing:
normalizer_id: google-drive-photo-normalizer
normalizer_version: 2.1.0
actor:
id: user_88
role: seller
promoted_at: 2026-06-19T18:05:01Z
field_provenance:
item_id: item_991
field: title
value_version: 4
origin:
type: seller_reviewed_candidate
candidate_id: candidate_771
source_record_id: record_991
transformation:
seller_edited: true
original_candidate_value: Vintage stereo receiver
actor_id: user_88
recorded_at: 2026-06-19T18:05:01Z
A later seller edit may become the current field value. It must not erase the historical fact that the item originally entered through a Google Drive record, CSV row, buyer submission, or other source.
Every connector, plugin, harness, MCP, normalization, review, and promotion action emits an append-only event.
plugin_event:
id: event_401
tenant_id: tenant_123
correlation_id: ingest_01J...
causation_id: event_399
plugin:
type: google_drive
id: connector_google_drive
version: 1.2.0
action: source_record_created
outcome: success
actor:
type: service
id: google-drive-sync
scope:
estate_sale_id: sale_221
source_connection_id: conn_google_drive_17
source_import_run_id: run_844
source_import_record_id: record_991
candidate_id: candidate_771
item_id: null
metadata:
source_object_type: image
source_object_id_hash: sha256:...
occurred_at: 2026-06-19T18:01:02Z
plugin_event_action:
- connection_created
- connection_authorized
- connection_reauthorized
- connection_paused
- connection_revoked
- import_run_started
- import_run_completed
- import_run_failed
- source_record_discovered
- source_record_unchanged
- source_record_created
- source_record_normalized
- source_record_rejected
- candidate_created
- candidate_updated
- candidate_duplicate_suggested
- candidate_reviewed
- candidate_approved
- candidate_rejected
- promotion_started
- promotion_completed
- promotion_failed
- source_link_attached
- status_event_imported
- media_copy_started
- media_copy_completed
- media_copy_failed
- outbound_link_clicked
- connector_mock_invoked
Append-only; tenant-scoped; correlated; causally linked where possible; free of credentials; free of raw private payloads unless explicitly encrypted and referenced; safe for observability and audit. writeEvent() must not serve as a disguised inventory-write mechanism.
21.1 Connector role — May: read its connection config; request credential use through an authorized server component; create import runs; create source records; create/update candidates; append plugin events; read its own run's records as needed. May not: insert/update/delete estate_sale_items; approve candidates; call unrestricted inventory-edit APIs; read unrelated tenants; read raw OAuth tokens.
21.2 MCP role — May: list authorized sales/items; create + list candidates; read permitted demand signals; append plugin events. May not: create canonical inventory; promote without an authorized seller-review token; mutate seller-approved fields directly; access secrets.
21.3 Desktop harness role — May: authenticate a registered device; create import runs; send normalized metadata or authorized files; create source records + candidates; resume interrupted runs; append plugin events. May not: write canonical inventory; approve candidates; read server OAuth credentials; access another device/tenant.
21.4 Buyer submission role — May: create a submission + associated source records; create a quarantined/untrusted candidate; upload permitted evidence; append submission events. May not: select the sale without authorization; create canonical inventory; approve the candidate; claim seller authority; expose private sale data.
21.5 Seller reviewer — May: review; edit proposed values; resolve duplicates; approve; reject; defer; request changes; initiate promotion through the application service.
21.6 Promotion service — the only application role allowed to execute the external-to-canonical promotion RPC. P6 must enforce this with database grants and RLS, not application convention. Even a broad service credential does not make direct inventory writes an approved integration path.
The six relevant ingestion surfaces must all conform to source_import_records → inventory_candidates → seller review → promoteInventoryCandidate(); none may write canonical inventory directly.
22.1 CSV/Excel Import Wizard (live, /trovesnap/harness/csv-import). Must create one import run; one import record per row/logical object; raw row payload; normalized row payload; candidate or reconciliation outcome; plugin events. A sold/status CSV may represent updates to canonical items — the one-door rule still applies: each row first becomes a source record, then a confidently matched item produces a status_update candidate; unmatched → create_item/unresolved; ambiguous → conflict review; seller approves; an authorized domain function appends inventory_status_events; the importer never directly overwrites status columns. Demo mode parses + previews + persists nothing. The run records source headers, suggested + seller-confirmed mapping, mapping-profile version, normalization version, row-level errors.
22.2 MCP Server (live; mock without env, packages/mcp-server). trove_create_candidate must resolve tenant/sale scope; create/reuse an import run; create a source record; preserve raw args; normalize into a candidate; return candidate ID + review status; log plugin_events; never create an estate_sale_item. Mock mode: ephemeral/namespaced candidates, no production mutation, labeled mock. Live mode should prefer a constrained server credential + narrow ingestion RPCs + tenant-scoped authorization + no arbitrary SQL/table access + no direct canonical grants. A service-role key does not authorize bypassing the promotion boundary.
22.3 Google Drive connector (built, OAuth setup required). Record Google file ID; file revision/version; folder ID; MIME; filename; size; capture/modification timestamp; content hash after download where practical; source URL only as an expiring access mechanism; durable copied asset before canonical attachment. OAuth callback finalizes the connection; stores only a credential reference; emits auth events; never exposes tokens to the browser after completion. Deployment config (GOOGLE_CLIENT_ID/SECRET, NEXT_PUBLIC_APP_URL, service key) are config, not import-record fields.
22.4 Desktop harness (live, apps/harness/harness.mjs, POST /api/harness/ingest). Record identity = tenant + registered device ID + normalized local path hash + file size + mtime (content hash strengthens it where available). Raw absolute local paths not broadly exposed (store display-safe relative path, hashed full path, device ID, source root ID, file metadata). Resumability: local queueing; retry after connection loss; stable client-generated record IDs; server idempotency; acknowledged checkpoints; --once + watch modes; safe replay. A repeated submission returns the previous record/candidate rather than a duplicate. Future upload: metadata ingestion separate from asset upload; short-lived upload authorization; content hash attaches to the existing record; retry doesn't create another candidate; not durable until storage confirms.
22.5 OneDrive connector (built, Azure setup required). OneDrive item ID as source-native identity; eTag/cTag as revision; drive + folder identity; provider timestamp; MIME + size; durable media-copy workflow; service-side token handling; record + candidate creation only. MS_CLIENT_ID/SECRET + redirect URI are deployment config, not candidate data.
22.6 Google Photos connector (built). Media item ID as source-native identity; album ID; filename; MIME; creation timestamp; dimensions; pagination cursor; media metadata. baseUrl expires: do not store as canonical media; do not hot-link in published listings; copy selected media during review or before promotion; mark candidate when copy pending; prevent promotion with an expiring URL presented as durable media.
23.1 Gmail alerts — each message/attachment becomes a source record (estate-sale alerts; marketplace alerts; attached CSVs/spreadsheets; listing links; images; addresses; dates). Must not make every email canonical; expose unrelated messages; store full mailbox credentials; or publish imported listings without seller review. Use message ID + attachment ID for identity.
23.2 EstateSales.net / AuctionNinja / HiBid exports — the external platform remains the authority for its native marketplace state. TroveSnap stores source facts, normalized proposed fields, source links, imported status candidates, provenance. It must not falsely claim bid/sale/settlement status beyond the imported record.
23.3 Buyer submissions — submission → source record → untrusted candidate → seller review → promotion. Buyer data may inform discovery but may not establish ownership, seller authorization, canonical price, condition, availability, or sale inclusion.
23.4 Vision scans — a promoted vision candidate follows the same boundary: trusted vision result → source import record or evidence reference → inventory candidate → seller review → promotion. A visual model may suggest item facts; it may not create canonical inventory. (See §37 — this is the P3/P4 → P5 join, source_type: vision_scan.)
External listing channels amplify reach rather than being systems TroveSnap must immediately replace. Sellers may add sale-level links (EstateSales.net; Facebook; HiBid; AuctionNinja; custom websites) stored in external_listing_urls. Outbound clicks pass through /api/out, which records privacy-safe cross-channel attribution (sale; item where applicable; channel; referring page; campaign; anonymous session; timestamp). An external listing URL is not proof synchronization exists; not permission to scrape a private account; does not make the external page canonical; may serve as a source reference / amplification channel; remains distinct from an authenticated connector.
status source row → source import record → normalized status proposal → item-match suggestion → seller review → authorized status event. The importer must not directly set estate_sale_items.status = sold; after approval it appends an inventory_status_event through an authorized function recording source row; source run; matching method; seller reviewer; old state; proposed state; accepted state; effective date; external price where supplied; whether price is asking/bid/sold/settlement; confidence + warnings. An external "sold price" must not be confused with listing price, starting bid, hammer price, buyer's premium, shipping-inclusive total, or payment settlement.
Candidates enter conflict when multiple canonical items are plausible matches; source fields conflict with seller-verified values; the same source identity maps to different items; a candidate was updated after seller review; an external status would create an invalid lifecycle transition; media appears to belong to another item; two sources claim incompatible identity; the candidate targets a locked/archived item.
candidate_conflict:
id: conflict_88
candidate_id: candidate_771
type: multiple_item_matches
options:
- item_id: item_201
confidence: 840
- item_id: item_244
confidence: 790
resolution: unresolved
No "highest confidence wins" automatic merge is permitted.
When the same external object changes: (1) create/identify the source record revision; (2) compare normalized revisions; (3) locate the existing candidate or promoted item link; (4) generate a proposed update candidate; (5) show changed fields; (6) preserve previous source values; (7) require seller review; (8) apply approved changes through promotion.
candidate_diff:
title:
before: Vintage receiver
proposed: Pioneer SX-780 stereo receiver
source_status:
before: active
proposed: sold
source_price:
before: 395
proposed: 325
Repeated imports must not silently overwrite seller-edited canonical fields.
ingestion_failure_class: [authentication, authorization, source_unavailable, rate_limited, malformed_source, unsupported_format, mapping_required, normalization_failed, duplicate, conflict, media_copy_failed, candidate_validation_failed, promotion_validation_failed, cancelled]
A connector failure updates the run; updates affected record status; emits a plugin event; preserves successful records; does not roll back unrelated successful records; does not create partial canonical items. Promotion remains atomic per candidate.
(1) No connector stores third-party passwords server-side. (2) OAuth tokens are service-side only. (3) credential_refs stores pointers, never secrets. (4) Signed URLs are short-lived and not broadly logged. (5) Raw payloads are tenant-scoped. (6) Buyer submissions are untrusted. (7) Imported HTML/email content is untrusted data. (8) Spreadsheet formulas are never executed by the server. (9) CSV formula-injection content is escaped in exports/previews. (10) Archive files are size/type-limited. (11) MIME type and extension are both checked. (12) Imported URLs do not trigger unrestricted server-side requests. (13) MCP arguments are untrusted. (14) Harness tokens are scoped + rotatable. (15) Device identity is recorded. (16) Service-role keys are never shipped to browser/desktop clients. (17) Promotion actor + tenant scope are revalidated server-side. (18) Cross-tenant source linking is prohibited. (19) Source-provided instructions cannot override ingestion policy. (20) Source media must pass validation before durable storage.
trovesnap.ingestion
├─ trovesnap.source.connect
├─ trovesnap.import.run
├─ trovesnap.source.discover
├─ trovesnap.source.record
├─ trovesnap.source.normalize
├─ trovesnap.source.asset
├─ trovesnap.candidate.create
├─ trovesnap.candidate.dedup
├─ trovesnap.candidate.review
├─ trovesnap.candidate.promote
├─ trovesnap.inventory.provenance
└─ trovesnap.plugin.event
Metrics: connections by type/status; import runs by source; discovered/unchanged/normalized records; normalization failures; candidates created/updated/rejected/deferred/promoted; candidate review time; batch-review size; duplicate suggestions + confirmation rate; source-to-candidate + candidate-to-item conversion; promotion failures; idempotent promotion replays; media-copy success/failure; connector auth failures; records per run; run duration; harness retry count; offline queue age; status-update approval rate; source-derived field conflict rate; external click-through by channel. Trace attributes must not include access/refresh tokens; passwords; complete email bodies; raw private source records; unredacted local absolute paths; or buyer PII beyond approved privacy-safe references.
All core P5 contract tests must run without live connector credentials.
31.1 One-door enforcement — direct canonical writes from connector/MCP/harness/buyer/import-worker roles all rejected. 31.2 Source flow — connection → run → record → normalized record → candidate → review → promotion → canonical item; every ID traceable. 31.3 Raw/normalized separation — raw immutable; normalization versioned; re-normalization preserves prior revisions; promotion references the exact reviewed revision. 31.4 Promotion — approved new-item; merge; link; media attachment; approved status update; rejected/deferred/stale-review/wrong-tenant/unauthorized cannot promote. 31.5 Promotion idempotency — promote same approved candidate twice → one item, one mutation, one provenance, no duplicate media/status/event, same result. 31.6 Source idempotency — repeated Drive/OneDrive/Photos/Gmail IDs, CSV row fingerprint, harness device+path, MCP idempotency key → no duplicate record/candidate. 31.7 Duplicate suggestions — exact source duplicate; matching content hash; probable candidate-to-item; two similar-but-distinct; ambiguous → probable matches require review, never auto-merge. 31.8 Credentials — secrets never enter connection/import records; OAuth tokens inaccessible to anonymous roles; local credentials stay local; plugin events redact secret material. 31.9 Media — temporary Photos URL; durable copied media; failed copy; metadata-only harness candidate; promotion with/without required durable media; repeated upload → temporary URLs never become canonical media. 31.10 CSV mapping — suggested + seller-confirmed mapping; unknown status; invalid date; duplicate/unmatched/ambiguous row; formula-like cell; demo no persistence. 31.11 Status import — source row becomes record; status proposal becomes candidate; unreviewed status doesn't mutate; approved status appends an event; invalid transition → conflict; repeated row idempotent. 31.12 MCP — candidate-writing tool creates candidate only; event logged; mock mode no production mutation; tool cannot promote or directly update inventory; tenant scope enforced. 31.13 Harness — token validation; device identity; path dedup; resend idempotency; offline replay; metadata-only; demo; interrupted-batch resume; original file stays local. 31.14 Buyer-submission — becomes untrusted candidate; seller review required; buyer cannot approve; private data hidden; rejected creates no item. 31.15 Events — all meaningful actions emit events; correlation IDs connect the chain; append-only; secrets absent; failures recorded; replay doesn't duplicate business events.
At least one live source must prove the full flow: live source → connection → run → records → candidates → seller review → promotion → canonical item → provenance → plugin events. Initial eligible: CSV/Excel import; desktop harness folder metadata; Google Drive folder; OneDrive folder; Google Photos album. Required live proof: source-native identity; repeated-sync idempotency; raw + normalized retention; candidate creation; no pre-review canonical item; seller approval; promotion; provenance to source; durable media behavior where applicable; plugin-event chain. Live tests use development tenants + non-sensitive source data.
33.1 Automatic promotion for high confidence? No external candidate is automatically promoted. High confidence may support ready-to-review state, sorting, field preselection, batch approval, reduced review effort. Seller approval remains required. 33.2 Candidate deduplication key? Layered identity: source-native object+revision; content fingerprint when native identity absent; candidate similarity for review suggestions; explicit seller duplicate resolution before canonical merge. No single universal dedup key. 33.3 Can a connector update an existing item? It may propose an update candidate; it may not update the item directly. The seller reviews the diff and promotion applies approved changes. 33.4 Can CSV sold reports write status events? Only after a source record exists; the item match is established; a status-update candidate exists; the seller approves; an authorized status-event function is called. The CSV parser may not directly mutate status. 33.5 Is a service-role key an exception? No. Technical ability is not architectural permission. Connector/MCP code using a service credential must still use narrow ingestion + candidate RPCs. 33.6 Can manual seller entry skip candidates? The UI may appear immediate, but manual external data still passes through the candidate + promotion boundary. 33.7 Can duplicate candidates be silently combined? Exact source replay may be idempotently reused. Probable semantic duplicates require seller review. 33.8 What happens to Google Photos temporary URLs? They are temporary source access references only; selected media is copied into owned storage before becoming canonical media. 33.9 Are external listing links connectors? Not automatically — amplification + attribution links unless an explicit authenticated/public import connector is separately configured.
(1) Source-type registry. (2) Source-connection contract. (3) Credential-reference contract. (4) Source-import-run contract. (5) Source-import-record contract. (6) Raw-payload retention contract. (7) Normalized-payload version contract. (8) Source-asset contract. (9) Inventory-candidate contract. (10) Candidate-lifecycle contract. (11) Candidate-review contract. (12) Promotion-decision contract. (13) promoteInventoryCandidate() service interface. (14) Source-derived update contract. (15) Status-update candidate contract. (16) Promotion idempotency contract. (17) Source-record identity rules. (18) Candidate duplicate-suggestion contract. (19) Conflict contract. (20) Item + field provenance contract. (21) Plugin-event contract. (22) Connector permission matrix. (23) Demo/live-mode rules. (24) Media durability rules. (25) External listing attribution boundary. (26) CSV/Excel adapter requirements. (27) MCP ingestion requirements. (28) Google Drive adapter requirements. (29) OneDrive adapter requirements. (30) Google Photos adapter requirements. (31) Desktop harness adapter requirements. (32) Buyer-submission ingestion requirements. (33) Observability requirements. (34) Golden contract fixtures. (35) Idempotency tests. (36) Permission-boundary tests. (37) Live end-to-end ingestion test. (38) P6 database enforcement requirements.
P5 is complete when: (1) every external source uses connection/run/record/candidate/review/promotion; (2) one-time sources still create runs + records; (3) raw payloads immutable; (4) normalized payloads versioned; (5) re-normalization doesn't silently mutate canonical inventory; (6) every candidate references its source records + run; (7) candidates explicitly noncanonical; (8) every external candidate requires seller/delegated-reviewer approval; (9) high confidence cannot bypass review; (10) promoteInventoryCandidate() is the only application path by which an external candidate creates/changes canonical inventory; (11) promotion supports creation, reviewed merge, source linking, media attachment, approved status update; (12) promotion executes atomically; (13) verifies candidate + item versions; (14) is idempotent; (15) re-promotion cannot create duplicate items; (16) connectors cannot directly insert/update canonical inventory; (17) MCP cannot; (18) harness cannot; (19) buyer submissions cannot; (20) broad service credentials don't exempt integrations; (21) source-native IDs prevent repeated imports; (22) fingerprints support sources without stable IDs; (23) similarity creates review suggestions, not silent merges; (24) every promoted item retains source provenance; (25) seller-edited fields retain both source + seller-review provenance; (26) provenance survives later edits; (27) every meaningful action emits an append-only plugin_event; (28) events contain no secrets; (29) OAuth tokens remain server-side; (30) credential_refs contain pointers only; (31) no third-party passwords stored server-side for scraping; (32) harness credentials + original files stay local unless explicitly uploaded; (33) temporary external image URLs cannot become permanent item media; (34) Google Photos media copied before canonical attachment; (35) demo mode persists no production data; (36) live CSV rows create source records before any candidate/status event; (37) status imports require reviewed status-update candidates; (38) repeated status imports idempotent; (39) MCP mock mode safely demoable; (40) Drive/OneDrive/Photos sync dedupe by provider identity; (43) harness dedupes by device + stable file identity; (44) harness ingestion supports retry + resumability; (45) buyer submissions untrusted until review; (46) external listing URLs distinct from authenticated connectors; (47) /api/out records privacy-safe attribution; (48) partial failures preserve valid records; (49) failed promotion leaves no partial mutation; (50) P6 can enforce all write boundaries via grants/RLS/constraints/narrow RPCs; (51) a live source flows end to end; (52) the promoted item traces back to promotion → review → candidate → source record → import run → connection → external object identity.
P5 is done when TroveSnap can accept information from spreadsheets, cloud drives, photo libraries, email, MCP, the desktop harness, marketplace exports, public links, vision scans, and buyer submissions without allowing any of those sources to become an uncontrolled canonical writer. For any source-derived item/update, TroveSnap can answer: which external source produced it? which connection + run handled it? what exact raw record arrived? which normalization version interpreted it? which candidate was created? was a duplicate suggested? what did the seller review? which fields did the seller change? who approved it? which promotion decision applied? new item or existing update? which media was copied to durable storage? which events recorded the process? can the operation be safely replayed without duplication? The P5 result is a seller-controlled source ingestion and canonical promotion boundary that is integration-friendly, spreadsheet-friendly, connector-neutral, review-first, idempotent, traceable, credential-safe, media-safe, offline-resilient, and enforceable at the database boundary.
P5 is the logical contract; P6 physically enforces it (dependency flip — see below). Seams to resolve before freeze:
source_connections, credential_refs (pointers only), source_import_runs, source_import_records (raw immutable + normalized versioned), source_assets, inventory_candidates, candidate_reviews, promotions, item_source_provenance, field_provenance, plugin_events — tenant-scoped, append-only where specified, with connector/MCP/harness/buyer roles granted candidate + event writes only, OAuth tokens unreadable by normal roles, and promoteInventoryCandidate() as the single canonical-write RPC. P6 depends on P5, not the reverse (D30 sequences P1 → P5 → P6). This also houses the P3 scan/validation storage and P4 ranking/profile/demand-snapshot storage.source_type: vision_scan → inventory_candidate (carrying vision_result_refs) → seller review → promotion (§23.4). The vision pipeline never writes canonical inventory directly — it produces candidates through the one door.promoteInventoryCandidate(). P7 consumes P5 (§21.2, §22.2).This is the heaviest gate in the sprint; it splits along contract vs conformance (both gate P5, owner Paul):
promoteInventoryCandidate() behavior + idempotency; layered dedup; item + field provenance; conflict + source-derived update handling; plugin-event audit; the connector permission matrix; deterministic contract + permission-boundary tests. This unblocks P6 + Spine.