jspg/LOOKUP_VERIFICATION.md

The Postgres Partial Index Claiming Pattern

This document outlines the architectural strategy for securely handling the deduplication, claiming, and verification of sensitive unique identifiers (like email addresses or phone numbers) strictly through PostgreSQL without requiring "magical" logic in the JSPG Merger.

The Denial of Service (DoS) Squatter Problem

If you enforce a standard UNIQUE constraint on an email address table:

1. Malicious User A signs up and adds jeff.bezos@amazon.com to their account but never verifies it.
2. The real Jeff Bezos signs up.
3. The Database blocks Jeff because the unique string already exists.

The squatter has effectively locked the legitimate owner out of the system.

The Anti-Patterns

1. Global Entity Flags: Adding a global verified boolean to the root entity table forces unrelated objects (like Widgets, Invoices, Orders) to carry verification logic that doesn't belong to them.
2. Magical Merger Logic: Making JSPG's Merger aware of a specific verified field breaks its pure structural translation model. The Merger shouldn't need hardcoded conditional logic to know if it's allowed to update an unverified row.

The Solution: Postgres Partial Unique Indexes

The holy grail is to defer all claiming logic natively to the database engine using a Partial Unique Index.

-- Remove any existing global unique constraint on address first
CREATE UNIQUE INDEX lk_email_address_verified 
ON email_address (address) 
WHERE verified_at IS NOT NULL;

How the Lifecycle Works Natively

1. Unverified Squatters (Isolated Rows): A hundred different users can send { "address": "jeff.bezos@amazon.com" } through the save_person Punc. Because the Punc isolates them and doesn't allow setting the verified_at property natively (enforced by the JSON schema), the JSPG Merger inserts NULL. Postgres permits all 100 INSERT commands to succeed because the Partial Index ignores rows where verified_at IS NULL. Every user gets their own isolated, unverified row acting as a placeholder on their contact edge.

2. The Verification Race (The Claim): The real Jeff clicks his magic verification link. The backend securely executes a specific verification Punc that runs: UPDATE email_address SET verified_at = now() WHERE id = <jeff's-real-uuid>

3. The Lockout: Because Jeff's row now strictly satisfies verified_at IS NOT NULL, that exact row enters the Partial Unique Index. If any of the other 99 squatters ever click their fake verification links (or if a new user tries to verify that same email), PostgreSQL hits the index and violently throws a Unique Constraint Violation, flawlessly blocking them. The winner has permanently claimed the slot across the entire environment!

Periodic Cleanup

Since unverified rows are allowed to accumulate without colliding, a simple Postgres pg_cron job or backend worker can sweep the table nightly to prune abandoned claims and preserve storage:

DELETE FROM email_address 
WHERE verified_at IS NULL 
AND created_at < NOW() - INTERVAL '24 hours';

Why this is the Ultimate Architecture

• The JSPG Merger remains mathematically pure. It doesn't know what verified_at is; it simply respects the database's structural limits (O(1) pure translation).
• Row-Level Security (RLS) naturally blocks users from seeing or claiming each other's unverified rows.
• You offload complex race-condition tracking entirely to the C-level PostgreSQL B-Tree indexing engine, guaranteeing absolute cluster-wide atomicity.