# JSPG: JSON Schema Postgres

**JSPG** is a high-performance PostgreSQL extension written in Rust (using `pgrx`) that transforms Postgres into a pre-compiled Semantic Engine. It serves as the core engine for the "Punc" architecture, where the database is the single source of truth for all data models, API contracts, validations, and reactive queries.

## 1. Overview & Architecture

JSPG operates by deeply integrating the JSON Schema Draft 2020-12 specification directly into the Postgres session lifecycle. It is built around three core pillars:
*   **Validator**: In-memory, near-instant JSON structural validation and type polymorphism routing.
*   **Merger**: Automatically traverse and UPSERT deeply nested JSON graphs into normalized relational tables.
*   **Queryer**: Compile JSON Schemas into static, cached SQL SPI `SELECT` plans for fetching full entities or isolated ad-hoc object boundaries.

### 🎯 Goals
1.  **Draft 2020-12 Based**: Attempt to adhere to the official JSON Schema Draft 2020-12 specification, while heavily augmenting it for strict structural typing.
2.  **Ultra-Fast Execution**: Compile schemas into optimized in-memory validation trees and cached SQL SPIs to bypass Postgres Query Builder overheads.
3.  **Connection-Bound Caching**: Leverage the PostgreSQL session lifecycle using an **Atomic Swap** pattern. Schemas are 100% frozen, completely eliminating locks during read access.
4.  **Structural Inheritance**: Support object-oriented schema design via Implicit Keyword Shadowing and virtual `family` references natively mapped to Postgres table constraints.
5.  **Reactive Beats**: Provide ultra-fast natively generated flat payloads mapping directly to the Dart topological state for dynamic websocket reactivity.

### Concurrency & Threading ("Immutable Graphs")
To support high-throughput operations while allowing for runtime updates (e.g., during hot-reloading), JSPG uses an **Atomic Swap** pattern:
1. **Parser Phase**: Schema JSONs are parsed into ordered `Schema` structs.
2. **Compiler Phase**: The database iterates all parsed schemas and pre-computes native optimization maps (Descendants Map, Depths Map, Variations Map).
3. **Immutable AST Caching**: The `Validator` struct immutably owns the `Database` registry. Schemas themselves are frozen structurally, but utilize `OnceLock` interior mutability during the Compilation Phase to permanently cache resolved `type` inheritances, properties, and `compiled_edges` directly onto their AST nodes. This guarantees strict `O(1)` relationship and property validation execution at runtime without locking or recursive DB polling.
4. **Lock-Free Reads**: Incoming operations acquire a read lock just long enough to clone the `Arc` inside an `RwLock<Option<Arc<Validator>>>`, ensuring zero blocking during schema updates.

### Global API Reference
These functions operate on the global `GLOBAL_JSPG` engine instance and provide administrative boundaries:

*   `jspg_setup(database jsonb) -> jsonb`: Initializes the engine. Deserializes the full database schema registry (types, enums, puncs, relations) from Postgres and compiles them into memory atomically.
*   `jspg_teardown() -> jsonb`: Clears the current session's engine instance from `GLOBAL_JSPG`, resetting the cache.
*   `jspg_database() -> jsonb`: Exports the fully compiled snapshot of the database registry (including Types, Puncs, Enums, and Relations) out of `GLOBAL_JSPG` into standard JSON Schema representations.

---

## 2. Schema Modeling (Punc Developer Guide)

JSPG augments standard JSON Schema 2020-12 to provide an opinionated, strict, and highly ergonomic Object-Oriented paradigm. Developers defining Punc Data Models should follow these conventions.

### Realms (Topological Boundaries)
JSPG strictly organizes schemas into three distinct topological boundaries called **Realms** to prevent cross-contamination and ensure secure API generation:
*   **Type Realm (`database.types`)**: Represents physical Postgres tables or structural JSONB bubbles. Table-backed entities here are strictly evaluated for their `type` or `kind` discriminators if they possess polymorphic variations.
*   **Punc Realm (`database.puncs`)**: Represents API endpoint Contracts (functions). Contains strictly `.request` and `.response` shapes. These cannot be inherited by standard data models.
*   **Enum Realm (`database.enums`)**: Represents simple restricted value lists. Handled universally across all lookups.

The core execution engines natively enforce these boundaries:
*   **Validator**: Routes dynamically using a single schema key, transparently switching domains to validate Punc requests/responses from the `Punc` realm, or raw instance payloads from the `Type` realm.
*   **Merger**: Strictly bounded to the `Type` Realm. It is philosophically impossible and mathematically illegal to attempt to UPSERT an API endpoint.
*   **Queryer**: Routes recursively. Safely evaluates API boundary inputs directly from the `Punc` realm, while tracing underlying table targets back through the `Type` realm to physically compile SQL `SELECT` statements.

### Types of Types
*   **Table-Backed (Entity Types)**: Primarily defined in root `types` schemas. These represent physical Postgres tables.
    *   They are implicitly registered in the Global Registry using their precise key name mapped from the database compilation phase.
    *   The schema conceptually requires a `type` discriminator at runtime so the engine knows what physical variation to interact with.
    *   Can inherit other entity types to build lineage (e.g. `person` -> `organization` -> `entity`) natively using the `type` property.
*   **Field-Backed (JSONB Bubbles)**: These are shapes that live entirely inside a Postgres JSONB column without being tied to a top-level table constraint.
    *   **Global Schema Registration**: Roots must be attached to the top-level keys mapped from the `types`, `enums`, or `puncs` database tables.
    *   They can re-use the standard `type` discriminator locally for `oneOf` polymorphism without conflicting with global Postgres Table constraints.

### Discriminators & The `<Variant>.<Base>` Convention
In Punc, polymorphic targets like explicit tagged unions or STI (Single Table Inheritance) rely on discriminators. The system heavily leverages a standard `<Variant>.<Base>` dot-notation to enforce topological boundaries deterministically.

**The 2-Tier Paradigm**: The system prevents "God Tables" by restricting routing to exactly two dimensions, guaranteeing absolute $O(1)$ lookups without ambiguity:
1. **Base (Vertical Routing)**: Represents the core physical lineage or foundational structural boundary. For entities, this is the table `type` (e.g. `person` or `widget`). For composed schemas, this is the root structural archetype (e.g., `filter`).
2. **Variant (Horizontal Routing)**: Represents the specific contextual projection or runtime mutation applied to the Base. For STI entities, this is the `kind` (e.g., `light`, `heavy`, `stock`). For composed filters, the variant identifies the entity it targets (e.g., `person`, `invoice`).

When an object is evaluated for STI polymorphism, the runtime natively extracts its `$kind` and `$type` values, dynamically concatenating them as `<Variant>.<Base>` (e.g. `light.person` or `stock.widget`) to yield the namespace-protected schema key.

Therefore, any schema that participates in polymorphic discrimination MUST explicitly define its discriminator properties natively inside its `properties` block. However, to stay DRY and maintain flexible APIs, you **DO NOT** need to hardcode `const` values, nor should you add them to your `required` array. The Punc engine treats `type` and `kind` as **magic properties**.

**Magic Validation Constraints**: 
* **Dynamically Required**: The system inherently drives the need for their requirement. The Validator dynamically expects the discriminators and structurally bubbles `MISSING_TYPE` ultimata ONLY when a polymorphic router (`family` / `oneOf`) dynamically requires them to resolve a path. You never manually put them in the JSON schema `required` block.
* **Implicit Resolution**: When wrapped in `family` or `oneOf`, the polymorphic router can mathematically parse the schema key (e.g. `light.person`) and natively validate that `type` equals `"person"` and `kind` equals `"light"`, bubbling `CONST_VIOLATED` if they mismatch, all without you ever hardcoding `const` limitations.
* **Generator Explicitness**: Because Postgres is the Single Source of Truth, forcing the explicit definition in `properties` initially guarantees the downstream Dart/Go code generators observe the fields and can cleanly serialize them dynamically back to the server.

For example, a schema registered under the exact key `"light.person"` inside the database registry must natively define its own structural boundaries:
```json
{
  "type": "person", 
  "properties": {
    "type": { "type": "string" },
    "kind": { "type": "string" }
  }
}
```

*   **The Object Contract (Presence)**: The Object enforces its own structural integrity mechanically. Standard JSON Validation natively ensures `type` and `kind` are dynamically present as expected.
*   **The Dynamic Values (`db.types`)**: Because the `type` and `kind` properties technically exist, the Punc engine dynamically intercepts them during `validate_object`. It mathematically parses the schema key (e.g. `light.person`) and natively validates that `type` equals `"person"` (or a valid descendant in `db.types`) and `kind` equals `"light"`, bubbling `CONST_VIOLATED` if they mismatch.
*   **The Routing Contract**: When wrapped in `family` or `oneOf`, the polymorphic router can execute Lightning Fast $O(1)$ fast-paths by reading the payload's `type`/`kind` identifiers, and gracefully fallback to standard structural failure if omitted.

### Composition & Inheritance (The `type` keyword)
Punc completely abandons the standard JSON Schema `$ref` keyword. Instead, it overloads the exact same `type` keyword used for primitives. A `"type"` in Punc is mathematically evaluated as either a Native Primitive (`"string"`, `"null"`) or a Custom Object Pointer (`"budget"`, `"user"`).
*   **Single Inheritance**: Setting `"type": "user"` acts exactly like an `extends` keyword. The schema borrows all fields and constraints from the `user` identity. During `jspg_setup`, the compiler recursively crawls the dependencies to map the physical Postgres table, permanently mapping its type restriction to `"object"` under the hood so JSON standards remain unbroken.
*   **Implicit Keyword Shadowing**: Unlike standard JSON Schema inheritance, local property definitions natively override and shadow inherited properties.
*   **Primitive Array Shorthand (Optionality)**: The `type` array syntax is heavily optimized for nullable fields. Defining `"type": ["budget", "null"]` natively builds a nullable strict, generating `Budget? budget;` in Dart. You can freely mix primitives like `["string", "number", "null"]`.
    *   **Strict Array Constraint**: To explicitly prevent mathematically ambiguous Multiple Inheritance, a `type` array is strictly constrained to at most **ONE** Custom Object Pointer. Defining `"type": ["person", "organization"]` will intentionally trigger a fatal database compilation error natively instructing developers to build a proper tagged union (`oneOf`) instead.
*   **Dynamic Type Bindings (`"$sibling.[suffix]"`)**: If a `type` string begins with a `$` (e.g., `"type": "$kind.filter"`), the JSPG engine treats it as a Dynamic Pointer. During compile time, it safely defers boundary checks. During runtime validation, the engine dynamically reads the literal string value of the referenced sibling property (`kind`) on the *current parent JSON object*, evaluates the substitution (e.g., `"person.filter"`), and instantly routes execution to that schema in $O(1)$ time. This enables incredibly powerful dynamic JSONB shapes (like a generic `filter` column inside a `search` table) without forcing downstream code generators to build unmaintainable unions.

### Polymorphism (`family` and `oneOf`)
Polymorphism is how an object boundary can dynamically take on entirely different shapes based on the payload provided at runtime. Punc utilizes the static database metadata generated from Postgres (`db.types`) to enforce these boundaries deterministically, rather than relying on ambiguous tree-traversals.

*   **`family` (Target-Based Polymorphism)**: An explicit Punc compiler macro instructing the engine to resolve dynamic options against the registered database `types` variations or its inner schema registry. It uses the exact physical constraints of the database to build SQL and validation routes.
    *   **Scenario A: Global Tables (Vertical Routing)**
        *   *Setup*: `{ "family": "organization" }`
        *   *Execution*: The engine queries `db.types.get("organization").variations` and finds `["bot", "organization", "person"]`. Because organizations are structurally table-backed, the `family` automatically uses `type` as the discriminator.
        *   *Options*: `bot` -> `bot`, `person` -> `person`, `organization` -> `organization`.
    *   **Scenario B: Prefixed Tables (Vertical Projection)**
        *   *Setup*: `{ "family": "light.organization" }`
        *   *Execution*: The engine sees the prefix `light.` and base `organization`. It queries `db.types.get("organization").variations` and dynamically prepends the prefix to discover the relevant UI schemas.
        *   *Options*: `person` -> `light.person`, `organization` -> `light.organization`. (If a projection like `light.bot` does not exist in the Type Registry, it is safely ignored).
    *   **Scenario C: Single Table Inheritance (Horizontal Routing)**
        *   *Setup*: `{ "family": "widget" }` (Where `widget` is a table type but has no external variations).
        *   *Execution*: The engine queries `db.types.get("widget").variations` and finds only `["widget"]`. Since it lacks table inheritance, it is treated as STI. The engine scans the specific, confined `schemas` array directly under `db.types.get("widget")` for any registered key terminating in the base `.widget` (e.g., `stock.widget`). The `family` automatically uses `kind` as the discriminator.
        *   *Options*: `stock` -> `stock.widget`, `tasks` -> `tasks.widget`.

*   **`oneOf` (Strict Tagged Unions)**: A hardcoded list of candidate schemas. Unlike `family` which relies on global DB metadata, `oneOf` forces pure mathematical structural evaluation of the provided candidates. It strictly bans typical JSON Schema "Union of Sets" fallback searches. Every candidate MUST possess a mathematically unique discriminator payload to allow $O(1)$ routing.
    *   **Disjoint Types**: `oneOf: [{ "type": "person" }, { "type": "widget" }]`. The engine succeeds because the native `type` acts as a unique discriminator (`"person"` vs `"widget"`).
    *   **STI Types**: `oneOf: [{ "type": "heavy.person" }, { "type": "light.person" }]`. The engine succeeds. Even though both share `"type": "person"`, their explicit discriminator is `kind` (`"heavy"` vs `"light"`), ensuring unique $O(1)$ fast-paths.
    *   **Conflicting Types**: `oneOf: [{ "type": "person" }, { "type": "light.person" }]`. The engine **fails compilation natively**. Both schemas evaluate to `"type": "person"` and neither provides a disjoint `kind` constraint, making them mathematically ambiguous and impossible to route in $O(1)$ time.

### Conditionals (`cases`)
Standard JSON Schema forces developers to write deeply nested `allOf` -> `if` -> `properties` blocks just to execute conditional branching. **JSPG completely abandons `allOf` and this practice.** For declarative business logic and structural mutations conditionally based upon property bounds, use the top-level `cases` array. 

It evaluates as an **Independent Declarative Rules Engine**. Every `Case` block within the array is evaluated independently in parallel. For a given rule, if the `when` condition evaluates to true, its `then` schema is executed. If it evaluates to false, its `else` schema is executed (if present). To maintain strict standard JSON Schema compatibility internally, the `when` block utilizes pure JSON Schema `properties` definitions (e.g. `enum`, `const`) rather than injecting unstandardized MongoDB operators. Because `when`, `then`, and `else` are themselves standard schemas, they natively support nested `cases` to handle mutually exclusive `else if` architectures.

```json
{
  "cases": [
    {
      "when": {
        "properties": {
          "status": { "const": "unverified" }
        },
        "required": ["status"]
      },
      "then": {
        "required": ["amount_1", "amount_2"]
      }
    },
    {
      "when": { 
        "properties": { "kind": { "const": "credit" } },
        "required": ["kind"]
      },
      "then": {
        "required": ["details"]
      },
      "else": {
        "cases": [
          {
            "when": { "properties": { "kind": { "const": "checking" } }, "required": ["kind"] },
            "then": { "required": ["routing_number"] }
          }
        ]
      }
    }
  ]
}
```

### Strict by Default & Extensibility
*   **Strictness**: By default, any property not explicitly defined in the schema causes a validation error (effectively enforcing `additionalProperties: false` globally). 
*   **Extensibility (`extensible: true`)**: To allow a free-for-all of undefined properties, schemas must explicitly declare `"extensible": true`. 
*   **Structured Additional Properties**: If `additionalProperties: {...}` is defined as a schema, arbitrary keys are allowed so long as their values match the defined type constraint.
*   **Inheritance Boundaries**: Strictness resets when crossing non-primitive `type` boundaries. A schema extending a strict parent remains strict unless it explicitly overrides with `"extensible": true`.

### Format Leniency for Empty Strings
To simplify frontend form validation, format validators specifically for `uuid`, `date-time`, and `email` explicitly allow empty strings (`""`), treating them as "present but unset".

### Filters & Conditions
In the Punc architecture, filters are automatically synthesized, strongly-typed JSON Schema boundaries that dictate the exact querying capabilities for any given entity or enum. They are completely generated for you; you never write them manually.

*   **Conditions**: A condition schema is the contract defining the mathematical operations allowed on a primitive field. For example, a `string.condition` allows `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$of` (IN), and `$nof` (NOT IN). 
    *   **Enum Conditions**: When JSPG synthesizes an enum, it dynamically generates an `<enum>.condition` (e.g., `address_kind.condition`). This strongly-typed condition perfectly mirrors the operations of a `string.condition`, but strictly limits the arrays and inputs of `$eq`, `$ne`, `$of`, and `$nof` to the exact variations defined by that Enum. This context ensures that UI generators know exactly when to render `<Select>` dropdowns instead of generic `<Text>` boxes.
*   **Filters**: A filter schema (e.g., `person.filter`) is an object containing condition properties used to filter entities. It natively supports structural composition:
    *   **Inherited Properties**: Filters automatically inherit all valid database columns from their base type schema, immediately converting them to their respective `.condition` schemas.
    *   **Relational Proxies**: If a table has a foreign key to another table, the filter automatically generates a proxy property pointing to the related entity's filter (e.g., the `person` filter automatically gains an `organization` property that points to `organization.filter`), allowing infinitely deep nested queries natively.
    *   **Logical Operators (`$and`, `$or`)**: Every filter automatically includes `$and` and `$or` arrays, which recursively accept the exact same filter schema, allowing complex logical grouping.
    *   **Ad-Hoc Extensions (`ad_hoc`)**: Fields stored purely in JSONB bubbles that lack formal database columns can still be queried using the `ad_hoc` object, which passes standard, unvalidated string conditions.

---

## 3. Database

The Database module manages the core execution graphs and structural compilation of the Postgres environment.

### Relational Edge Resolution
When compiling nested object graphs or arrays, the JSPG engine must dynamically infer which Postgres Foreign Key constraint correctly bridges the parent to the nested schema. To guarantee deterministic SQL generation, it utilizes a strict, multi-step algebraic resolution process applied during the `OnceLock` Compilation phase:

1.  **Graph Locality Boundary**: Before evaluating constraints, the engine ensures the parent and child types do not belong strictly to the same inheritance lineage (e.g., `invoice` -> `activity`). Structural inheritance edges are handled natively by the payload merger, so relational edge discovery is intentionally bypassed.
2.  **Structural Cardinality Filtration**: If the JSON Schema requires an Array collection (`{"type": "array"}`), JSPG mathematically rejects pure scalar Forward constraints (where the parent holds a single UUID pointer), logically narrowing the possibilities to Reverse (1:N) or Junction (M:M) constraints.
3.  **Exact Prefix Match**: If an explicitly prefixed Foreign Key (e.g. `fk_invoice_counterparty_entity` -> `prefix: "counterparty"`) directly matches the name of the requested schema property (e.g. `{"counterparty": {...}}`), it is instantly selected.
4.  **Ambiguity Elimination (M:M Twin Deduction)**: If multiple explicitly prefixed relations remain (which happens by design in Many-to-Many junction tables like `contact` or `role`), the compiler inspects the actual compiled child JSON schema AST. If it observes the child natively consumes one of the prefixes as an explicit outbound property (e.g. `contact` explicitly defining `{ "target": ... }`), it considers that arrow "used up". It mathematically deduces that its exact twin providing reverse ownership (`"source"`) MUST be the inbound link mapping from the parent. 
5.  **Implicit Base Fallback (1:M)**: If no explicit prefix matches, and M:M deduction fails, the compiler filters for exactly one remaining relation with a `null` prefix (e.g. `fk_invoice_line_invoice` -> `prefix: null`). A `null` prefix mathematically denotes the core structural parent-child ownership edge and is used safely as a fallback.
6.  **Deterministic Abort**: If the engine exhausts all deduction pathways and the edge remains ambiguous, it explicitly aborts schema compilation (`returns None`) rather than silently generating unpredictable SQL.

### Subschema Promotion
To seamlessly support deeply nested Object and Array structures, JSPG aggressively promotes them to standalone topological entities during the database compilation phase. 
*   **Path Generation:** While evaluating a unified graph originating from a base `types`, `enums`, or `puncs` key, the compiler tracks its exact path descent into nested objects and arrays. It dynamically calculates a localized alias string by appending a `/` pathing syntax (e.g., `base_schema_key/nested/path`) representing exactly its structural constraints.
*   **Promotion:** This nested subschema chunk is mathematically elevated to an independent subschema entry natively within its parent's internal scope (e.g. inside `db.types.get("base").schemas`) using its full path. This guarantees that $O(1)$ WebSockets or isolated queries can natively target any arbitrary nested sub-object of a massive database topology directly without recursively re-parsing its parent's AST block every read. Note that you cannot use the `type` attribute to statically inherit from these automatically promoted subschemas.
*   **Primitive Confinement:** Purely scalar or primitive branches (like `oneOf: [{type: "string"}, {type: "null"}]`) bypass global topological promotion. They are evaluated directly within the execution engine via isolated Tuple Indexes to explicitly protect the global DB Registry and Go Mixer from memory bloat.

---

## 4. Validator

The Validator provides strict, schema-driven evaluation for the "Punc" architecture.

### API Reference
*   `jspg_validate(schema_id text, instance jsonb) -> jsonb`: Validates the `instance` JSON payload strictly against the constraints of the registered `schema_id`. Returns boolean-like success or structured error codes.

### Custom Features & Deviations
JSPG implements specific extensions to the Draft 2020-12 standard to support the Punc architecture's object-oriented needs while heavily optimizing for zero-runtime lookups.

*   **Caching Strategy**: The Validator caches the pre-compiled `Database` registry in memory upon initialization (`jspg_setup`). This registry holds the comprehensive graph of schema boundaries, Types, ENUMs, and Foreign Key relationships, acting as the Single Source of Truth for all validation operations without polling Postgres.
*   **Discriminator Fast Paths & Extraction**: When executing a polymorphic node (`oneOf` or `family`), the engine statically analyzes the incoming JSON payload for the literal `type` and `kind` string coordinates. It routes the evaluation specifically to matching candidates in $O(1)$ while returning `MISSING_TYPE` ultimata directly.
*   **Missing Type Ultimatum**: If an entity logically requires a discriminator and the JSON payload omits it, JSPG short-circuits branch execution entirely, bubbling a single, perfectly-pathed `MISSING_TYPE` error back to the UI natively to prevent confusing cascading failures.
*   **Golden Match Context**: When exactly one structural candidate perfectly maps a discriminator, the Validator exclusively cascades that specific structural error context directly to the user, stripping away all noise generated by other parallel schemas.
*   **Topological Array Pathing**: Instead of relying on explicit `$id` references or injected properties, array iteration paths are dynamically typed based on their compiler boundary constraints. If the array's `items` schema resolves to a topological table-backed entity (e.g., inheriting via a `family` macro tracked in the global DB catalog), the array locks paths and derives element indexes from their actual UUID paths (`array/widget-1/name`), natively enforcing database continuity. If evaluating isolated ad-hoc JSONB elements, strict numeric indexing is enforced natively (`array/1/name`) preventing synthetic payload manipulation.

---

## 5. Merger

The Merger provides an automated, high-performance graph synchronization engine. It orchestrates the complex mapping of nested JSON objects into normalized Postgres relational tables, honoring all inheritance and graph constraints.

### API Reference
*   `jspg_merge(schema_id text, data jsonb) -> jsonb`: Traverses the provided JSON payload according to the compiled relational map of `schema_id`. Dynamically builds and executes relational SQL UPSERT paths natively.

### Core Features

*   **Caching Strategy**: The Merger leverages the native `compiled_edges` permanently cached onto the Schema AST via `OnceLock` to instantly resolve Foreign Key mapping graphs natively in absolute `O(1)` time. It additionally utilizes the concurrent `GLOBAL_JSPG` application memory (`DashMap`) to cache statically constructed SQL `SELECT` strings used during deduplication (`lk_`) and difference tracking calculations.
*   **Deep Graph Merging**: The Merger walks arbitrary levels of deeply nested JSON schemas (e.g. tracking an `order`, its `customer`, and an array of its `lines`). It intelligently discovers the correct parent-to-child or child-to-parent Foreign Keys stored in the registry and automatically maps the UUIDs across the relationships during UPSERT.
*   **Prefix Foreign Key Matching**: Handles scenario where multiple relations point to the same table by using database Foreign Key constraint prefixes (`fk_`). For example, if a schema has `shipping_address` and `billing_address`, the merger resolves against `fk_shipping_address_entity` vs `fk_billing_address_entity` automatically to correctly route object properties.
*   **Dynamic Deduplication & Lookups**: If a nested object is provided without an `id`, the Merger utilizes Postgres `lk_` index constraints defined in the schema registry (e.g. `lk_person` mapped to `first_name` and `last_name`). It dynamically queries these unique matching constraints to discover the correct UUID to perform an UPDATE, preventing data duplication.
*   **Hierarchical Table Inheritance**: The Punc system uses distributed table inheritance (e.g. `person` inherits `user` inherits `organization` inherits `entity`). The Merger splits the incoming JSON payload and performs atomic row updates across *all* relevant tables in the lineage map.
*   **The Archive Paradigm**: Data is never deleted in the Punc system. The Merger securely enforces referential integrity by toggling the `archived` Boolean flag on the base `entity` table rather than issuing SQL `DELETE` commands.
*   **Change Tracking & Reactivity**: The Merger diffs the incoming JSON against the existing database row (utilizing static, `DashMap`-cached `lk_` SELECT string templates). Every detected change is recorded into the `agreego.change` audit table, tracking the user mapping. It then natively uses `pg_notify` to broadcast a completely flat row-level diff out to the Go WebSocket server for O(1) routing.
*   **Flat Structural Beats (Unidirectional Flow)**: The Merger purposefully DOES NOT trace or hydrate outbound Foreign Keys or nested parent structures during writes. It emits completely flat, mathematically perfect structural deltas via `pg_notify` representing only the exact Postgres rows that changed. This guarantees the write-path remains O(1) lightning fast. It is the strict responsibility of the upstream Punc Framework (the Go `Speaker`) to intercept these flat beats, evaluate them against active Websocket Schema Topologies, and dynamically issue targeted `jspg_query` reads to hydrate the exact contextual subgraphs required by listening clients.
*   **Pre-Order Notification Traversal**: To support proper topological hydration on the upstream Go Framework, the Merger decouples the `pg_notify` execution from the physical database write execution. The engine collects structural changes and explicitly fires `pg_notify` SQL statements in strict **Pre-Order** (Parent -> Relations -> Children). This guarantees that WebSocket clients receive the parent entity `Beat` prior to any nested child entities, ensuring stable unidirectional data flows without hydration race conditions.
*   **Many-to-Many Graph Edge Management**: Operates seamlessly with the global `agreego.relationship` table, allowing the system to represent and merge arbitrary reified M:M relationships directionally between any two entities.
*   **Sparse Updates**: Empty JSON strings `""` are directly bound as explicit SQL `NULL` directives to clear data, whilst omitted (missing) properties skip UPDATE execution entirely, ensuring partial UI submissions do not wipe out sibling fields.
*   **Unified Return Structure**: To eliminate UI hydration race conditions and multi-user duplication, `jspg_merge` explicitly strips the response graph and returns only the root `{ "id": "uuid" }` (or an array of IDs for list insertions). External APIs can then explicitly call read APIs to fetch the resulting graph, while the UI relies 100% implicitly on the flat `pg_notify` pipeline for reactive state synchronization.
*   **Decoupled SQL Generation**: Because Writes (INSERT/UPDATE) are inherently highly dynamic based on partial payload structures, the Merger generates raw SQL strings dynamically per execution without caching, guaranteeing a minimal memory footprint while scaling optimally.

---

## 6. Queryer

The Queryer transforms Postgres into a pre-compiled Semantic Query Engine, designed to serve the exact shape of Punc responses directly via SQL.

### API Reference
*   `jspg_query(schema_id text, filter jsonb) -> jsonb`: Compiles the JSON Schema AST of `schema_id` directly into pre-planned, nested multi-JOIN SQL execution trees. Processes the `filter` structurally.

### Core Features

*   **Caching Strategy (DashMap SQL Caching)**: The Queryer securely caches its compiled, static SQL string templates per schema permutation inside the `GLOBAL_JSPG` concurrent `DashMap`. This eliminates recursive AST schema crawling on consecutive requests. Furthermore, it evaluates the strings via Postgres SPI (Server Programming Interface) Prepared Statements, leveraging native database caching of execution plans for extreme performance.
*   **Schema-to-SQL Compilation**: Compiles JSON Schema ASTs spanning deep arrays directly into static, pre-planned SQL multi-JOIN queries. This explicitly features the `Smart Merge` evaluation engine which natively translates properties through `type` inheritances, mapping JSON fields specifically to their physical database table aliases during translation.
*   **Root Null-Stripping Optimization**: Unlike traditional nested document builders, the Queryer intelligently defers Postgres' natively recursive `jsonb_strip_nulls` execution to the absolute apex of the compiled query pipeline. The compiler organically layers millions of rapid `jsonb_build_object()` sub-query allocations instantly, wrapping them in a singular overarching pass. This strips all empty optionals uniformly before exiting the database, maximizing CPU throughput.
*   **Dynamic Filter Execution**: Evaluates the structured `filter` payload and recursively traverses and flattens its paths at AST compilation time. It safely binds parameter constraints using standard operations (e.g., mapping `$eq` to `=`, `$of` to `IN`, `$gt` to `>`) and automatically casts values instantly into the physical database types mapped in the schema (e.g. parsing `uuid` bindings to `::uuid`, formatting DateTimes to `::timestamptz`, and numbers to `::numeric`). Text matching naturally evaluates `$eq` or `$ne` against string fields containing the `%` character natively into Postgres `ILIKE` and `NOT ILIKE`.
*   **Polymorphic SQL Generation (`family`)**: Compiles `family` properties by analyzing the **Physical Database Variations**, *not* the schema descendants.
    *   **The Dot Convention**: When a schema requests `family: "target.schema"`, the compiler extracts the base type (e.g. `schema`) and looks up its Physical Table definition.
    *   **Multi-Table Branching**: If the Physical Table is a parent to other tables (e.g. `organization` has variations `["organization", "bot", "person"]`), the compiler generates a dynamic `CASE WHEN type = '...' THEN ...` query, expanding into sub-queries for each variation. To ensure safe resolution, the compiler dynamically evaluates correlation boundaries: it attempts standard Relational Edge discovery first. If no explicit relational edge exists (indicating pure Table Inheritance rather than a standard foreign-key graph relationship), it safely invokes a **Table Parity Fallback**. This generates an explicit ID correlation constraint (`AND inner.id = outer.id`), perfectly binding the structural variations back to the parent row to eliminate Cartesian products.
    *   **Single-Table Bypass**: If the Physical Table is a leaf node with only one variation (e.g. `person` has variations `["person"]`), the compiler cleanly bypasses `CASE` generation and compiles a simple `SELECT` across the base table, as all schema extensions (e.g. `light.person`, `full.person`) are guaranteed to reside in the exact same physical row.

---

## 7. Testing & Execution Architecture

JSPG implements a strict separation of concerns to bypass the need to boot a full PostgreSQL cluster for unit and integration testing. Because `pgrx::spi::Spi` directly links to PostgreSQL C-headers, building the library with `cargo test` on macOS natively normally results in fatal `dyld` crashes.

To solve this, JSPG introduces the `DatabaseExecutor` trait inside `src/database/executors/`:

*   **`SpiExecutor` (`pgrx.rs`)**: The production evaluator that is conditionally compiled (`#[cfg(not(test))]`). It unwraps standard `pgrx::spi` connections to the database.
*   **`MockExecutor` (`mock.rs`)**: The testing evaluator that is conditionally compiled (`#[cfg(test)]`). It absorbs SQL calls and captures parameter bindings in memory without executing them.

### Universal Test Harness (`src/tests/`)
JSPG abandons the standard `cargo pgrx test` model in favor of native OS testing for a >1000x speed increase (`~0.05s` execution).

1.  **JSON Fixtures**: All core interactions are defined abstractly as JSON arrays in `fixtures/`. Each file contains suites of `TestCase` objects with an `action` flag (`compile`, `validate`, `merge`, `query`).
2.  **`build.rs` Generator**: The build script traverses the JSON fixtures, extracts their structural identities, and generates standard `#[test]` blocks into `src/tests/fixtures.rs`.
3.  **Modular Test Dispatcher**: The `src/tests/types/` module deserializes the abstract JSON test payloads into `Suite`, `Case`, and `Expect` data structures.
    *   The `compile` action natively asserts the exact output shape of `jspg_stems`, allowing structural and relationship mapping logic to be tested purely through JSON without writing brute-force manual tests in Rust.
4.  **Unit Context Execution**: When `cargo test` executes, the runner iterates the JSON payloads. Because the tests run natively inside the module via `#cfg(test)`, the Rust compiler globally erases `pgrx` C-linkage, instantiates the `MockExecutor`, and allows for pure structural evaluation of complex database logic completely in memory in parallel.