queryer merger test progress

This commit is contained in:
2026-03-11 05:18:01 -04:00
parent 1c08a8f2b8
commit 44be75f5d4
104 changed files with 22563 additions and 18859 deletions

View File

@ -44,18 +44,22 @@ JSPG implements specific extensions to the Draft 2020-12 standard to support the
* **Shape Polymorphism (`$family`)**: Auto-expands polymorphic API lists based on an abstract Descendants Graph. If `{"$family": "widget"}` is used, JSPG evaluates the JSON against every schema that `$ref`s widget.
* **Strict Matches & Depth Heuristic**: Polymorphic structures MUST match exactly **one** schema permutation. If multiple inherited struct permutations pass, JSPG applies the **Depth Heuristic Tie-Breaker**, selecting the candidate deepest in the inheritance tree.
#### B. Strict by Default & Extensibility
#### B. Dot-Notation Schema Resolution & Database Mapping
* **The Dot Convention**: When a schema represents a specific variation or shape of an underlying physical database `Type` (e.g., a "summary" of a "person"), its `$id` must adhere to a dot-notation suffix convention (e.g., `summary.person` or `full.person`).
* **Entity Resolution**: The framework (Validator, Queryer, Merger) dynamically determines the backing PostgreSQL table structure by splitting the schema's `$id` (or `$ref`) by `.` and extracting the **last segment** (`next_back()`). If the last segment matches a known Database Type (like `person`), the framework natively applies that table's inheritance rules, variations, and physical foreign keys to the schema graph, regardless of the prefix.
#### C. Strict by Default & Extensibility
* **Strictness**: By default, any property not explicitly defined in the schema causes a validation error (effectively enforcing `additionalProperties: false` globally).
* **Extensibility (`extensible: true`)**: To allow a free-for-all of undefined properties, schemas must explicitly declare `"extensible": true`.
* **Structured Additional Properties**: If `additionalProperties: {...}` is defined as a schema, arbitrary keys are allowed so long as their values match the defined type constraint.
* **Inheritance Boundaries**: Strictness resets when crossing `$ref` boundaries. A schema extending a strict parent remains strict unless it explicitly overrides with `"extensible": true`.
#### C. Implicit Keyword Shadowing
#### D. Implicit Keyword Shadowing
* **Inheritance (`$ref` + properties)**: Unlike standard JSON Schema, when a schema uses `$ref` alongside local properties, JSPG implements **Smart Merge**. Local constraints natively take precedence over (shadow) inherited constraints for the same keyword.
* *Example*: If `entity` has `type: {const: "entity"}`, but `person` defines `type: {const: "person"}`, the local `person` const cleanly overrides the inherited one.
* **Composition (`allOf`)**: When evaluating `allOf`, standard intersection rules apply seamlessly. No shadowing occurs, meaning all constraints from all branches must pass.
#### D. Format Leniency for Empty Strings
#### E. Format Leniency for Empty Strings
To simplify frontend form validation, format validators specifically for `uuid`, `date-time`, and `email` explicitly allow empty strings (`""`), treating them as "present but unset".
---
@ -85,10 +89,36 @@ The Merger provides an automated, high-performance graph synchronization engine
The Queryer transforms Postgres into a pre-compiled Semantic Query Engine via the `jspg_query(schema_id text, cue jsonb)` API, designed to serve the exact shape of Punc responses directly via SQL.
### Core Features
* **Schema-to-SQL Compilation**: Compiles JSON Schema ASTs spanning deep arrays directly into static, pre-planned SQL multi-JOIN queries.
* **Schema-to-SQL Compilation**: Compiles JSON Schema ASTs spanning deep arrays directly into static, pre-planned SQL multi-JOIN queries. This explicitly features the `Smart Merge` evaluation engine which natively translates properties through `allOf` and `$ref` inheritances, mapping JSON fields specifically to their physical database table aliases during translation.
* **DashMap SQL Caching**: Executes compiled SQL via Postgres SPI execution, securely caching the static string compilation templates per schema permutation inside the `GLOBAL_JSPG` application memory, drastically reducing repetitive schema crawling.
* **Dynamic Filtering**: Binds parameters natively through `cue.filters` objects. Dynamically handles string formatting (e.g. parsing `uuid` or formatting date-times) and safely escapes complex combinations utilizing `ILIKE` operations correctly mapped to the originating structural table.
* **The Stem Engine**: Rather than over-fetching heavy Entity payloads and trimming them, Punc Framework Websockets depend on isolated subgraphs defined as **Stems**.
* During initialization, the generator auto-discovers graph boundaries (Stems) inside the schema tree.
* The Queryer prepares dedicated SQL execution templates tailored precisely for that exact `Stem` path (e.g. executing `get_dashboard` queried specifically for the `/owner` stem).
* These Stem outputs instantly hydrate targeted Go Bitsets, providing `O(1)` real-time routing for fractional data payloads without any application-layer overhead.
### 4. The Stem Engine
Rather than over-fetching heavy Entity payloads and trimming them, Punc Framework Websockets depend on isolated subgraphs defined as **Stems**.
A `Stem` is **not a JSON Pointer** or a physical path string (like `/properties/contacts/items/phone_number`). It is simply a declaration of an **Entity Type boundary** that exists somewhere within the compiled JSON Schema graph.
Because `pg_notify` (Beats) fire rigidly from physical Postgres tables (e.g. `{"type": "phone_number"}`), the Go Framework only ever needs to know: "Does the schema `with_contacts.person` contain the `phone_number` Entity anywhere inside its tree?"
* **Initialization:** During startup (`jspg_stems()`), the database crawls all Schemas and maps out every physical Entity Type it references. It builds a flat dictionary of `Schema ID -> [Entity Types]` (e.g. `with_contacts.person -> ["person", "contact", "phone_number", "email_address"]`).
* **Relationship Path Squashing:** When calculating nested string paths structurally to discover these boundaries, JSPG intentionally **omits** properties natively named `target` or `source` if they belong to a native database `relationship` table override. This ensures paths like `phone_numbers/contact/target` correctly register their beat resolution pattern as `phone_numbers/contact/phone_number`.
* **The Go Router**: The Golang Punc framework uses this exact mapping to register WebSocket Beat frequencies exclusively on the Entity types discovered.
* **The Queryer Execution**: When the Go framework asks JSPG to hydrate a partial `phone_number` stem for the `with_contacts.person` schema, instead of jumping through string paths, the SQL Compiler simply reaches into the Schema's AST using the `phone_number` Type string, pulls out exactly that entity's mapping rules, and returns a fully correlated `SELECT` block! This natively handles nested array properties injected via `oneOf` or array references efficiently bypassing runtime powerset expansion.
* **Performance:** These Stem execution structures are fully statically compiled via SPI and map perfectly to `O(1)` real-time routing logic on the application tier.
## 5. Testing & Execution Architecture
JSPG implements a strict separation of concerns to bypass the need to boot a full PostgreSQL cluster for unit and integration testing. Because `pgrx::spi::Spi` directly links to PostgreSQL C-headers, building the library with `cargo test` on macOS natively normally results in fatal `dyld` crashes.
To solve this, JSPG introduces the `DatabaseExecutor` trait inside `src/database/executors/`:
* **`SpiExecutor` (`pgrx.rs`)**: The production evaluator that is conditionally compiled (`#[cfg(not(test))]`). It unwraps standard `pgrx::spi` connections to the database.
* **`MockExecutor` (`mock.rs`)**: The testing evaluator that is conditionally compiled (`#[cfg(test)]`). It absorbs SQL calls and captures parameter bindings in memory without executing them.
### Universal Test Harness (`src/tests/`)
JSPG abandons the standard `cargo pgrx test` model in favor of native OS testing for a >1000x speed increase (`~0.05s` execution).
1. **JSON Fixtures**: All core interactions are defined abstractly as JSON arrays in `fixtures/`. Each file contains suites of `TestCase` objects with an `action` flag (`validate`, `merge`, `query`).
2. **`build.rs` Generator**: The build script traverses the JSON fixtures, extracts their structural identities, and generates standard `#[test]` blocks into `src/tests/fixtures.rs`.
3. **Modular Test Dispatcher**: The `src/tests/types/` module deserializes the abstract JSON test payloads into `Suite`, `Case`, and `Expect` data structures.
4. **Unit Context Execution**: When `cargo test` executes, the `Runner` feeds the JSON payloads directly into `case.execute(db)`. Because the tests run natively inside the module via `#cfg(test)`, the Rust compiler globally erases `pgrx` C-linkage, instantiates the `MockExecutor`, and allows for pure structural evaluation of complex database logic completely in memory.