diff --git a/GEMINI.md b/GEMINI.md index 35fe777..2c9a21a 100644 --- a/GEMINI.md +++ b/GEMINI.md @@ -38,6 +38,8 @@ The Validator provides strict, schema-driven evaluation for the "Punc" architect ### Custom Features & Deviations JSPG implements specific extensions to the Draft 2020-12 standard to support the Punc architecture's object-oriented needs while heavily optimizing for zero-runtime lookups. +* **Caching Strategy**: The Validator caches the pre-compiled `Database` registry in memory upon initialization (`jspg_setup`). This registry holds the comprehensive graph of schema boundaries, Types, ENUMs, and Foreign Key relationships, acting as the Single Source of Truth for all validation operations without polling Postgres. + #### A. Polymorphism & Referencing (`$ref`, `$family`, and Native Types) * **Native Type Discrimination (`variations`)**: Schemas defined inside a Postgres `type` are Entities. The validator securely and implicitly manages their `"type"` property. If an entity inherits from `user`, incoming JSON can safely define `{"type": "person"}` without errors, thanks to `compiled_variations` inheritance. * **Structural Inheritance & Viral Infection (`$ref`)**: `$ref` is used exclusively for structural inheritance, *never* for union creation. A Punc request schema that `$ref`s an Entity virally inherits all physical database polymorphism rules for that target. @@ -71,6 +73,7 @@ The Merger provides an automated, high-performance graph synchronization engine ### Core Features +* **Caching Strategy**: The Merger leverages the `Validator`'s in-memory `Database` registry to instantly resolve Foreign Key mapping graphs. It additionally utilizes the concurrent `GLOBAL_JSPG` application memory (`DashMap`) to cache statically constructed SQL `SELECT` strings used during deduplication (`lk_`) and difference tracking calculations. * **Deep Graph Merging**: The Merger walks arbitrary levels of deeply nested JSON schemas (e.g. tracking an `order`, its `customer`, and an array of its `lines`). It intelligently discovers the correct parent-to-child or child-to-parent Foreign Keys stored in the registry and automatically maps the UUIDs across the relationships during UPSERT. * **Prefix Foreign Key Matching**: Handles scenario where multiple relations point to the same table by using database Foreign Key constraint prefixes (`fk_`). For example, if a schema has `shipping_address` and `billing_address`, the merger resolves against `fk_shipping_address_entity` vs `fk_billing_address_entity` automatically to correctly route object properties. * **Dynamic Deduplication & Lookups**: If a nested object is provided without an `id`, the Merger utilizes Postgres `lk_` index constraints defined in the schema registry (e.g. `lk_person` mapped to `first_name` and `last_name`). It dynamically queries these unique matching constraints to discover the correct UUID to perform an UPDATE, preventing data duplication. @@ -89,8 +92,9 @@ The Merger provides an automated, high-performance graph synchronization engine The Queryer transforms Postgres into a pre-compiled Semantic Query Engine via the `jspg_query(schema_id text, cue jsonb)` API, designed to serve the exact shape of Punc responses directly via SQL. ### Core Features + +* **Caching Strategy (DashMap SQL Caching)**: The Queryer securely caches its compiled, static SQL string templates per schema permutation inside the `GLOBAL_JSPG` concurrent `DashMap`. This eliminates recursive AST schema crawling on consecutive requests. Furthermore, it evaluates the strings via Postgres SPI (Server Programming Interface) Prepared Statements, leveraging native database caching of execution plans for extreme performance. * **Schema-to-SQL Compilation**: Compiles JSON Schema ASTs spanning deep arrays directly into static, pre-planned SQL multi-JOIN queries. This explicitly features the `Smart Merge` evaluation engine which natively translates properties through `allOf` and `$ref` inheritances, mapping JSON fields specifically to their physical database table aliases during translation. -* **DashMap SQL Caching**: Executes compiled SQL via Postgres SPI execution, securely caching the static string compilation templates per schema permutation inside the `GLOBAL_JSPG` application memory, drastically reducing repetitive schema crawling. * **Dynamic Filtering**: Binds parameters natively through `cue.filters` objects. The queryer enforces a strict, structured, MongoDB-style operator syntax to map incoming JSON request paths directly to their originating structural table columns. * **Equality / Inequality**: `{"$eq": value}`, `{"$ne": value}` automatically map to `=` and `!=`. * **Comparison**: `{"$gt": ...}`, `{"$gte": ...}`, `{"$lt": ...}`, `{"$lte": ...}` directly compile to Postgres comparison operators (`> `, `>=`, `<`, `<=`). diff --git a/fixtures/queryer.json b/fixtures/queryer.json index 73a7404..4a9a948 100644 --- a/fixtures/queryer.json +++ b/fixtures/queryer.json @@ -36,6 +36,34 @@ "type" ], "prefix": "target" + }, + { + "id": "22222222-2222-2222-2222-222222222222", + "type": "relation", + "constraint": "fk_order_customer", + "source_type": "order", + "source_columns": [ + "customer_id" + ], + "destination_type": "person", + "destination_columns": [ + "id" + ], + "prefix": "customer" + }, + { + "id": "33333333-3333-3333-3333-333333333333", + "type": "relation", + "constraint": "fk_order_line_order", + "source_type": "order_line", + "source_columns": [ + "order_id" + ], + "destination_type": "order", + "destination_columns": [ + "id" + ], + "prefix": "lines" } ], "types": [ @@ -462,6 +490,155 @@ } } ] + }, + { + "name": "order", + "schemas": [ + { + "$id": "order", + "$ref": "entity", + "properties": { + "total": { + "type": "number" + }, + "customer_id": { + "type": "string" + } + } + }, + { + "$id": "full.order", + "$ref": "order", + "properties": { + "customer": { + "$ref": "base.person" + }, + "lines": { + "type": "array", + "items": { + "$ref": "order_line" + } + } + } + } + ], + "hierarchy": [ + "order", + "entity" + ], + "fields": [ + "id", + "type", + "total", + "customer_id", + "created_at", + "created_by", + "modified_at", + "modified_by", + "archived" + ], + "grouped_fields": { + "order": [ + "id", + "type", + "total", + "customer_id" + ], + "entity": [ + "id", + "type", + "created_at", + "created_by", + "modified_at", + "modified_by", + "archived" + ] + }, + "lookup_fields": [ + "id" + ], + "historical": true, + "relationship": false, + "field_types": { + "id": "uuid", + "type": "text", + "archived": "boolean", + "total": "numeric", + "customer_id": "uuid", + "created_at": "timestamptz", + "created_by": "uuid", + "modified_at": "timestamptz", + "modified_by": "uuid" + } + }, + { + "name": "order_line", + "schemas": [ + { + "$id": "order_line", + "$ref": "entity", + "properties": { + "order_id": { + "type": "string" + }, + "product": { + "type": "string" + }, + "price": { + "type": "number" + } + } + } + ], + "hierarchy": [ + "order_line", + "entity" + ], + "fields": [ + "id", + "type", + "order_id", + "product", + "price", + "created_at", + "created_by", + "modified_at", + "modified_by", + "archived" + ], + "grouped_fields": { + "order_line": [ + "id", + "type", + "order_id", + "product", + "price" + ], + "entity": [ + "id", + "type", + "created_at", + "created_by", + "modified_at", + "modified_by", + "archived" + ] + }, + "lookup_fields": [], + "historical": true, + "relationship": false, + "field_types": { + "id": "uuid", + "type": "text", + "archived": "boolean", + "order_id": "uuid", + "product": "text", + "price": "numeric", + "created_at": "timestamptz", + "created_by": "uuid", + "modified_at": "timestamptz", + "modified_by": "uuid" + } } ] }, @@ -480,8 +657,8 @@ " 'id', t1_obj_t1.id,", " 'name', t1_obj_t1.name,", " 'type', t1_obj_t1.type)", - " FROM agreego.entity t1_obj_t1", - " WHERE NOT t1_obj_t1.archived)" + "FROM agreego.entity t1_obj_t1", + "WHERE NOT t1_obj_t1.archived)" ] ] } @@ -501,8 +678,8 @@ " 'id', t1_obj_t1.id,", " 'name', t1_obj_t1.name,", " 'type', t1_obj_t1.type)", - " FROM agreego.entity t1_obj_t1", - " WHERE NOT t1_obj_t1.archived)" + "FROM agreego.entity t1_obj_t1", + "WHERE NOT t1_obj_t1.archived)" ] ] } @@ -607,9 +784,9 @@ " 'last_name', t1_obj_t1.last_name,", " 'name', t1_obj_t2.name,", " 'type', t1_obj_t2.type)", - " FROM agreego.person t1_obj_t1", - " JOIN agreego.entity t1_obj_t2 ON t1_obj_t2.id = t1_obj_t1.id", - " WHERE NOT t1_obj_t1.archived)" + "FROM agreego.person t1_obj_t1", + "JOIN agreego.entity t1_obj_t2 ON t1_obj_t2.id = t1_obj_t1.id", + "WHERE NOT t1_obj_t1.archived)" ] ] } @@ -1036,6 +1213,62 @@ ] ] } + }, + { + "description": "Order select with customer and lines", + "action": "query", + "schema_id": "full.order", + "expect": { + "success": true, + "sql": [ + [ + "(SELECT jsonb_build_object(", + " 'archived', t1_obj_t2.archived,", + " 'created_at', t1_obj_t2.created_at,", + " 'customer',", + " (SELECT jsonb_build_object(", + " 'age', t1_obj_t2_customer_t1.age,", + " 'archived', t1_obj_t2_customer_t2.archived,", + " 'created_at', t1_obj_t2_customer_t2.created_at,", + " 'first_name', t1_obj_t2_customer_t1.first_name,", + " 'id', t1_obj_t2_customer_t2.id,", + " 'last_name', t1_obj_t2_customer_t1.last_name,", + " 'name', t1_obj_t2_customer_t2.name,", + " 'type', t1_obj_t2_customer_t2.type", + " )", + " FROM agreego.person t1_obj_t2_customer_t1", + " JOIN agreego.entity t1_obj_t2_customer_t2 ON t1_obj_t2_customer_t2.id = t1_obj_t2_customer_t1.id", + " WHERE", + " NOT t1_obj_t2_customer_t1.archived", + " AND t1_obj_t2_customer_t1.parent_id = t1_obj_t2.id),", + " 'customer_id', t1_obj_t1.customer_id,", + " 'id', t1_obj_t2.id,", + " 'lines',", + " (SELECT COALESCE(jsonb_agg(jsonb_build_object(", + " 'archived', t1_obj_t2_lines_t2.archived,", + " 'created_at', t1_obj_t2_lines_t2.created_at,", + " 'id', t1_obj_t2_lines_t2.id,", + " 'name', t1_obj_t2_lines_t2.name,", + " 'order_id', t1_obj_t2_lines_t1.order_id,", + " 'price', t1_obj_t2_lines_t1.price,", + " 'product', t1_obj_t2_lines_t1.product,", + " 'type', t1_obj_t2_lines_t2.type", + " )), '[]'::jsonb)", + " FROM agreego.order_line t1_obj_t2_lines_t1", + " JOIN agreego.entity t1_obj_t2_lines_t2 ON t1_obj_t2_lines_t2.id = t1_obj_t2_lines_t1.id", + " WHERE", + " NOT t1_obj_t2_lines_t1.archived", + " AND t1_obj_t2_lines_t1.parent_id = t1_obj_t2.id),", + " 'name', t1_obj_t2.name,", + " 'total', t1_obj_t1.total,", + " 'type', t1_obj_t2.type", + ")", + "FROM agreego.order t1_obj_t1", + "JOIN agreego.entity t1_obj_t2 ON t1_obj_t2.id = t1_obj_t1.id", + "WHERE NOT t1_obj_t1.archived)" + ] + ] + } } ] } diff --git a/src/tests/fixtures.rs b/src/tests/fixtures.rs index b040004..7d54c11 100644 --- a/src/tests/fixtures.rs +++ b/src/tests/fixtures.rs @@ -1463,6 +1463,12 @@ fn test_queryer_0_8() { crate::tests::runner::run_test_case(&path, 0, 8).unwrap(); } +#[test] +fn test_queryer_0_9() { + let path = format!("{}/fixtures/queryer.json", env!("CARGO_MANIFEST_DIR")); + crate::tests::runner::run_test_case(&path, 0, 9).unwrap(); +} + #[test] fn test_not_0_0() { let path = format!("{}/fixtures/not.json", env!("CARGO_MANIFEST_DIR"));