diff --git a/GEMINI.md b/GEMINI.md index 3bf2adb..f5dc233 100644 --- a/GEMINI.md +++ b/GEMINI.md @@ -184,6 +184,24 @@ It evaluates as an **Independent Declarative Rules Engine**. Every `Case` block * **`array`**: Homogeneous collection of items matching the `items` schema. Validation is homogeneous, so strictness checking does not apply. `"extensible"` is not applicable at the `array` level (tuple-like `prefixItems` are removed). * **Inheritance Boundaries**: Strictness resets when crossing non-primitive `type` boundaries. A schema extending a strict parent remains strict unless it explicitly overrides with `"extensible": true`. +### Immutable Properties (`"immutable": true`) +To distinguish read-only hydrated endpoint references, computed properties, system-managed timestamps (`created_at`, `modified_at`), or audit fields from writable properties, JSPG introduces the universal `"immutable": true` schema keyword. + +* **Developer Perspective**: Annotate properties in database schemas or trait definitions with `"immutable": true` when the property should be visible on reads (`jspg_query` / `.response`), but rejected or ignored on writes (`jspg_merge` / `.request`). + ```json + "properties": { + "source": { + "family": "lite.organization", + "immutable": true, + "description": "Read-only hydrated member entity summary on a membership edge." + } + } + ``` +* **Behavior Across Pillars**: + * **Queryer (`jspg_query`)**: Hydrates and includes `immutable` properties in output read responses without restriction. + * **Validator (`jspg_validate`)**: Context-aware. Returns `IMMUTABLE_PROPERTY_VIOLATION` if an `immutable` property is supplied in a write/request payload (schema IDs not ending in `.response`). Response schemas (`.response`) permit `immutable` fields. + * **Merger (`jspg_merge`)**: Automatically skips `immutable` properties during object graph merging so client payloads can never mutate database columns or relationship edges. + ### Format Leniency for Empty Strings To simplify frontend form validation, format validators specifically for `uuid`, `date-time`, and `email` explicitly allow empty strings (`""`), treating them as "present but unset". @@ -302,6 +320,7 @@ JSPG implements specific extensions to the Draft 2020-12 standard to support the * **Missing Type Ultimatum**: If an entity logically requires a discriminator and the JSON payload omits it, JSPG short-circuits branch execution entirely, bubbling a single, perfectly-pathed `MISSING_TYPE` error back to the UI natively to prevent confusing cascading failures. * **Golden Match Context**: When exactly one structural candidate perfectly maps a discriminator, the Validator exclusively cascades that specific structural error context directly to the user, stripping away all noise generated by other parallel schemas. * **Topological Array Pathing**: Instead of relying on explicit `$id` references or injected properties, array iteration paths are dynamically typed based on their compiler boundary constraints. If the array's `items` schema resolves to a topological table-backed entity (e.g., inheriting via a `family` macro tracked in the global DB catalog), the array locks paths and derives element indexes from their actual UUID paths (`array/widget-1/name`), natively enforcing database continuity. If evaluating isolated ad-hoc JSONB elements, strict numeric indexing is enforced natively (`array/1/name`) preventing synthetic payload manipulation. +* **Context-Aware Immutability Validation**: `jspg_validate` evaluates the target schema ID context. If the target schema ID does not end with `.response` (inbound write/request context), any property present in the payload marked with `"immutable": true` raises an `IMMUTABLE_PROPERTY_VIOLATION` error. Response schemas (`.response`) permit `immutable` fields for read hydration. --- @@ -314,13 +333,16 @@ The Merger provides an automated, high-performance graph synchronization engine. ### Core Features -* **Caching Strategy**: The Merger leverages the native `compiled_edges` permanently cached onto the Schema AST via `OnceLock` to instantly resolve Foreign Key mapping graphs natively in absolute `O(1)` time. It additionally utilizes the concurrent `GLOBAL_JSPG` application memory (`DashMap`) to cache statically constructed SQL `SELECT` strings used during deduplication (`lk_`) and difference tracking calculations. +* **Caching Strategy**: The Merger leverages the native `compiled_edges` permanently cached onto the Schema AST via `OnceLock` to instantly resolve Foreign Key mapping graphs natively in absolute `O(1)` time. It additionally utilizes the concurrent `GLOBAL_JSPG` application memory (`DashMap`) to cache statically constructed SQL `SELECT` strings used during deduplication (`lookup_fields`) and difference tracking calculations. * **Deep Graph Merging**: The Merger walks arbitrary levels of deeply nested JSON schemas (e.g. tracking an `order`, its `customer`, and an array of its `lines`). It intelligently discovers the correct parent-to-child or child-to-parent Foreign Keys stored in the registry and automatically maps the UUIDs across the relationships during UPSERT. +* **Immutable Property Filtering**: Properties declaring `"immutable": true` are filtered out during schema property traversal prior to edge classification or column assembly, guaranteeing client payloads cannot write or mutate read-only/computed properties or hydrated endpoint references. +* **Factual Creation Dependency Validation**: When `stage_entity` confirms `kind == "create"` via database lookup, `jspg_merge` evaluates `schema.obj.dependencies.get("created")`. If required creation fields are missing from `entity_fields`, `entity_objects`, or `entity_arrays`, `jspg_merge` halts and returns a structured `REQUIRED_FIELD_MISSING` `Drop`. +* **Structured Fail-Fast Error Propagation**: Internal merger operations return typed `Result` instances, allowing clean fail-fast `?` traversal and returning `Drop::with_errors(vec![err])` directly at the top-level API boundary on failure. * **Prefix Foreign Key Matching**: Handles scenario where multiple relations point to the same table by using database Foreign Key constraint prefixes (`fk_`). For example, if a schema has `shipping_address` and `billing_address`, the merger resolves against `fk_shipping_address_entity` vs `fk_billing_address_entity` automatically to correctly route object properties. -* **Dynamic Deduplication & Lookups**: If a nested object is provided without an `id`, the Merger utilizes Postgres `lk_` index constraints defined in the schema registry (e.g. `lk_person` mapped to `first_name` and `last_name`). It dynamically queries these unique matching constraints to discover the correct UUID to perform an UPDATE, preventing data duplication. +* **Dynamic Deduplication & Lookups**: If a nested object is provided without an `id`, the Merger utilizes custom `lookup_fields` declared directly in the schema registry JSON comments. It validates at setup compile-time that a corresponding unique index exists in PostgreSQL for these fields. When merging, it dynamically builds query predicates for any satisfied `lookup_fields` sets in the entity's type hierarchy (checking child-to-parent hierarchies order-independently and combining satisfied keys with `UNION` queries) to discover the correct UUID to perform an UPDATE, preventing data duplication. * **Hierarchical Table Inheritance**: The Punc system uses distributed table inheritance (e.g. `person` inherits `user` inherits `organization` inherits `entity`). The Merger splits the incoming JSON payload and performs atomic row updates across *all* relevant tables in the lineage map. * **The Archive Paradigm**: Data is never deleted in the Punc system. The Merger securely enforces referential integrity by toggling the `archived` Boolean flag on the base `entity` table rather than issuing SQL `DELETE` commands. -* **Change Tracking & Reactivity**: The Merger diffs the incoming JSON against the existing database row (utilizing static, `DashMap`-cached `lk_` SELECT string templates). Every detected change is recorded into the `agreego.change` audit table, tracking the user mapping. It then natively uses `pg_notify` to broadcast a completely flat row-level diff out to the Go WebSocket server for O(1) routing. +* **Change Tracking & Reactivity**: The Merger diffs the incoming JSON against the existing database row (utilizing static, `DashMap`-cached `lookup` SELECT string templates). Every detected change is recorded into the `agreego.change` audit table, tracking the user mapping. It then natively uses `pg_notify` to broadcast a completely flat row-level diff out to the Go WebSocket server for O(1) routing. * **Flat Structural Beats (Unidirectional Flow)**: The Merger purposefully DOES NOT trace or hydrate outbound Foreign Keys or nested parent structures during writes. It emits completely flat, mathematically perfect structural deltas via `pg_notify` representing only the exact Postgres rows that changed. This guarantees the write-path remains O(1) lightning fast. It is the strict responsibility of the upstream Punc Framework (the Go `Speaker`) to intercept these flat beats, evaluate them against active Websocket Schema Topologies, and dynamically issue targeted `jspg_query` reads to hydrate the exact contextual subgraphs required by listening clients. * **Pre-Order Notification Traversal**: To support proper topological hydration on the upstream Go Framework, the Merger decouples the `pg_notify` execution from the physical database write execution. The engine collects structural changes and explicitly fires `pg_notify` SQL statements in strict **Pre-Order** (Parent -> Relations -> Children). This guarantees that WebSocket clients receive the parent entity `Beat` prior to any nested child entities, ensuring stable unidirectional data flows without hydration race conditions. * **Many-to-Many Graph Edge Management**: Operates seamlessly with the global `agreego.relationship` table, allowing the system to represent and merge arbitrary reified M:M relationships directionally between any two entities. diff --git a/fixtures/database.json b/fixtures/database.json index 8a7024f..b532814 100644 --- a/fixtures/database.json +++ b/fixtures/database.json @@ -911,9 +911,7 @@ "archived" ] }, - "lookup_fields": [ - "id" - ], + "lookup_fields": [], "historical": true, "relationship": false, "field_types": { @@ -1018,5 +1016,100 @@ } } ] + }, + { + "description": "Validation - lookup fields without matching index", + "database": { + "types": [ + { + "id": "22222222-2222-2222-2222-222222222222", + "type": "type", + "name": "user", + "module": "test", + "source": "test", + "hierarchy": ["user"], + "variations": ["user"], + "fields": ["id", "email"], + "lookup_fields": ["email"], + "schemas": { + "user": { + "type": "object", + "properties": { + "id": { "type": "string", "format": "uuid" }, + "email": { "type": "string", "format": "email" } + } + } + } + } + ] + }, + "tests": [ + { + "description": "Fails setup compilation with MISSING_LOOKUP_INDEX if unique index is missing", + "action": "compile", + "expect": { + "success": false, + "errors": [ + { + "code": "MISSING_LOOKUP_INDEX", + "values": { + "type": "user", + "lookup_fields": "[\"email\"]" + }, + "details": { + "path": "/types/user", + "schema": "user" + } + } + ] + } + } + ] + }, + { + "description": "Validation - lookup fields with matching index", + "database": { + "indexes": [ + { + "table": "user", + "columns": ["email"] + } + ], + "types": [ + { + "id": "22222222-2222-2222-2222-222222222222", + "type": "type", + "name": "user", + "module": "test", + "source": "test", + "hierarchy": ["user"], + "variations": ["user"], + "fields": ["id", "email"], + "lookup_fields": ["email"], + "schemas": { + "user": { + "type": "object", + "properties": { + "id": { "type": "string", "format": "uuid" }, + "email": { "type": "string", "format": "email" } + } + } + } + } + ] + }, + "tests": [ + { + "description": "Compiles successfully when unique index matches lookup_fields", + "action": "compile", + "expect": { + "success": true, + "schemas": { + "user": {}, + "user.filter": {} + } + } + } + ] } ] \ No newline at end of file diff --git a/fixtures/merger.json b/fixtures/merger.json index aabe9e1..9154a16 100644 --- a/fixtures/merger.json +++ b/fixtures/merger.json @@ -117,6 +117,16 @@ ] } ], + "indexes": [ + { + "table": "person", + "columns": ["first_name", "last_name", "date_of_birth", "pronouns"] + }, + { + "table": "user", + "columns": ["name"] + } + ], "types": [ { "name": "entity", @@ -296,7 +306,9 @@ "archived" ] }, - "lookup_fields": [], + "lookup_fields": [ + "name" + ], "historical": true, "notify": true, "relationship": false @@ -478,9 +490,7 @@ "organization_id" ] }, - "lookup_fields": [ - "id" - ], + "lookup_fields": [], "historical": true, "notify": true, "relationship": false @@ -1068,9 +1078,7 @@ "archived" ] }, - "lookup_fields": [ - "id" - ], + "lookup_fields": [], "historical": true, "relationship": false, "field_types": { @@ -1593,6 +1601,180 @@ ] } }, + { + "description": "Update existing person with id and multiple inherited lookup keys", + "action": "merge", + "data": { + "id": "33333333-3333-3333-3333-333333333333", + "type": "person", + "name": "LookupName", + "first_name": "LookupFirst", + "last_name": "LookupLast", + "date_of_birth": "1990-01-01T00:00:00Z", + "pronouns": "they/them", + "contact_id": "abc-contact" + }, + "mocks": [ + { + "id": "22222222-2222-2222-2222-222222222222", + "type": "person", + "name": "LookupName", + "first_name": "LookupFirst", + "last_name": "LookupLast", + "date_of_birth": "1990-01-01T00:00:00Z", + "pronouns": "they/them", + "contact_id": "old-contact" + } + ], + "schema_id": "person", + "expect": { + "success": true, + "sql": [ + [ + "(SELECT to_jsonb(t1.*) || to_jsonb(t2.*) || to_jsonb(t3.*) || to_jsonb(t4.*)", + "FROM agreego.\"entity\" t1", + "LEFT JOIN agreego.\"organization\" t2 ON t2.id = t1.id", + "LEFT JOIN agreego.\"user\" t3 ON t3.id = t1.id", + "LEFT JOIN agreego.\"person\" t4 ON t4.id = t1.id", + "WHERE", + " t1.id = '{{uuid:data.id}}'", + "UNION SELECT to_jsonb(t1.*) || to_jsonb(t2.*) || to_jsonb(t3.*) || to_jsonb(t4.*)", + "FROM agreego.\"entity\" t1", + "LEFT JOIN agreego.\"organization\" t2 ON t2.id = t1.id", + "LEFT JOIN agreego.\"user\" t3 ON t3.id = t1.id", + "LEFT JOIN agreego.\"person\" t4 ON t4.id = t1.id", + "WHERE", + " (\"first_name\" = 'LookupFirst'", + " AND \"last_name\" = 'LookupLast'", + " AND \"date_of_birth\" = '{{timestamp}}'", + " AND \"pronouns\" = 'they/them')", + "UNION SELECT to_jsonb(t1.*) || to_jsonb(t2.*) || to_jsonb(t3.*) || to_jsonb(t4.*)", + "FROM agreego.\"entity\" t1", + "LEFT JOIN agreego.\"organization\" t2 ON t2.id = t1.id", + "LEFT JOIN agreego.\"user\" t3 ON t3.id = t1.id", + "LEFT JOIN agreego.\"person\" t4 ON t4.id = t1.id", + "WHERE", + " (\"name\" = 'LookupName'))" + ], + [ + "INSERT INTO agreego.\"entity\" (", + " \"created_at\",", + " \"created_by\",", + " \"id\",", + " \"modified_at\",", + " \"modified_by\",", + " \"type\"", + ")", + "VALUES (", + " '{{timestamp}}',", + " '00000000-0000-0000-0000-000000000000',", + " '{{uuid:data.id}}',", + " '{{timestamp}}',", + " '00000000-0000-0000-0000-000000000000',", + " 'person'", + ")" + ], + [ + "INSERT INTO agreego.\"organization\" (", + " \"id\",", + " \"name\",", + " \"type\"", + ")", + "VALUES (", + " '{{uuid:data.id}}',", + " 'LookupName',", + " 'person'", + ")" + ], + [ + "INSERT INTO agreego.\"user\" (", + " \"id\",", + " \"type\"", + ")", + "VALUES (", + " '{{uuid:data.id}}',", + " 'person'", + ")" + ], + [ + "INSERT INTO agreego.\"person\" (", + " \"contact_id\",", + " \"date_of_birth\",", + " \"first_name\",", + " \"id\",", + " \"last_name\",", + " \"pronouns\",", + " \"type\"", + ")", + "VALUES (", + " 'abc-contact',", + " '{{timestamp}}',", + " 'LookupFirst',", + " '{{uuid:data.id}}',", + " 'LookupLast',", + " 'they/them',", + " 'person'", + ")" + ], + [ + "INSERT INTO agreego.change (", + " \"old\",", + " \"new\",", + " \"entity_id\",", + " \"id\",", + " \"kind\",", + " \"modified_at\",", + " \"modified_by\"", + ")", + "VALUES (", + " NULL,", + " '{", + " \"name\": \"LookupName\",", + " \"first_name\": \"LookupFirst\",", + " \"last_name\": \"LookupLast\",", + " \"date_of_birth\": \"{{timestamp}}\",", + " \"pronouns\": \"they/them\",", + " \"contact_id\": \"abc-contact\",", + " \"type\": \"person\"", + " }',", + " '{{uuid:data.id}}',", + " '{{uuid:generated_0}}',", + " 'create',", + " '{{timestamp}}',", + " '00000000-0000-0000-0000-000000000000'", + ")" + ], + [ + "(SELECT pg_notify('entity', '{", + " \"kind\": \"create\",", + " \"complete\": {", + " \"name\": \"LookupName\",", + " \"first_name\": \"LookupFirst\",", + " \"last_name\": \"LookupLast\",", + " \"date_of_birth\": \"{{timestamp}}\",", + " \"pronouns\": \"they/them\",", + " \"contact_id\": \"abc-contact\",", + " \"id\": \"{{uuid:data.id}}\",", + " \"type\": \"person\",", + " \"created_by\": \"00000000-0000-0000-0000-000000000000\",", + " \"created_at\": \"{{timestamp}}\",", + " \"modified_by\": \"00000000-0000-0000-0000-000000000000\",", + " \"modified_at\": \"{{timestamp}}\"", + " },", + " \"new\": {", + " \"name\": \"LookupName\",", + " \"first_name\": \"LookupFirst\",", + " \"last_name\": \"LookupLast\",", + " \"date_of_birth\": \"{{timestamp}}\",", + " \"pronouns\": \"they/them\",", + " \"contact_id\": \"abc-contact\",", + " \"type\": \"person\"", + " }", + "}'))" + ] + ] + } + }, { "description": "Replace existing person with id and no changes (lookup)", "action": "merge", @@ -2235,12 +2417,7 @@ "FROM agreego.\"entity\" t1", "LEFT JOIN agreego.\"order\" t2 ON t2.id = t1.id", "WHERE", - " t1.id = 'abc'", - "UNION SELECT to_jsonb(t1.*) || to_jsonb(t2.*)", - "FROM agreego.\"entity\" t1", - "LEFT JOIN agreego.\"order\" t2 ON t2.id = t1.id", - "WHERE", - " (\"id\" = 'abc'))" + " t1.id = 'abc')" ], [ "INSERT INTO agreego.\"entity\" (", @@ -3519,12 +3696,7 @@ "FROM agreego.\"entity\" t1", "LEFT JOIN agreego.\"invoice\" t2 ON t2.id = t1.id", "WHERE", - " t1.id = '{{uuid:data.id}}'", - "UNION SELECT to_jsonb(t1.*) || to_jsonb(t2.*)", - "FROM agreego.\"entity\" t1", - "LEFT JOIN agreego.\"invoice\" t2 ON t2.id = t1.id", - "WHERE", - " (\"id\" = '{{uuid:data.id}}'))" + " t1.id = '{{uuid:data.id}}')" ], [ "INSERT INTO agreego.\"entity\" (", diff --git a/fixtures/properties.json b/fixtures/properties.json index ceb7b3f..f7637f5 100644 --- a/fixtures/properties.json +++ b/fixtures/properties.json @@ -873,5 +873,77 @@ } } ] + }, + { + "description": "immutable property validation in request vs response context", + "database": { + "types": [ + { + "name": "item", + "schemas": { + "save_item.request": { + "properties": { + "id": { + "type": "string" + }, + "created_at": { + "type": "string", + "immutable": true + } + } + }, + "get_item.response": { + "properties": { + "id": { + "type": "string" + }, + "created_at": { + "type": "string", + "immutable": true + } + } + } + } + } + ] + }, + "tests": [ + { + "description": "immutable property in request context causes IMMUTABLE_PROPERTY_VIOLATION", + "data": { + "id": "123", + "created_at": "2026-07-21T00:00:00Z" + }, + "schema_id": "save_item.request", + "action": "validate", + "expect": { + "success": false, + "errors": [ + { + "code": "IMMUTABLE_PROPERTY_VIOLATION", + "values": { + "property_name": "created_at" + }, + "details": { + "path": "created_at", + "schema": "save_item.request" + } + } + ] + } + }, + { + "description": "immutable property in response context is allowed", + "data": { + "id": "123", + "created_at": "2026-07-21T00:00:00Z" + }, + "schema_id": "get_item.response", + "action": "validate", + "expect": { + "success": true + } + } + ] } ] \ No newline at end of file diff --git a/fixtures/queryer.json b/fixtures/queryer.json index a906a42..ed22ca9 100644 --- a/fixtures/queryer.json +++ b/fixtures/queryer.json @@ -258,9 +258,7 @@ "name": "text", "created_at": "timestamptz" }, - "lookup_fields": [ - "id" - ], + "lookup_fields": [], "null_fields": [], "default_fields": [ "id", @@ -806,9 +804,7 @@ "archived" ] }, - "lookup_fields": [ - "id" - ], + "lookup_fields": [], "historical": true, "relationship": false, "field_types": { @@ -1106,9 +1102,7 @@ "archived" ] }, - "lookup_fields": [ - "id" - ], + "lookup_fields": [], "historical": true, "relationship": false, "field_types": { diff --git a/src/database/mod.rs b/src/database/mod.rs index 64d889d..5b724ab 100644 --- a/src/database/mod.rs +++ b/src/database/mod.rs @@ -21,16 +21,22 @@ use executors::pgrx::SpiExecutor; #[cfg(test)] use executors::mock::MockExecutor; +use indexmap::IndexMap; use punc::Punc; use relation::Relation; use schema::Schema; use serde_json::Value; -use indexmap::IndexMap; use std::sync::Arc; use r#type::Type; use crate::drop::{Drop, Error, ErrorDetails}; +#[derive(serde::Serialize, serde::Deserialize, Debug, Clone)] +pub struct IndexInfo { + pub table: String, + pub columns: Vec, +} + #[derive(serde::Serialize)] pub struct Database { pub enums: IndexMap, @@ -38,6 +44,8 @@ pub struct Database { pub puncs: IndexMap, pub relations: IndexMap, #[serde(skip)] + pub indexes: Vec, + #[serde(skip)] pub schemas: IndexMap>, #[serde(skip)] pub executor: Box, @@ -50,6 +58,7 @@ impl Database { types: IndexMap::new(), relations: IndexMap::new(), puncs: IndexMap::new(), + indexes: Vec::new(), schemas: IndexMap::new(), #[cfg(not(test))] executor: Box::new(SpiExecutor::new()), @@ -177,6 +186,25 @@ impl Database { } } } + + if let Some(serde_json::Value::Array(arr)) = map.remove("indexes") { + for item in arr { + match serde_json::from_value::(item) { + Ok(idx) => { + db.indexes.push(idx); + } + Err(e) => { + errors.push(Error { + code: "DATABASE_INDEX_PARSE_FAILED".to_string(), + values: Some(IndexMap::from([("reason".to_string(), e.to_string())])), + details: ErrorDetails { + ..Default::default() + }, + }); + } + } + } + } } db.compile(&mut errors); @@ -214,7 +242,6 @@ impl Database { self.executor.timestamp() } - pub fn compile(&mut self, errors: &mut Vec) { // Phase 1: Registration self.collect_schemas(errors); @@ -260,6 +287,43 @@ impl Database { .compile(self, root_id, id.clone(), errors); } } + + // Phase 5: Verify unique indexes for defined lookup fields + for (_, type_def) in &self.types { + if !type_def.lookup_fields.is_empty() { + let mut index_found = false; + for index in &self.indexes { + if index.table == type_def.name { + if index.columns.len() == type_def.lookup_fields.len() + && index + .columns + .iter() + .all(|c| type_def.lookup_fields.contains(c)) + { + index_found = true; + break; + } + } + } + if !index_found { + errors.push(Error { + code: "MISSING_LOOKUP_INDEX".to_string(), + values: Some(IndexMap::from([ + ("type".to_string(), type_def.name.clone()), + ( + "lookup_fields".to_string(), + format!("{:?}", type_def.lookup_fields), + ), + ])), + details: ErrorDetails { + path: Some(format!("/types/{}", type_def.name)), + schema: Some(type_def.name.clone()), + ..Default::default() + }, + }); + } + } + } } /// Synthesizes Composed Filter References for all table-backed boundaries. @@ -395,7 +459,6 @@ impl Database { } } - /// Inspects the Postgres pg_constraint relations catalog to securely identify /// the precise Foreign Key connecting a parent and child hierarchy path. pub fn resolve_relation<'a>( diff --git a/src/database/object.rs b/src/database/object.rs index 5540ca1..fcb883a 100644 --- a/src/database/object.rs +++ b/src/database/object.rs @@ -149,6 +149,9 @@ pub struct SchemaObject { #[serde(default)] #[serde(skip_serializing_if = "Option::is_none")] pub extensible: Option, + #[serde(default)] + #[serde(skip_serializing_if = "Option::is_none")] + pub immutable: Option, // Contains ALL structural fields perfectly flattened from the ENTIRE Database inheritance tree (e.g. `entity` fields like `id`) as well as local fields hidden inside conditional `cases` blocks. // This JSON exported array gives clients absolute deterministic visibility to O(1) validation and masking bounds without duplicating structural memory. @@ -306,4 +309,8 @@ impl SchemaObject { false } + + pub fn is_immutable(&self) -> bool { + self.immutable == Some(true) + } } diff --git a/src/merger/mod.rs b/src/merger/mod.rs index 10cac99..4e5ae68 100644 --- a/src/merger/mod.rs +++ b/src/merger/mod.rs @@ -48,47 +48,7 @@ impl Merger { let val_resolved = match result { Ok(val) => val, - Err(msg) => { - let mut final_code = "MERGE_FAILED".to_string(); - let mut final_message = msg.clone(); - let mut final_cause = None; - - if let Ok(Value::Object(map)) = serde_json::from_str::(&msg) { - if let (Some(Value::String(e_msg)), Some(Value::String(e_code))) = - (map.get("error"), map.get("code")) - { - final_message = e_msg.clone(); - final_code = e_code.clone(); - let mut cause_parts = Vec::new(); - if let Some(Value::String(d)) = map.get("detail") { - if !d.is_empty() { - cause_parts.push(d.clone()); - } - } - if let Some(Value::String(h)) = map.get("hint") { - if !h.is_empty() { - cause_parts.push(h.clone()); - } - } - if !cause_parts.is_empty() { - final_cause = Some(cause_parts.join("\n")); - } - } - } - - return Drop::with_errors(vec![Error { - code: final_code, - values: Some(IndexMap::from([ - ("error".to_string(), final_message), - ])), - details: ErrorDetails { - path: None, - cause: final_cause, - context: None, - schema: None, - }, - }]); - } + Err(err) => return Drop::with_errors(vec![err]), }; // Execute the globally collected, pre-ordered notifications last! @@ -97,11 +57,11 @@ impl Merger { return Drop::with_errors(vec![Error { code: "MERGE_FAILED".to_string(), values: Some(IndexMap::from([ - ("error".to_string(), e.to_string()), + ("error".to_string(), e.clone()), ])), details: ErrorDetails { path: None, - cause: None, + cause: Some(e), context: None, schema: None, }, @@ -144,7 +104,7 @@ impl Merger { notifications: &mut Vec, parent_org_id: Option, is_child: bool, - ) -> Result { + ) -> Result { match data { Value::Array(items) => { self.merge_array(schema, items, notifications, parent_org_id, is_child) @@ -159,10 +119,14 @@ impl Merger { if let Some(target_schema) = self.db.schemas.get(target_id) { schema = target_schema.clone(); } else { - return Err(format!( - "Polymorphic mapped target '{}' not found in database registry", - target_id - )); + return Err(Error { + code: "TARGET_SCHEMA_NOT_FOUND".to_string(), + values: Some(IndexMap::from([("target_id".to_string(), target_id.clone())])), + details: ErrorDetails { + cause: Some(format!("Polymorphic mapped target '{}' not found in database registry", target_id)), + ..Default::default() + }, + }); } } else if let Some(idx) = idx_opt { if let Some(target_schema) = schema @@ -173,31 +137,60 @@ impl Merger { { schema = Arc::clone(target_schema); } else { - return Err(format!( - "Polymorphic index target '{}' not found in local oneOf array", - idx - )); + return Err(Error { + code: "ONE_OF_INDEX_NOT_FOUND".to_string(), + values: Some(IndexMap::from([("index".to_string(), idx.to_string())])), + details: ErrorDetails { + cause: Some(format!("Polymorphic index target '{}' not found in local oneOf array", idx)), + ..Default::default() + }, + }); } } else { - return Err(format!("Polymorphic mapped target has no path")); + return Err(Error { + code: "INVALID_POLYMORPHIC_TARGET".to_string(), + values: None, + details: ErrorDetails { + cause: Some("Polymorphic mapped target has no path".to_string()), + ..Default::default() + }, + }); } } else { - return Err(format!( - "Polymorphic discriminator {}='{}' matched no compiled options", - disc, v - )); + return Err(Error { + code: "DISCRIMINATOR_MISMATCH".to_string(), + values: Some(IndexMap::from([ + ("discriminator".to_string(), disc.to_string()), + ("value".to_string(), v.to_string()), + ])), + details: ErrorDetails { + cause: Some(format!("Polymorphic discriminator {}='{}' matched no compiled options", disc, v)), + ..Default::default() + }, + }); } } else { - return Err(format!( - "Polymorphic merging failed: missing required discriminator '{}'", - disc - )); + return Err(Error { + code: "MISSING_DISCRIMINATOR".to_string(), + values: Some(IndexMap::from([("discriminator".to_string(), disc.to_string())])), + details: ErrorDetails { + cause: Some(format!("Polymorphic merging failed: missing required discriminator '{}'", disc)), + ..Default::default() + }, + }); } } } self.merge_object(schema, map, notifications, parent_org_id, is_child) } - _ => Err("Invalid merge payload: root must be an Object or Array".to_string()), + _ => Err(Error { + code: "INVALID_MERGE_PAYLOAD".to_string(), + values: None, + details: ErrorDetails { + cause: Some("Invalid merge payload: root must be an Object or Array".to_string()), + ..Default::default() + }, + }), } } @@ -208,7 +201,7 @@ impl Merger { notifications: &mut Vec, parent_org_id: Option, is_child: bool, - ) -> Result { + ) -> Result { let mut item_schema = schema.clone(); if let Some(crate::database::object::SchemaTypeOrArray::Single(t)) = &schema.obj.type_ { if t == "array" { @@ -239,22 +232,49 @@ impl Merger { notifications: &mut Vec, parent_org_id: Option, is_child: bool, - ) -> Result { + ) -> Result { let queue_start = notifications.len(); let type_name = match obj.get("type").and_then(|v| v.as_str()) { Some(t) => t.to_string(), - None => return Err("Missing required 'type' field on object".to_string()), + None => { + return Err(Error { + code: "MISSING_TYPE".to_string(), + values: None, + details: ErrorDetails { + cause: Some("Missing required 'type' field on object".to_string()), + ..Default::default() + }, + }); + } }; let type_def = match self.db.types.get(&type_name) { Some(t) => t, - None => return Err(format!("Unknown entity type: {}", type_name)), + None => { + return Err(Error { + code: "UNKNOWN_ENTITY_TYPE".to_string(), + values: Some(IndexMap::from([("type".to_string(), type_name.clone())])), + details: ErrorDetails { + cause: Some(format!("Unknown entity type: {}", type_name)), + ..Default::default() + }, + }); + } }; let compiled_props = match schema.obj.compiled_properties.get() { Some(props) => props, - None => return Err("Schema has no compiled properties for merging".to_string()), + None => { + return Err(Error { + code: "UNCOMPILED_SCHEMA".to_string(), + values: None, + details: ErrorDetails { + cause: Some("Schema has no compiled properties for merging".to_string()), + ..Default::default() + }, + }); + } }; let mut entity_fields = serde_json::Map::new(); @@ -269,6 +289,10 @@ impl Merger { } if let Some(prop_schema) = compiled_props.get(&k) { + if prop_schema.is_immutable() { + continue; + } + let mut is_edge = false; if let Some(edges) = schema.obj.compiled_edges.get() { if edges.contains_key(&k) { @@ -312,8 +336,22 @@ impl Merger { current_org_id = parent_org_id.clone(); } - let user_id = self.db.auth_user_id()?; - let timestamp = self.db.timestamp()?; + let user_id = self.db.auth_user_id().map_err(|e| Error { + code: "AUTH_USER_FAILED".to_string(), + values: Some(IndexMap::from([("error".to_string(), e.clone())])), + details: ErrorDetails { + cause: Some(e), + ..Default::default() + }, + })?; + let timestamp = self.db.timestamp().map_err(|e| Error { + code: "TIMESTAMP_FAILED".to_string(), + values: Some(IndexMap::from([("error".to_string(), e.clone())])), + details: ErrorDetails { + cause: Some(e), + ..Default::default() + }, + })?; let mut entity_change_kind = None; let mut entity_fetched = None; @@ -328,6 +366,27 @@ impl Merger { entity_replaces = replaces; if entity_change_kind.as_deref() == Some("create") { + if let Some(deps) = &schema.obj.dependencies { + if let Some(crate::database::object::Dependency::Props(req_props)) = deps.get("created") { + for req in req_props { + if !entity_fields.contains_key(req) && !entity_objects.contains_key(req) && !entity_arrays.contains_key(req) { + return Err(Error { + code: "REQUIRED_FIELD_MISSING".to_string(), + values: Some(IndexMap::from([ + ("property_name".to_string(), req.to_string()), + ("entity_type".to_string(), type_name.clone()), + ])), + details: ErrorDetails { + path: Some(req.to_string()), + cause: Some(format!("Missing required creation field '{}' for entity {}", req, type_name)), + ..Default::default() + }, + }); + } + } + } + } + if is_child { if !entity_fields.contains_key("organization_id") { if let Some(ref org_id) = current_org_id { @@ -538,7 +597,7 @@ impl Merger { Option>, Option, ), - String, + Error, > { let type_name = type_def.name.as_str(); @@ -673,29 +732,36 @@ impl Merger { &self, entity_fields: &serde_json::Map, entity_type: &crate::database::r#type::Type, - ) -> Result>, String> { + ) -> Result>, Error> { let id_val = entity_fields.get("id"); let entity_type_name = entity_type.name.as_str(); - let mut lookup_complete = false; - if !entity_type.lookup_fields.is_empty() { - lookup_complete = true; - for column in &entity_type.lookup_fields { - match entity_fields.get(column) { - Some(Value::Null) | None => { - lookup_complete = false; - break; + let mut lookup_satisfied_keys = Vec::new(); + for parent_type_name in entity_type.hierarchy.iter().rev() { + if let Some(parent_type) = self.db.types.get(parent_type_name) { + if !parent_type.lookup_fields.is_empty() { + let mut lookup_complete = true; + for column in &parent_type.lookup_fields { + match entity_fields.get(column) { + Some(Value::Null) | None => { + lookup_complete = false; + break; + } + Some(Value::String(s)) if s.is_empty() => { + lookup_complete = false; + break; + } + _ => {} + } } - Some(Value::String(s)) if s.is_empty() => { - lookup_complete = false; - break; + if lookup_complete { + lookup_satisfied_keys.push(&parent_type.lookup_fields); } - _ => {} } } } - if id_val.is_none() && !lookup_complete { + if id_val.is_none() && lookup_satisfied_keys.is_empty() { return Ok(None); } @@ -727,9 +793,9 @@ impl Merger { where_parts.push(format!("t1.id = {}", Self::quote_literal(id))); } - if lookup_complete { + for lookup_fields in lookup_satisfied_keys { let mut lookup_predicates = Vec::new(); - for column in &entity_type.lookup_fields { + for column in lookup_fields { let val = entity_fields.get(column).unwrap_or(&Value::Null); if column == "type" { lookup_predicates.push(format!("t1.\"{}\" = {}", column, Self::quote_literal(val))); @@ -757,22 +823,47 @@ impl Merger { let fetched = match self.db.query(&final_sql, None) { Ok(Value::Array(table)) => { if table.len() > 1 { - Err(format!( - "TOO_MANY_LOOKUP_ROWS: Lookup for {} found too many existing rows", - entity_type_name - )) + Err(Error { + code: "TOO_MANY_LOOKUP_ROWS".to_string(), + values: Some(IndexMap::from([("entity_type".to_string(), entity_type_name.to_string())])), + details: ErrorDetails { + cause: Some(format!("Lookup for {} found too many existing rows", entity_type_name)), + ..Default::default() + }, + }) } else if table.is_empty() { Ok(None) } else { let row = table.first().unwrap(); match row { Value::Object(map) => Ok(Some(map.clone())), - other => Err(format!("Expected JSON object, got: {:?}", other)), + other => Err(Error { + code: "UNEXPECTED_QUERY_RESULT".to_string(), + values: None, + details: ErrorDetails { + cause: Some(format!("Expected JSON object, got: {:?}", other)), + ..Default::default() + }, + }), } } } - Ok(_) => Err("Expected array from query in fetch_entity".to_string()), - Err(e) => Err(format!("SPI error in fetch_entity: {:?}", e)), + Ok(_) => Err(Error { + code: "UNEXPECTED_QUERY_RESULT".to_string(), + values: None, + details: ErrorDetails { + cause: Some("Expected array from query in fetch_entity".to_string()), + ..Default::default() + }, + }), + Err(e) => Err(Error { + code: "DATABASE_SPI_ERROR".to_string(), + values: Some(IndexMap::from([("error".to_string(), e.clone())])), + details: ErrorDetails { + cause: Some(format!("SPI error in fetch_entity: {:?}", e)), + ..Default::default() + }, + }), }?; Ok(fetched) @@ -785,23 +876,36 @@ impl Merger { entity_type: &crate::database::r#type::Type, entity_fields: &serde_json::Map, _entity_fetched: Option<&serde_json::Map>, - ) -> Result<(), String> { + ) -> Result<(), Error> { if change_kind.is_empty() { return Ok(()); } let id_str = match entity_fields.get("id").and_then(|v| v.as_str()) { Some(id) => id, - None => return Err("Missing 'id' for merge execution".to_string()), + None => { + return Err(Error { + code: "MISSING_ENTITY_ID".to_string(), + values: None, + details: ErrorDetails { + cause: Some("Missing 'id' for merge execution".to_string()), + ..Default::default() + }, + }); + } }; let grouped_fields = match &entity_type.grouped_fields { Some(Value::Object(map)) => map, _ => { - return Err(format!( - "Grouped fields missing for type {}", - entity_type_name - )); + return Err(Error { + code: "MISSING_GROUPED_FIELDS".to_string(), + values: Some(IndexMap::from([("type".to_string(), entity_type_name.to_string())])), + details: ErrorDetails { + cause: Some(format!("Grouped fields missing for type {}", entity_type_name)), + ..Default::default() + }, + }); } }; @@ -854,7 +958,16 @@ impl Merger { columns.join(", "), values.join(", ") ); - self.db.execute(&sql, None)?; + if let Err(e) = self.db.execute(&sql, None) { + return Err(Error { + code: "DATABASE_SPI_ERROR".to_string(), + values: Some(IndexMap::from([("error".to_string(), e.clone())])), + details: ErrorDetails { + cause: Some(e), + ..Default::default() + }, + }); + } } else if change_kind == "update" || change_kind == "delete" { entity_pairs.remove("id"); entity_pairs.remove("type"); @@ -886,7 +999,16 @@ impl Merger { set_clauses.join(", "), Self::quote_literal(&Value::String(id_str.to_string())) ); - self.db.execute(&sql, None)?; + if let Err(e) = self.db.execute(&sql, None) { + return Err(Error { + code: "DATABASE_SPI_ERROR".to_string(), + values: Some(IndexMap::from([("error".to_string(), e.clone())])), + details: ErrorDetails { + cause: Some(e), + ..Default::default() + }, + }); + } } } @@ -902,7 +1024,7 @@ impl Merger { user_id: &str, timestamp: &str, replaces_id: Option<&str>, - ) -> Result, String> { + ) -> Result, Error> { let change_kind = match entity_change_kind { Some(k) => k, None => return Ok(None), @@ -995,7 +1117,16 @@ impl Merger { Self::quote_literal(&Value::String(user_id.to_string())) ); - self.db.execute(&change_sql, None)?; + if let Err(e) = self.db.execute(&change_sql, None) { + return Err(Error { + code: "DATABASE_SPI_ERROR".to_string(), + values: Some(IndexMap::from([("error".to_string(), e.clone())])), + details: ErrorDetails { + cause: Some(e), + ..Default::default() + }, + }); + } } if type_obj.notify { diff --git a/src/tests/fixtures.rs b/src/tests/fixtures.rs index ab10a57..a0efae1 100644 --- a/src/tests/fixtures.rs +++ b/src/tests/fixtures.rs @@ -2519,6 +2519,18 @@ fn test_properties_12_0() { crate::tests::runner::run_test_case(&path, 12, 0).unwrap(); } +#[test] +fn test_properties_13_0() { + let path = format!("{}/fixtures/properties.json", env!("CARGO_MANIFEST_DIR")); + crate::tests::runner::run_test_case(&path, 13, 0).unwrap(); +} + +#[test] +fn test_properties_13_1() { + let path = format!("{}/fixtures/properties.json", env!("CARGO_MANIFEST_DIR")); + crate::tests::runner::run_test_case(&path, 13, 1).unwrap(); +} + #[test] fn test_max_contains_0_0() { let path = format!("{}/fixtures/maxContains.json", env!("CARGO_MANIFEST_DIR")); @@ -3407,6 +3419,18 @@ fn test_database_6_0() { crate::tests::runner::run_test_case(&path, 6, 0).unwrap(); } +#[test] +fn test_database_7_0() { + let path = format!("{}/fixtures/database.json", env!("CARGO_MANIFEST_DIR")); + crate::tests::runner::run_test_case(&path, 7, 0).unwrap(); +} + +#[test] +fn test_database_8_0() { + let path = format!("{}/fixtures/database.json", env!("CARGO_MANIFEST_DIR")); + crate::tests::runner::run_test_case(&path, 8, 0).unwrap(); +} + #[test] fn test_cases_0_0() { let path = format!("{}/fixtures/cases.json", env!("CARGO_MANIFEST_DIR")); @@ -7666,3 +7690,9 @@ fn test_merger_0_15() { let path = format!("{}/fixtures/merger.json", env!("CARGO_MANIFEST_DIR")); crate::tests::runner::run_test_case(&path, 0, 15).unwrap(); } + +#[test] +fn test_merger_0_16() { + let path = format!("{}/fixtures/merger.json", env!("CARGO_MANIFEST_DIR")); + crate::tests::runner::run_test_case(&path, 0, 16).unwrap(); +} diff --git a/src/validator/context.rs b/src/validator/context.rs index 94fa39a..4991b65 100644 --- a/src/validator/context.rs +++ b/src/validator/context.rs @@ -16,6 +16,7 @@ pub struct ValidationContext<'a> { pub reporter: bool, pub overrides: HashSet, pub parents: Vec<&'a serde_json::Value>, + pub response: bool, } impl<'a> ValidationContext<'a> { @@ -27,6 +28,7 @@ impl<'a> ValidationContext<'a> { overrides: HashSet, extensible: bool, reporter: bool, + response: bool, ) -> Self { let effective_extensible = schema.extensible.unwrap_or(extensible); Self { @@ -40,6 +42,7 @@ impl<'a> ValidationContext<'a> { reporter, overrides, parents: Vec::new(), + response, } } @@ -79,6 +82,7 @@ impl<'a> ValidationContext<'a> { reporter, overrides, parents, + response: self.response, } } diff --git a/src/validator/mod.rs b/src/validator/mod.rs index 3e82064..58b682b 100644 --- a/src/validator/mod.rs +++ b/src/validator/mod.rs @@ -48,6 +48,7 @@ impl Validator { let schema_opt = self.db.schemas.get(schema_id); if let Some(schema) = schema_opt { + let response = schema_id.ends_with(".response"); let ctx = ValidationContext::new( &self.db, &schema, @@ -56,6 +57,7 @@ impl Validator { HashSet::new(), false, false, + response, ); match ctx.validate_scoped() { Ok(result) => { diff --git a/src/validator/rules/object.rs b/src/validator/rules/object.rs index e7d1675..08ea9fe 100644 --- a/src/validator/rules/object.rs +++ b/src/validator/rules/object.rs @@ -179,6 +179,34 @@ impl<'a> ValidationContext<'a> { } } + if !self.response { + if let Some(compiled_props) = self.schema.compiled_properties.get() { + for (key, sub_schema) in compiled_props { + if sub_schema.is_immutable() && obj.contains_key(key) { + result.errors.push(ValidationError { + code: "IMMUTABLE_PROPERTY_VIOLATION".to_string(), + values: Some(IndexMap::from([ + ("property_name".to_string(), key.to_string()), + ])), + path: self.join_path(key), + }); + } + } + } else if let Some(props) = &self.schema.properties { + for (key, sub_schema) in props { + if sub_schema.is_immutable() && obj.contains_key(key) { + result.errors.push(ValidationError { + code: "IMMUTABLE_PROPERTY_VIOLATION".to_string(), + values: Some(IndexMap::from([ + ("property_name".to_string(), key.to_string()), + ])), + path: self.join_path(key), + }); + } + } + } + } + if let Some(props) = &self.schema.properties { for (key, sub_schema) in props { if self.overrides.contains(key) { @@ -228,6 +256,7 @@ impl<'a> ValidationContext<'a> { HashSet::new(), self.extensible, self.reporter, + self.response, ); result.merge(ctx.validate()?); diff --git a/version b/version index c2667db..d077126 100644 --- a/version +++ b/version @@ -1 +1 @@ -1.0.181 +1.0.183