Compare commits

..

27 Commits

Author SHA1 Message Date
15e826cbcb version: 1.0.199 2026-09-01 12:22:03 -04:00
f17ee3b543 fixed invalid use of postgres auth and punc session vars 2026-09-01 12:21:48 -04:00
90262fc82b version: 1.0.198 2026-08-25 13:11:15 -04:00
8a35b8eba4 fixed test issues with lookups 2026-08-25 13:11:10 -04:00
61d07b3c68 version: 1.0.197 2026-08-22 21:48:46 -04:00
d08181e1d7 added alias names to lookup queries for merger 2026-08-22 21:48:40 -04:00
a232bd4727 doc updates for field_defaults usage 2026-08-03 23:33:40 -04:00
8865123b1b version: 1.0.196 2026-08-03 22:20:49 -04:00
f8bdac3428 added defaults support for lookups properly 2026-08-03 22:20:39 -04:00
99ae2a7e89 version: 1.0.195 2026-08-03 21:23:09 -04:00
53776fc696 added defaults support for lookups 2026-08-03 21:22:27 -04:00
d43be1def4 doc update 2026-08-03 19:35:07 -04:00
37bce3ce61 version: 1.0.194 2026-08-03 19:18:55 -04:00
ab4df37fd0 fixed more query volatility issues 2026-08-03 19:18:50 -04:00
3414a32bd6 version: 1.0.193 2026-08-03 19:09:54 -04:00
06432cf0f5 fixed volatile issues with merge 2026-08-03 19:09:46 -04:00
0d4b7c3dec version: 1.0.192 2026-08-03 18:59:16 -04:00
b2f9fe387c added returning statements to inserts and updates for trigger-based mutations 2026-08-03 18:57:08 -04:00
020286d603 version: 1.0.191 2026-08-03 16:40:36 -04:00
1619ca4400 fixed hierarchy null jsonb merging issue with merger when a part of the more specific hierarchy does not exist yet 2026-08-03 16:40:26 -04:00
580fa0bc9f version: 1.0.190 2026-08-03 15:00:13 -04:00
2dfb4e68e4 fixed immutability effects in merger 2026-08-03 15:00:03 -04:00
ae31230ef2 version: 1.0.189 2026-08-03 12:08:46 -04:00
ce542a31cc version: 1.0.188 2026-08-03 12:08:26 -04:00
75296ab107 version: 1.0.187 2026-08-03 12:05:53 -04:00
1a4b328b45 merged in main 2026-08-03 12:05:39 -04:00
c5e103c867 immutable tri-state 2026-08-03 12:03:03 -04:00
18 changed files with 830 additions and 702 deletions

View File

@ -184,23 +184,27 @@ It evaluates as an **Independent Declarative Rules Engine**. Every `Case` block
* **`array`**: Homogeneous collection of items matching the `items` schema. Validation is homogeneous, so strictness checking does not apply. `"extensible"` is not applicable at the `array` level (tuple-like `prefixItems` are removed). * **`array`**: Homogeneous collection of items matching the `items` schema. Validation is homogeneous, so strictness checking does not apply. `"extensible"` is not applicable at the `array` level (tuple-like `prefixItems` are removed).
* **Inheritance Boundaries**: Strictness resets when crossing non-primitive `type` boundaries. A schema extending a strict parent remains strict unless it explicitly overrides with `"extensible": true`. * **Inheritance Boundaries**: Strictness resets when crossing non-primitive `type` boundaries. A schema extending a strict parent remains strict unless it explicitly overrides with `"extensible": true`.
### Immutable Properties (`"immutable": true`) ### Immutable Properties (`"immutable": "always" | "external"`)
To distinguish read-only hydrated endpoint references, computed properties, system-managed timestamps (`created_at`, `modified_at`), or audit fields from writable properties, JSPG introduces the universal `"immutable": true` schema keyword. To distinguish read-only hydrated endpoint references, computed properties, system-managed timestamps (`created_at`, `modified_at`), or audit fields from writable properties, JSPG supports the `"immutable"` property schema attribute, which takes string enum values (`"always"`, `"external"`, or omitted).
* **Developer Perspective**: Annotate properties in database schemas or trait definitions with `"immutable": true` when the property should be visible on reads (`jspg_query` / `.response`), but rejected or ignored on writes (`jspg_merge` / `.request`). * **Enum Values**:
* `"always"`: Property is permanently read-only across all boundaries (e.g. system-managed audit timestamps like `created_at`, `modified_at`, or computed fields).
* `"external"`: Property is read-only from external client API requests (`.request` payloads), but can be populated or mutated by internal system operations.
* Omitted: Property is fully writable.
* **Developer Perspective**: Annotate properties in database schemas or trait definitions with `"immutable": "always"` or `"immutable": "external"` when the property should be visible on reads (`jspg_query` / `.response`), but rejected or ignored on writes (`jspg_merge` / `.request`).
```json ```json
"properties": { "properties": {
"source": { "source": {
"family": "lite.organization", "family": "lite.organization",
"immutable": true, "immutable": "always",
"description": "Read-only hydrated member entity summary on a membership edge." "description": "Read-only hydrated member entity summary on a membership edge."
} }
} }
``` ```
* **Behavior Across Pillars**: * **Behavior Across Pillars**:
* **Queryer (`jspg_query`)**: Hydrates and includes `immutable` properties in output read responses without restriction. * **Queryer (`jspg_query`)**: Hydrates and includes `"immutable"` properties in output read responses without restriction.
* **Validator (`jspg_validate`)**: Context-aware. Returns `IMMUTABLE_PROPERTY_VIOLATION` if an `immutable` property is supplied in a write/request payload (schema IDs not ending in `.response`). Response schemas (`.response`) permit `immutable` fields. * **Validator (`jspg_validate`)**: Context-aware. Rejects specified `"immutable"` properties supplied in write/request payloads. Response schemas (`.response`) permit `"immutable"` fields.
* **Merger (`jspg_merge`)**: Automatically skips `immutable` properties during object graph merging so client payloads can never mutate database columns or relationship edges. * **Merger (`jspg_merge`)**: Automatically skips `"immutable"` properties during object graph merging so client payloads can never mutate database columns or relationship edges.
### Format Leniency for Empty Strings ### Format Leniency for Empty Strings
To simplify frontend form validation, format validators specifically for `uuid`, `date-time`, and `email` explicitly allow empty strings (`""`), treating them as "present but unset". To simplify frontend form validation, format validators specifically for `uuid`, `date-time`, and `email` explicitly allow empty strings (`""`), treating them as "present but unset".
@ -339,7 +343,7 @@ The Merger provides an automated, high-performance graph synchronization engine.
* **Factual Creation Dependency Validation**: When `stage_entity` confirms `kind == "create"` via database lookup, `jspg_merge` evaluates `schema.obj.dependencies.get("created")`. If required creation fields are missing from `entity_fields`, `entity_objects`, or `entity_arrays`, `jspg_merge` halts and returns a structured `REQUIRED_FIELD_MISSING` `Drop`. * **Factual Creation Dependency Validation**: When `stage_entity` confirms `kind == "create"` via database lookup, `jspg_merge` evaluates `schema.obj.dependencies.get("created")`. If required creation fields are missing from `entity_fields`, `entity_objects`, or `entity_arrays`, `jspg_merge` halts and returns a structured `REQUIRED_FIELD_MISSING` `Drop`.
* **Structured Fail-Fast Error Propagation**: Internal merger operations return typed `Result<T, crate::drop::Error>` instances, allowing clean fail-fast `?` traversal and returning `Drop::with_errors(vec![err])` directly at the top-level API boundary on failure. * **Structured Fail-Fast Error Propagation**: Internal merger operations return typed `Result<T, crate::drop::Error>` instances, allowing clean fail-fast `?` traversal and returning `Drop::with_errors(vec![err])` directly at the top-level API boundary on failure.
* **Prefix Foreign Key Matching**: Handles scenario where multiple relations point to the same table by using database Foreign Key constraint prefixes (`fk_`). For example, if a schema has `shipping_address` and `billing_address`, the merger resolves against `fk_shipping_address_entity` vs `fk_billing_address_entity` automatically to correctly route object properties. * **Prefix Foreign Key Matching**: Handles scenario where multiple relations point to the same table by using database Foreign Key constraint prefixes (`fk_`). For example, if a schema has `shipping_address` and `billing_address`, the merger resolves against `fk_shipping_address_entity` vs `fk_billing_address_entity` automatically to correctly route object properties.
* **Dynamic Deduplication & Lookups**: If a nested object is provided without an `id`, the Merger utilizes custom `lookup_fields` declared directly in the schema registry JSON comments. It validates at setup compile-time that a corresponding unique index exists in PostgreSQL for these fields. When merging, it dynamically builds query predicates for any satisfied `lookup_fields` sets in the entity's type hierarchy (checking child-to-parent hierarchies order-independently and combining satisfied keys with `UNION` queries) to discover the correct UUID to perform an UPDATE, preventing data duplication. * **Dynamic Deduplication & Lookups (`lookup_fields` & `field_defaults`)**: If an entity payload is provided without an `id`, the Merger utilizes custom `lookup_fields` declared directly in the schema metadata (`COMMENT ON TABLE`). When searching for an existing database record (`fetch_entity`), it iterates candidate `lookup_fields` sets across the entity's type hierarchy. For any column in a lookup set that is omitted from the input payload, `fetch_entity` checks for a default value in `field_defaults` (populated from Postgres table column `DEFAULT` expressions during setup). If a default is found (e.g. `start_date: "0001-01-01T00:00:00+00:00"`), `fetch_entity` uses that default value to satisfy the lookup query predicate (`WHERE ... AND start_date = ...`). Crucially, `entity_fields` itself is **never mutated** with defaults during `merge_entity`, ensuring input payloads remain pure, preventing false `modified_at` updates or spurious change notifications, and allowing PostgreSQL to handle column default generation natively server-side on `INSERT`.
* **Hierarchical Table Inheritance**: The Punc system uses distributed table inheritance (e.g. `person` inherits `user` inherits `organization` inherits `entity`). The Merger splits the incoming JSON payload and performs atomic row updates across *all* relevant tables in the lineage map. * **Hierarchical Table Inheritance**: The Punc system uses distributed table inheritance (e.g. `person` inherits `user` inherits `organization` inherits `entity`). The Merger splits the incoming JSON payload and performs atomic row updates across *all* relevant tables in the lineage map.
* **The Archive Paradigm**: Data is never deleted in the Punc system. The Merger securely enforces referential integrity by toggling the `archived` Boolean flag on the base `entity` table rather than issuing SQL `DELETE` commands. * **The Archive Paradigm**: Data is never deleted in the Punc system. The Merger securely enforces referential integrity by toggling the `archived` Boolean flag on the base `entity` table rather than issuing SQL `DELETE` commands.
* **Change Tracking & Reactivity**: The Merger diffs the incoming JSON against the existing database row (utilizing static, `DashMap`-cached `lookup` SELECT string templates). Every detected change is recorded into the `agreego.change` audit table, tracking the user mapping. It then natively uses `pg_notify` to broadcast a completely flat row-level diff out to the Go WebSocket server for O(1) routing. * **Change Tracking & Reactivity**: The Merger diffs the incoming JSON against the existing database row (utilizing static, `DashMap`-cached `lookup` SELECT string templates). Every detected change is recorded into the `agreego.change` audit table, tracking the user mapping. It then natively uses `pg_notify` to broadcast a completely flat row-level diff out to the Go WebSocket server for O(1) routing.
@ -370,6 +374,7 @@ The Queryer transforms Postgres into a pre-compiled Semantic Query Engine, desig
* **Multi-Table Branching**: If the Physical Table is a parent to other tables (e.g. `organization` has variations `["organization", "bot", "person"]`), the compiler generates a dynamic `CASE WHEN type = '...' THEN ...` query, expanding into sub-queries for each variation. To ensure safe resolution, the compiler dynamically evaluates correlation boundaries: it attempts standard Relational Edge discovery first. If no explicit relational edge exists (indicating pure Table Inheritance rather than a standard foreign-key graph relationship), it safely invokes a **Table Parity Fallback**. This generates an explicit ID correlation constraint (`AND inner.id = outer.id`), perfectly binding the structural variations back to the parent row to eliminate Cartesian products. * **Multi-Table Branching**: If the Physical Table is a parent to other tables (e.g. `organization` has variations `["organization", "bot", "person"]`), the compiler generates a dynamic `CASE WHEN type = '...' THEN ...` query, expanding into sub-queries for each variation. To ensure safe resolution, the compiler dynamically evaluates correlation boundaries: it attempts standard Relational Edge discovery first. If no explicit relational edge exists (indicating pure Table Inheritance rather than a standard foreign-key graph relationship), it safely invokes a **Table Parity Fallback**. This generates an explicit ID correlation constraint (`AND inner.id = outer.id`), perfectly binding the structural variations back to the parent row to eliminate Cartesian products.
* **Single-Table Bypass**: If the Physical Table is a leaf node with only one variation (e.g. `person` has variations `["person"]`), the compiler cleanly bypasses `CASE` generation and compiles a simple `SELECT` across the base table, as all schema extensions (e.g. `light.person`, `full.person`) are guaranteed to reside in the exact same physical row. * **Single-Table Bypass**: If the Physical Table is a leaf node with only one variation (e.g. `person` has variations `["person"]`), the compiler cleanly bypasses `CASE` generation and compiles a simple `SELECT` across the base table, as all schema extensions (e.g. `light.person`, `full.person`) are guaranteed to reside in the exact same physical row.
* **Polymorphic Relation Type Filtering**: When a relationship maps to a polymorphic target with variations, the Queryer compiles an `IN` clause containing all allowed table variations (e.g., `counterparty_type IN ('bot', 'organization', 'person')`) rather than matching the base type literal, ensuring all polymorphic types are loaded correctly. * **Polymorphic Relation Type Filtering**: When a relationship maps to a polymorphic target with variations, the Queryer compiles an `IN` clause containing all allowed table variations (e.g., `counterparty_type IN ('bot', 'organization', 'person')`) rather than matching the base type literal, ensuring all polymorphic types are loaded correctly.
* **Tenant Scoping & ReBAC Filtering**: When `jspg_query` compiles root-level or relationship-level entity queries, the generated SQL queries evaluate within the calling PostgreSQL transaction's session configuration (e.g. `punc.cue.organization_ids`, `punc.cue.roles`). The compiled queries seamlessly interact with PostgreSQL Row Level Security (RLS) policies on base tables (such as `agreego.entity`) and pre-materialized Zanzibar graph edges (`agreego.relationship`), ensuring multi-tenant isolation and $O(1)$ query evaluation without requiring manual WHERE scoping logic across every application punc.
* **Static Relation Constraints (Kind Constraints)**: When a relationship (such as a nested object or array) is defined with a schema that constrains a field value statically using a `const` or `enum` keyword (for example, `kind` constrained to `"cover"` in a `cover_attachment`), the Queryer automatically extracts these static assertions during AST compilation. It injects them directly as static filters into the SQL subquery's `WHERE` clause (e.g. `AND attachment.kind = 'cover'`), allowing developers to query pre-filtered subsets of related tables natively through the schema. * **Static Relation Constraints (Kind Constraints)**: When a relationship (such as a nested object or array) is defined with a schema that constrains a field value statically using a `const` or `enum` keyword (for example, `kind` constrained to `"cover"` in a `cover_attachment`), the Queryer automatically extracts these static assertions during AST compilation. It injects them directly as static filters into the SQL subquery's `WHERE` clause (e.g. `AND attachment.kind = 'cover'`), allowing developers to query pre-filtered subsets of related tables natively through the schema.
* **Proxy Schema Dereferencing / Resolution**: To support punc endpoints that return non-polymorphic table-backed shapes (using `type: "full.X"` proxy schemas at the root response level), the Queryer compiler automatically dereferences non-table schema pointers to their target schemas prior to checking the types. This allows the Queryer to correctly resolve the table relationship edges pre-compiled on the full schema, while avoiding polluting the database registry with relations on ad-hoc punc response schemas during setup. * **Proxy Schema Dereferencing / Resolution**: To support punc endpoints that return non-polymorphic table-backed shapes (using `type: "full.X"` proxy schemas at the root response level), the Queryer compiler automatically dereferences non-table schema pointers to their target schemas prior to checking the types. This allows the Queryer to correctly resolve the table relationship edges pre-compiled on the full schema, while avoiding polluting the database registry with relations on ad-hoc punc response schemas during setup.

View File

@ -1027,16 +1027,31 @@
"name": "user", "name": "user",
"module": "test", "module": "test",
"source": "test", "source": "test",
"hierarchy": ["user"], "hierarchy": [
"variations": ["user"], "user"
"fields": ["id", "email"], ],
"lookup_fields": ["email"], "variations": [
"user"
],
"fields": [
"id",
"email"
],
"lookup_fields": [
"email"
],
"schemas": { "schemas": {
"user": { "user": {
"type": "object", "type": "object",
"properties": { "properties": {
"id": { "type": "string", "format": "uuid" }, "id": {
"email": { "type": "string", "format": "email" } "type": "string",
"format": "uuid"
},
"email": {
"type": "string",
"format": "email"
}
} }
} }
} }
@ -1072,7 +1087,9 @@
"indexes": [ "indexes": [
{ {
"table": "user", "table": "user",
"columns": ["email"] "columns": [
"email"
]
} }
], ],
"types": [ "types": [
@ -1082,16 +1099,31 @@
"name": "user", "name": "user",
"module": "test", "module": "test",
"source": "test", "source": "test",
"hierarchy": ["user"], "hierarchy": [
"variations": ["user"], "user"
"fields": ["id", "email"], ],
"lookup_fields": ["email"], "variations": [
"user"
],
"fields": [
"id",
"email"
],
"lookup_fields": [
"email"
],
"schemas": { "schemas": {
"user": { "user": {
"type": "object", "type": "object",
"properties": { "properties": {
"id": { "type": "string", "format": "uuid" }, "id": {
"email": { "type": "string", "format": "email" } "type": "string",
"format": "uuid"
},
"email": {
"type": "string",
"format": "email"
}
} }
} }
} }

File diff suppressed because it is too large Load Diff

View File

@ -888,7 +888,7 @@
}, },
"created_at": { "created_at": {
"type": "string", "type": "string",
"immutable": true "immutable": "always"
} }
} }
}, },
@ -899,7 +899,7 @@
}, },
"created_at": { "created_at": {
"type": "string", "type": "string",
"immutable": true "immutable": "always"
} }
} }
} }
@ -945,5 +945,70 @@
} }
} }
] ]
},
{
"description": "immutable external vs always property validation",
"database": {
"types": [
{
"name": "invoice",
"schemas": {
"save_invoice.request": {
"properties": {
"id": {
"type": "string"
},
"status": {
"type": "string",
"immutable": "external"
},
"created_at": {
"type": "string",
"immutable": "always"
}
}
}
}
}
]
},
"tests": [
{
"description": "immutable external property in request context allowed when internal (punc.external = false)",
"data": {
"id": "123",
"status": "paid"
},
"schema_id": "save_invoice.request",
"action": "validate",
"expect": {
"success": true
}
},
{
"description": "immutable always property in request context rejected even when internal (punc.external = false)",
"data": {
"id": "123",
"created_at": "2026-07-21T00:00:00Z"
},
"schema_id": "save_invoice.request",
"action": "validate",
"expect": {
"success": false,
"errors": [
{
"code": "IMMUTABLE_PROPERTY_VIOLATION",
"values": {
"property_name": "created_at"
},
"details": {
"path": "created_at",
"schema": "save_invoice.request"
}
}
]
}
}
]
} }
] ]

View File

@ -260,12 +260,9 @@
}, },
"lookup_fields": [], "lookup_fields": [],
"null_fields": [], "null_fields": [],
"default_fields": [ "field_defaults": {
"id", "archived": false
"type", },
"created_at",
"archived"
],
"variations": [ "variations": [
"bot", "bot",
"organization", "organization",
@ -1739,20 +1736,20 @@
" AND entity_1.created_at <= ($16 #>> '{}')::TIMESTAMPTZ", " AND entity_1.created_at <= ($16 #>> '{}')::TIMESTAMPTZ",
" AND entity_1.created_at <> ($17 #>> '{}')::TIMESTAMPTZ", " AND entity_1.created_at <> ($17 #>> '{}')::TIMESTAMPTZ",
" AND person_3.first_name ILIKE $18 #>> '{}'", " AND person_3.first_name ILIKE $18 #>> '{}'",
" AND person_3.first_name > ($19 #>> '{)", " AND person_3.first_name > ($19 #>> '{}')",
" AND person_3.first_name >= ($20 #>> '{)", " AND person_3.first_name >= ($20 #>> '{}')",
" AND person_3.first_name < ($21 #>> '{)", " AND person_3.first_name < ($21 #>> '{}')",
" AND person_3.first_name <= ($22 #>> '{)", " AND person_3.first_name <= ($22 #>> '{}')",
" AND person_3.first_name NOT ILIKE $23 #>> '{}'", " AND person_3.first_name NOT ILIKE $23 #>> '{}'",
" AND person_3.first_name NOT IN (SELECT value FROM jsonb_array_elements_text(($24 #>> '{}')::jsonb))", " AND person_3.first_name NOT IN (SELECT value FROM jsonb_array_elements_text(($24 #>> '{}')::jsonb))",
" AND person_3.first_name IN (SELECT value FROM jsonb_array_elements_text(($25 #>> '{}')::jsonb))", " AND person_3.first_name IN (SELECT value FROM jsonb_array_elements_text(($25 #>> '{}')::jsonb))",
" AND entity_1.id = ($26 #>> '{}')::UUID", " AND entity_1.id = ($26 #>> '{}')::UUID",
" AND entity_1.id <> ($27 #>> '{}')::UUID", " AND entity_1.id <> ($27 #>> '{}')::UUID",
" AND entity_1.id NOT IN (SELECT value::UUID FROM jsonb_array_elements_text(($28 #>> '{}')::jsonb))", " AND entity_1.id NOT IN (SELECT value::UUID FROM jsonb_array_elements_text(($28 #>> '{}')::jsonb))",
" AND entity_1.id IN (SELECT value::UUID FROM jsonb_array_elements_text(($29 #>> '{}')::jsonb))", " AND entity_1.id IN (SELECT value::UUID FROM jsonb_array_elements_text(($29 #>> '{}')::jsonb))",
" AND person_3.last_name ILIKE $30 #>> '{}'", " AND person_3.last_name ILIKE $30 #>> '{}'",
" AND person_3.last_name NOT ILIKE $31 #>> '{}'", " AND person_3.last_name NOT ILIKE $31 #>> '{}'",
" ))))" "))))"
] ]
] ]
} }

View File

@ -13,6 +13,8 @@ pub struct MockState {
pub query_responses: Vec<Result<Value, String>>, pub query_responses: Vec<Result<Value, String>>,
pub execute_responses: Vec<Result<(), String>>, pub execute_responses: Vec<Result<(), String>>,
pub mocks: Vec<Value>, pub mocks: Vec<Value>,
pub cue_created_by: Option<String>,
pub cue_external: bool,
} }
#[cfg(test)] #[cfg(test)]
@ -23,6 +25,8 @@ impl MockState {
query_responses: Default::default(), query_responses: Default::default(),
execute_responses: Default::default(), execute_responses: Default::default(),
mocks: Default::default(), mocks: Default::default(),
cue_created_by: Some("00000000-0000-0000-0000-000000000000".to_string()),
cue_external: false,
} }
} }
} }
@ -77,14 +81,24 @@ impl DatabaseExecutor for MockExecutor {
}) })
} }
fn auth_user_id(&self) -> Result<String, String> { fn cue_created_by(&self) -> Result<String, String> {
Ok("00000000-0000-0000-0000-000000000000".to_string()) MOCK_STATE.with(|state| {
state
.borrow()
.cue_created_by
.clone()
.ok_or_else(|| "Missing cue.created_by in session context".to_string())
})
} }
fn timestamp(&self) -> Result<String, String> { fn timestamp(&self) -> Result<String, String> {
Ok("2026-03-10T00:00:00Z".to_string()) Ok("2026-03-10T00:00:00Z".to_string())
} }
fn cue_external(&self) -> Result<bool, String> {
Ok(MOCK_STATE.with(|state| state.borrow().cue_external))
}
#[cfg(test)] #[cfg(test)]
fn get_queries(&self) -> Vec<String> { fn get_queries(&self) -> Vec<String> {
MOCK_STATE.with(|state| state.borrow().captured_queries.clone()) MOCK_STATE.with(|state| state.borrow().captured_queries.clone())
@ -105,6 +119,23 @@ impl DatabaseExecutor for MockExecutor {
s.query_responses.clear(); s.query_responses.clear();
s.execute_responses.clear(); s.execute_responses.clear();
s.mocks.clear(); s.mocks.clear();
s.cue_created_by = Some("00000000-0000-0000-0000-00000000000".to_string());
s.cue_external = false;
});
}
}
#[cfg(test)]
impl MockExecutor {
pub fn set_cue_external(&self, external: bool) {
MOCK_STATE.with(|state| {
state.borrow_mut().cue_external = external;
});
}
pub fn set_cue_created_by(&self, created_by: Option<String>) {
MOCK_STATE.with(|state| {
state.borrow_mut().cue_created_by = created_by;
}); });
} }
} }
@ -112,56 +143,72 @@ impl DatabaseExecutor for MockExecutor {
#[cfg(test)] #[cfg(test)]
fn parse_and_match_mocks(sql: &str, mocks: &[Value]) -> Option<Vec<Value>> { fn parse_and_match_mocks(sql: &str, mocks: &[Value]) -> Option<Vec<Value>> {
let sql_upper = sql.to_uppercase(); let sql_upper = sql.to_uppercase();
if !sql_upper.starts_with("SELECT") { if !sql_upper.starts_with("SELECT") && !sql_upper.starts_with("(SELECT") {
return None; return None;
} }
// 1. Extract table name let union_regex = Regex::new(r"(?i)\s+UNION\s+").ok()?;
let table_regex = Regex::new(r#"(?i)\s+FROM\s+(?:[a-zA-Z_]\w*\.)?"?([a-zA-Z_]\w*)"?"#).ok()?; let queries: Vec<&str> = union_regex.split(sql).collect();
let table = if let Some(caps) = table_regex.captures(sql) {
caps.get(1)?.as_str()
} else {
return None;
};
// 2. Extract WHERE conditions string
let mut where_clause = String::new();
if let Some(where_idx) = sql_upper.find(" WHERE ") {
let mut where_end = sql_upper.find(" ORDER BY ").unwrap_or(sql_upper.len());
if let Some(limit_idx) = sql_upper.find(" LIMIT ") {
if limit_idx < where_end {
where_end = limit_idx;
}
}
where_clause = sql[where_idx + 7..where_end].to_string();
}
// 3. Find matching mocks
let mut matches = Vec::new(); let mut matches = Vec::new();
let or_regex = Regex::new(r"(?i)\s+OR\s+").ok()?; let or_regex = Regex::new(r"(?i)\s+OR\s+").ok()?;
let and_regex = Regex::new(r"(?i)\s+AND\s+").ok()?; let and_regex = Regex::new(r"(?i)\s+AND\s+").ok()?;
for mock in mocks { for mock in mocks {
if let Some(mock_obj) = mock.as_object() { let mock_obj = match mock.as_object() {
if let Some(t) = mock_obj.get("type") { Some(obj) => obj,
if t.as_str() != Some(table) { None => continue,
continue; };
}
}
if where_clause.is_empty() { let mock_type = mock_obj.get("type").and_then(|v| v.as_str()).unwrap_or("");
matches.push(mock.clone());
let mut mock_matched = false;
for query in &queries {
let q_upper = query.to_uppercase();
// Check if mock type matches the table or any joined tables in this query
let table_regex =
Regex::new(r#"(?i)\s+(?:FROM|JOIN)\s+(?:[a-zA-Z_]\w*\.)?"?([a-zA-Z_]\w*)"?"#).ok()?;
let tables: Vec<String> = table_regex
.captures_iter(query)
.filter_map(|c| c.get(1).map(|m| m.as_str().to_string()))
.collect();
if !mock_type.is_empty() && !tables.is_empty() && !tables.iter().any(|t| t == mock_type) {
continue; continue;
} }
let or_parts = or_regex.split(&where_clause); // Extract WHERE clause
let mut any_branch_matched = false; let mut where_clause = String::new();
if let Some(where_idx) = q_upper.find(" WHERE ") {
let mut where_end = q_upper.find(" ORDER BY ").unwrap_or(q_upper.len());
if let Some(limit_idx) = q_upper.find(" LIMIT ") {
if limit_idx < where_end {
where_end = limit_idx;
}
}
where_clause = query[where_idx + 7..where_end]
.trim_end_matches(')')
.to_string();
}
if where_clause.is_empty() {
mock_matched = true;
break;
}
let or_parts = or_regex.split(&where_clause);
for or_part in or_parts { for or_part in or_parts {
let branch_str = or_part.replace('(', "").replace(')', ""); let branch_str = or_part.replace('(', "").replace(')', "");
let mut branch_matches = true; let mut branch_matches = true;
for part in and_regex.split(&branch_str) { for part in and_regex.split(&branch_str) {
let part = part.trim();
if part.is_empty() {
continue;
}
if let Some(eq_idx) = part.find('=') { if let Some(eq_idx) = part.find('=') {
let left = part[..eq_idx] let left = part[..eq_idx]
.trim() .trim()
@ -175,7 +222,7 @@ fn parse_and_match_mocks(sql: &str, mocks: &[Value]) -> Option<Vec<Value>> {
Some(Value::String(s)) => s.clone(), Some(Value::String(s)) => s.clone(),
Some(Value::Number(n)) => n.to_string(), Some(Value::Number(n)) => n.to_string(),
Some(Value::Bool(b)) => b.to_string(), Some(Value::Bool(b)) => b.to_string(),
Some(Value::Null) => "null".to_string(), Some(Value::Null) | None => "null".to_string(),
_ => "".to_string(), _ => "".to_string(),
}; };
if mock_val_str != right { if mock_val_str != right {
@ -183,19 +230,21 @@ fn parse_and_match_mocks(sql: &str, mocks: &[Value]) -> Option<Vec<Value>> {
break; break;
} }
} else if part.to_uppercase().contains(" IS NULL") { } else if part.to_uppercase().contains(" IS NULL") {
let left = part[..part.to_uppercase().find(" IS NULL").unwrap()] let is_null_idx = part.to_uppercase().find(" IS NULL").unwrap();
let left = part[..is_null_idx]
.trim() .trim()
.split('.') .split('.')
.last() .last()
.unwrap_or("") .unwrap_or("")
.trim_matches('"'); .trim_matches('"');
let mock_val_str = match mock_obj.get(left) { let is_null_val = match mock_obj.get(left) {
Some(Value::Null) => "null".to_string(), Some(Value::Null) | None => true,
_ => "".to_string(), Some(Value::String(s)) if s.is_empty() => true,
_ => false,
}; };
if mock_val_str != "null" { if !is_null_val {
branch_matches = false; branch_matches = false;
break; break;
} }
@ -203,15 +252,19 @@ fn parse_and_match_mocks(sql: &str, mocks: &[Value]) -> Option<Vec<Value>> {
} }
if branch_matches { if branch_matches {
any_branch_matched = true; mock_matched = true;
break; break;
} }
} }
if any_branch_matched { if mock_matched {
matches.push(mock.clone()); break;
} }
} }
if mock_matched {
matches.push(mock.clone());
}
} }
Some(matches) Some(matches)

View File

@ -14,12 +14,15 @@ pub trait DatabaseExecutor: Send + Sync {
/// Executes an operation (INSERT, UPDATE, DELETE, or pg_notify) that does not return rows. /// Executes an operation (INSERT, UPDATE, DELETE, or pg_notify) that does not return rows.
fn execute(&self, sql: &str, args: Option<Vec<Value>>) -> Result<(), String>; fn execute(&self, sql: &str, args: Option<Vec<Value>>) -> Result<(), String>;
/// Returns the current authenticated user's ID /// Returns the current authenticated user's ID (from cue.created_by)
fn auth_user_id(&self) -> Result<String, String>; fn cue_created_by(&self) -> Result<String, String>;
/// Returns the current transaction timestamp /// Returns the current transaction timestamp
fn timestamp(&self) -> Result<String, String>; fn timestamp(&self) -> Result<String, String>;
/// Returns true if the current execution context is marked as an external client API cue (cue.external = true)
fn cue_external(&self) -> Result<bool, String>;
#[cfg(test)] #[cfg(test)]
fn get_queries(&self) -> Vec<String>; fn get_queries(&self) -> Vec<String>;

View File

@ -77,8 +77,8 @@ impl DatabaseExecutor for SpiExecutor {
pgrx::debug1!("JSPG_SQL: {}", sql); pgrx::debug1!("JSPG_SQL: {}", sql);
self.transact(|| { self.transact(|| {
Spi::connect(|client| { Spi::connect_mut(|client| {
match client.select(sql, Some(args_with_oid.len() as i64), &args_with_oid) { match client.update(sql, Some(args_with_oid.len() as i64), &args_with_oid) {
Ok(tup_table) => { Ok(tup_table) => {
let mut results = Vec::new(); let mut results = Vec::new();
for row in tup_table { for row in tup_table {
@ -113,12 +113,12 @@ impl DatabaseExecutor for SpiExecutor {
}) })
} }
fn auth_user_id(&self) -> Result<String, String> { fn cue_created_by(&self) -> Result<String, String> {
self.transact(|| { self.transact(|| {
Spi::connect(|client| { Spi::connect(|client| {
let mut tup_table = client let mut tup_table = client
.select( .select(
"SELECT COALESCE(current_setting('auth.user_id', true), 'ffffffff-ffff-ffff-ffff-ffffffffffff')", "SELECT NULLIF(current_setting('punc.cue', true), '')::jsonb->>'created_by'",
None, None,
&[], &[],
) )
@ -126,10 +126,10 @@ impl DatabaseExecutor for SpiExecutor {
let row = tup_table let row = tup_table
.next() .next()
.ok_or("No user id setting returned from context".to_string())?; .ok_or("No cue setting returned from context".to_string())?;
let user_id: Option<String> = row.get(1).map_err(|e| e.to_string())?; let user_id: Option<String> = row.get(1).map_err(|e| e.to_string())?;
user_id.ok_or("Missing user_id".to_string()) user_id.ok_or("Missing cue.created_by in session context".to_string())
}) })
}) })
} }
@ -150,4 +150,25 @@ impl DatabaseExecutor for SpiExecutor {
}) })
}) })
} }
fn cue_external(&self) -> Result<bool, String> {
self.transact(|| {
Spi::connect(|client| {
let mut tup_table = client
.select(
"SELECT COALESCE((NULLIF(current_setting('punc.cue', true), '')::jsonb->>'external')::boolean, false)",
None,
&[],
)
.map_err(|e| format!("SPI Select Error: {}", e))?;
let row = tup_table
.next()
.ok_or("No setting returned from context".to_string())?;
let is_external: Option<bool> = row.get(1).map_err(|e| e.to_string())?;
Ok(is_external.unwrap_or(false))
})
})
}
} }

View File

@ -232,9 +232,9 @@ impl Database {
self.executor.execute(sql, args) self.executor.execute(sql, args)
} }
/// Returns the current authenticated user's ID /// Returns the current authenticated user's ID (from cue.created_by)
pub fn auth_user_id(&self) -> Result<String, String> { pub fn cue_created_by(&self) -> Result<String, String> {
self.executor.auth_user_id() self.executor.cue_created_by()
} }
/// Returns the current transaction timestamp /// Returns the current transaction timestamp
@ -242,6 +242,11 @@ impl Database {
self.executor.timestamp() self.executor.timestamp()
} }
/// Returns true if the current execution context is marked as an external client API cue (cue.external = true)
pub fn cue_external(&self) -> Result<bool, String> {
self.executor.cue_external()
}
pub fn compile(&mut self, errors: &mut Vec<crate::drop::Error>) { pub fn compile(&mut self, errors: &mut Vec<crate::drop::Error>) {
// Phase 1: Registration // Phase 1: Registration
self.collect_schemas(errors); self.collect_schemas(errors);

View File

@ -151,19 +151,7 @@ pub struct SchemaObject {
pub extensible: Option<bool>, pub extensible: Option<bool>,
#[serde(default)] #[serde(default)]
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
pub immutable: Option<bool>, pub immutable: Option<ImmutableMode>,
// readOnly = the CLIENT may not write it; the SERVER still may. Deliberately NOT
// enforced here (see validator/rules/object.rs, which rejects `immutable` on any
// request) — enforcing it would break legitimate server writers such as the
// Mercury feed matcher stamping `reconciliation_status`, or the rollup triggers
// that own `total_amount`/`amount_paid`. It is carried purely so it survives
// compilation and reaches the punc code generator, which turns it into a
// client-side immutable Reactor. Without this field the key was silently dropped
// at this boundary and ten authored declarations did nothing for months.
#[serde(default)]
#[serde(rename = "readOnly")]
#[serde(skip_serializing_if = "Option::is_none")]
pub read_only: Option<bool>,
// Contains ALL structural fields perfectly flattened from the ENTIRE Database inheritance tree (e.g. `entity` fields like `id`) as well as local fields hidden inside conditional `cases` blocks. // Contains ALL structural fields perfectly flattened from the ENTIRE Database inheritance tree (e.g. `entity` fields like `id`) as well as local fields hidden inside conditional `cases` blocks.
// This JSON exported array gives clients absolute deterministic visibility to O(1) validation and masking bounds without duplicating structural memory. // This JSON exported array gives clients absolute deterministic visibility to O(1) validation and masking bounds without duplicating structural memory.
@ -275,6 +263,13 @@ pub fn is_primitive_type(t: &str) -> bool {
) )
} }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum ImmutableMode {
Always,
External,
}
impl SchemaObject { impl SchemaObject {
pub fn get_discriminator_value(&self, dim: &str, schema_id: &str) -> Option<String> { pub fn get_discriminator_value(&self, dim: &str, schema_id: &str) -> Option<String> {
let is_split = self let is_split = self
@ -322,7 +317,11 @@ impl SchemaObject {
false false
} }
pub fn is_immutable(&self) -> bool { pub fn is_immutable(&self, is_external: bool) -> bool {
self.immutable == Some(true) match self.immutable {
Some(ImmutableMode::Always) => true,
Some(ImmutableMode::External) => is_external,
None => false,
}
} }
} }

View File

@ -5,6 +5,14 @@ use serde::{Deserialize, Serialize};
use std::sync::Arc; use std::sync::Arc;
use serde_json::Value; use serde_json::Value;
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct Roles {
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub read: Vec<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub write: Vec<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, Default)] #[derive(Debug, Clone, Serialize, Deserialize, Default)]
#[serde(default)] #[serde(default)]
pub struct Type { pub struct Type {
@ -36,8 +44,10 @@ pub struct Type {
#[serde(default)] #[serde(default)]
pub null_fields: Vec<String>, pub null_fields: Vec<String>,
#[serde(default)] #[serde(default)]
pub default_fields: Vec<String>, pub field_defaults: IndexMap<String, Value>,
pub field_types: Option<Value>, pub field_types: Option<Value>,
#[serde(skip_serializing_if = "Option::is_none")]
pub roles: Option<Roles>,
#[serde(default)] #[serde(default)]
pub schemas: IndexMap<String, Arc<Schema>>, pub schemas: IndexMap<String, Arc<Schema>>,
} }

View File

@ -57,7 +57,7 @@ pub fn jspg_setup(database: Json) -> Json {
Json(serde_json::to_value(drop).unwrap()) Json(serde_json::to_value(drop).unwrap())
} }
#[cfg_attr(not(test), pg_extern)] #[cfg_attr(not(test), pg_extern(volatile))]
pub fn jspg_merge(schema_id: &str, data: JsonB) -> JsonB { pub fn jspg_merge(schema_id: &str, data: JsonB) -> JsonB {
// Try to acquire a read lock to get a clone of the Engine Arc // Try to acquire a read lock to get a clone of the Engine Arc
let engine_opt = { let engine_opt = {
@ -74,7 +74,7 @@ pub fn jspg_merge(schema_id: &str, data: JsonB) -> JsonB {
} }
} }
#[cfg_attr(not(test), pg_extern)] #[cfg_attr(not(test), pg_extern(volatile))]
pub fn jspg_merge_ordered(schema_id: &str, data: Json) -> Json { pub fn jspg_merge_ordered(schema_id: &str, data: Json) -> Json {
let engine_opt = { let engine_opt = {
let lock = GLOBAL_JSPG.read().unwrap(); let lock = GLOBAL_JSPG.read().unwrap();

View File

@ -6,8 +6,8 @@ pub mod cache;
use crate::database::Database; use crate::database::Database;
use crate::database::r#type::Type; use crate::database::r#type::Type;
use crate::drop::{Drop, Error, ErrorDetails}; use crate::drop::{Drop, Error, ErrorDetails};
use serde_json::Value;
use indexmap::IndexMap; use indexmap::IndexMap;
use serde_json::Value;
use std::sync::Arc; use std::sync::Arc;
pub struct Merger { pub struct Merger {
@ -31,9 +31,10 @@ impl Merger {
None => { None => {
return Drop::with_errors(vec![Error { return Drop::with_errors(vec![Error {
code: "SCHEMA_NOT_FOUND".to_string(), code: "SCHEMA_NOT_FOUND".to_string(),
values: Some(IndexMap::from([ values: Some(IndexMap::from([(
("schema".to_string(), schema_id.to_string()), "schema".to_string(),
])), schema_id.to_string(),
)])),
details: ErrorDetails { details: ErrorDetails {
path: None, path: None,
cause: None, cause: None,
@ -56,9 +57,7 @@ impl Merger {
if let Err(e) = self.db.execute(&notify_sql, None) { if let Err(e) = self.db.execute(&notify_sql, None) {
return Drop::with_errors(vec![Error { return Drop::with_errors(vec![Error {
code: "MERGE_FAILED".to_string(), code: "MERGE_FAILED".to_string(),
values: Some(IndexMap::from([ values: Some(IndexMap::from([("error".to_string(), e.clone())])),
("error".to_string(), e.clone()),
])),
details: ErrorDetails { details: ErrorDetails {
path: None, path: None,
cause: Some(e), cause: Some(e),
@ -121,9 +120,15 @@ impl Merger {
} else { } else {
return Err(Error { return Err(Error {
code: "TARGET_SCHEMA_NOT_FOUND".to_string(), code: "TARGET_SCHEMA_NOT_FOUND".to_string(),
values: Some(IndexMap::from([("target_id".to_string(), target_id.clone())])), values: Some(IndexMap::from([(
"target_id".to_string(),
target_id.clone(),
)])),
details: ErrorDetails { details: ErrorDetails {
cause: Some(format!("Polymorphic mapped target '{}' not found in database registry", target_id)), cause: Some(format!(
"Polymorphic mapped target '{}' not found in database registry",
target_id
)),
..Default::default() ..Default::default()
}, },
}); });
@ -141,7 +146,10 @@ impl Merger {
code: "ONE_OF_INDEX_NOT_FOUND".to_string(), code: "ONE_OF_INDEX_NOT_FOUND".to_string(),
values: Some(IndexMap::from([("index".to_string(), idx.to_string())])), values: Some(IndexMap::from([("index".to_string(), idx.to_string())])),
details: ErrorDetails { details: ErrorDetails {
cause: Some(format!("Polymorphic index target '{}' not found in local oneOf array", idx)), cause: Some(format!(
"Polymorphic index target '{}' not found in local oneOf array",
idx
)),
..Default::default() ..Default::default()
}, },
}); });
@ -164,7 +172,10 @@ impl Merger {
("value".to_string(), v.to_string()), ("value".to_string(), v.to_string()),
])), ])),
details: ErrorDetails { details: ErrorDetails {
cause: Some(format!("Polymorphic discriminator {}='{}' matched no compiled options", disc, v)), cause: Some(format!(
"Polymorphic discriminator {}='{}' matched no compiled options",
disc, v
)),
..Default::default() ..Default::default()
}, },
}); });
@ -172,9 +183,15 @@ impl Merger {
} else { } else {
return Err(Error { return Err(Error {
code: "MISSING_DISCRIMINATOR".to_string(), code: "MISSING_DISCRIMINATOR".to_string(),
values: Some(IndexMap::from([("discriminator".to_string(), disc.to_string())])), values: Some(IndexMap::from([(
"discriminator".to_string(),
disc.to_string(),
)])),
details: ErrorDetails { details: ErrorDetails {
cause: Some(format!("Polymorphic merging failed: missing required discriminator '{}'", disc)), cause: Some(format!(
"Polymorphic merging failed: missing required discriminator '{}'",
disc
)),
..Default::default() ..Default::default()
}, },
}); });
@ -289,10 +306,6 @@ impl Merger {
} }
if let Some(prop_schema) = compiled_props.get(&k) { if let Some(prop_schema) = compiled_props.get(&k) {
if prop_schema.is_immutable() {
continue;
}
let mut is_edge = false; let mut is_edge = false;
if let Some(edges) = schema.obj.compiled_edges.get() { if let Some(edges) = schema.obj.compiled_edges.get() {
if edges.contains_key(&k) { if edges.contains_key(&k) {
@ -322,6 +335,8 @@ impl Merger {
} }
} }
let mut current_org_id = None; let mut current_org_id = None;
if let Some(compiled_props) = schema.obj.compiled_properties.get() { if let Some(compiled_props) = schema.obj.compiled_properties.get() {
if let Some(org_schema) = compiled_props.get("organization_id") { if let Some(org_schema) = compiled_props.get("organization_id") {
@ -336,8 +351,8 @@ impl Merger {
current_org_id = parent_org_id.clone(); current_org_id = parent_org_id.clone();
} }
let user_id = self.db.auth_user_id().map_err(|e| Error { let user_id = self.db.cue_created_by().map_err(|e| Error {
code: "AUTH_USER_FAILED".to_string(), code: "CUE_CREATED_BY_FAILED".to_string(),
values: Some(IndexMap::from([("error".to_string(), e.clone())])), values: Some(IndexMap::from([("error".to_string(), e.clone())])),
details: ErrorDetails { details: ErrorDetails {
cause: Some(e), cause: Some(e),
@ -369,7 +384,10 @@ impl Merger {
if let Some(deps) = &schema.obj.dependencies { if let Some(deps) = &schema.obj.dependencies {
if let Some(crate::database::object::Dependency::Props(req_props)) = deps.get("created") { if let Some(crate::database::object::Dependency::Props(req_props)) = deps.get("created") {
for req in req_props { for req in req_props {
if !entity_fields.contains_key(req) && !entity_objects.contains_key(req) && !entity_arrays.contains_key(req) { if !entity_fields.contains_key(req)
&& !entity_objects.contains_key(req)
&& !entity_arrays.contains_key(req)
{
return Err(Error { return Err(Error {
code: "REQUIRED_FIELD_MISSING".to_string(), code: "REQUIRED_FIELD_MISSING".to_string(),
values: Some(IndexMap::from([ values: Some(IndexMap::from([
@ -378,7 +396,10 @@ impl Merger {
])), ])),
details: ErrorDetails { details: ErrorDetails {
path: Some(req.to_string()), path: Some(req.to_string()),
cause: Some(format!("Missing required creation field '{}' for entity {}", req, type_name)), cause: Some(format!(
"Missing required creation field '{}' for entity {}",
req, type_name
)),
..Default::default() ..Default::default()
}, },
}); });
@ -489,7 +510,7 @@ impl Merger {
entity_change_kind.as_deref().unwrap_or(""), entity_change_kind.as_deref().unwrap_or(""),
&type_name, &type_name,
type_def, type_def,
&entity_fields, &mut entity_fields,
entity_fetched.as_ref(), entity_fetched.as_ref(),
)?; )?;
@ -741,21 +762,34 @@ impl Merger {
if let Some(parent_type) = self.db.types.get(parent_type_name) { if let Some(parent_type) = self.db.types.get(parent_type_name) {
if !parent_type.lookup_fields.is_empty() { if !parent_type.lookup_fields.is_empty() {
let mut lookup_complete = true; let mut lookup_complete = true;
let mut has_provided_fields = false;
for column in &parent_type.lookup_fields { for column in &parent_type.lookup_fields {
match entity_fields.get(column) { let is_nullable = parent_type.null_fields.contains(column);
let val = entity_fields.get(column).or_else(|| {
parent_type.field_defaults.get(column)
});
match val {
Some(Value::Null) | None => { Some(Value::Null) | None => {
lookup_complete = false; if !is_nullable {
break; lookup_complete = false;
break;
}
} }
Some(Value::String(s)) if s.is_empty() => { Some(Value::String(s)) if s.is_empty() => {
lookup_complete = false; if !is_nullable {
break; lookup_complete = false;
break;
}
}
_ => {
if entity_fields.contains_key(column) {
has_provided_fields = true;
}
} }
_ => {}
} }
} }
if lookup_complete { if lookup_complete && has_provided_fields {
lookup_satisfied_keys.push(&parent_type.lookup_fields); lookup_satisfied_keys.push((&parent_type.lookup_fields, parent_type));
} }
} }
} }
@ -768,7 +802,7 @@ impl Merger {
let fetch_sql_template = if let Some(cached) = self.cache.get(entity_type_name) { let fetch_sql_template = if let Some(cached) = self.cache.get(entity_type_name) {
cached cached
} else { } else {
let mut select_list = String::from("to_jsonb(t1.*)"); let mut select_list = String::from("COALESCE(to_jsonb(t1.*), '{}')");
let mut join_clauses = format!("FROM agreego.\"{}\" t1", entity_type.hierarchy[0]); let mut join_clauses = format!("FROM agreego.\"{}\" t1", entity_type.hierarchy[0]);
for (i, table_name) in entity_type.hierarchy.iter().enumerate().skip(1) { for (i, table_name) in entity_type.hierarchy.iter().enumerate().skip(1) {
@ -777,7 +811,7 @@ impl Merger {
" LEFT JOIN agreego.\"{}\" {} ON {}.id = t1.id", " LEFT JOIN agreego.\"{}\" {} ON {}.id = t1.id",
table_name, t_alias, t_alias table_name, t_alias, t_alias
)); ));
select_list.push_str(&format!(" || to_jsonb({}.*)", t_alias)); select_list.push_str(&format!(" || COALESCE(to_jsonb({}.*), '{{}}')", t_alias));
} }
let template = format!("SELECT {} {}", select_list, join_clauses); let template = format!("SELECT {} {}", select_list, join_clauses);
@ -793,14 +827,24 @@ impl Merger {
where_parts.push(format!("t1.id = {}", Self::quote_literal(id))); where_parts.push(format!("t1.id = {}", Self::quote_literal(id)));
} }
for lookup_fields in lookup_satisfied_keys { for (lookup_fields, parent_type) in lookup_satisfied_keys {
let t_alias = entity_type
.hierarchy
.iter()
.position(|name| name == &parent_type.name)
.map(|idx| format!("t{}", idx + 1))
.unwrap_or_else(|| "t1".to_string());
let mut lookup_predicates = Vec::new(); let mut lookup_predicates = Vec::new();
for column in lookup_fields { for column in lookup_fields {
let val = entity_fields.get(column).unwrap_or(&Value::Null); let val = entity_fields
if column == "type" { .get(column)
lookup_predicates.push(format!("t1.\"{}\" = {}", column, Self::quote_literal(val))); .or_else(|| parent_type.field_defaults.get(column))
.unwrap_or(&Value::Null);
if val.is_null() || val.as_str() == Some("") {
lookup_predicates.push(format!("{}.\"{}\" IS NULL", t_alias, column));
} else { } else {
lookup_predicates.push(format!("\"{}\" = {}", column, Self::quote_literal(val))); lookup_predicates.push(format!("{}.\"{}\" = {}", t_alias, column, Self::quote_literal(val)));
} }
} }
where_parts.push(format!("({})", lookup_predicates.join(" AND "))); where_parts.push(format!("({})", lookup_predicates.join(" AND ")));
@ -825,9 +869,15 @@ impl Merger {
if table.len() > 1 { if table.len() > 1 {
Err(Error { Err(Error {
code: "TOO_MANY_LOOKUP_ROWS".to_string(), code: "TOO_MANY_LOOKUP_ROWS".to_string(),
values: Some(IndexMap::from([("entity_type".to_string(), entity_type_name.to_string())])), values: Some(IndexMap::from([(
"entity_type".to_string(),
entity_type_name.to_string(),
)])),
details: ErrorDetails { details: ErrorDetails {
cause: Some(format!("Lookup for {} found too many existing rows", entity_type_name)), cause: Some(format!(
"Lookup for {} found too many existing rows",
entity_type_name
)),
..Default::default() ..Default::default()
}, },
}) })
@ -874,7 +924,7 @@ impl Merger {
change_kind: &str, change_kind: &str,
entity_type_name: &str, entity_type_name: &str,
entity_type: &crate::database::r#type::Type, entity_type: &crate::database::r#type::Type,
entity_fields: &serde_json::Map<String, Value>, entity_fields: &mut serde_json::Map<String, Value>,
_entity_fetched: Option<&serde_json::Map<String, Value>>, _entity_fetched: Option<&serde_json::Map<String, Value>>,
) -> Result<(), Error> { ) -> Result<(), Error> {
if change_kind.is_empty() { if change_kind.is_empty() {
@ -882,7 +932,7 @@ impl Merger {
} }
let id_str = match entity_fields.get("id").and_then(|v| v.as_str()) { let id_str = match entity_fields.get("id").and_then(|v| v.as_str()) {
Some(id) => id, Some(id) => id.to_string(),
None => { None => {
return Err(Error { return Err(Error {
code: "MISSING_ENTITY_ID".to_string(), code: "MISSING_ENTITY_ID".to_string(),
@ -900,9 +950,15 @@ impl Merger {
_ => { _ => {
return Err(Error { return Err(Error {
code: "MISSING_GROUPED_FIELDS".to_string(), code: "MISSING_GROUPED_FIELDS".to_string(),
values: Some(IndexMap::from([("type".to_string(), entity_type_name.to_string())])), values: Some(IndexMap::from([(
"type".to_string(),
entity_type_name.to_string(),
)])),
details: ErrorDetails { details: ErrorDetails {
cause: Some(format!("Grouped fields missing for type {}", entity_type_name)), cause: Some(format!(
"Grouped fields missing for type {}",
entity_type_name
)),
..Default::default() ..Default::default()
}, },
}); });
@ -919,7 +975,7 @@ impl Merger {
}; };
let mut entity_pairs = serde_json::Map::new(); let mut entity_pairs = serde_json::Map::new();
for (k, v) in entity_fields { for (k, v) in entity_fields.iter() {
if table_fields.contains(k) { if table_fields.contains(k) {
entity_pairs.insert(k.clone(), v.clone()); entity_pairs.insert(k.clone(), v.clone());
} }
@ -953,20 +1009,33 @@ impl Merger {
} }
let sql = format!( let sql = format!(
"INSERT INTO agreego.\"{}\" ({}) VALUES ({})", "INSERT INTO agreego.\"{}\" ({}) VALUES ({}) RETURNING to_jsonb(\"{}\".*)",
table_name, table_name,
columns.join(", "), columns.join(", "),
values.join(", ") values.join(", "),
table_name
); );
if let Err(e) = self.db.execute(&sql, None) { match self.db.query(&sql, None) {
return Err(Error { Ok(Value::Array(rows)) => {
code: "DATABASE_SPI_ERROR".to_string(), if let Some(Value::Object(row_map)) = rows.into_iter().next() {
values: Some(IndexMap::from([("error".to_string(), e.clone())])), for (k, v) in row_map {
details: ErrorDetails { if !v.is_null() {
cause: Some(e), entity_fields.insert(k, v);
..Default::default() }
}, }
}); }
}
Err(e) => {
return Err(Error {
code: "DATABASE_SPI_ERROR".to_string(),
values: Some(IndexMap::from([("error".to_string(), e.clone())])),
details: ErrorDetails {
cause: Some(e),
..Default::default()
},
});
}
_ => {}
} }
} else if change_kind == "update" || change_kind == "delete" { } else if change_kind == "update" || change_kind == "delete" {
entity_pairs.remove("id"); entity_pairs.remove("id");
@ -994,20 +1063,33 @@ impl Merger {
} }
let sql = format!( let sql = format!(
"UPDATE agreego.\"{}\" SET {} WHERE id = {}", "UPDATE agreego.\"{}\" SET {} WHERE id = {} RETURNING to_jsonb(\"{}\".*)",
table_name, table_name,
set_clauses.join(", "), set_clauses.join(", "),
Self::quote_literal(&Value::String(id_str.to_string())) Self::quote_literal(&Value::String(id_str.to_string())),
table_name
); );
if let Err(e) = self.db.execute(&sql, None) { match self.db.query(&sql, None) {
return Err(Error { Ok(Value::Array(rows)) => {
code: "DATABASE_SPI_ERROR".to_string(), if let Some(Value::Object(row_map)) = rows.into_iter().next() {
values: Some(IndexMap::from([("error".to_string(), e.clone())])), for (k, v) in row_map {
details: ErrorDetails { if !v.is_null() {
cause: Some(e), entity_fields.insert(k, v);
..Default::default() }
}, }
}); }
}
Err(e) => {
return Err(Error {
code: "DATABASE_SPI_ERROR".to_string(),
values: Some(IndexMap::from([("error".to_string(), e.clone())])),
details: ErrorDetails {
cause: Some(e),
..Default::default()
},
});
}
_ => {}
} }
} }
} }

View File

@ -2531,6 +2531,18 @@ fn test_properties_13_1() {
crate::tests::runner::run_test_case(&path, 13, 1).unwrap(); crate::tests::runner::run_test_case(&path, 13, 1).unwrap();
} }
#[test]
fn test_properties_14_0() {
let path = format!("{}/fixtures/properties.json", env!("CARGO_MANIFEST_DIR"));
crate::tests::runner::run_test_case(&path, 14, 0).unwrap();
}
#[test]
fn test_properties_14_1() {
let path = format!("{}/fixtures/properties.json", env!("CARGO_MANIFEST_DIR"));
crate::tests::runner::run_test_case(&path, 14, 1).unwrap();
}
#[test] #[test]
fn test_max_contains_0_0() { fn test_max_contains_0_0() {
let path = format!("{}/fixtures/maxContains.json", env!("CARGO_MANIFEST_DIR")); let path = format!("{}/fixtures/maxContains.json", env!("CARGO_MANIFEST_DIR"));

View File

@ -97,7 +97,14 @@ impl SqlFormatter {
self.push_line("VALUES ("); self.push_line("VALUES (");
self.indent += 2; self.indent += 2;
let vals = if suffix.ends_with(")") { &suffix[..suffix.len() - 1] } else { suffix }; let (vals, returning_clause) = if let Some(ret_idx) = suffix.rfind(") RETURNING ") {
(&suffix[..ret_idx], Some(&suffix[ret_idx + 2..]))
} else if suffix.ends_with(")") {
(&suffix[..suffix.len() - 1], None)
} else {
(suffix, None)
};
let mut val_tokens = Vec::new(); let mut val_tokens = Vec::new();
let mut curr = String::new(); let mut curr = String::new();
let mut in_str = false; let mut in_str = false;
@ -119,7 +126,7 @@ impl SqlFormatter {
for (i, val) in val_tokens.iter().enumerate() { for (i, val) in val_tokens.iter().enumerate() {
let comma = if i < val_tokens.len() - 1 { "," } else { "" }; let comma = if i < val_tokens.len() - 1 { "," } else { "" };
if val.starts_with("'{") && val.ends_with("}'") { if val.starts_with("'{") && val.ends_with("}'") && val.len() > 4 {
let inner = &val[1..val.len() - 1]; let inner = &val[1..val.len() - 1];
// Unescape single quotes from SQL strings // Unescape single quotes from SQL strings
let unescaped = inner.replace("''", "'"); let unescaped = inner.replace("''", "'");
@ -146,6 +153,9 @@ impl SqlFormatter {
} }
self.indent -= 2; self.indent -= 2;
self.push_line(")"); self.push_line(")");
if let Some(ret) = returning_clause {
self.push_line(ret);
}
} else { } else {
self.push_line(&s); self.push_line(&s);
} }
@ -168,10 +178,19 @@ impl SqlFormatter {
self.indent -= 2; self.indent -= 2;
if let Some(w) = where_idx { if let Some(w) = where_idx {
self.push_line("WHERE"); let where_clause = &after_set[w + 7..];
self.indent += 2; if let Some(ret_idx) = where_clause.find(" RETURNING ") {
self.push_line(&after_set[w + 7..]); self.push_line("WHERE");
self.indent -= 2; self.indent += 2;
self.push_line(&where_clause[..ret_idx]);
self.indent -= 2;
self.push_line(&where_clause[ret_idx + 1..]);
} else {
self.push_line("WHERE");
self.indent += 2;
self.push_line(where_clause);
self.indent -= 2;
}
} }
} else { } else {
self.push_line(&s); self.push_line(&s);
@ -341,7 +360,7 @@ impl SqlFormatter {
} }
Expr::Value(sqlparser::ast::ValueWithSpan { value: Value::SingleQuotedString(s), .. }) | Expr::Value(sqlparser::ast::ValueWithSpan { value: Value::EscapedStringLiteral(s), .. }) => { Expr::Value(sqlparser::ast::ValueWithSpan { value: Value::SingleQuotedString(s), .. }) | Expr::Value(sqlparser::ast::ValueWithSpan { value: Value::EscapedStringLiteral(s), .. }) => {
if s.starts_with('{') && s.ends_with('}') { if s.starts_with('{') && s.ends_with('}') && s.len() > 2 {
if let Ok(json) = serde_json::from_str::<serde_json::Value>(s) { if let Ok(json) = serde_json::from_str::<serde_json::Value>(s) {
if let Ok(pretty) = serde_json::to_string_pretty(&json) { if let Ok(pretty) = serde_json::to_string_pretty(&json) {
let lines: Vec<&str> = pretty.split('\n').collect(); let lines: Vec<&str> = pretty.split('\n').collect();

View File

@ -104,7 +104,7 @@ fn test_library_api() {
}, },
"types": { "types": {
"source_schema": { "source_schema": {
"default_fields": [], "field_defaults": {},
"field_types": null, "field_types": null,
"fields": [], "fields": [],
"grouped_fields": null, "grouped_fields": null,
@ -169,7 +169,7 @@ fn test_library_api() {
"variations": ["source_schema"] "variations": ["source_schema"]
}, },
"target_schema": { "target_schema": {
"default_fields": [], "field_defaults": {},
"field_types": null, "field_types": null,
"fields": [], "fields": [],
"grouped_fields": null, "grouped_fields": null,

View File

@ -180,9 +180,10 @@ impl<'a> ValidationContext<'a> {
} }
if !self.response { if !self.response {
let is_external = self.db.cue_external().unwrap_or(false);
if let Some(compiled_props) = self.schema.compiled_properties.get() { if let Some(compiled_props) = self.schema.compiled_properties.get() {
for (key, sub_schema) in compiled_props { for (key, sub_schema) in compiled_props {
if sub_schema.is_immutable() && obj.contains_key(key) { if sub_schema.is_immutable(is_external) && obj.contains_key(key) {
result.errors.push(ValidationError { result.errors.push(ValidationError {
code: "IMMUTABLE_PROPERTY_VIOLATION".to_string(), code: "IMMUTABLE_PROPERTY_VIOLATION".to_string(),
values: Some(IndexMap::from([ values: Some(IndexMap::from([
@ -194,7 +195,7 @@ impl<'a> ValidationContext<'a> {
} }
} else if let Some(props) = &self.schema.properties { } else if let Some(props) = &self.schema.properties {
for (key, sub_schema) in props { for (key, sub_schema) in props {
if sub_schema.is_immutable() && obj.contains_key(key) { if sub_schema.is_immutable(is_external) && obj.contains_key(key) {
result.errors.push(ValidationError { result.errors.push(ValidationError {
code: "IMMUTABLE_PROPERTY_VIOLATION".to_string(), code: "IMMUTABLE_PROPERTY_VIOLATION".to_string(),
values: Some(IndexMap::from([ values: Some(IndexMap::from([
@ -207,6 +208,8 @@ impl<'a> ValidationContext<'a> {
} }
} }
if let Some(props) = &self.schema.properties { if let Some(props) = &self.schema.properties {
for (key, sub_schema) in props { for (key, sub_schema) in props {
if self.overrides.contains(key) { if self.overrides.contains(key) {

View File

@ -1 +1 @@
1.0.186 1.0.199