Compare commits

...

15 Commits

Author SHA1 Message Date
9dcafed406 version: 1.0.90 2026-03-25 19:32:02 -04:00
ffd6c27da3 more pg try catching and error handling 2026-03-25 19:31:51 -04:00
4941dc6069 doc update 2026-03-23 19:07:45 -04:00
a8a15a82ef version: 1.0.89 2026-03-23 16:41:41 -04:00
8dcc714963 fixed target_type restrictions in where clauses 2026-03-23 16:41:34 -04:00
f87ac81f3b pre-script-fix 2026-03-23 16:34:45 -04:00
8ca9017cc4 version: 1.0.88 2026-03-23 14:37:29 -04:00
10c57e59ec fixed nested filtering syntax 2026-03-23 14:37:22 -04:00
ef4571767c version: 1.0.87 2026-03-23 12:49:36 -04:00
29bd25eaff fixed filter override for archived 2026-03-23 12:49:30 -04:00
4d9b510819 version: 1.0.86 2026-03-23 12:26:03 -04:00
3c4b1066df fixed merger with anchor test issue 2026-03-23 12:25:55 -04:00
4c59d9ba7f version: 1.0.85 2026-03-23 12:05:47 -04:00
a1038490dd tested nested merging with anchors 2026-03-23 12:05:34 -04:00
14707330a7 subschema id queryer test added 2026-03-22 05:54:31 -04:00
12 changed files with 546 additions and 950 deletions

View File

@ -7,14 +7,14 @@
JSPG operates by deeply integrating the JSON Schema Draft 2020-12 specification directly into the Postgres session lifecycle. It is built around three core pillars:
* **Validator**: In-memory, near-instant JSON structural validation and type polymorphism routing.
* **Merger**: Automatically traverse and UPSERT deeply nested JSON graphs into normalized relational tables.
* **Queryer**: Compile JSON Schemas into static, cached SQL SPI `SELECT` plans for fetching full entities or isolated "Stems".
* **Queryer**: Compile JSON Schemas into static, cached SQL SPI `SELECT` plans for fetching full entities or isolated ad-hoc object boundaries.
### 🎯 Goals
1. **Draft 2020-12 Compliance**: Attempt to adhere to the official JSON Schema Draft 2020-12 specification.
2. **Ultra-Fast Execution**: Compile schemas into optimized in-memory validation trees and cached SQL SPIs to bypass Postgres Query Builder overheads.
3. **Connection-Bound Caching**: Leverage the PostgreSQL session lifecycle using an **Atomic Swap** pattern. Schemas are 100% frozen, completely eliminating locks during read access.
4. **Structural Inheritance**: Support object-oriented schema design via Implicit Keyword Shadowing and virtual `$family` references natively mapped to Postgres table constraints.
5. **Reactive Beats**: Provide natively generated "Stems" (isolated payload fragments) for dynamic websocket reactivity.
5. **Reactive Beats**: Provide ultra-fast natively generated flat payloads mapping directly to the Dart topological state for dynamic websocket reactivity.
### Concurrency & Threading ("Immutable Graphs")
To support high-throughput operations while allowing for runtime updates (e.g., during hot-reloading), JSPG uses an **Atomic Swap** pattern:
@ -118,22 +118,11 @@ The Queryer transforms Postgres into a pre-compiled Semantic Query Engine, desig
* **Multi-Table Branching**: If the Physical Table is a parent to other tables (e.g. `organization` has variations `["organization", "bot", "person"]`), the compiler generates a dynamic `CASE WHEN type = '...' THEN ...` query, expanding into `JOIN`s for each variation.
* **Single-Table Bypass**: If the Physical Table is a leaf node with only one variation (e.g. `person` has variations `["person"]`), the compiler cleanly bypasses `CASE` generation and compiles a simple `SELECT` across the base table, as all schema extensions (e.g. `light.person`, `full.person`) are guaranteed to reside in the exact same physical row.
### The Stem Engine
### Ad-Hoc Schema Promotion
Rather than over-fetching heavy Entity payloads and trimming them, Punc Framework Websockets depend on isolated subgraphs defined as **Stems**.
A `Stem` is a declaration of an **Entity Type boundary** that exists somewhere within the compiled JSON Schema graph, expressed using **`gjson` multipath syntax** (e.g., `contacts.#.phone_numbers.#`).
Because `pg_notify` (Beats) fire rigidly from physical Postgres tables (e.g. `{"type": "phone_number"}`), the Go Framework only ever needs to know: "Does the schema `with_contacts.person` contain the `phone_number` Entity anywhere inside its tree, and if so, what is the gjson path to iterate its payload?"
* **Initialization:** During startup (`jspg_stems()`), the database crawls all Schemas and maps out every physical Entity Type it references. It builds a highly optimized `HashMap<String, HashMap<String, Arc<Stem>>>` providing strictly `O(1)` memory lookups mapping `Schema ID -> { Stem Path -> Entity Type }`.
* **GJSON Pathing:** Unlike standard JSON Pointers, stems utilize `.#` array iterator syntax. The Go web server consumes this native path (e.g. `lines.#`) across the raw Postgres JSON byte payload, extracting all active UUIDs in one massive sub-millisecond sweep without unmarshaling Go ASTs.
* **Polymorphic Condition Selectors:** When trailing paths would otherwise collide because of abstract polymorphic type definitions (e.g., a `target` property bounded by a `oneOf` taking either `phone_number` or `email_address`), JSPG natively appends evaluated `gjson` type conditions into the path (e.g. `contacts.#.target#(type=="phone_number")`). This guarantees `O(1)` key uniqueness in the HashMap while retaining extreme array extraction speeds natively without runtime AST evaluation.
* **Identifier Prioritization:** When determining if a nested object boundary is an Entity, JSPG natively prioritizes defined `$id` tags over `$ref` inheritance pointers to prevent polymorphic boundaries from devolving into their generic base classes.
* **Cyclical Deduplication:** Because Punc relationships often reference back on themselves via deeply nested classes, the Stem Engine applies intelligent path deduplication. If the active `current_path` already ends with the target entity string, it traverses the inheritance properties without appending the entity to the stem path again, eliminating infinite powerset loops.
* **Relationship Path Squashing:** When calculating string paths structurally, JSPG intentionally **omits** properties natively named `target` or `source` if they belong to a native database `relationship` table override.
* **The Go Router**: The Golang Punc framework uses this exact mapping to register WebSocket Beat frequencies exclusively on the Entity types discovered.
* **The Queryer Execution**: When the Go framework asks JSPG to hydrate a partial `phone_number` stem for the `with_contacts.person` schema, instead of jumping through string paths, the SQL Compiler simply reaches into the Schema's AST using the `phone_number` Type string, pulls out exactly that entity's mapping rules, and returns a fully correlated `SELECT` block! This natively handles nested array properties injected via `oneOf` or array references efficiently bypassing runtime powerset expansion.
* **Performance:** These Stem execution structures are fully statically compiled via SPI and map perfectly to `O(1)` real-time routing logic on the application tier.
To seamlessly support deeply nested, inline Object definitions that don't declare an explicit `$id`, JSPG aggressively promotes them to standalone topological entities during the database compilation phase.
* **Hash Generation:** While evaluating the unified graph, if the compiler enters an `Object` or `Array` structure completely lacking an `$id`, it dynamically calculates a localized hash alias representing exactly its structural constraints.
* **Promotion:** This inline chunk is mathematically elevated to its own `$id` in the `db.schemas` cache registry. This guarantees that $O(1)$ WebSockets or isolated queries can natively target any arbitrary sub-object of a massive database topology directly without recursively re-parsing its parent's AST block every read.
## 5. Testing & Execution Architecture

819
d1.json
View File

@ -1,819 +0,0 @@
{
"database": {
"puncs": [],
"enums": [
{
"id": "11111111-1111-1111-1111-111111111111",
"type": "relation_type",
"enum": "relation_type",
"values": [
"foreign_key",
"polymorphic",
"graph"
]
}
],
"relations": [
{
"id": "22222222-2222-2222-2222-222222222222",
"type": "relation",
"constraint": "fk_order_customer",
"source_type": "order",
"source_columns": [
"customer_id"
],
"destination_type": "person",
"destination_columns": [
"id"
],
"prefix": "customer"
},
{
"id": "33333333-3333-3333-3333-333333333333",
"type": "relation",
"constraint": "fk_order_line_order",
"source_type": "order_line",
"source_columns": [
"order_id"
],
"destination_type": "order",
"destination_columns": [
"id"
],
"prefix": "lines"
},
{
"id": "44444444-4444-4444-4444-444444444444",
"type": "relation",
"constraint": "fk_relationship_source_entity",
"source_type": "relationship",
"source_columns": [
"source_id",
"source_type"
],
"destination_type": "entity",
"destination_columns": [
"id",
"type"
],
"prefix": "source"
},
{
"id": "55555555-5555-5555-5555-555555555555",
"type": "relation",
"constraint": "fk_relationship_target_entity",
"source_type": "relationship",
"source_columns": [
"target_id",
"target_type"
],
"destination_type": "entity",
"destination_columns": [
"id",
"type"
],
"prefix": "target"
}
],
"types": [
{
"name": "entity",
"schemas": [
{
"$id": "entity",
"type": "object",
"properties": {
"id": {
"type": "string"
},
"type": {
"type": "string"
},
"archived": {
"type": "boolean"
},
"created_by": {
"type": "string"
},
"modified_by": {
"type": "string"
},
"created_at": {
"type": "string",
"format": "date-time"
},
"modified_at": {
"type": "string",
"format": "date-time"
}
},
"required": [
"id",
"type",
"created_by",
"created_at",
"modified_by",
"modified_at"
]
}
],
"hierarchy": [
"entity"
],
"fields": [
"id",
"type",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
],
"grouped_fields": {
"entity": [
"id",
"type",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
]
},
"lookup_fields": [],
"historical": true,
"notify": true,
"relationship": false
},
{
"name": "organization",
"schemas": [
{
"$id": "organization",
"$ref": "entity",
"properties": {
"name": {
"type": "string"
}
}
}
],
"hierarchy": [
"organization",
"entity"
],
"fields": [
"id",
"type",
"name",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
],
"grouped_fields": {
"organization": [
"id",
"type",
"name"
],
"entity": [
"id",
"type",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
]
},
"lookup_fields": [],
"historical": true,
"notify": true,
"relationship": false
},
{
"name": "user",
"schemas": [
{
"$id": "user",
"$ref": "organization",
"properties": {}
}
],
"hierarchy": [
"user",
"organization",
"entity"
],
"fields": [
"id",
"type",
"name",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
],
"grouped_fields": {
"user": [
"id",
"type"
],
"organization": [
"id",
"type",
"name"
],
"entity": [
"id",
"type",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
]
},
"lookup_fields": [],
"historical": true,
"notify": true,
"relationship": false
},
{
"name": "person",
"schemas": [
{
"$id": "person",
"$ref": "user",
"properties": {
"first_name": {
"type": "string"
},
"last_name": {
"type": "string"
},
"date_of_birth": {
"type": "string"
},
"pronouns": {
"type": "string"
},
"contact_id": {
"type": "string"
},
"contacts": {
"type": "array",
"items": {
"$ref": "contact",
"properties": {
"target": {
"oneOf": [
{
"$ref": "phone_number"
},
{
"$ref": "email_address"
}
]
}
}
}
}
}
}
],
"hierarchy": [
"person",
"user",
"organization",
"entity"
],
"fields": [
"id",
"type",
"first_name",
"last_name",
"date_of_birth",
"pronouns",
"contact_id",
"name",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
],
"grouped_fields": {
"person": [
"id",
"type",
"first_name",
"last_name",
"date_of_birth",
"pronouns",
"contact_id"
],
"user": [
"id",
"type"
],
"organization": [
"id",
"type",
"name"
],
"entity": [
"id",
"type",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
]
},
"lookup_fields": [
"first_name",
"last_name",
"date_of_birth",
"pronouns"
],
"historical": true,
"notify": true,
"relationship": false
},
{
"name": "order",
"schemas": [
{
"$id": "order",
"$ref": "entity",
"properties": {
"total": {
"type": "number"
},
"customer_id": {
"type": "string"
},
"customer": {
"$ref": "person"
},
"lines": {
"type": "array",
"items": {
"$ref": "order_line"
}
}
}
}
],
"hierarchy": [
"order",
"entity"
],
"fields": [
"id",
"type",
"total",
"customer_id",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
],
"grouped_fields": {
"order": [
"id",
"type",
"total",
"customer_id"
],
"entity": [
"id",
"type",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
]
},
"lookup_fields": [
"id"
],
"historical": true,
"notify": true,
"relationship": false
},
{
"name": "order_line",
"schemas": [
{
"$id": "order_line",
"$ref": "entity",
"properties": {
"order_id": {
"type": "string"
},
"product": {
"type": "string"
},
"price": {
"type": "number"
}
}
}
],
"hierarchy": [
"order_line",
"entity"
],
"fields": [
"id",
"type",
"order_id",
"product",
"price",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
],
"grouped_fields": {
"order_line": [
"id",
"type",
"order_id",
"product",
"price"
],
"entity": [
"id",
"type",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
]
},
"lookup_fields": [],
"historical": true,
"notify": true,
"relationship": false
},
{
"name": "relationship",
"relationship": true,
"hierarchy": [
"relationship",
"entity"
],
"fields": [
"source_id",
"source_type",
"target_id",
"target_type",
"id",
"type",
"name",
"archived",
"created_at",
"created_by",
"modified_at",
"modified_by"
],
"grouped_fields": {
"entity": [
"id",
"type",
"name",
"archived",
"created_at",
"created_by",
"modified_at",
"modified_by"
],
"relationship": [
"source_id",
"source_type",
"target_id",
"target_type"
]
},
"field_types": {
"id": "uuid",
"type": "text",
"archived": "boolean",
"source_id": "uuid",
"source_type": "text",
"target_id": "uuid",
"target_type": "text",
"name": "text",
"created_at": "timestamptz",
"created_by": "uuid",
"modified_at": "timestamptz",
"modified_by": "uuid"
},
"schemas": [
{
"$id": "relationship",
"$ref": "entity",
"properties": {}
}
],
"lookup_fields": [],
"historical": true,
"notify": true
},
{
"name": "contact",
"relationship": true,
"hierarchy": [
"contact",
"relationship",
"entity"
],
"fields": [
"is_primary",
"source_id",
"source_type",
"target_id",
"target_type",
"id",
"type",
"name",
"archived",
"created_at",
"created_by",
"modified_at",
"modified_by"
],
"grouped_fields": {
"entity": [
"id",
"type",
"name",
"archived",
"created_at",
"created_by",
"modified_at",
"modified_by"
],
"relationship": [
"source_id",
"source_type",
"target_id",
"target_type"
],
"contact": [
"is_primary"
]
},
"field_types": {
"id": "uuid",
"type": "text",
"archived": "boolean",
"source_id": "uuid",
"source_type": "text",
"target_id": "uuid",
"target_type": "text",
"is_primary": "boolean",
"name": "text",
"created_at": "timestamptz",
"created_by": "uuid",
"modified_at": "timestamptz",
"modified_by": "uuid"
},
"schemas": [
{
"$id": "contact",
"$ref": "relationship",
"properties": {
"is_primary": {
"type": "boolean"
}
}
}
],
"lookup_fields": [],
"historical": true,
"notify": true
},
{
"name": "phone_number",
"hierarchy": [
"phone_number",
"entity"
],
"fields": [
"number",
"id",
"type",
"name",
"archived",
"created_at",
"created_by",
"modified_at",
"modified_by"
],
"grouped_fields": {
"entity": [
"id",
"type",
"name",
"archived",
"created_at",
"created_by",
"modified_at",
"modified_by"
],
"phone_number": [
"number"
]
},
"field_types": {
"id": "uuid",
"type": "text",
"archived": "boolean",
"number": "text",
"name": "text",
"created_at": "timestamptz",
"created_by": "uuid",
"modified_at": "timestamptz",
"modified_by": "uuid"
},
"schemas": [
{
"$id": "phone_number",
"$ref": "entity",
"properties": {
"number": {
"type": "string"
}
}
}
],
"lookup_fields": [],
"historical": true,
"notify": true,
"relationship": false
},
{
"name": "email_address",
"hierarchy": [
"email_address",
"entity"
],
"fields": [
"address",
"id",
"type",
"name",
"archived",
"created_at",
"created_by",
"modified_at",
"modified_by"
],
"grouped_fields": {
"entity": [
"id",
"type",
"name",
"archived",
"created_at",
"created_by",
"modified_at",
"modified_by"
],
"email_address": [
"address"
]
},
"field_types": {
"id": "uuid",
"type": "text",
"archived": "boolean",
"address": "text",
"name": "text",
"created_at": "timestamptz",
"created_by": "uuid",
"modified_at": "timestamptz",
"modified_by": "uuid"
},
"schemas": [
{
"$id": "email_address",
"$ref": "entity",
"properties": {
"address": {
"type": "string"
}
}
}
],
"lookup_fields": [],
"historical": true,
"notify": true,
"relationship": false
},
{
"name": "attachment",
"schemas": [
{
"$id": "type_metadata",
"type": "object",
"properties": {
"type": {
"type": "string"
}
}
},
{
"$id": "other_metadata",
"type": "object",
"properties": {
"other": {
"type": "string"
}
}
},
{
"$id": "attachment",
"$ref": "entity",
"properties": {
"flags": {
"type": "array",
"items": {
"type": "string"
}
},
"type_metadata": {
"$ref": "type_metadata"
},
"other_metadata": {
"$ref": "other_metadata"
}
}
}
],
"hierarchy": [
"attachment",
"entity"
],
"fields": [
"id",
"type",
"flags",
"type_metadata",
"other_metadata",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
],
"grouped_fields": {
"attachment": [
"id",
"type",
"flags",
"type_metadata",
"other_metadata"
],
"entity": [
"id",
"type",
"created_at",
"created_by",
"modified_at",
"modified_by",
"archived"
]
},
"field_types": {
"id": "uuid",
"type": "text",
"flags": "_text",
"type_metadata": "jsonb",
"other_metadata": "jsonb",
"created_at": "timestamptz",
"created_by": "uuid",
"modified_at": "timestamptz",
"modified_by": "uuid",
"archived": "boolean"
},
"lookup_fields": [],
"historical": true,
"notify": true,
"relationship": false
}
]
}
}

View File

@ -2401,6 +2401,215 @@
]
]
}
},
{
"description": "Anchor order and insert new line (no line id)",
"action": "merge",
"data": {
"id": "abc",
"type": "order",
"lines": [
{
"type": "order_line",
"product": "Widget",
"price": 99.0
}
]
},
"schema_id": "order",
"expect": {
"success": true,
"sql": [
[
"INSERT INTO agreego.\"entity\" (",
" \"created_at\",",
" \"created_by\",",
" \"id\",",
" \"modified_at\",",
" \"modified_by\",",
" \"type\"",
")",
"VALUES (",
" '{{timestamp}}',",
" '00000000-0000-0000-0000-000000000000',",
" '{{uuid:line_id}}',",
" '{{timestamp}}',",
" '00000000-0000-0000-0000-000000000000',",
" 'order_line'",
")"
],
[
"INSERT INTO agreego.\"order_line\" (",
" \"id\",",
" \"order_id\",",
" \"price\",",
" \"product\",",
" \"type\"",
")",
"VALUES (",
" '{{uuid:line_id}}',",
" 'abc',",
" 99,",
" 'Widget',",
" 'order_line'",
")"
],
[
"INSERT INTO agreego.change (",
" \"old\",",
" \"new\",",
" entity_id,",
" id,",
" kind,",
" modified_at,",
" modified_by",
")",
"VALUES (",
" NULL,",
" '{",
" \"order_id\":\"abc\",",
" \"price\":99.0,",
" \"product\":\"Widget\",",
" \"type\":\"order_line\"",
" }',",
" '{{uuid:line_id}}',",
" '{{uuid}}',",
" 'create',",
" '{{timestamp}}',",
" '00000000-0000-0000-0000-000000000000'",
")"
],
[
"SELECT pg_notify('entity', '{",
" \"complete\":{",
" \"created_at\":\"{{timestamp}}\",",
" \"created_by\":\"00000000-0000-0000-0000-000000000000\",",
" \"id\":\"{{uuid:line_id}}\",",
" \"modified_at\":\"{{timestamp}}\",",
" \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
" \"order_id\":\"abc\",",
" \"price\":99.0,",
" \"product\":\"Widget\",",
" \"type\":\"order_line\"",
" },",
" \"new\":{",
" \"order_id\":\"abc\",",
" \"price\":99.0,",
" \"product\":\"Widget\",",
" \"type\":\"order_line\"",
" }",
" }')"
]
]
}
},
{
"description": "Anchor order and insert new line (with line id)",
"action": "merge",
"data": {
"id": "abc",
"type": "order",
"lines": [
{
"id": "11111111-2222-3333-4444-555555555555",
"type": "order_line",
"product": "Widget",
"price": 99.0
}
]
},
"schema_id": "order",
"expect": {
"success": true,
"sql": [
[
"SELECT to_jsonb(t1.*) || to_jsonb(t2.*)",
"FROM agreego.\"order_line\" t1",
"LEFT JOIN agreego.\"entity\" t2 ON t2.id = t1.id",
"WHERE t1.id = '11111111-2222-3333-4444-555555555555'"
],
[
"INSERT INTO agreego.\"entity\" (",
" \"created_at\",",
" \"created_by\",",
" \"id\",",
" \"modified_at\",",
" \"modified_by\",",
" \"type\"",
")",
"VALUES (",
" '{{timestamp}}',",
" '00000000-0000-0000-0000-000000000000',",
" '11111111-2222-3333-4444-555555555555',",
" '{{timestamp}}',",
" '00000000-0000-0000-0000-000000000000',",
" 'order_line'",
")"
],
[
"INSERT INTO agreego.\"order_line\" (",
" \"id\",",
" \"order_id\",",
" \"price\",",
" \"product\",",
" \"type\"",
")",
"VALUES (",
" '11111111-2222-3333-4444-555555555555',",
" 'abc',",
" 99,",
" 'Widget',",
" 'order_line'",
")"
],
[
"INSERT INTO agreego.change (",
" \"old\",",
" \"new\",",
" entity_id,",
" id,",
" kind,",
" modified_at,",
" modified_by",
")",
"VALUES (",
" NULL,",
" '{",
" \"order_id\":\"abc\",",
" \"price\":99.0,",
" \"product\":\"Widget\",",
" \"type\":\"order_line\"",
" }',",
" '11111111-2222-3333-4444-555555555555',",
" '{{uuid}}',",
" 'create',",
" '{{timestamp}}',",
" '00000000-0000-0000-0000-000000000000'",
")"
],
[
"SELECT pg_notify('entity', '{",
" \"complete\":{",
" \"created_at\":\"{{timestamp}}\",",
" \"created_by\":\"00000000-0000-0000-0000-000000000000\",",
" \"id\":\"11111111-2222-3333-4444-555555555555\",",
" \"modified_at\":\"{{timestamp}}\",",
" \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
" \"order_id\":\"abc\",",
" \"price\":99.0,",
" \"product\":\"Widget\",",
" \"type\":\"order_line\"",
" },",
" \"new\":{",
" \"order_id\":\"abc\",",
" \"price\":99.0,",
" \"product\":\"Widget\",",
" \"type\":\"order_line\"",
" }",
" }')"
]
]
}
}
]
}

View File

@ -1003,6 +1003,7 @@
" JOIN agreego.entity entity_6 ON entity_6.id = relationship_5.id",
" WHERE",
" NOT entity_6.archived",
" AND relationship_5.target_type = 'address'",
" AND relationship_5.source_id = entity_3.id),",
" 'age', person_1.age,",
" 'archived', entity_3.archived,",
@ -1094,6 +1095,7 @@
" JOIN agreego.entity entity_20 ON entity_20.id = relationship_19.id",
" WHERE",
" NOT entity_20.archived",
" AND relationship_19.target_type = 'email_address'",
" AND relationship_19.source_id = entity_3.id),",
" 'first_name', person_1.first_name,",
" 'id', entity_3.id,",
@ -1127,6 +1129,7 @@
" JOIN agreego.entity entity_25 ON entity_25.id = relationship_24.id",
" WHERE",
" NOT entity_25.archived",
" AND relationship_24.target_type = 'phone_number'",
" AND relationship_24.source_id = entity_3.id),",
" 'type', entity_3.type",
")",
@ -1163,7 +1166,7 @@
"$eq": true,
"$ne": false
},
"contacts.#.is_primary": {
"contacts/is_primary": {
"$eq": true
},
"created_at": {
@ -1203,7 +1206,7 @@
"$eq": "%Doe%",
"$ne": "%Smith%"
},
"phone_numbers.#.target.number": {
"phone_numbers/target/number": {
"$eq": "555-1234"
}
},
@ -1240,6 +1243,7 @@
" JOIN agreego.entity entity_6 ON entity_6.id = relationship_5.id",
" WHERE",
" NOT entity_6.archived",
" AND relationship_5.target_type = 'address'",
" AND relationship_5.source_id = entity_3.id),",
" 'age', person_1.age,",
" 'archived', entity_3.archived,",
@ -1332,6 +1336,7 @@
" JOIN agreego.entity entity_20 ON entity_20.id = relationship_19.id",
" WHERE",
" NOT entity_20.archived",
" AND relationship_19.target_type = 'email_address'",
" AND relationship_19.source_id = entity_3.id),",
" 'first_name', person_1.first_name,",
" 'id', entity_3.id,",
@ -1366,6 +1371,7 @@
" JOIN agreego.entity entity_25 ON entity_25.id = relationship_24.id",
" WHERE",
" NOT entity_25.archived",
" AND relationship_24.target_type = 'phone_number'",
" AND relationship_24.source_id = entity_3.id),",
" 'type', entity_3.type",
")",
@ -1408,6 +1414,46 @@
]
}
},
{
"description": "Person ad-hoc email addresses select",
"action": "query",
"schema_id": "full.person/email_addresses",
"expect": {
"success": true,
"sql": [
[
"(SELECT jsonb_build_object(",
" 'archived', entity_3.archived,",
" 'created_at', entity_3.created_at,",
" 'id', entity_3.id,",
" 'is_primary', contact_1.is_primary,",
" 'name', entity_3.name,",
" 'target',",
" (SELECT jsonb_build_object(",
" 'address', email_address_4.address,",
" 'archived', entity_5.archived,",
" 'created_at', entity_5.created_at,",
" 'id', entity_5.id,",
" 'name', entity_5.name,",
" 'type', entity_5.type",
" )",
" FROM agreego.email_address email_address_4",
" JOIN agreego.entity entity_5 ON entity_5.id = email_address_4.id",
" WHERE",
" NOT entity_5.archived",
" AND relationship_2.target_id = entity_5.id),",
" 'type', entity_3.type",
")",
"FROM agreego.contact contact_1",
"JOIN agreego.relationship relationship_2 ON relationship_2.id = contact_1.id",
"JOIN agreego.entity entity_3 ON entity_3.id = relationship_2.id",
"WHERE",
" NOT entity_3.archived",
" AND relationship_2.target_type = 'email_address')"
]
]
}
},
{
"description": "Order select with customer and lines",
"action": "query",

View File

@ -9,6 +9,61 @@ impl SpiExecutor {
pub fn new() -> Self {
Self {}
}
fn transact<F, R>(&self, f: F) -> Result<R, String>
where
F: FnOnce() -> Result<R, String>,
{
unsafe {
let oldcontext = pgrx::pg_sys::CurrentMemoryContext;
let oldowner = pgrx::pg_sys::CurrentResourceOwner;
pgrx::pg_sys::BeginInternalSubTransaction(std::ptr::null());
pgrx::pg_sys::MemoryContextSwitchTo(oldcontext);
let runner = std::panic::AssertUnwindSafe(move || {
let res = f();
pgrx::pg_sys::ReleaseCurrentSubTransaction();
pgrx::pg_sys::MemoryContextSwitchTo(oldcontext);
pgrx::pg_sys::CurrentResourceOwner = oldowner;
res
});
pgrx::PgTryBuilder::new(runner)
.catch_rust_panic(|cause| {
pgrx::pg_sys::RollbackAndReleaseCurrentSubTransaction();
pgrx::pg_sys::MemoryContextSwitchTo(oldcontext);
pgrx::pg_sys::CurrentResourceOwner = oldowner;
// Rust panics are fatal bugs, not validation errors. Rethrow so they bubble up.
cause.rethrow()
})
.catch_others(|cause| {
pgrx::pg_sys::RollbackAndReleaseCurrentSubTransaction();
pgrx::pg_sys::MemoryContextSwitchTo(oldcontext);
pgrx::pg_sys::CurrentResourceOwner = oldowner;
let error_msg = match &cause {
pgrx::pg_sys::panic::CaughtError::PostgresError(e)
| pgrx::pg_sys::panic::CaughtError::ErrorReport(e) => {
let json_err = serde_json::json!({
"error": e.message(),
"code": format!("{:?}", e.sql_error_code()),
"detail": e.detail(),
"hint": e.hint()
});
json_err.to_string()
}
_ => format!("{:?}", cause),
};
pgrx::warning!("JSPG Caught Native Postgres Error: {}", error_msg);
Err(error_msg)
})
.execute()
}
}
}
impl DatabaseExecutor for SpiExecutor {
@ -24,7 +79,7 @@ impl DatabaseExecutor for SpiExecutor {
}
}
pgrx::PgTryBuilder::new(|| {
self.transact(|| {
Spi::connect(|client| {
pgrx::notice!("JSPG_SQL: {}", sql);
match client.select(sql, Some(args_with_oid.len() as i64), &args_with_oid) {
@ -41,11 +96,6 @@ impl DatabaseExecutor for SpiExecutor {
}
})
})
.catch_others(|cause| {
pgrx::warning!("JSPG Caught Native Postgres Error: {:?}", cause);
Err(format!("{:?}", cause))
})
.execute()
}
fn execute(&self, sql: &str, args: Option<&[Value]>) -> Result<(), String> {
@ -60,7 +110,7 @@ impl DatabaseExecutor for SpiExecutor {
}
}
pgrx::PgTryBuilder::new(|| {
self.transact(|| {
Spi::connect_mut(|client| {
pgrx::notice!("JSPG_SQL: {}", sql);
match client.update(sql, Some(args_with_oid.len() as i64), &args_with_oid) {
@ -69,44 +119,43 @@ impl DatabaseExecutor for SpiExecutor {
}
})
})
.catch_others(|cause| {
pgrx::warning!("JSPG Caught Native Postgres Error: {:?}", cause);
Err(format!("{:?}", cause))
})
.execute()
}
fn auth_user_id(&self) -> Result<String, String> {
Spi::connect(|client| {
let mut tup_table = client
.select(
"SELECT COALESCE(current_setting('auth.user_id', true), 'ffffffff-ffff-ffff-ffff-ffffffffffff')",
None,
&[],
)
.map_err(|e| format!("SPI Select Error: {}", e))?;
self.transact(|| {
Spi::connect(|client| {
let mut tup_table = client
.select(
"SELECT COALESCE(current_setting('auth.user_id', true), 'ffffffff-ffff-ffff-ffff-ffffffffffff')",
None,
&[],
)
.map_err(|e| format!("SPI Select Error: {}", e))?;
let row = tup_table
.next()
.ok_or("No user id setting returned from context".to_string())?;
let user_id: Option<String> = row.get(1).map_err(|e| e.to_string())?;
let row = tup_table
.next()
.ok_or("No user id setting returned from context".to_string())?;
let user_id: Option<String> = row.get(1).map_err(|e| e.to_string())?;
user_id.ok_or("Missing user_id".to_string())
user_id.ok_or("Missing user_id".to_string())
})
})
}
fn timestamp(&self) -> Result<String, String> {
Spi::connect(|client| {
let mut tup_table = client
.select("SELECT clock_timestamp()::text", None, &[])
.map_err(|e| format!("SPI Select Error: {}", e))?;
self.transact(|| {
Spi::connect(|client| {
let mut tup_table = client
.select("SELECT clock_timestamp()::text", None, &[])
.map_err(|e| format!("SPI Select Error: {}", e))?;
let row = tup_table
.next()
.ok_or("No clock timestamp returned".to_string())?;
let timestamp: Option<String> = row.get(1).map_err(|e| e.to_string())?;
let row = tup_table
.next()
.ok_or("No clock timestamp returned".to_string())?;
let timestamp: Option<String> = row.get(1).map_err(|e| e.to_string())?;
timestamp.ok_or("Missing timestamp".to_string())
timestamp.ok_or("Missing timestamp".to_string())
})
})
}
}

View File

@ -507,10 +507,8 @@ impl Schema {
let mut parent_type_name = None;
if let Some(family) = &self.obj.family {
parent_type_name = Some(family.split('.').next_back().unwrap_or(family).to_string());
} else if let Some(id) = &self.obj.id {
parent_type_name = Some(id.split('.').next_back().unwrap_or("").to_string());
} else if let Some(ref_id) = &self.obj.r#ref {
parent_type_name = Some(ref_id.split('.').next_back().unwrap_or("").to_string());
} else if let Some(identifier) = self.obj.identifier() {
parent_type_name = Some(identifier);
}
if let Some(p_type) = parent_type_name {
@ -531,12 +529,12 @@ impl Schema {
if let Some(family) = &target_schema.obj.family {
child_type_name = Some(family.split('.').next_back().unwrap_or(family).to_string());
} else if let Some(ref_id) = target_schema.obj.r#ref.as_ref() {
child_type_name = Some(ref_id.split('.').next_back().unwrap_or("").to_string());
} else if let Some(ref_id) = target_schema.obj.identifier() {
child_type_name = Some(ref_id);
} else if let Some(arr) = &target_schema.obj.one_of {
if let Some(first) = arr.first() {
if let Some(ref_id) = first.obj.id.as_ref().or(first.obj.r#ref.as_ref()) {
child_type_name = Some(ref_id.split('.').next_back().unwrap_or("").to_string());
if let Some(ref_id) = first.obj.identifier() {
child_type_name = Some(ref_id);
}
}
}
@ -697,6 +695,16 @@ impl<'de> Deserialize<'de> for Schema {
}
}
impl SchemaObject {
pub fn identifier(&self) -> Option<String> {
if let Some(lookup_key) = self.id.as_ref().or(self.r#ref.as_ref()) {
Some(lookup_key.split('.').next_back().unwrap_or("").to_string())
} else {
None
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(untagged)]
pub enum SchemaTypeOrArray {

View File

@ -45,12 +45,33 @@ impl Merger {
let val_resolved = match result {
Ok(val) => val,
Err(msg) => {
let mut final_code = "MERGE_FAILED".to_string();
let mut final_message = msg.clone();
let mut final_cause = None;
if let Ok(Value::Object(map)) = serde_json::from_str::<Value>(&msg) {
if let (Some(Value::String(e_msg)), Some(Value::String(e_code))) = (map.get("error"), map.get("code")) {
final_message = e_msg.clone();
final_code = e_code.clone();
let mut cause_parts = Vec::new();
if let Some(Value::String(d)) = map.get("detail") {
if !d.is_empty() { cause_parts.push(d.clone()); }
}
if let Some(Value::String(h)) = map.get("hint") {
if !h.is_empty() { cause_parts.push(h.clone()); }
}
if !cause_parts.is_empty() {
final_cause = Some(cause_parts.join("\n"));
}
}
}
return crate::drop::Drop::with_errors(vec![crate::drop::Error {
code: "MERGE_FAILED".to_string(),
message: msg,
code: final_code,
message: final_message,
details: crate::drop::ErrorDetails {
path: "".to_string(),
cause: None,
cause: final_cause,
context: Some(data),
schema: None,
},
@ -403,6 +424,23 @@ impl Merger {
> {
let type_name = type_def.name.as_str();
// 🚀 Anchor Short-Circuit Optimization
// An anchor is STRICTLY a struct containing merely an `id` and `type`.
// We aggressively bypass Database SPI `SELECT` fetches because there are no primitive
// mutations to apply to the row. PostgreSQL inherently protects relationships via Foreign Keys downstream.
let is_anchor = entity_fields.len() == 2
&& entity_fields.contains_key("id")
&& entity_fields.contains_key("type");
let has_valid_id = entity_fields
.get("id")
.and_then(|v| v.as_str())
.map_or(false, |s| !s.is_empty());
if is_anchor && has_valid_id {
return Ok((entity_fields, None, None));
}
let entity_fetched = self.fetch_entity(&entity_fields, type_def)?;
let system_keys = vec![
@ -674,8 +712,7 @@ impl Merger {
);
self
.db
.execute(&sql, None)
.map_err(|e| format!("SPI Error in INSERT: {:?}", e))?;
.execute(&sql, None)?;
} else if change_kind == "update" || change_kind == "delete" {
entity_pairs.remove("id");
entity_pairs.remove("type");
@ -709,8 +746,7 @@ impl Merger {
);
self
.db
.execute(&sql, None)
.map_err(|e| format!("SPI Error in UPDATE: {:?}", e))?;
.execute(&sql, None)?;
}
}
@ -813,10 +849,7 @@ impl Merger {
Self::quote_literal(&Value::String(user_id.to_string()))
);
self
.db
.execute(&change_sql, None)
.map_err(|e| format!("Executor Error in change: {:?}", e))?;
self.db.execute(&change_sql, None)?;
}
if type_obj.notify {

View File

@ -64,11 +64,7 @@ impl<'a> Compiler<'a> {
fn compile_array(&mut self, node: Node<'a>) -> Result<(String, String), String> {
if let Some(items) = &node.schema.obj.items {
let next_path = if node.ast_path.is_empty() {
String::from("#")
} else {
format!("{}.#", node.ast_path)
};
let next_path = node.ast_path.clone();
if let Some(ref_id) = &items.obj.r#ref {
if let Some(type_def) = self.db.types.get(ref_id) {
@ -102,14 +98,7 @@ impl<'a> Compiler<'a> {
if let Some(family_target) = node.schema.obj.family.as_ref() {
resolved_type = self.db.types.get(family_target);
} else if let Some(lookup_key) = node
.schema
.obj
.id
.as_ref()
.or(node.schema.obj.r#ref.as_ref())
{
let base_type_name = lookup_key.split('.').next_back().unwrap_or("").to_string();
} else if let Some(base_type_name) = node.schema.obj.identifier() {
resolved_type = self.db.types.get(&base_type_name);
}
@ -448,22 +437,21 @@ impl<'a> Compiler<'a> {
}
}
let mut child_node = node.clone();
child_node.parent_alias = owner_alias.clone();
let arc_aliases = std::sync::Arc::new(table_aliases.clone());
child_node.parent_type_aliases = Some(arc_aliases);
child_node.parent_type = Some(r#type);
child_node.parent_schema = Some(std::sync::Arc::clone(&node.schema));
child_node.property_name = Some(prop_key.clone());
child_node.depth += 1;
let next_path = if node.ast_path.is_empty() {
prop_key.clone()
} else {
format!("{}.{}", node.ast_path, prop_key)
let child_node = Node {
schema: std::sync::Arc::clone(prop_schema),
parent_alias: owner_alias.clone(),
parent_type_aliases: Some(std::sync::Arc::new(table_aliases.clone())),
parent_type: Some(r#type),
parent_schema: Some(std::sync::Arc::clone(&node.schema)),
property_name: Some(prop_key.clone()),
depth: node.depth + 1,
ast_path: if node.ast_path.is_empty() {
prop_key.clone()
} else {
format!("{}/{}", node.ast_path, prop_key)
},
};
child_node.ast_path = next_path;
child_node.schema = std::sync::Arc::clone(prop_schema);
let (val_sql, val_type) = self.compile_node(child_node)?;
@ -491,9 +479,17 @@ impl<'a> Compiler<'a> {
.unwrap_or_else(|| base_alias.clone());
let mut where_clauses = Vec::new();
where_clauses.push(format!("NOT {}.archived", entity_alias));
// Dynamically apply the 'active-only' default ONLY if the client
// didn't explicitly request to filter on 'archived' themselves!
let has_archived_override = self.filter_keys.iter().any(|k| k == "archived");
if !has_archived_override {
where_clauses.push(format!("NOT {}.archived", entity_alias));
}
self.compile_filter_conditions(r#type, type_aliases, &node, &base_alias, &mut where_clauses);
self.compile_polymorphic_bounds(r#type, type_aliases, &node, &mut where_clauses);
self.compile_relation_conditions(
r#type,
type_aliases,
@ -505,6 +501,54 @@ impl<'a> Compiler<'a> {
Ok(where_clauses)
}
fn compile_polymorphic_bounds(
&self,
_type: &crate::database::r#type::Type,
type_aliases: &std::collections::HashMap<String, String>,
node: &Node,
where_clauses: &mut Vec<String>,
) {
if let Some(edges) = node.schema.obj.compiled_edges.get() {
if let Some(props) = node.schema.obj.compiled_properties.get() {
for (prop_name, edge) in edges {
if let Some(prop_schema) = props.get(prop_name) {
// Determine if the property schema resolves to a physical Database Entity
let mut bound_type_name = None;
if let Some(family_target) = prop_schema.obj.family.as_ref() {
bound_type_name = Some(family_target.split('.').next_back().unwrap_or(family_target).to_string());
} else if let Some(lookup_key) = prop_schema.obj.identifier() {
bound_type_name = Some(lookup_key);
}
if let Some(type_name) = bound_type_name {
// Ensure this type actually exists
if self.db.types.contains_key(&type_name) {
if let Some(relation) = self.db.relations.get(&edge.constraint) {
let mut poly_col = None;
let mut table_to_alias = "";
if edge.forward && relation.source_columns.len() > 1 {
poly_col = Some(&relation.source_columns[1]); // e.g., target_type
table_to_alias = &relation.source_type; // e.g., relationship
} else if !edge.forward && relation.destination_columns.len() > 1 {
poly_col = Some(&relation.destination_columns[1]); // e.g., source_type
table_to_alias = &relation.destination_type; // e.g., relationship
}
if let Some(col) = poly_col {
if let Some(alias) = type_aliases.get(table_to_alias).or_else(|| type_aliases.get(&node.parent_alias)) {
where_clauses.push(format!("{}.{} = '{}'", alias, col, type_name));
}
}
}
}
}
}
}
}
}
}
fn resolve_filter_alias(
r#type: &crate::database::r#type::Type,
type_aliases: &std::collections::HashMap<String, String>,
@ -580,15 +624,15 @@ impl<'a> Compiler<'a> {
let op = parts.next().unwrap_or("$eq");
let field_name = if node.ast_path.is_empty() {
if full_field_path.contains('.') || full_field_path.contains('#') {
if full_field_path.contains('/') {
continue;
}
full_field_path
} else {
let prefix = format!("{}.", node.ast_path);
let prefix = format!("{}/", node.ast_path);
if full_field_path.starts_with(&prefix) {
let remainder = &full_field_path[prefix.len()..];
if remainder.contains('.') || remainder.contains('#') {
if remainder.contains('/') {
continue;
}
remainder

View File

@ -54,6 +54,45 @@ impl Queryer {
self.execute_sql(schema_id, &sql, &args)
}
fn extract_filters(
prefix: String,
val: &serde_json::Value,
entries: &mut Vec<(String, serde_json::Value)>,
) -> Result<(), String> {
if let Some(obj) = val.as_object() {
let mut is_op_obj = false;
if let Some(first_key) = obj.keys().next() {
if first_key.starts_with('$') {
is_op_obj = true;
}
}
if is_op_obj {
for (op, op_val) in obj {
if !op.starts_with('$') {
return Err(format!("Filter operator must start with '$', got: {}", op));
}
entries.push((format!("{}:{}", prefix, op), op_val.clone()));
}
} else {
for (k, v) in obj {
let next_prefix = if prefix.is_empty() {
k.clone()
} else {
format!("{}/{}", prefix, k)
};
Self::extract_filters(next_prefix, v, entries)?;
}
}
} else {
return Err(format!(
"Filter for path '{}' must be an operator object like {{$eq: ...}} or a nested map.",
prefix
));
}
Ok(())
}
fn parse_filter_entries(
&self,
filters_map: Option<&serde_json::Map<String, serde_json::Value>>,
@ -61,19 +100,7 @@ impl Queryer {
let mut filter_entries: Vec<(String, serde_json::Value)> = Vec::new();
if let Some(fm) = filters_map {
for (key, val) in fm {
if let Some(obj) = val.as_object() {
for (op, op_val) in obj {
if !op.starts_with('$') {
return Err(format!("Filter operator must start with '$', got: {}", op));
}
filter_entries.push((format!("{}:{}", key, op), op_val.clone()));
}
} else {
return Err(format!(
"Filter for field '{}' must be an object with operators like $eq, $in, etc.",
key
));
}
Self::extract_filters(key.clone(), val, &mut filter_entries)?;
}
}
filter_entries.sort_by(|a, b| a.0.cmp(&b.0));

View File

@ -1451,6 +1451,12 @@ fn test_queryer_0_6() {
crate::tests::runner::run_test_case(&path, 0, 6).unwrap();
}
#[test]
fn test_queryer_0_7() {
let path = format!("{}/fixtures/queryer.json", env!("CARGO_MANIFEST_DIR"));
crate::tests::runner::run_test_case(&path, 0, 7).unwrap();
}
#[test]
fn test_not_0_0() {
let path = format!("{}/fixtures/not.json", env!("CARGO_MANIFEST_DIR"));
@ -8542,3 +8548,15 @@ fn test_merger_0_8() {
let path = format!("{}/fixtures/merger.json", env!("CARGO_MANIFEST_DIR"));
crate::tests::runner::run_test_case(&path, 0, 8).unwrap();
}
#[test]
fn test_merger_0_9() {
let path = format!("{}/fixtures/merger.json", env!("CARGO_MANIFEST_DIR"));
crate::tests::runner::run_test_case(&path, 0, 9).unwrap();
}
#[test]
fn test_merger_0_10() {
let path = format!("{}/fixtures/merger.json", env!("CARGO_MANIFEST_DIR"));
crate::tests::runner::run_test_case(&path, 0, 10).unwrap();
}

View File

@ -14,16 +14,13 @@ impl<'a> ValidationContext<'a> {
let current = self.instance;
if let Some(obj) = current.as_object() {
// Entity implicit type validation
// Use the specific schema id or ref as a fallback
if let Some(identifier) = self.schema.id.as_ref().or(self.schema.r#ref.as_ref()) {
if let Some(schema_identifier) = self.schema.identifier() {
// Kick in if the data object has a type field
if let Some(type_val) = obj.get("type")
&& let Some(type_str) = type_val.as_str()
{
// Get the string or the final segment as the base
let base = identifier.split('.').next_back().unwrap_or("").to_string();
// Check if the base is a global type name
if let Some(type_def) = self.db.types.get(&base) {
// Check if the identifier is a global type name
if let Some(type_def) = self.db.types.get(&schema_identifier) {
// Ensure the instance type is a variation of the global type
if type_def.variations.contains(type_str) {
// Ensure it passes strict mode
@ -40,7 +37,7 @@ impl<'a> ValidationContext<'a> {
}
} else {
// Ad-Hoc schemas natively use strict schema discriminator strings instead of variation inheritance
if type_str == identifier {
if type_str == schema_identifier.as_str() {
result.evaluated_keys.insert("type".to_string());
}
}
@ -128,14 +125,9 @@ impl<'a> ValidationContext<'a> {
// Entity Bound Implicit Type Interception
if key == "type"
&& let Some(schema_bound) = sub_schema.id.as_ref().or(sub_schema.r#ref.as_ref())
&& let Some(schema_bound) = sub_schema.identifier()
{
let physical_type_name = schema_bound
.split('.')
.next_back()
.unwrap_or("")
.to_string();
if let Some(type_def) = self.db.types.get(&physical_type_name)
if let Some(type_def) = self.db.types.get(&schema_bound)
&& let Some(instance_type) = child_instance.as_str()
&& type_def.variations.contains(instance_type)
{

View File

@ -1 +1 @@
1.0.84
1.0.90