version: 1.0.80

.test/tests.md

@@ -0,0 +1,11 @@
+# 🗒️ Test Report (punc/framework)
+
+_Generated at Wed Mar 18 05:21:40 EDT 2026_
+
+## Summary
+
+| Lang | Status | Tests | Passed | Failed | Duration |
+| :--- | :---: | :---: | :---: | :---: | ---: |
+
+## Results
+

.vscode/extensions.json

@@ -0,0 +1,5 @@
+{
+    "recommendations": [
+        "rust-lang.rust-analyzer"
+    ]
+}

Cargo.toml

@@ -39,10 +39,6 @@ crate-type = ["cdylib", "lib"]
 name = "pgrx_embed_jspg"
 path = "src/bin/pgrx_embed.rs"
 
-[[bin]]
-name = "ast_explore"
-path = "src/bin/ast_explore.rs"
-
 [features]
 default = ["pg18"]
 pg18 = ["pgrx/pg18", "pgrx-tests/pg18" ]

GEMINI.md

@@ -43,7 +43,7 @@ JSPG implements specific extensions to the Draft 2020-12 standard to support the
 #### A. Polymorphism & Referencing (`$ref`, `$family`, and Native Types)
 *   **Native Type Discrimination (`variations`)**: Schemas defined inside a Postgres `type` are Entities. The validator securely and implicitly manages their `"type"` property. If an entity inherits from `user`, incoming JSON can safely define `{"type": "person"}` without errors, thanks to `compiled_variations` inheritance.
 *   **Structural Inheritance & Viral Infection (`$ref`)**: `$ref` is used exclusively for structural inheritance, *never* for union creation. A Punc request schema that `$ref`s an Entity virally inherits all physical database polymorphism rules for that target.
-*   **Shape Polymorphism (`$family`)**: Auto-expands polymorphic API lists based on an abstract Descendants Graph. If `{"$family": "widget"}` is used, JSPG evaluates the JSON against every schema that `$ref`s widget.
+*   **Shape Polymorphism (`$family`)**: Auto-expands polymorphic API lists based on an abstract **Descendants Graph**. If `{"$family": "widget"}` is used, the Validator dynamically identifies *every* schema in the registry that `$ref`s `widget` (e.g., `stock.widget`, `task.widget`) and evaluates the JSON against all of them.
 *   **Strict Matches & Depth Heuristic**: Polymorphic structures MUST match exactly **one** schema permutation. If multiple inherited struct permutations pass, JSPG applies the **Depth Heuristic Tie-Breaker**, selecting the candidate deepest in the inheritance tree. 
 
 #### B. Dot-Notation Schema Resolution & Database Mapping
@@ -103,6 +103,10 @@ The Queryer transforms Postgres into a pre-compiled Semantic Query Engine via th
     *   **Array Inclusion**: `{"$in": [values]}`, `{"$nin": [values]}` use native `jsonb_array_elements_text()` bindings to enforce `IN` and `NOT IN` logic without runtime SQL injection risks.
     *   **Text Matching (ILIKE)**: Evaluates `$eq` or `$ne` against string fields containing the `%` character natively into Postgres `ILIKE` and `NOT ILIKE` partial substring matches.
     *   **Type Casting**: Safely resolves dynamic combinations by casting values instantly into the physical database types mapped in the schema (e.g. parsing `uuid` bindings to `::uuid`, formatting DateTimes to `::timestamptz`, and numbers to `::numeric`).
+*   **Polymorphic SQL Generation (`$family`)**: Compiles `$family` properties by analyzing the **Physical Database Variations**, *not* the schema descendants.
+    *   **The Dot Convention**: When a schema requests `$family: "target.schema"`, the compiler extracts the base type (e.g. `schema`) and looks up its Physical Table definition.
+    *   **Multi-Table Branching**: If the Physical Table is a parent to other tables (e.g. `organization` has variations `["organization", "bot", "person"]`), the compiler generates a dynamic `CASE WHEN type = '...' THEN ...` query, expanding into `JOIN`s for each variation.
+    *   **Single-Table Bypass**: If the Physical Table is a leaf node with only one variation (e.g. `person` has variations `["person"]`), the compiler cleanly bypasses `CASE` generation and compiles a simple `SELECT` across the base table, as all schema extensions (e.g. `light.person`, `full.person`) are guaranteed to reside in the exact same physical row.
 
 ### The Stem Engine
 

fixtures/merger.json

@@ -143,7 +143,8 @@
                         ]
                     },
                     "lookup_fields": [],
-                    "historical": false,
+                    "historical": true,
+                    "notify": true,
                     "relationship": false
                 },
                 {
@@ -191,6 +192,7 @@
                     },
                     "lookup_fields": [],
                     "historical": true,
+                    "notify": true,
                     "relationship": false
                 },
                 {
@@ -239,6 +241,7 @@
                     },
                     "lookup_fields": [],
                     "historical": true,
+                    "notify": true,
                     "relationship": false
                 },
                 {
@@ -341,6 +344,7 @@
                         "pronouns"
                     ],
                     "historical": true,
+                    "notify": true,
                     "relationship": false
                 },
                 {
@@ -395,6 +399,7 @@
                         "id"
                     ],
                     "historical": true,
+                    "notify": true,
                     "relationship": false
                 },
                 {
@@ -452,6 +457,7 @@
                     },
                     "lookup_fields": [],
                     "historical": true,
+                    "notify": true,
                     "relationship": false
                 },
                 {
@@ -515,7 +521,8 @@
                         }
                     ],
                     "lookup_fields": [],
-                    "historical": true
+                    "historical": true,
+                    "notify": true
                 },
                 {
                     "name": "contact",
@@ -588,7 +595,8 @@
                         }
                     ],
                     "lookup_fields": [],
-                    "historical": true
+                    "historical": true,
+                    "notify": true
                 },
                 {
                     "name": "phone_number",
@@ -646,6 +654,7 @@
                     ],
                     "lookup_fields": [],
                     "historical": true,
+                    "notify": true,
                     "relationship": false
                 },
                 {
@@ -704,6 +713,75 @@
                     ],
                     "lookup_fields": [],
                     "historical": true,
+                    "notify": true,
+                    "relationship": false
+                },
+                {
+                    "name": "attachment",
+                    "schemas": [
+                        {
+                            "$id": "attachment",
+                            "$ref": "entity",
+                            "properties": {
+                                "flags": {
+                                    "type": "array",
+                                    "items": {
+                                        "type": "string"
+                                    }
+                                },
+                                "metadata": {
+                                    "type": "object",
+                                    "additionalProperties": true
+                                }
+                            }
+                        }
+                    ],
+                    "hierarchy": [
+                        "attachment",
+                        "entity"
+                    ],
+                    "fields": [
+                        "id",
+                        "type",
+                        "flags",
+                        "metadata",
+                        "created_at",
+                        "created_by",
+                        "modified_at",
+                        "modified_by",
+                        "archived"
+                    ],
+                    "grouped_fields": {
+                        "attachment": [
+                            "id",
+                            "type",
+                            "flags",
+                            "metadata"
+                        ],
+                        "entity": [
+                            "id",
+                            "type",
+                            "created_at",
+                            "created_by",
+                            "modified_at",
+                            "modified_by",
+                            "archived"
+                        ]
+                    },
+                    "field_types": {
+                        "id": "uuid",
+                        "type": "text",
+                        "flags": "_text",
+                        "metadata": "jsonb",
+                        "created_at": "timestamptz",
+                        "created_by": "uuid",
+                        "modified_at": "timestamptz",
+                        "modified_by": "uuid",
+                        "archived": "boolean"
+                    },
+                    "lookup_fields": [],
+                    "historical": true,
+                    "notify": true,
                     "relationship": false
                 }
             ]
@@ -785,7 +863,8 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
@@ -793,6 +872,7 @@
                             "    modified_by",
                             ")",
                             "VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"first_name\":\"IncompleteFirst\",",
                             "        \"last_name\":\"IncompleteLast\",",
@@ -816,6 +896,11 @@
                             "            \"modified_at\":\"{{timestamp}}\",",
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"type\":\"person\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"first_name\":\"IncompleteFirst\",",
+                            "            \"last_name\":\"IncompleteLast\",",
+                            "            \"type\":\"person\"",
                             "        }",
                             "    }')"
                         ]
@@ -872,7 +957,8 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
@@ -881,6 +967,9 @@
                             ")",
                             "VALUES (",
                             "    '{",
+                            "        \"contact_id\":\"old-contact\"",
+                            "    }',",
+                            "    '{",
                             "        \"contact_id\":\"abc-contact\",",
                             "        \"type\":\"person\"",
                             "    }',",
@@ -893,10 +982,6 @@
                         ],
                         [
                             "SELECT pg_notify('entity', '{",
-                            "    \"changes\":{",
-                            "        \"contact_id\":\"abc-contact\",",
-                            "        \"type\":\"person\"",
-                            "    },",
                             "        \"complete\":{",
                             "            \"contact_id\":\"abc-contact\",",
                             "            \"date_of_birth\":\"1990-01-01T00:00:00Z\",",
@@ -907,6 +992,13 @@
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"pronouns\":\"they/them\",",
                             "            \"type\":\"person\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"contact_id\":\"abc-contact\",",
+                            "            \"type\":\"person\"",
+                            "        },",
+                            "        \"old\":{",
+                            "            \"contact_id\":\"old-contact\"",
                             "        }",
                             "    }')"
                         ]
@@ -959,7 +1051,8 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
@@ -968,6 +1061,10 @@
                             ")",
                             "VALUES (",
                             "    '{",
+                            "        \"first_name\":\"OldFirst\",",
+                            "        \"last_name\":\"OldLast\"",
+                            "    }',",
+                            "    '{",
                             "        \"first_name\":\"NewFirst\",",
                             "        \"last_name\":\"NewLast\",",
                             "        \"type\":\"person\"",
@@ -981,11 +1078,6 @@
                         ],
                         [
                             "SELECT pg_notify('entity', '{",
-                            "    \"changes\":{",
-                            "        \"first_name\":\"NewFirst\",",
-                            "        \"last_name\":\"NewLast\",",
-                            "        \"type\":\"person\"",
-                            "    },",
                             "        \"complete\":{",
                             "            \"first_name\":\"NewFirst\",",
                             "            \"id\":\"11111111-1111-1111-1111-111111111111\",",
@@ -993,6 +1085,15 @@
                             "            \"modified_at\":\"2026-03-10T00:00:00Z\",",
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"type\":\"person\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"first_name\":\"NewFirst\",",
+                            "            \"last_name\":\"NewLast\",",
+                            "            \"type\":\"person\"",
+                            "        },",
+                            "        \"old\":{",
+                            "            \"first_name\":\"OldFirst\",",
+                            "            \"last_name\":\"OldLast\"",
                             "        }",
                             "    }')"
                         ]
@@ -1079,7 +1180,8 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
@@ -1087,6 +1189,7 @@
                             "    modified_by",
                             ")",
                             "VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"date_of_birth\":\"1990-01-01T00:00:00Z\",",
                             "        \"first_name\":\"John\",",
@@ -1114,6 +1217,13 @@
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"pronouns\":\"\",",
                             "            \"type\":\"person\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"date_of_birth\":\"1990-01-01T00:00:00Z\",",
+                            "            \"first_name\":\"John\",",
+                            "            \"last_name\":\"Doe\",",
+                            "            \"pronouns\":\"\",",
+                            "            \"type\":\"person\"",
                             "        }",
                             "    }')"
                         ]
@@ -1192,7 +1302,8 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
@@ -1200,6 +1311,7 @@
                             "    modified_by",
                             ")",
                             "VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"date_of_birth\":\"2000-01-01\",",
                             "        \"first_name\":\"Bob\",",
@@ -1247,7 +1359,8 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
@@ -1255,6 +1368,7 @@
                             "    modified_by",
                             ")",
                             "VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"customer_id\":\"{{uuid:customer_id}}\",",
                             "        \"total\":100.0,",
@@ -1278,6 +1392,11 @@
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"total\":100.0,",
                             "            \"type\":\"order\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"customer_id\":\"{{uuid:customer_id}}\",",
+                            "            \"total\":100.0,",
+                            "            \"type\":\"order\"",
                             "        }",
                             "    }')"
                         ],
@@ -1293,6 +1412,12 @@
                             "            \"modified_at\":\"{{timestamp}}\",",
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"type\":\"person\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"date_of_birth\":\"2000-01-01\",",
+                            "            \"first_name\":\"Bob\",",
+                            "            \"last_name\":\"Smith\",",
+                            "            \"type\":\"person\"",
                             "        }",
                             "    }')"
                         ]
@@ -1388,7 +1513,8 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
@@ -1396,6 +1522,7 @@
                             "    modified_by",
                             ")",
                             "VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"order_id\":\"abc\",",
                             "        \"price\":99.0,",
@@ -1411,7 +1538,8 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
@@ -1419,6 +1547,7 @@
                             "    modified_by",
                             ")",
                             "VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"total\":99.0,",
                             "        \"type\":\"order\"",
@@ -1440,6 +1569,10 @@
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"total\":99.0,",
                             "            \"type\":\"order\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"total\":99.0,",
+                            "            \"type\":\"order\"",
                             "        }",
                             "    }')"
                         ],
@@ -1455,6 +1588,12 @@
                             "            \"price\":99.0,",
                             "            \"product\":\"Widget\",",
                             "            \"type\":\"order_line\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"order_id\":\"abc\",",
+                            "            \"price\":99.0,",
+                            "            \"product\":\"Widget\",",
+                            "            \"type\":\"order_line\"",
                             "        }",
                             "    }')"
                         ]
@@ -1572,13 +1711,15 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
                             "    modified_at,",
                             "    modified_by",
                             ") VALUES (",
+                            "    NULL,",
                             "    '{\"number\":\"555-0001\",\"type\":\"phone_number\"}',",
                             "    '{{uuid:phone1_id}}',",
                             "    '{{uuid}}',",
@@ -1626,13 +1767,15 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
                             "    modified_at,",
                             "    modified_by",
                             ") VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"is_primary\":true,",
                             "        \"source_id\":\"{{uuid:person_id}}\",",
@@ -1674,13 +1817,15 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
                             "    modified_at,",
                             "    modified_by",
                             ") VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"number\":\"555-0002\",",
                             "        \"type\":\"phone_number\"",
@@ -1731,13 +1876,15 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
                             "    modified_at,",
                             "    modified_by",
                             ") VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"is_primary\":false,",
                             "        \"source_id\":\"{{uuid:person_id}}\",",
@@ -1779,13 +1926,15 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
                             "    modified_at,",
                             "    modified_by",
                             ") VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"address\":\"test@example.com\",",
                             "        \"type\":\"email_address\"",
@@ -1836,13 +1985,15 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
                             "    modified_at,",
                             "    modified_by",
                             ") VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"is_primary\":false,",
                             "        \"source_id\":\"{{uuid:person_id}}\",",
@@ -1860,13 +2011,15 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
                             "    modified_at,",
                             "    modified_by",
                             ") VALUES (",
+                            "    NULL,",
                             "    '{",
                             "        \"first_name\":\"Relation\",",
                             "        \"last_name\":\"Test\",",
@@ -1890,6 +2043,11 @@
                             "            \"modified_at\":\"{{timestamp}}\",",
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"type\":\"person\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"first_name\":\"Relation\",",
+                            "            \"last_name\":\"Test\",",
+                            "            \"type\":\"person\"",
                             "        }",
                             "    }')"
                         ],
@@ -1907,6 +2065,14 @@
                             "            \"target_id\":\"{{uuid:phone1_id}}\",",
                             "            \"target_type\":\"phone_number\",",
                             "            \"type\":\"contact\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"is_primary\":true,",
+                            "            \"source_id\":\"{{uuid:person_id}}\",",
+                            "            \"source_type\":\"person\",",
+                            "            \"target_id\":\"{{uuid:phone1_id}}\",",
+                            "            \"target_type\":\"phone_number\",",
+                            "            \"type\":\"contact\"",
                             "        }",
                             "    }')"
                         ],
@@ -1920,6 +2086,10 @@
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"number\":\"555-0001\",",
                             "            \"type\":\"phone_number\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"number\":\"555-0001\",",
+                            "            \"type\":\"phone_number\"",
                             "        }",
                             "    }')"
                         ],
@@ -1937,6 +2107,14 @@
                             "            \"target_id\":\"{{uuid:phone2_id}}\",",
                             "            \"target_type\":\"phone_number\",",
                             "            \"type\":\"contact\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"is_primary\":false,",
+                            "            \"source_id\":\"{{uuid:person_id}}\",",
+                            "            \"source_type\":\"person\",",
+                            "            \"target_id\":\"{{uuid:phone2_id}}\",",
+                            "            \"target_type\":\"phone_number\",",
+                            "            \"type\":\"contact\"",
                             "        }",
                             "    }')"
                         ],
@@ -1950,6 +2128,10 @@
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"number\":\"555-0002\",",
                             "            \"type\":\"phone_number\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"number\":\"555-0002\",",
+                            "            \"type\":\"phone_number\"",
                             "        }",
                             "    }')"
                         ],
@@ -1967,6 +2149,14 @@
                             "            \"target_id\":\"{{uuid:email1_id}}\",",
                             "            \"target_type\":\"email_address\",",
                             "            \"type\":\"contact\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"is_primary\":false,",
+                            "            \"source_id\":\"{{uuid:person_id}}\",",
+                            "            \"source_type\":\"person\",",
+                            "            \"target_id\":\"{{uuid:email1_id}}\",",
+                            "            \"target_type\":\"email_address\",",
+                            "            \"type\":\"contact\"",
                             "        }",
                             "    }')"
                         ],
@@ -1980,6 +2170,10 @@
                             "            \"modified_at\":\"{{timestamp}}\",",
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"type\":\"email_address\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"address\":\"test@example.com\",",
+                            "            \"type\":\"email_address\"",
                             "        }",
                             "    }')"
                         ]
@@ -2023,13 +2217,17 @@
                         ],
                         [
                             "INSERT INTO agreego.change (",
-                            "    changes,",
+                            "    \"old\",",
+                            "    \"new\",",
                             "    entity_id,",
                             "    id,",
                             "    kind,",
                             "    modified_at,",
                             "    modified_by",
                             ") VALUES (",
+                            "    '{",
+                            "        \"archived\":false",
+                            "    }',",
                             "    '{\"archived\":true,\"type\":\"person\"}',",
                             "    'abc-archived',",
                             "    '{{uuid}}',",
@@ -2040,8 +2238,6 @@
                         ],
                         [
                             "SELECT pg_notify('entity', '{",
-                            "    \"changes\":{",
-                            "        \"archived\":true,\"type\":\"person\"},",
                             "        \"complete\":{",
                             "            \"archived\":true,",
                             "            \"first_name\":\"ArchivedFirst\",",
@@ -2050,6 +2246,108 @@
                             "            \"modified_at\":\"{{timestamp}}\",",
                             "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
                             "            \"type\":\"person\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"archived\":true,",
+                            "            \"type\":\"person\"",
+                            "        },",
+                            "        \"old\":{",
+                            "            \"archived\":false",
+                            "        }",
+                            "    }')"
+                        ]
+                    ]
+                }
+            },
+            {
+                "description": "Insert attachment displaying side-by-side array literal and jsonb formatting translations",
+                "action": "merge",
+                "data": {
+                    "type": "attachment",
+                    "flags": [
+                        "urgent",
+                        "reviewed"
+                    ],
+                    "metadata": {
+                        "size": 1024,
+                        "source": "upload"
+                    }
+                },
+                "expect": {
+                    "success": true,
+                    "sql": [
+                        [
+                            "INSERT INTO agreego.\"entity\" (",
+                            "    \"created_at\",",
+                            "    \"created_by\",",
+                            "    \"id\",",
+                            "    \"modified_at\",",
+                            "    \"modified_by\",",
+                            "    \"type\"",
+                            ")",
+                            "VALUES (",
+                            "    '{{timestamp}}',",
+                            "    '00000000-0000-0000-0000-000000000000',",
+                            "    '{{uuid:attachment_id}}',",
+                            "    '{{timestamp}}',",
+                            "    '00000000-0000-0000-0000-000000000000',",
+                            "    'attachment'",
+                            ")"
+                        ],
+                        [
+                            "INSERT INTO agreego.\"attachment\" (",
+                            "    \"flags\",",
+                            "    \"id\",",
+                            "    \"metadata\",",
+                            "    \"type\"",
+                            ")",
+                            "VALUES (",
+                            "    '{\"urgent\",\"reviewed\"}',",
+                            "    '{{uuid:attachment_id}}',",
+                            "    '{\"size\":1024,\"source\":\"upload\"}',",
+                            "    'attachment'",
+                            ")"
+                        ],
+                        [
+                            "INSERT INTO agreego.change (",
+                            "    \"old\",",
+                            "    \"new\",",
+                            "    entity_id,",
+                            "    id,",
+                            "    kind,",
+                            "    modified_at,",
+                            "    modified_by",
+                            ")",
+                            "VALUES (",
+                            "    NULL,",
+                            "    '{",
+                            "        \"flags\":[\"urgent\",\"reviewed\"],",
+                            "        \"metadata\":{\"size\":1024,\"source\":\"upload\"},",
+                            "        \"type\":\"attachment\"",
+                            "    }',",
+                            "    '{{uuid:attachment_id}}',",
+                            "    '{{uuid}}',",
+                            "    'create',",
+                            "    '{{timestamp}}',",
+                            "    '00000000-0000-0000-0000-000000000000'",
+                            ")"
+                        ],
+                        [
+                            "SELECT pg_notify('entity', '{",
+                            "        \"complete\":{",
+                            "            \"created_at\":\"{{timestamp}}\",",
+                            "            \"created_by\":\"00000000-0000-0000-0000-000000000000\",",
+                            "            \"flags\":[\"urgent\",\"reviewed\"],",
+                            "            \"id\":\"{{uuid:attachment_id}}\",",
+                            "            \"metadata\":{\"size\":1024,\"source\":\"upload\"},",
+                            "            \"modified_at\":\"{{timestamp}}\",",
+                            "            \"modified_by\":\"00000000-0000-0000-0000-000000000000\",",
+                            "            \"type\":\"attachment\"",
+                            "        },",
+                            "        \"new\":{",
+                            "            \"flags\":[\"urgent\",\"reviewed\"],",
+                            "            \"metadata\":{\"size\":1024,\"source\":\"upload\"},",
+                            "            \"type\":\"attachment\"",
                             "        }",
                             "    }')"
                         ]

fixtures/stems.json

@@ -1,312 +0,0 @@
-[
-  {
-    "description": "Stem Engine Unit Tests",
-    "database": {
-      "puncs": [],
-      "enums": [],
-      "relations": [
-        {
-          "id": "rel1",
-          "type": "relation",
-          "constraint": "fk_contact_entity",
-          "source_type": "contact",
-          "source_columns": [
-            "entity_id"
-          ],
-          "destination_type": "person",
-          "destination_columns": [
-            "id"
-          ],
-          "prefix": null
-        },
-        {
-          "id": "rel2",
-          "type": "relation",
-          "constraint": "fk_relationship_target",
-          "source_type": "relationship",
-          "source_columns": [
-            "target_id",
-            "target_type"
-          ],
-          "destination_type": "entity",
-          "destination_columns": [
-            "id",
-            "type"
-          ],
-          "prefix": "target"
-        }
-      ],
-      "types": [
-        {
-          "name": "entity",
-          "hierarchy": [
-            "entity"
-          ],
-          "schemas": [
-            {
-              "$id": "entity",
-              "type": "object",
-              "properties": {}
-            }
-          ]
-        },
-        {
-          "name": "person",
-          "hierarchy": [
-            "person",
-            "entity"
-          ],
-          "schemas": [
-            {
-              "$id": "person",
-              "$ref": "entity",
-              "properties": {}
-            }
-          ]
-        },
-        {
-          "name": "email_address",
-          "hierarchy": [
-            "email_address",
-            "entity"
-          ],
-          "schemas": [
-            {
-              "$id": "email_address",
-              "$ref": "entity",
-              "properties": {}
-            }
-          ]
-        },
-        {
-          "name": "phone_number",
-          "hierarchy": [
-            "phone_number",
-            "entity"
-          ],
-          "schemas": [
-            {
-              "$id": "phone_number",
-              "$ref": "entity",
-              "properties": {}
-            }
-          ]
-        },
-        {
-          "name": "relationship",
-          "relationship": true,
-          "hierarchy": [
-            "relationship",
-            "entity"
-          ],
-          "schemas": [
-            {
-              "$id": "relationship",
-              "$ref": "entity",
-              "properties": {}
-            }
-          ]
-        },
-        {
-          "name": "contact",
-          "relationship": true,
-          "hierarchy": [
-            "contact",
-            "relationship",
-            "entity"
-          ],
-          "schemas": [
-            {
-              "$id": "contact",
-              "$ref": "relationship",
-              "properties": {
-                "target": {
-                  "oneOf": [
-                    {
-                      "$ref": "phone_number"
-                    },
-                    {
-                      "$ref": "email_address"
-                    }
-                  ]
-                }
-              }
-            }
-          ]
-        },
-        {
-          "name": "save_person",
-          "schemas": [
-            {
-              "$id": "save_person.response",
-              "$ref": "person",
-              "properties": {
-                "contacts": {
-                  "type": "array",
-                  "items": {
-                    "$ref": "contact"
-                  }
-                }
-              }
-            }
-          ]
-        }
-      ]
-    },
-    "tests": [
-      {
-        "description": "correctly squashes deep oneOf refs through array paths",
-        "action": "compile",
-        "expect": {
-          "success": true,
-          "stems": {
-            "contact": {
-              "": {
-                "schema": {
-                  "$id": "contact",
-                  "$ref": "relationship",
-                  "properties": {
-                    "target": {
-                      "oneOf": [
-                        {
-                          "$ref": "phone_number"
-                        },
-                        {
-                          "$ref": "email_address"
-                        }
-                      ]
-                    }
-                  }
-                },
-                "type": "contact"
-              },
-              "target#(type==\"email_address\")": {
-                "relation": "target_id",
-                "schema": {
-                  "$id": "email_address",
-                  "$ref": "entity",
-                  "properties": {}
-                },
-                "type": "email_address"
-              },
-              "target#(type==\"phone_number\")": {
-                "relation": "target_id",
-                "schema": {
-                  "$id": "phone_number",
-                  "$ref": "entity",
-                  "properties": {}
-                },
-                "type": "phone_number"
-              }
-            },
-            "email_address": {
-              "": {
-                "schema": {
-                  "$id": "email_address",
-                  "$ref": "entity",
-                  "properties": {}
-                },
-                "type": "email_address"
-              }
-            },
-            "entity": {
-              "": {
-                "schema": {
-                  "$id": "entity",
-                  "properties": {},
-                  "type": "object"
-                },
-                "type": "entity"
-              }
-            },
-            "person": {
-              "": {
-                "schema": {
-                  "$id": "person",
-                  "$ref": "entity",
-                  "properties": {}
-                },
-                "type": "person"
-              }
-            },
-            "phone_number": {
-              "": {
-                "schema": {
-                  "$id": "phone_number",
-                  "$ref": "entity",
-                  "properties": {}
-                },
-                "type": "phone_number"
-              }
-            },
-            "relationship": {
-              "": {
-                "schema": {
-                  "$id": "relationship",
-                  "$ref": "entity",
-                  "properties": {}
-                },
-                "type": "relationship"
-              }
-            },
-            "save_person.response": {
-              "": {
-                "schema": {
-                  "$id": "save_person.response",
-                  "$ref": "person",
-                  "properties": {
-                    "contacts": {
-                      "items": {
-                        "$ref": "contact"
-                      },
-                      "type": "array"
-                    }
-                  }
-                },
-                "type": "person"
-              },
-              "contacts.#": {
-                "relation": "contacts_id",
-                "schema": {
-                  "$id": "contact",
-                  "$ref": "relationship",
-                  "properties": {
-                    "target": {
-                      "oneOf": [
-                        {
-                          "$ref": "phone_number"
-                        },
-                        {
-                          "$ref": "email_address"
-                        }
-                      ]
-                    }
-                  }
-                },
-                "type": "contact"
-              },
-              "contacts.#.target#(type==\"email_address\")": {
-                "relation": "target_id",
-                "schema": {
-                  "$id": "email_address",
-                  "$ref": "entity",
-                  "properties": {}
-                },
-                "type": "email_address"
-              },
-              "contacts.#.target#(type==\"phone_number\")": {
-                "relation": "target_id",
-                "schema": {
-                  "$id": "phone_number",
-                  "$ref": "entity",
-                  "properties": {}
-                },
-                "type": "phone_number"
-              }
-            }
-          }
-        }
-      }
-    ]
-  }
-]

src/bin/ast_explore.rs

@@ -1,17 +0,0 @@
-use sqlparser::dialect::PostgreSqlDialect;
-use sqlparser::parser::Parser;
-use std::env;
-
-fn main() {
-    let sql = "SELECT t1_obj_t1_addresses_t1_target_t2.archived, t1.id FROM person t1 JOIN address t1_obj_t1_addresses ON true";
-    let dialect = PostgreSqlDialect {};
-    
-    match Parser::parse_sql(&dialect, sql) {
-        Ok(ast) => {
-            println!("{:#?}", ast);
-        }
-        Err(e) => {
-            println!("Error: {:?}", e);
-        }
-    }
-}

src/database/executors/pgrx.rs

@@ -24,6 +24,7 @@ impl DatabaseExecutor for SpiExecutor {
       }
     }
 
+    pgrx::PgTryBuilder::new(|| {
       Spi::connect(|client| {
         pgrx::notice!("JSPG_SQL: {}", sql);
         match client.select(sql, Some(args_with_oid.len() as i64), &args_with_oid) {
@@ -39,6 +40,12 @@ impl DatabaseExecutor for SpiExecutor {
           Err(e) => Err(format!("SPI Query Fetch Failure: {}", e)),
         }
       })
+    })
+    .catch_others(|cause| {
+      pgrx::warning!("JSPG Caught Native Postgres Error: {:?}", cause);
+      Err(format!("{:?}", cause))
+    })
+    .execute()
   }
 
   fn execute(&self, sql: &str, args: Option<&[Value]>) -> Result<(), String> {
@@ -53,6 +60,7 @@ impl DatabaseExecutor for SpiExecutor {
       }
     }
 
+    pgrx::PgTryBuilder::new(|| {
       Spi::connect_mut(|client| {
         pgrx::notice!("JSPG_SQL: {}", sql);
         match client.update(sql, Some(args_with_oid.len() as i64), &args_with_oid) {
@@ -60,6 +68,12 @@ impl DatabaseExecutor for SpiExecutor {
           Err(e) => Err(format!("SPI Execution Failure: {}", e)),
         }
       })
+    })
+    .catch_others(|cause| {
+      pgrx::warning!("JSPG Caught Native Postgres Error: {:?}", cause);
+      Err(format!("{:?}", cause))
+    })
+    .execute()
   }
 
   fn auth_user_id(&self) -> Result<String, String> {

src/database/mod.rs

@@ -18,24 +18,19 @@ use executors::pgrx::SpiExecutor;
 #[cfg(test)]
 use executors::mock::MockExecutor;
 
-pub mod stem;
 use punc::Punc;
 use relation::Relation;
 use schema::Schema;
 use serde_json::Value;
 use std::collections::{HashMap, HashSet};
-use std::sync::Arc;
-use stem::Stem;
 use r#type::Type;
 
 pub struct Database {
   pub enums: HashMap<String, Enum>,
   pub types: HashMap<String, Type>,
   pub puncs: HashMap<String, Punc>,
-  pub relations: HashMap<String, Relation>,
+  pub relations: Vec<Relation>,
   pub schemas: HashMap<String, Schema>,
-  // Map of Schema ID -> { Entity Type -> Target Subschema Arc }
-  pub stems: HashMap<String, HashMap<String, Arc<Stem>>>,
   pub descendants: HashMap<String, Vec<String>>,
   pub depths: HashMap<String, usize>,
   pub executor: Box<dyn DatabaseExecutor + Send + Sync>,
@@ -46,10 +41,9 @@ impl Database {
     let mut db = Self {
       enums: HashMap::new(),
       types: HashMap::new(),
-      relations: HashMap::new(),
+      relations: Vec::new(),
       puncs: HashMap::new(),
       schemas: HashMap::new(),
-      stems: HashMap::new(),
       descendants: HashMap::new(),
       depths: HashMap::new(),
       #[cfg(not(test))]
@@ -78,7 +72,11 @@ impl Database {
       for item in arr {
         match serde_json::from_value::<Relation>(item.clone()) {
           Ok(def) => {
-            db.relations.insert(def.constraint.clone(), def);
+            if db.types.contains_key(&def.source_type)
+              && db.types.contains_key(&def.destination_type)
+            {
+              db.relations.push(def);
+            }
           }
           Err(e) => println!("DATABASE RELATION PARSE FAILED: {:?}", e),
         }
@@ -137,13 +135,11 @@ impl Database {
     self.executor.timestamp()
   }
 
-  /// Organizes the graph of the database, compiling regex, format functions, and caching relationships.
   pub fn compile(&mut self) -> Result<(), crate::drop::Drop> {
     self.collect_schemas();
     self.collect_depths();
     self.collect_descendants();
     self.compile_schemas();
-    self.collect_stems()?;
 
     Ok(())
   }
@@ -226,6 +222,79 @@ impl Database {
     self.descendants = descendants;
   }
 
+  pub fn get_relation(
+    &self,
+    parent_type: &str,
+    child_type: &str,
+    prop_name: &str,
+    relative_keys: Option<&Vec<String>>,
+  ) -> Option<(&Relation, bool)> {
+    if parent_type == "entity" && child_type == "entity" {
+      return None; // Ignore entity <-> entity generic fallbacks, they aren't useful edges
+    }
+
+    let p_def = self.types.get(parent_type)?;
+    let c_def = self.types.get(child_type)?;
+
+    let mut matching_rels = Vec::new();
+    let mut directions = Vec::new();
+
+    for rel in &self.relations {
+      let is_forward = p_def.hierarchy.contains(&rel.source_type)
+        && c_def.hierarchy.contains(&rel.destination_type);
+      let is_reverse = p_def.hierarchy.contains(&rel.destination_type)
+        && c_def.hierarchy.contains(&rel.source_type);
+
+      if is_forward {
+        matching_rels.push(rel);
+        directions.push(true);
+      } else if is_reverse {
+        matching_rels.push(rel);
+        directions.push(false);
+      }
+    }
+
+    if matching_rels.is_empty() {
+      return None;
+    }
+
+    if matching_rels.len() == 1 {
+      return Some((matching_rels[0], directions[0]));
+    }
+
+    let mut chosen_idx = 0;
+    let mut resolved = false;
+
+    // Reduce ambiguity with prefix
+    for (i, rel) in matching_rels.iter().enumerate() {
+      if let Some(prefix) = &rel.prefix {
+        if prefix == prop_name {
+          chosen_idx = i;
+          resolved = true;
+          break;
+        }
+      }
+    }
+
+    // Reduce ambiguity by checking if relative payload OMITS the prefix (M:M heuristic)
+    if !resolved && relative_keys.is_some() {
+      let keys = relative_keys.unwrap();
+      let mut missing_prefix_ids = Vec::new();
+      for (i, rel) in matching_rels.iter().enumerate() {
+        if let Some(prefix) = &rel.prefix {
+          if !keys.contains(prefix) {
+            missing_prefix_ids.push(i);
+          }
+        }
+      }
+      if missing_prefix_ids.len() == 1 {
+        chosen_idx = missing_prefix_ids[0];
+      }
+    }
+
+    Some((matching_rels[chosen_idx], directions[chosen_idx]))
+  }
+
   fn collect_descendants_recursively(
     target: &str,
     direct_refs: &HashMap<String, Vec<String>>,
@@ -249,239 +318,4 @@ impl Database {
       }
     }
   }
-
-  fn collect_stems(&mut self) -> Result<(), crate::drop::Drop> {
-    let mut db_stems: HashMap<String, HashMap<String, Arc<Stem>>> = HashMap::new();
-    let mut errors: Vec<crate::drop::Error> = Vec::new();
-
-    let schema_ids: Vec<String> = self.schemas.keys().cloned().collect();
-    for schema_id in schema_ids {
-      if let Some(schema) = self.schemas.get(&schema_id) {
-        let mut inner_map = HashMap::new();
-        Self::discover_stems(
-          self,
-          &schema_id,
-          schema,
-          String::from(""),
-          None,
-          None,
-          false,
-          &mut inner_map,
-          Vec::new(),
-          &mut errors,
-        );
-        if !inner_map.is_empty() {
-          db_stems.insert(schema_id, inner_map);
-        }
-      }
-    }
-
-    self.stems = db_stems;
-
-    if !errors.is_empty() {
-      return Err(crate::drop::Drop::with_errors(errors));
-    }
-    Ok(())
-  }
-
-  fn discover_stems(
-    db: &Database,
-    root_schema_id: &str,
-    schema: &Schema,
-    current_path: String,
-    parent_type: Option<String>,
-    property_name: Option<String>,
-    is_polymorphic: bool,
-    inner_map: &mut HashMap<String, Arc<Stem>>,
-    seen_entities: Vec<String>,
-    errors: &mut Vec<crate::drop::Error>,
-  ) {
-    let mut is_entity = false;
-    let mut entity_type = String::new();
-
-    // First check if the Schema's $id is a native Database Type
-    if let Some(ref id) = schema.obj.id {
-      let parts: Vec<&str> = id.split('.').collect();
-      if let Some(last_seg) = parts.last() {
-        if db.types.contains_key(*last_seg) {
-          is_entity = true;
-          entity_type = last_seg.to_string();
-        }
-      }
-    }
-
-    // If not found via $id, check the $ref pointer
-    // This allows ad-hoc schemas (like `save_person.response`) to successfully adopt the Type of what they $ref
-    if !is_entity {
-      if let Some(ref r) = schema.obj.r#ref {
-        let parts: Vec<&str> = r.split('.').collect();
-        if let Some(last_seg) = parts.last() {
-          if db.types.contains_key(*last_seg) {
-            is_entity = true;
-            entity_type = last_seg.to_string();
-          }
-        }
-      }
-    }
-
-    if is_entity {
-      if seen_entities.contains(&entity_type) {
-        return; // Break cyclical schemas!
-      }
-    }
-
-    let mut relation_col = None;
-    if is_entity {
-      if let (Some(pt), Some(prop)) = (&parent_type, &property_name) {
-        let expected_col = format!("{}_id", prop);
-        let mut found = false;
-        for rel in db.relations.values() {
-          if (rel.source_type == *pt && rel.destination_type == entity_type)
-            || (rel.source_type == entity_type && rel.destination_type == *pt)
-          {
-            if rel.source_columns.contains(&expected_col) {
-              relation_col = Some(expected_col.clone());
-              found = true;
-              break;
-            }
-          }
-        }
-        if !found {
-          relation_col = Some(expected_col);
-        }
-      }
-
-      let mut final_path = current_path.clone();
-      if is_polymorphic && !final_path.is_empty() && !final_path.ends_with(&entity_type) {
-        if final_path.ends_with(".#") {
-          final_path = format!("{}(type==\"{}\")", final_path, entity_type);
-        } else {
-          final_path = format!("{}#(type==\"{}\")", final_path, entity_type);
-        }
-      }
-
-      let stem = Stem {
-        r#type: entity_type.clone(),
-        relation: relation_col,
-        schema: Arc::new(schema.clone()),
-      };
-      inner_map.insert(final_path, Arc::new(stem));
-    }
-
-    let next_parent = if is_entity {
-      Some(entity_type.clone())
-    } else {
-      parent_type.clone()
-    };
-
-    let pass_seen = if is_entity {
-      let mut ns = seen_entities.clone();
-      ns.push(entity_type.clone());
-      ns
-    } else {
-      seen_entities.clone()
-    };
-
-    // Properties branch
-    if let Some(props) = &schema.obj.properties {
-      for (k, v) in props {
-        // Standard Property Pathing
-        let next_path = if current_path.is_empty() {
-          k.clone()
-        } else {
-          format!("{}.{}", current_path, k)
-        };
-
-        Self::discover_stems(
-          db,
-          root_schema_id,
-          v,
-          next_path,
-          next_parent.clone(),
-          Some(k.clone()),
-          false,
-          inner_map,
-          pass_seen.clone(),
-          errors,
-        );
-      }
-    }
-
-    // Array Item branch
-    if let Some(items) = &schema.obj.items {
-      let next_path = if current_path.is_empty() {
-        String::from("#")
-      } else {
-        format!("{}.#", current_path)
-      };
-
-      Self::discover_stems(
-        db,
-        root_schema_id,
-        items,
-        next_path,
-        next_parent.clone(),
-        property_name.clone(),
-        false,
-        inner_map,
-        pass_seen.clone(),
-        errors,
-      );
-    }
-
-    // Follow external reference if we didn't just crawl local properties
-    if schema.obj.properties.is_none() && schema.obj.items.is_none() && schema.obj.one_of.is_none()
-    {
-      if let Some(ref r) = schema.obj.r#ref {
-        if let Some(target_schema) = db.schemas.get(r) {
-          Self::discover_stems(
-            db,
-            root_schema_id,
-            target_schema,
-            current_path.clone(),
-            next_parent.clone(),
-            property_name.clone(),
-            is_polymorphic,
-            inner_map,
-            seen_entities.clone(),
-            errors,
-          );
-        }
-      }
-    }
-
-    // Polymorphism branch
-    if let Some(arr) = &schema.obj.one_of {
-      for v in arr {
-        Self::discover_stems(
-          db,
-          root_schema_id,
-          v.as_ref(),
-          current_path.clone(),
-          next_parent.clone(),
-          property_name.clone(),
-          true,
-          inner_map,
-          pass_seen.clone(),
-          errors,
-        );
-      }
-    }
-    if let Some(arr) = &schema.obj.all_of {
-      for v in arr {
-        Self::discover_stems(
-          db,
-          root_schema_id,
-          v.as_ref(),
-          current_path.clone(),
-          next_parent.clone(),
-          property_name.clone(),
-          is_polymorphic,
-          inner_map,
-          pass_seen.clone(),
-          errors,
-        );
-      }
-    }
-  }
 }

src/database/stem.rs

@@ -1,12 +0,0 @@
-use crate::database::schema::Schema;
-use serde::{Deserialize, Serialize};
-use std::sync::Arc;
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct Stem {
-  pub r#type: String,
-  #[serde(skip_serializing_if = "Option::is_none")]
-  pub relation: Option<String>,
-
-  pub schema: Arc<Schema>,
-}

src/database/type.rs

@@ -15,6 +15,8 @@ pub struct Type {
   #[serde(default)]
   pub historical: bool,
   #[serde(default)]
+  pub notify: bool,
+  #[serde(default)]
   pub sensitive: bool,
   #[serde(default)]
   pub ownable: bool,

src/drop.rs

@@ -70,7 +70,7 @@ pub struct ErrorDetails {
   #[serde(skip_serializing_if = "Option::is_none")]
   pub cause: Option<String>,
   #[serde(skip_serializing_if = "Option::is_none")]
-  pub context: Option<Vec<String>>,
+  pub context: Option<Value>,
   #[serde(skip_serializing_if = "Option::is_none")]
   pub schema: Option<String>,
 }

src/lib.rs

@@ -77,7 +77,7 @@ pub fn jspg_merge(data: JsonB) -> JsonB {
 }
 
 #[cfg_attr(not(test), pg_extern)]
-pub fn jspg_query(schema_id: &str, stem: Option<&str>, filters: Option<JsonB>) -> JsonB {
+pub fn jspg_query(schema_id: &str, filters: Option<JsonB>) -> JsonB {
   let engine_opt = {
     let lock = GLOBAL_JSPG.read().unwrap();
     lock.clone()
@@ -87,7 +87,7 @@ pub fn jspg_query(schema_id: &str, stem: Option<&str>, filters: Option<JsonB>) -
     Some(engine) => {
       let drop = engine
         .queryer
-        .query(schema_id, stem, filters.as_ref().map(|f| &f.0));
+        .query(schema_id, filters.as_ref().map(|f| &f.0));
       JsonB(serde_json::to_value(drop).unwrap())
     }
     None => jspg_failure(),
@@ -131,24 +131,6 @@ pub fn jspg_schemas() -> JsonB {
   }
 }
 
-#[cfg_attr(not(test), pg_extern)]
-pub fn jspg_stems() -> JsonB {
-  let engine_opt = {
-    let lock = GLOBAL_JSPG.read().unwrap();
-    lock.clone()
-  };
-
-  match engine_opt {
-    Some(engine) => {
-      let stems_json = serde_json::to_value(&engine.database.stems)
-        .unwrap_or(serde_json::Value::Object(serde_json::Map::new()));
-      let drop = crate::drop::Drop::success_with_val(stems_json);
-      JsonB(serde_json::to_value(drop).unwrap())
-    }
-    None => jspg_failure(),
-  }
-}
-
 #[cfg_attr(not(test), pg_extern(strict))]
 pub fn jspg_teardown() -> JsonB {
   let mut lock = GLOBAL_JSPG.write().unwrap();

src/merger/mod.rs

@@ -3,6 +3,7 @@
 
 pub mod cache;
 
+use crate::database::r#type::Type;
 use crate::database::Database;
 use serde_json::Value;
 use std::sync::Arc;
@@ -21,15 +22,12 @@ impl Merger {
   }
 
   pub fn merge(&self, data: Value) -> crate::drop::Drop {
-    let mut val_resolved = Value::Null;
     let mut notifications_queue = Vec::new();
 
-    let result = self.merge_internal(data, &mut notifications_queue);
+    let result = self.merge_internal(data.clone(), &mut notifications_queue);
 
-    match result {
-      Ok(val) => {
-        val_resolved = val;
-      }
+    let val_resolved = match result {
+      Ok(val) => val,
       Err(msg) => {
         return crate::drop::Drop::with_errors(vec![crate::drop::Error {
           code: "MERGE_FAILED".to_string(),
@@ -37,7 +35,7 @@ impl Merger {
           details: crate::drop::ErrorDetails {
             path: "".to_string(),
             cause: None,
-            context: None,
+            context: Some(data),
             schema: None,
           },
         }]);
@@ -88,7 +86,11 @@ impl Merger {
     crate::drop::Drop::success_with_val(stripped_val)
   }
 
-  pub(crate) fn merge_internal(&self, data: Value, notifications: &mut Vec<String>) -> Result<Value, String> {
+  pub(crate) fn merge_internal(
+    &self,
+    data: Value,
+    notifications: &mut Vec<String>,
+  ) -> Result<Value, String> {
     match data {
       Value::Array(items) => self.merge_array(items, notifications),
       Value::Object(map) => self.merge_object(map, notifications),
@@ -96,7 +98,11 @@ impl Merger {
     }
   }
 
-  fn merge_array(&self, items: Vec<Value>, notifications: &mut Vec<String>) -> Result<Value, String> {
+  fn merge_array(
+    &self,
+    items: Vec<Value>,
+    notifications: &mut Vec<String>,
+  ) -> Result<Value, String> {
     let mut resolved_items = Vec::new();
     for item in items {
       let resolved = self.merge_internal(item, notifications)?;
@@ -105,7 +111,11 @@ impl Merger {
     Ok(Value::Array(resolved_items))
   }
 
-  fn merge_object(&self, obj: serde_json::Map<String, Value>, notifications: &mut Vec<String>) -> Result<Value, String> {
+  fn merge_object(
+    &self,
+    obj: serde_json::Map<String, Value>,
+    notifications: &mut Vec<String>,
+  ) -> Result<Value, String> {
     let queue_start = notifications.len();
 
     let type_name = match obj.get("type").and_then(|v| v.as_str()) {
@@ -164,10 +174,23 @@ impl Merger {
         _ => continue,
       };
 
-      let relative_relation = self.get_entity_relation(type_def, &relative, &relation_name)?;
+      // Attempt to extract relative object type name
+      let relative_type_name = match relative.get("type").and_then(|v| v.as_str()) {
+        Some(t) => t.to_string(),
+        None => continue,
+      };
 
-      if let Some(relation) = relative_relation {
-        let parent_is_source = type_def.hierarchy.contains(&relation.source_type);
+      let relative_keys: Vec<String> = relative.keys().cloned().collect();
+
+      // Call central Database O(1) graph logic
+      let relative_relation = self.db.get_relation(
+        &type_def.name,
+        &relative_type_name,
+        &relation_name,
+        Some(&relative_keys),
+      );
+
+      if let Some((relation, parent_is_source)) = relative_relation {
 
         if parent_is_source {
           // Parent holds FK to Child. Child MUST be generated FIRST.
@@ -177,11 +200,16 @@ impl Merger {
             }
           }
 
-          let merged_relative = match self.merge_internal(Value::Object(relative), notifications)? {
+          let mut merged_relative = match self.merge_internal(Value::Object(relative), notifications)? {
             Value::Object(m) => m,
             _ => continue,
           };
 
+          merged_relative.insert(
+            "type".to_string(),
+            Value::String(relative_type_name),
+          );
+
           Self::apply_entity_relation(
             &mut entity_fields,
             &relation.source_columns,
@@ -253,9 +281,23 @@ impl Merger {
         _ => continue,
       };
 
-      let relative_relation = self.get_entity_relation(type_def, first_relative, &relation_name)?;
+      // Attempt to extract relative object type name
+      let relative_type_name = match first_relative.get("type").and_then(|v| v.as_str()) {
+        Some(t) => t,
+        None => continue,
+      };
 
-      if let Some(relation) = relative_relation {
+      let relative_keys: Vec<String> = first_relative.keys().cloned().collect();
+
+      // Call central Database O(1) graph logic
+      let relative_relation = self.db.get_relation(
+        &type_def.name,
+        relative_type_name,
+        &relation_name,
+        Some(&relative_keys),
+      );
+
+      if let Some((relation, _)) = relative_relation {
         let mut relative_responses = Vec::new();
         for relative_item_val in relative_arr {
           if let Value::Object(mut relative_item) = relative_item_val {
@@ -272,7 +314,8 @@ impl Merger {
               &entity_fields,
             );
 
-            let merged_relative = match self.merge_internal(Value::Object(relative_item), notifications)? {
+            let merged_relative =
+              match self.merge_internal(Value::Object(relative_item), notifications)? {
                 Value::Object(m) => m,
                 _ => continue,
               };
@@ -284,8 +327,9 @@ impl Merger {
       }
     }
 
-    // 7. Perform change tracking
+    // 7. Perform change tracking dynamically suppressing noise based on type bounds!
     let notify_sql = self.merge_entity_change(
+      type_def,
       &entity_fields,
       entity_fetched.as_ref(),
       entity_change_kind.as_deref(),
@@ -583,11 +627,7 @@ impl Merger {
         for key in &sorted_keys {
           columns.push(format!("\"{}\"", key));
           let val = entity_pairs.get(key).unwrap();
-          if val.as_str() == Some("") {
-            values.push("NULL".to_string());
-          } else {
-            values.push(Self::quote_literal(val));
-          }
+          values.push(Self::format_sql_value(val, key, entity_type));
         }
 
         if columns.is_empty() {
@@ -621,7 +661,11 @@ impl Merger {
           if val.as_str() == Some("") {
             set_clauses.push(format!("\"{}\" = NULL", key));
           } else {
-            set_clauses.push(format!("\"{}\" = {}", key, Self::quote_literal(val)));
+            set_clauses.push(format!(
+              "\"{}\" = {}",
+              key,
+              Self::format_sql_value(val, key, entity_type)
+            ));
           }
         }
 
@@ -643,6 +687,7 @@ impl Merger {
 
   fn merge_entity_change(
     &self,
+    type_obj: &Type,
     entity_fields: &serde_json::Map<String, Value>,
     entity_fetched: Option<&serde_json::Map<String, Value>>,
     entity_change_kind: Option<&str>,
@@ -657,7 +702,8 @@ impl Merger {
     let id_str = entity_fields.get("id").unwrap();
     let type_name = entity_fields.get("type").unwrap();
 
-    let mut changes = serde_json::Map::new();
+    let mut old_vals = serde_json::Map::new();
+    let mut new_vals = serde_json::Map::new();
     let is_update = change_kind == "update" || change_kind == "delete";
 
     if !is_update {
@@ -670,7 +716,7 @@ impl Merger {
       ];
       for (k, v) in entity_fields {
         if !system_keys.contains(k) {
-          changes.insert(k.clone(), v.clone());
+          new_vals.insert(k.clone(), v.clone());
         }
       }
     } else {
@@ -687,12 +733,13 @@ impl Merger {
           if let Some(fetched) = entity_fetched {
             let old_val = fetched.get(k).unwrap_or(&Value::Null);
             if v != old_val {
-              changes.insert(k.clone(), v.clone());
+              new_vals.insert(k.clone(), v.clone());
+              old_vals.insert(k.clone(), old_val.clone());
             }
           }
         }
       }
-      changes.insert("type".to_string(), type_name.clone());
+      new_vals.insert("type".to_string(), type_name.clone());
     }
 
     let mut complete = entity_fields.clone();
@@ -706,15 +753,27 @@ impl Merger {
       }
     }
 
+    let new_val_obj = Value::Object(new_vals);
+    let old_val_obj = if old_vals.is_empty() {
+      Value::Null
+    } else {
+      Value::Object(old_vals)
+    };
+
     let mut notification = serde_json::Map::new();
     notification.insert("complete".to_string(), Value::Object(complete));
-    if is_update {
-      notification.insert("changes".to_string(), Value::Object(changes.clone()));
+    notification.insert("new".to_string(), new_val_obj.clone());
+    
+    if old_val_obj != Value::Null {
+        notification.insert("old".to_string(), old_val_obj.clone());
     }
 
+    let mut notify_sql = None;
+    if type_obj.historical {
       let change_sql = format!(
-      "INSERT INTO agreego.change (changes, entity_id, id, kind, modified_at, modified_by) VALUES ({}, {}, {}, {}, {}, {})",
-      Self::quote_literal(&Value::Object(changes)),
+        "INSERT INTO agreego.change (\"old\", \"new\", entity_id, id, kind, modified_at, modified_by) VALUES ({}, {}, {}, {}, {}, {}, {})",
+        Self::quote_literal(&old_val_obj),
+        Self::quote_literal(&new_val_obj),
         Self::quote_literal(id_str),
         Self::quote_literal(&Value::String(uuid::Uuid::new_v4().to_string())),
         Self::quote_literal(&Value::String(change_kind.to_string())),
@@ -722,17 +781,20 @@ impl Merger {
         Self::quote_literal(&Value::String(user_id.to_string()))
       );
 
-    let notify_sql = format!(
-      "SELECT pg_notify('entity', {})",
-      Self::quote_literal(&Value::String(Value::Object(notification).to_string()))
-    );
-
       self
         .db
         .execute(&change_sql, None)
         .map_err(|e| format!("Executor Error in change: {:?}", e))?;
+    }
 
-    Ok(Some(notify_sql))
+    if type_obj.notify {
+      notify_sql = Some(format!(
+        "SELECT pg_notify('entity', {})",
+        Self::quote_literal(&Value::String(Value::Object(notification).to_string()))
+      ));
+    }
+
+    Ok(notify_sql)
   }
 
   fn compare_entities(
@@ -766,101 +828,7 @@ impl Merger {
     changes
   }
 
-  fn reduce_entity_relations(
-    &self,
-    mut matching_relations: Vec<crate::database::relation::Relation>,
-    relative: &serde_json::Map<String, Value>,
-    relation_name: &str,
-  ) -> Result<Option<crate::database::relation::Relation>, String> {
-    if matching_relations.is_empty() {
-      return Ok(None);
-    }
-    if matching_relations.len() == 1 {
-      return Ok(Some(matching_relations.pop().unwrap()));
-    }
-
-    let exact_match: Vec<_> = matching_relations
-      .iter()
-      .filter(|r| r.prefix.as_deref() == Some(relation_name))
-      .cloned()
-      .collect();
-    if exact_match.len() == 1 {
-      return Ok(Some(exact_match.into_iter().next().unwrap()));
-    }
-
-    matching_relations.retain(|r| {
-      if let Some(prefix) = &r.prefix {
-        !relative.contains_key(prefix)
-      } else {
-        true
-      }
-    });
-
-    if matching_relations.len() == 1 {
-      Ok(Some(matching_relations.pop().unwrap()))
-    } else {
-      let constraints: Vec<_> = matching_relations
-        .iter()
-        .map(|r| r.constraint.clone())
-        .collect();
-      Err(format!(
-        "AMBIGUOUS_TYPE_RELATIONS: Could not reduce ambiguous type relations: {}",
-        constraints.join(", ")
-      ))
-    }
-  }
-
-  fn get_entity_relation(
-    &self,
-    entity_type: &crate::database::r#type::Type,
-    relative: &serde_json::Map<String, Value>,
-    relation_name: &str,
-  ) -> Result<Option<crate::database::relation::Relation>, String> {
-    let relative_type_name = match relative.get("type").and_then(|v| v.as_str()) {
-      Some(t) => t,
-      None => return Ok(None),
-    };
-
-    let relative_type = match self.db.types.get(relative_type_name) {
-      Some(t) => t,
-      None => return Ok(None),
-    };
-
-    let mut relative_relations: Vec<crate::database::relation::Relation> = Vec::new();
-
-    for r in self.db.relations.values() {
-      if r.source_type != "entity" && r.destination_type != "entity" {
-        let condition1 = relative_type.hierarchy.contains(&r.source_type)
-          && entity_type.hierarchy.contains(&r.destination_type);
-        let condition2 = entity_type.hierarchy.contains(&r.source_type)
-          && relative_type.hierarchy.contains(&r.destination_type);
-
-        if condition1 || condition2 {
-          relative_relations.push(r.clone());
-        }
-      }
-    }
-
-    let mut relative_relation =
-      self.reduce_entity_relations(relative_relations, relative, relation_name)?;
-
-    if relative_relation.is_none() {
-      let mut poly_relations: Vec<crate::database::relation::Relation> = Vec::new();
-      for r in self.db.relations.values() {
-        if r.destination_type == "entity" {
-          let condition1 = relative_type.hierarchy.contains(&r.source_type);
-          let condition2 = entity_type.hierarchy.contains(&r.source_type);
-
-          if condition1 || condition2 {
-            poly_relations.push(r.clone());
-          }
-        }
-      }
-      relative_relation = self.reduce_entity_relations(poly_relations, relative, relation_name)?;
-    }
-
-    Ok(relative_relation)
-  }
+  // Helper Functions
 
   fn apply_entity_relation(
     source_entity: &mut serde_json::Map<String, Value>,
@@ -878,6 +846,34 @@ impl Merger {
     }
   }
 
+  fn format_sql_value(val: &Value, key: &str, entity_type: &Type) -> String {
+    if val.as_str() == Some("") {
+      return "NULL".to_string();
+    }
+
+    let mut is_pg_array = false;
+    if let Some(field_types_map) = entity_type.field_types.as_ref().and_then(|v| v.as_object()) {
+      if let Some(t_val) = field_types_map.get(key) {
+        if let Some(t_str) = t_val.as_str() {
+          if t_str.starts_with('_') {
+            is_pg_array = true;
+          }
+        }
+      }
+    }
+
+    if is_pg_array && val.is_array() {
+      let mut s = val.to_string();
+      if s.starts_with('[') && s.ends_with(']') {
+        s.replace_range(0..1, "{");
+        s.replace_range(s.len() - 1..s.len(), "}");
+      }
+      Self::quote_literal(&Value::String(s))
+    } else {
+      Self::quote_literal(val)
+    }
+  }
+
   fn quote_literal(val: &Value) -> String {
     match val {
       Value::Null => "NULL".to_string(),

src/queryer/mod.rs

@@ -21,7 +21,6 @@ impl Queryer {
   pub fn query(
     &self,
     schema_id: &str,
-    stem_opt: Option<&str>,
     filters: Option<&serde_json::Value>,
   ) -> crate::drop::Drop {
     let filters_map = filters.and_then(|f| f.as_object());
@@ -36,18 +35,17 @@ impl Queryer {
           details: crate::drop::ErrorDetails {
             path: "".to_string(), // filters apply to the root query
             cause: Some(msg),
-            context: filters.map(|f| vec![f.to_string()]),
+            context: filters.cloned(),
             schema: Some(schema_id.to_string()),
           },
         }]);
       }
     };
 
-    let stem_key = stem_opt.unwrap_or("/");
-    let cache_key = format!("{}(Stem:{}):{}", schema_id, stem_key, filter_keys.join(","));
+    let cache_key = format!("{}:{}", schema_id, filter_keys.join(","));
 
     // 2. Fetch from cache or compile
-    let sql = match self.get_or_compile_sql(&cache_key, schema_id, stem_opt, &filter_keys) {
+    let sql = match self.get_or_compile_sql(&cache_key, schema_id, &filter_keys) {
       Ok(sql) => sql,
       Err(drop) => return drop,
     };
@@ -90,15 +88,19 @@ impl Queryer {
     &self,
     cache_key: &str,
     schema_id: &str,
-    stem_opt: Option<&str>,
     filter_keys: &[String],
   ) -> Result<String, crate::drop::Drop> {
     if let Some(cached_sql) = self.cache.get(cache_key) {
       return Ok(cached_sql.value().clone());
     }
 
-    let compiler = compiler::SqlCompiler::new(self.db.clone());
-    match compiler.compile(schema_id, stem_opt, filter_keys) {
+    let compiler = compiler::Compiler {
+      db: &self.db,
+      filter_keys: filter_keys,
+      alias_counter: 0,
+    };
+
+    match compiler.compile(schema_id, filter_keys) {
       Ok(compiled_sql) => {
         self
           .cache
@@ -138,7 +140,7 @@ impl Queryer {
         details: crate::drop::ErrorDetails {
           path: "".to_string(),
           cause: Some(format!("Expected array, got {}", other)),
-          context: Some(vec![sql.to_string()]),
+          context: Some(serde_json::json!([sql])),
           schema: Some(schema_id.to_string()),
         },
       }]),
@@ -148,7 +150,7 @@ impl Queryer {
         details: crate::drop::ErrorDetails {
           path: "".to_string(),
           cause: Some(format!("SPI error in queryer: {}", e)),
-          context: Some(vec![sql.to_string()]),
+          context: Some(serde_json::json!([sql])),
           schema: Some(schema_id.to_string()),
         },
       }]),

src/tests/fixtures.rs

@@ -1451,30 +1451,6 @@ fn test_queryer_0_6() {
     crate::tests::runner::run_test_case(&path, 0, 6).unwrap();
 }
 
-#[test]
-fn test_queryer_0_7() {
-    let path = format!("{}/fixtures/queryer.json", env!("CARGO_MANIFEST_DIR"));
-    crate::tests::runner::run_test_case(&path, 0, 7).unwrap();
-}
-
-#[test]
-fn test_queryer_0_8() {
-    let path = format!("{}/fixtures/queryer.json", env!("CARGO_MANIFEST_DIR"));
-    crate::tests::runner::run_test_case(&path, 0, 8).unwrap();
-}
-
-#[test]
-fn test_queryer_0_9() {
-    let path = format!("{}/fixtures/queryer.json", env!("CARGO_MANIFEST_DIR"));
-    crate::tests::runner::run_test_case(&path, 0, 9).unwrap();
-}
-
-#[test]
-fn test_queryer_0_10() {
-    let path = format!("{}/fixtures/queryer.json", env!("CARGO_MANIFEST_DIR"));
-    crate::tests::runner::run_test_case(&path, 0, 10).unwrap();
-}
-
 #[test]
 fn test_not_0_0() {
     let path = format!("{}/fixtures/not.json", env!("CARGO_MANIFEST_DIR"));
@@ -3449,12 +3425,6 @@ fn test_if_then_else_13_1() {
     crate::tests::runner::run_test_case(&path, 13, 1).unwrap();
 }
 
-#[test]
-fn test_stems_0_0() {
-    let path = format!("{}/fixtures/stems.json", env!("CARGO_MANIFEST_DIR"));
-    crate::tests::runner::run_test_case(&path, 0, 0).unwrap();
-}
-
 #[test]
 fn test_empty_string_0_0() {
     let path = format!("{}/fixtures/emptyString.json", env!("CARGO_MANIFEST_DIR"));
@@ -8566,3 +8536,9 @@ fn test_merger_0_7() {
     let path = format!("{}/fixtures/merger.json", env!("CARGO_MANIFEST_DIR"));
     crate::tests::runner::run_test_case(&path, 0, 7).unwrap();
 }
+
+#[test]
+fn test_merger_0_8() {
+    let path = format!("{}/fixtures/merger.json", env!("CARGO_MANIFEST_DIR"));
+    crate::tests::runner::run_test_case(&path, 0, 8).unwrap();
+}

src/tests/mod.rs

@@ -2,7 +2,6 @@ use crate::*;
 pub mod runner;
 pub mod types;
 use serde_json::json;
-pub mod sql_validator;
 
 // Database module tests moved to src/database/executors/mock.rs
 

src/tests/runner.rs

@@ -1,19 +1,10 @@
+use crate::tests::types::Suite;
 use serde::Deserialize;
+use serde_json::Value;
 use std::collections::HashMap;
 use std::fs;
 use std::sync::{Arc, OnceLock, RwLock};
 
-#[derive(Debug, Deserialize)]
-pub struct TestSuite {
-  #[allow(dead_code)]
-  pub description: String,
-  pub database: serde_json::Value,
-  pub tests: Vec<TestCase>,
-}
-
-use crate::tests::types::TestCase;
-use serde_json::Value;
-
 pub fn deserialize_some<'de, D>(deserializer: D) -> Result<Option<Value>, D::Error>
 where
   D: serde::Deserializer<'de>,
@@ -23,7 +14,7 @@ where
 }
 
 // Type alias for easier reading
-type CompiledSuite = Arc<Vec<(TestSuite, Arc<crate::database::Database>)>>;
+type CompiledSuite = Arc<Vec<(Suite, Arc<crate::database::Database>)>>;
 
 // Global cache mapping filename -> Vector of (Parsed JSON suite, Compiled Database)
 static CACHE: OnceLock<RwLock<HashMap<String, CompiledSuite>>> = OnceLock::new();
@@ -46,7 +37,7 @@ fn get_cached_file(path: &str) -> CompiledSuite {
       } else {
         let content =
           fs::read_to_string(path).unwrap_or_else(|_| panic!("Failed to read file: {}", path));
-        let suites: Vec<TestSuite> = serde_json::from_str(&content)
+        let suites: Vec<Suite> = serde_json::from_str(&content)
           .unwrap_or_else(|e| panic!("Failed to parse JSON in {}: {}", path, e));
 
         let mut compiled_suites = Vec::new();

src/tests/sql_validator.rs

@@ -1,156 +0,0 @@
-use sqlparser::ast::{
-    Expr, Join, JoinConstraint, JoinOperator, Query, Select, SelectItem, SetExpr, Statement,
-    TableFactor, TableWithJoins, Ident,
-};
-use sqlparser::dialect::PostgreSqlDialect;
-use sqlparser::parser::Parser;
-use std::collections::HashSet;
-
-pub fn validate_semantic_sql(sql: &str) -> Result<(), String> {
-    let dialect = PostgreSqlDialect {};
-    let statements = match Parser::parse_sql(&dialect, sql) {
-        Ok(s) => s,
-        Err(e) => return Err(format!("SQL Syntax Error: {}\nSQL: {}", e, sql)),
-    };
-
-    for statement in statements {
-        validate_statement(&statement, sql)?;
-    }
-
-    Ok(())
-}
-
-fn validate_statement(stmt: &Statement, original_sql: &str) -> Result<(), String> {
-    match stmt {
-        Statement::Query(query) => validate_query(query, original_sql)?,
-        Statement::Insert(insert) => {
-            if let Some(query) = &insert.source {
-                 validate_query(query, original_sql)?
-            }
-        }
-        Statement::Update(update) => {
-            if let Some(expr) = &update.selection {
-                validate_expr(expr, &HashSet::new(), original_sql)?;
-            }
-        }
-        Statement::Delete(delete) => {
-            if let Some(expr) = &delete.selection {
-                validate_expr(expr, &HashSet::new(), original_sql)?;
-            }
-        }
-        _ => {}
-    }
-    Ok(())
-}
-
-fn validate_query(query: &Query, original_sql: &str) -> Result<(), String> {
-    if let SetExpr::Select(select) = &*query.body {
-        validate_select(select, original_sql)?;
-    }
-    Ok(())
-}
-
-fn validate_select(select: &Select, original_sql: &str) -> Result<(), String> {
-    let mut available_aliases = HashSet::new();
-
-    // 1. Collect all declared table aliases in the FROM clause and JOINs
-    for table_with_joins in &select.from {
-        collect_aliases_from_table_factor(&table_with_joins.relation, &mut available_aliases);
-        for join in &table_with_joins.joins {
-            collect_aliases_from_table_factor(&join.relation, &mut available_aliases);
-        }
-    }
-
-    // 2. Validate all SELECT projection fields
-    for projection in &select.projection {
-        if let SelectItem::UnnamedExpr(expr) | SelectItem::ExprWithAlias { expr, .. } = projection {
-            validate_expr(expr, &available_aliases, original_sql)?;
-        }
-    }
-
-    // 3. Validate ON conditions in joins 
-    for table_with_joins in &select.from {
-        for join in &table_with_joins.joins {
-            if let JoinOperator::Inner(JoinConstraint::On(expr))
-            | JoinOperator::LeftOuter(JoinConstraint::On(expr))
-            | JoinOperator::RightOuter(JoinConstraint::On(expr))
-            | JoinOperator::FullOuter(JoinConstraint::On(expr))
-            | JoinOperator::Join(JoinConstraint::On(expr)) = &join.join_operator
-            {
-                validate_expr(expr, &available_aliases, original_sql)?;
-            }
-        }
-    }
-
-    // 4. Validate WHERE conditions
-    if let Some(selection) = &select.selection {
-        validate_expr(selection, &available_aliases, original_sql)?;
-    }
-
-    Ok(())
-}
-
-fn collect_aliases_from_table_factor(tf: &TableFactor, aliases: &mut HashSet<String>) {
-    match tf {
-        TableFactor::Table { name, alias, .. } => {
-            if let Some(table_alias) = alias {
-                aliases.insert(table_alias.name.value.clone());
-            } else if let Some(last) = name.0.last() {
-                 match last {
-                     sqlparser::ast::ObjectNamePart::Identifier(i) => {
-                         aliases.insert(i.value.clone());
-                     }
-                     _ => {}
-                 }
-            }
-        }
-        TableFactor::Derived { alias: Some(table_alias), .. } => {
-            aliases.insert(table_alias.name.value.clone());
-        }
-        _ => {}
-    }
-}
-
-fn validate_expr(expr: &Expr, available_aliases: &HashSet<String>, sql: &str) -> Result<(), String> {
-    match expr {
-        Expr::CompoundIdentifier(idents) => {
-            if idents.len() == 2 {
-                let alias = &idents[0].value;
-                if !available_aliases.is_empty() && !available_aliases.contains(alias) {
-                    return Err(format!(
-                        "Semantic Error: Orchestrated query referenced table alias '{}' but it was not declared in the query's FROM/JOIN clauses.\nAvailable aliases: {:?}\nSQL: {}",
-                        alias, available_aliases, sql
-                    ));
-                }
-            } else if idents.len() > 2 {
-                let alias = &idents[1].value; // In form schema.table.column, 'table' is idents[1]
-                if !available_aliases.is_empty() && !available_aliases.contains(alias) {
-                    return Err(format!(
-                        "Semantic Error: Orchestrated query referenced table '{}' but it was not mapped.\nAvailable aliases: {:?}\nSQL: {}",
-                        alias, available_aliases, sql
-                    ));
-                }
-            }
-        }
-        Expr::BinaryOp { left, right, .. } => {
-            validate_expr(left, available_aliases, sql)?;
-            validate_expr(right, available_aliases, sql)?;
-        }
-        Expr::IsFalse(e) | Expr::IsNotFalse(e) | Expr::IsTrue(e) | Expr::IsNotTrue(e) 
-        | Expr::IsNull(e) | Expr::IsNotNull(e) | Expr::InList { expr: e, .. }
-        | Expr::Nested(e) | Expr::UnaryOp { expr: e, .. } | Expr::Cast { expr: e, .. }
-        | Expr::Like { expr: e, .. } | Expr::ILike { expr: e, .. } | Expr::AnyOp { left: e, .. }
-        | Expr::AllOp { left: e, .. } => {
-             validate_expr(e, available_aliases, sql)?;
-        }
-        Expr::Function(func) => {
-            if let sqlparser::ast::FunctionArguments::List(args) = &func.args {
-                if let Some(sqlparser::ast::FunctionArg::Unnamed(sqlparser::ast::FunctionArgExpr::Expr(e))) = args.args.get(0) {
-                    validate_expr(e, available_aliases, sql)?;
-                }
-            }
-        }
-        _ => {}
-    }
-    Ok(())
-}

src/tests/types/case.rs

@@ -1,11 +1,11 @@
-use super::expect::ExpectBlock;
+use super::expect::Expect;
 use crate::database::Database;
 use serde::Deserialize;
 use serde_json::Value;
 use std::sync::Arc;
 
 #[derive(Debug, Deserialize)]
-pub struct TestCase {
+pub struct Case {
   pub description: String,
 
   #[serde(default = "default_action")]
@@ -16,9 +16,6 @@ pub struct TestCase {
   pub schema_id: String,
 
   // For Query
-  #[serde(default)]
-  pub stem: Option<String>,
-
   #[serde(default)]
   pub filters: Option<serde_json::Value>,
 
@@ -30,15 +27,15 @@ pub struct TestCase {
   #[serde(default)]
   pub mocks: Option<serde_json::Value>,
 
-  pub expect: Option<ExpectBlock>,
+  pub expect: Option<Expect>,
 }
 
 fn default_action() -> String {
   "validate".to_string()
 }
 
-impl TestCase {
-  pub fn run_compile(&self, db: Arc<Database>) -> Result<(), String> {
+impl Case {
+  pub fn run_compile(&self, _db: Arc<Database>) -> Result<(), String> {
     let expected_success = self.expect.as_ref().map(|e| e.success).unwrap_or(false);
 
     // We assume db has already been setup and compiled successfully by runner.rs's `jspg_setup`
@@ -52,24 +49,6 @@ impl TestCase {
       ));
     }
 
-    // Assert stems
-    if let Some(expect) = &self.expect {
-      if let Some(expected_stems) = &expect.stems {
-        // Convert the Db stems (HashMap<String, HashMap<String, Arc<Stem>>>) to matching JSON shape
-        let db_stems_json = serde_json::to_value(&db.stems).unwrap();
-        let expect_stems_json = serde_json::to_value(expected_stems).unwrap();
-
-        if db_stems_json != expect_stems_json {
-          let expected_pretty = serde_json::to_string_pretty(&expect_stems_json).unwrap();
-          let got_pretty = serde_json::to_string_pretty(&db_stems_json).unwrap();
-          return Err(format!(
-            "Stem validation failed.\nExpected:\n{}\n\nGot:\n{}",
-            expected_pretty, got_pretty
-          ));
-        }
-      }
-    }
-
     Ok(())
   }
 
@@ -138,6 +117,7 @@ impl TestCase {
       ))
     } else if let Some(expect) = &self.expect {
       let queries = db.executor.get_queries();
+      expect.assert_pattern(&queries)?;
       expect.assert_sql(&queries)
     } else {
       Ok(())
@@ -157,8 +137,7 @@ impl TestCase {
     use crate::queryer::Queryer;
     let queryer = Queryer::new(db.clone());
 
-    let stem_opt = self.stem.as_deref();
-    let result = queryer.query(&self.schema_id, stem_opt, self.filters.as_ref());
+    let result = queryer.query(&self.schema_id, self.filters.as_ref());
 
     let expected_success = self.expect.as_ref().map(|e| e.success).unwrap_or(false);
     let got_success = result.errors.is_empty();
@@ -176,6 +155,7 @@ impl TestCase {
       ))
     } else if let Some(expect) = &self.expect {
       let queries = db.executor.get_queries();
+      expect.assert_pattern(&queries)?;
       expect.assert_sql(&queries)
     } else {
       Ok(())

src/tests/types/expect/mod.rs

@@ -0,0 +1,20 @@
+pub mod pattern;
+pub mod sql;
+
+use serde::Deserialize;
+
+#[derive(Debug, Deserialize)]
+#[serde(untagged)]
+pub enum SqlExpectation {
+  Single(String),
+  Multi(Vec<String>),
+}
+
+#[derive(Debug, Deserialize)]
+pub struct Expect {
+  pub success: bool,
+  pub result: Option<serde_json::Value>,
+  pub errors: Option<Vec<serde_json::Value>>,
+  #[serde(default)]
+  pub sql: Option<Vec<SqlExpectation>>,
+}

src/tests/types/expect/pattern.rs

@@ -1,30 +1,13 @@
+use super::Expect;
 use regex::Regex;
-use serde::Deserialize;
 use std::collections::HashMap;
 
-#[derive(Debug, Deserialize)]
-#[serde(untagged)]
-pub enum SqlExpectation {
-  Single(String),
-  Multi(Vec<String>),
-}
-
-#[derive(Debug, Deserialize)]
-pub struct ExpectBlock {
-  pub success: bool,
-  pub result: Option<serde_json::Value>,
-  pub errors: Option<Vec<serde_json::Value>>,
-  pub stems: Option<HashMap<String, HashMap<String, serde_json::Value>>>,
-  #[serde(default)]
-  pub sql: Option<Vec<SqlExpectation>>,
-}
-
-impl ExpectBlock {
+impl Expect {
   /// Advanced SQL execution assertion algorithm ported from `assert.go`.
   /// This compares two arrays of strings, one containing {{uuid:name}} or {{timestamp}} placeholders,
   /// and the other containing actual executed database queries. It ensures that placeholder UUIDs
   /// are consistently mapped to the same actual UUIDs across all lines, and strictly validates line-by-line sequences.
-  pub fn assert_sql(&self, actual: &[String]) -> Result<(), String> {
+  pub fn assert_pattern(&self, actual: &[String]) -> Result<(), String> {
     let patterns = match &self.sql {
       Some(s) => s,
       None => return Ok(()),
@@ -39,12 +22,6 @@ impl ExpectBlock {
       ));
     }
 
-    for query in actual {
-      if let Err(e) = crate::tests::sql_validator::validate_semantic_sql(query) {
-        return Err(e);
-      }
-    }
-
     let ws_re = Regex::new(r"\s+").unwrap();
 
     let types = HashMap::from([
@@ -82,8 +59,8 @@ impl ExpectBlock {
       let aline = clean_str(aline_raw);
 
       let pattern_str_raw = match pattern_expect {
-        SqlExpectation::Single(s) => s.clone(),
-        SqlExpectation::Multi(m) => m.join(" "),
+        super::SqlExpectation::Single(s) => s.clone(),
+        super::SqlExpectation::Multi(m) => m.join(" "),
       };
 
       let pattern_str = clean_str(&pattern_str_raw);

src/tests/types/expect/sql.rs

@@ -0,0 +1,206 @@
+use super::Expect;
+use sqlparser::ast::{Expr, Query, SelectItem, Statement, TableFactor};
+use sqlparser::dialect::PostgreSqlDialect;
+use sqlparser::parser::Parser;
+use std::collections::HashSet;
+
+impl Expect {
+  pub fn assert_sql(&self, actual: &[String]) -> Result<(), String> {
+    for query in actual {
+      if let Err(e) = Self::validate_semantic_sql(query) {
+        return Err(e);
+      }
+    }
+    Ok(())
+  }
+
+  pub fn validate_semantic_sql(sql: &str) -> Result<(), String> {
+    let dialect = PostgreSqlDialect {};
+    let statements = match Parser::parse_sql(&dialect, sql) {
+      Ok(s) => s,
+      Err(e) => return Err(format!("SQL Syntax Error: {}\nSQL: {}", e, sql)),
+    };
+
+    for statement in statements {
+      Self::validate_statement(&statement, sql)?;
+    }
+
+    Ok(())
+  }
+
+  fn validate_statement(stmt: &Statement, original_sql: &str) -> Result<(), String> {
+    match stmt {
+      Statement::Query(query) => Self::validate_query(query, &HashSet::new(), original_sql)?,
+      Statement::Insert(insert) => {
+        if let Some(query) = &insert.source {
+          Self::validate_query(query, &HashSet::new(), original_sql)?
+        }
+      }
+      Statement::Update(update) => {
+        if let Some(expr) = &update.selection {
+          Self::validate_expr(expr, &HashSet::new(), original_sql)?;
+        }
+      }
+      Statement::Delete(delete) => {
+        if let Some(expr) = &delete.selection {
+          Self::validate_expr(expr, &HashSet::new(), original_sql)?;
+        }
+      }
+      _ => {}
+    }
+    Ok(())
+  }
+
+  fn validate_query(
+    query: &Query,
+    available_aliases: &HashSet<String>,
+    original_sql: &str,
+  ) -> Result<(), String> {
+    if let sqlparser::ast::SetExpr::Select(select) = &*query.body {
+      Self::validate_select(&select, available_aliases, original_sql)?;
+    }
+    Ok(())
+  }
+
+  fn validate_select(
+    select: &sqlparser::ast::Select,
+    parent_aliases: &HashSet<String>,
+    original_sql: &str,
+  ) -> Result<(), String> {
+    let mut available_aliases = parent_aliases.clone();
+
+    // 1. Collect all declared table aliases in the FROM clause and JOINs
+    for table_with_joins in &select.from {
+      Self::collect_aliases_from_table_factor(&table_with_joins.relation, &mut available_aliases);
+      for join in &table_with_joins.joins {
+        Self::collect_aliases_from_table_factor(&join.relation, &mut available_aliases);
+      }
+    }
+
+    // 2. Validate all SELECT projection fields
+    for projection in &select.projection {
+      if let SelectItem::UnnamedExpr(expr) | SelectItem::ExprWithAlias { expr, .. } = projection {
+        Self::validate_expr(expr, &available_aliases, original_sql)?;
+      }
+    }
+
+    // 3. Validate ON conditions in joins
+    for table_with_joins in &select.from {
+      for join in &table_with_joins.joins {
+        if let sqlparser::ast::JoinOperator::Inner(sqlparser::ast::JoinConstraint::On(expr))
+        | sqlparser::ast::JoinOperator::LeftOuter(sqlparser::ast::JoinConstraint::On(expr))
+        | sqlparser::ast::JoinOperator::RightOuter(sqlparser::ast::JoinConstraint::On(expr))
+        | sqlparser::ast::JoinOperator::FullOuter(sqlparser::ast::JoinConstraint::On(expr))
+        | sqlparser::ast::JoinOperator::Join(sqlparser::ast::JoinConstraint::On(expr)) =
+          &join.join_operator
+        {
+          Self::validate_expr(expr, &available_aliases, original_sql)?;
+        }
+      }
+    }
+
+    // 4. Validate WHERE conditions
+    if let Some(selection) = &select.selection {
+      Self::validate_expr(selection, &available_aliases, original_sql)?;
+    }
+
+    Ok(())
+  }
+
+  fn collect_aliases_from_table_factor(tf: &TableFactor, aliases: &mut HashSet<String>) {
+    match tf {
+      TableFactor::Table { name, alias, .. } => {
+        if let Some(table_alias) = alias {
+          aliases.insert(table_alias.name.value.clone());
+        } else if let Some(last) = name.0.last() {
+          match last {
+            sqlparser::ast::ObjectNamePart::Identifier(i) => {
+              aliases.insert(i.value.clone());
+            }
+            _ => {}
+          }
+        }
+      }
+      TableFactor::Derived {
+        subquery,
+        alias: Some(table_alias),
+        ..
+      } => {
+        aliases.insert(table_alias.name.value.clone());
+        // A derived table is technically a nested scope which is opaque outside, but for pure semantic checks
+        // its internal contents should be validated purely within its own scope (not leaking external aliases in, usually)
+        // but Postgres allows lateral correlation. We will validate its interior with an empty scope.
+        let _ = Self::validate_query(subquery, &HashSet::new(), "");
+      }
+      _ => {}
+    }
+  }
+
+  fn validate_expr(
+    expr: &Expr,
+    available_aliases: &HashSet<String>,
+    sql: &str,
+  ) -> Result<(), String> {
+    match expr {
+      Expr::CompoundIdentifier(idents) => {
+        if idents.len() == 2 {
+          let alias = &idents[0].value;
+          if !available_aliases.is_empty() && !available_aliases.contains(alias) {
+            return Err(format!(
+              "Semantic Error: Orchestrated query referenced table alias '{}' but it was not declared in the query's FROM/JOIN clauses.\nAvailable aliases: {:?}\nSQL: {}",
+              alias, available_aliases, sql
+            ));
+          }
+        } else if idents.len() > 2 {
+          let alias = &idents[1].value; // In form schema.table.column, 'table' is idents[1]
+          if !available_aliases.is_empty() && !available_aliases.contains(alias) {
+            return Err(format!(
+              "Semantic Error: Orchestrated query referenced table '{}' but it was not mapped.\nAvailable aliases: {:?}\nSQL: {}",
+              alias, available_aliases, sql
+            ));
+          }
+        }
+      }
+      Expr::Subquery(subquery) => Self::validate_query(subquery, available_aliases, sql)?,
+      Expr::Exists { subquery, .. } => Self::validate_query(subquery, available_aliases, sql)?,
+      Expr::InSubquery {
+        expr: e, subquery, ..
+      } => {
+        Self::validate_expr(e, available_aliases, sql)?;
+        Self::validate_query(subquery, available_aliases, sql)?;
+      }
+      Expr::BinaryOp { left, right, .. } => {
+        Self::validate_expr(left, available_aliases, sql)?;
+        Self::validate_expr(right, available_aliases, sql)?;
+      }
+      Expr::IsFalse(e)
+      | Expr::IsNotFalse(e)
+      | Expr::IsTrue(e)
+      | Expr::IsNotTrue(e)
+      | Expr::IsNull(e)
+      | Expr::IsNotNull(e)
+      | Expr::InList { expr: e, .. }
+      | Expr::Nested(e)
+      | Expr::UnaryOp { expr: e, .. }
+      | Expr::Cast { expr: e, .. }
+      | Expr::Like { expr: e, .. }
+      | Expr::ILike { expr: e, .. }
+      | Expr::AnyOp { left: e, .. }
+      | Expr::AllOp { left: e, .. } => {
+        Self::validate_expr(e, available_aliases, sql)?;
+      }
+      Expr::Function(func) => {
+        if let sqlparser::ast::FunctionArguments::List(args) = &func.args {
+          if let Some(sqlparser::ast::FunctionArg::Unnamed(
+            sqlparser::ast::FunctionArgExpr::Expr(e),
+          )) = args.args.get(0)
+          {
+            Self::validate_expr(e, available_aliases, sql)?;
+          }
+        }
+      }
+      _ => {}
+    }
+    Ok(())
+  }
+}

src/tests/types/mod.rs

@@ -2,6 +2,6 @@ pub mod case;
 pub mod expect;
 pub mod suite;
 
-pub use case::TestCase;
-pub use expect::ExpectBlock;
-pub use suite::TestSuite;
+pub use case::Case;
+pub use expect::Expect;
+pub use suite::Suite;

src/tests/types/suite.rs

@@ -1,10 +1,10 @@
-use super::case::TestCase;
+use super::case::Case;
 use serde::Deserialize;
 
 #[derive(Debug, Deserialize)]
-pub struct TestSuite {
+pub struct Suite {
   #[allow(dead_code)]
   pub description: String,
   pub database: serde_json::Value,
-  pub tests: Vec<TestCase>,
+  pub tests: Vec<Case>,
 }

version

@@ -1 +1 @@
-1.0.68
+1.0.80