Compare commits
9 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0b072d66e7 | |||
| 41649766db | |||
| 61a8c5eed7 | |||
| 77af67aef5 | |||
| cd85a8a2c3 | |||
| d3cb72a5e2 | |||
| 57baa389b6 | |||
| 8ceb4f05a2 | |||
| a3bd79deef |
32
GEMINI.md
32
GEMINI.md
@ -13,7 +13,7 @@ JSPG operates by deeply integrating the JSON Schema Draft 2020-12 specification
|
||||
1. **Draft 2020-12 Based**: Attempt to adhere to the official JSON Schema Draft 2020-12 specification, while heavily augmenting it for strict structural typing.
|
||||
2. **Ultra-Fast Execution**: Compile schemas into optimized in-memory validation trees and cached SQL SPIs to bypass Postgres Query Builder overheads.
|
||||
3. **Connection-Bound Caching**: Leverage the PostgreSQL session lifecycle using an **Atomic Swap** pattern. Schemas are 100% frozen, completely eliminating locks during read access.
|
||||
4. **Structural Inheritance**: Support object-oriented schema design via Implicit Keyword Shadowing and virtual `$family` references natively mapped to Postgres table constraints.
|
||||
4. **Structural Inheritance**: Support object-oriented schema design via Implicit Keyword Shadowing and virtual `family` references natively mapped to Postgres table constraints.
|
||||
5. **Reactive Beats**: Provide ultra-fast natively generated flat payloads mapping directly to the Dart topological state for dynamic websocket reactivity.
|
||||
|
||||
### Concurrency & Threading ("Immutable Graphs")
|
||||
@ -55,8 +55,8 @@ In Punc, polymorphic targets like explicit tagged unions or STI (Single Table In
|
||||
Therefore, any schema that participates in polymorphic discrimination MUST explicitly define its discriminator properties natively inside its `properties` block. However, to stay DRY and maintain flexible APIs, you **DO NOT** need to hardcode `const` values, nor should you add them to your `required` array. The Punc engine treats `type` and `kind` as **magic properties**.
|
||||
|
||||
**Magic Validation Constraints**:
|
||||
* **Dynamically Required**: The system inherently drives the need for their requirement. The Validator dynamically expects the discriminators and structurally bubbles `MISSING_TYPE` ultimata ONLY when a polymorphic router (`$family` / `oneOf`) dynamically requires them to resolve a path. You never manually put them in the JSON schema `required` block.
|
||||
* **Implicit Resolution**: When wrapped in `$family` or `oneOf`, the polymorphic router can mathematically parse the schema key (e.g. `light.person`) and natively validate that `type` equals `"person"` and `kind` equals `"light"`, bubbling `CONST_VIOLATED` if they mismatch, all without you ever hardcoding `const` limitations.
|
||||
* **Dynamically Required**: The system inherently drives the need for their requirement. The Validator dynamically expects the discriminators and structurally bubbles `MISSING_TYPE` ultimata ONLY when a polymorphic router (`family` / `oneOf`) dynamically requires them to resolve a path. You never manually put them in the JSON schema `required` block.
|
||||
* **Implicit Resolution**: When wrapped in `family` or `oneOf`, the polymorphic router can mathematically parse the schema key (e.g. `light.person`) and natively validate that `type` equals `"person"` and `kind` equals `"light"`, bubbling `CONST_VIOLATED` if they mismatch, all without you ever hardcoding `const` limitations.
|
||||
* **Generator Explicitness**: Because Postgres is the Single Source of Truth, forcing the explicit definition in `properties` initially guarantees the downstream Dart/Go code generators observe the fields and can cleanly serialize them dynamically back to the server.
|
||||
|
||||
For example, a schema registered under the exact key `"light.person"` inside the database registry must natively define its own structural boundaries:
|
||||
@ -72,7 +72,7 @@ For example, a schema registered under the exact key `"light.person"` inside the
|
||||
|
||||
* **The Object Contract (Presence)**: The Object enforces its own structural integrity mechanically. Standard JSON Validation natively ensures `type` and `kind` are dynamically present as expected.
|
||||
* **The Dynamic Values (`db.types`)**: Because the `type` and `kind` properties technically exist, the Punc engine dynamically intercepts them during `validate_object`. It mathematically parses the schema key (e.g. `light.person`) and natively validates that `type` equals `"person"` (or a valid descendant in `db.types`) and `kind` equals `"light"`, bubbling `CONST_VIOLATED` if they mismatch.
|
||||
* **The Routing Contract**: When wrapped in `$family` or `oneOf`, the polymorphic router can execute Lightning Fast $O(1)$ fast-paths by reading the payload's `type`/`kind` identifiers, and gracefully fallback to standard structural failure if omitted.
|
||||
* **The Routing Contract**: When wrapped in `family` or `oneOf`, the polymorphic router can execute Lightning Fast $O(1)$ fast-paths by reading the payload's `type`/`kind` identifiers, and gracefully fallback to standard structural failure if omitted.
|
||||
|
||||
### Composition & Inheritance (The `type` keyword)
|
||||
Punc completely abandons the standard JSON Schema `$ref` keyword. Instead, it overloads the exact same `type` keyword used for primitives. A `"type"` in Punc is mathematically evaluated as either a Native Primitive (`"string"`, `"null"`) or a Custom Object Pointer (`"budget"`, `"user"`).
|
||||
@ -81,24 +81,24 @@ Punc completely abandons the standard JSON Schema `$ref` keyword. Instead, it ov
|
||||
* **Primitive Array Shorthand (Optionality)**: The `type` array syntax is heavily optimized for nullable fields. Defining `"type": ["budget", "null"]` natively builds a nullable strict, generating `Budget? budget;` in Dart. You can freely mix primitives like `["string", "number", "null"]`.
|
||||
* **Strict Array Constraint**: To explicitly prevent mathematically ambiguous Multiple Inheritance, a `type` array is strictly constrained to at most **ONE** Custom Object Pointer. Defining `"type": ["person", "organization"]` will intentionally trigger a fatal database compilation error natively instructing developers to build a proper tagged union (`oneOf`) instead.
|
||||
|
||||
### Polymorphism (`$family` and `oneOf`)
|
||||
### Polymorphism (`family` and `oneOf`)
|
||||
Polymorphism is how an object boundary can dynamically take on entirely different shapes based on the payload provided at runtime. Punc utilizes the static database metadata generated from Postgres (`db.types`) to enforce these boundaries deterministically, rather than relying on ambiguous tree-traversals.
|
||||
|
||||
* **`$family` (Target-Based Polymorphism)**: An explicit Punc compiler macro instructing the engine to resolve dynamic options against the registered database `types` variations or its inner schema registry. It uses the exact physical constraints of the database to build SQL and validation routes.
|
||||
* **`family` (Target-Based Polymorphism)**: An explicit Punc compiler macro instructing the engine to resolve dynamic options against the registered database `types` variations or its inner schema registry. It uses the exact physical constraints of the database to build SQL and validation routes.
|
||||
* **Scenario A: Global Tables (Vertical Routing)**
|
||||
* *Setup*: `{ "$family": "organization" }`
|
||||
* *Execution*: The engine queries `db.types.get("organization").variations` and finds `["bot", "organization", "person"]`. Because organizations are structurally table-backed, the `$family` automatically uses `type` as the discriminator.
|
||||
* *Setup*: `{ "family": "organization" }`
|
||||
* *Execution*: The engine queries `db.types.get("organization").variations` and finds `["bot", "organization", "person"]`. Because organizations are structurally table-backed, the `family` automatically uses `type` as the discriminator.
|
||||
* *Options*: `bot` -> `bot`, `person` -> `person`, `organization` -> `organization`.
|
||||
* **Scenario B: Prefixed Tables (Vertical Projection)**
|
||||
* *Setup*: `{ "$family": "light.organization" }`
|
||||
* *Setup*: `{ "family": "light.organization" }`
|
||||
* *Execution*: The engine sees the prefix `light.` and base `organization`. It queries `db.types.get("organization").variations` and dynamically prepends the prefix to discover the relevant UI schemas.
|
||||
* *Options*: `person` -> `light.person`, `organization` -> `light.organization`. (If a projection like `light.bot` does not exist in `db.schemas`, it is safely ignored).
|
||||
* **Scenario C: Single Table Inheritance (Horizontal Routing)**
|
||||
* *Setup*: `{ "$family": "widget" }` (Where `widget` is a table type but has no external variations).
|
||||
* *Execution*: The engine queries `db.types.get("widget").variations` and finds only `["widget"]`. Since it lacks table inheritance, it is treated as STI. The engine scans the specific, confined `schemas` array directly under `db.types.get("widget")` for any registered key terminating in the base `.widget` (e.g., `stock.widget`). The `$family` automatically uses `kind` as the discriminator.
|
||||
* *Setup*: `{ "family": "widget" }` (Where `widget` is a table type but has no external variations).
|
||||
* *Execution*: The engine queries `db.types.get("widget").variations` and finds only `["widget"]`. Since it lacks table inheritance, it is treated as STI. The engine scans the specific, confined `schemas` array directly under `db.types.get("widget")` for any registered key terminating in the base `.widget` (e.g., `stock.widget`). The `family` automatically uses `kind` as the discriminator.
|
||||
* *Options*: `stock` -> `stock.widget`, `tasks` -> `tasks.widget`.
|
||||
|
||||
* **`oneOf` (Strict Tagged Unions)**: A hardcoded list of candidate schemas. Unlike `$family` which relies on global DB metadata, `oneOf` forces pure mathematical structural evaluation of the provided candidates. It strictly bans typical JSON Schema "Union of Sets" fallback searches. Every candidate MUST possess a mathematically unique discriminator payload to allow $O(1)$ routing.
|
||||
* **`oneOf` (Strict Tagged Unions)**: A hardcoded list of candidate schemas. Unlike `family` which relies on global DB metadata, `oneOf` forces pure mathematical structural evaluation of the provided candidates. It strictly bans typical JSON Schema "Union of Sets" fallback searches. Every candidate MUST possess a mathematically unique discriminator payload to allow $O(1)$ routing.
|
||||
* **Disjoint Types**: `oneOf: [{ "type": "person" }, { "type": "widget" }]`. The engine succeeds because the native `type` acts as a unique discriminator (`"person"` vs `"widget"`).
|
||||
* **STI Types**: `oneOf: [{ "type": "heavy.person" }, { "type": "light.person" }]`. The engine succeeds. Even though both share `"type": "person"`, their explicit discriminator is `kind` (`"heavy"` vs `"light"`), ensuring unique $O(1)$ fast-paths.
|
||||
* **Conflicting Types**: `oneOf: [{ "type": "person" }, { "type": "light.person" }]`. The engine **fails compilation natively**. Both schemas evaluate to `"type": "person"` and neither provides a disjoint `kind` constraint, making them mathematically ambiguous and impossible to route in $O(1)$ time.
|
||||
@ -187,10 +187,10 @@ The Validator provides strict, schema-driven evaluation for the "Punc" architect
|
||||
JSPG implements specific extensions to the Draft 2020-12 standard to support the Punc architecture's object-oriented needs while heavily optimizing for zero-runtime lookups.
|
||||
|
||||
* **Caching Strategy**: The Validator caches the pre-compiled `Database` registry in memory upon initialization (`jspg_setup`). This registry holds the comprehensive graph of schema boundaries, Types, ENUMs, and Foreign Key relationships, acting as the Single Source of Truth for all validation operations without polling Postgres.
|
||||
* **Discriminator Fast Paths & Extraction**: When executing a polymorphic node (`oneOf` or `$family`), the engine statically analyzes the incoming JSON payload for the literal `type` and `kind` string coordinates. It routes the evaluation specifically to matching candidates in $O(1)$ while returning `MISSING_TYPE` ultimata directly.
|
||||
* **Discriminator Fast Paths & Extraction**: When executing a polymorphic node (`oneOf` or `family`), the engine statically analyzes the incoming JSON payload for the literal `type` and `kind` string coordinates. It routes the evaluation specifically to matching candidates in $O(1)$ while returning `MISSING_TYPE` ultimata directly.
|
||||
* **Missing Type Ultimatum**: If an entity logically requires a discriminator and the JSON payload omits it, JSPG short-circuits branch execution entirely, bubbling a single, perfectly-pathed `MISSING_TYPE` error back to the UI natively to prevent confusing cascading failures.
|
||||
* **Golden Match Context**: When exactly one structural candidate perfectly maps a discriminator, the Validator exclusively cascades that specific structural error context directly to the user, stripping away all noise generated by other parallel schemas.
|
||||
* **Topological Array Pathing**: Instead of relying on explicit `$id` references or injected properties, array iteration paths are dynamically typed based on their compiler boundary constraints. If the array's `items` schema resolves to a topological table-backed entity (e.g., inheriting via a `$family` macro tracked in the global DB catalog), the array locks paths and derives element indexes from their actual UUID paths (`array/widget-1/name`), natively enforcing database continuity. If evaluating isolated ad-hoc JSONB elements, strict numeric indexing is enforced natively (`array/1/name`) preventing synthetic payload manipulation.
|
||||
* **Topological Array Pathing**: Instead of relying on explicit `$id` references or injected properties, array iteration paths are dynamically typed based on their compiler boundary constraints. If the array's `items` schema resolves to a topological table-backed entity (e.g., inheriting via a `family` macro tracked in the global DB catalog), the array locks paths and derives element indexes from their actual UUID paths (`array/widget-1/name`), natively enforcing database continuity. If evaluating isolated ad-hoc JSONB elements, strict numeric indexing is enforced natively (`array/1/name`) preventing synthetic payload manipulation.
|
||||
|
||||
---
|
||||
|
||||
@ -237,8 +237,8 @@ The Queryer transforms Postgres into a pre-compiled Semantic Query Engine, desig
|
||||
* **Array Inclusion**: `{"$in": [values]}`, `{"$nin": [values]}` use native `jsonb_array_elements_text()` bindings to enforce `IN` and `NOT IN` logic without runtime SQL injection risks.
|
||||
* **Text Matching (ILIKE)**: Evaluates `$eq` or `$ne` against string fields containing the `%` character natively into Postgres `ILIKE` and `NOT ILIKE` partial substring matches.
|
||||
* **Type Casting**: Safely resolves dynamic combinations by casting values instantly into the physical database types mapped in the schema (e.g. parsing `uuid` bindings to `::uuid`, formatting DateTimes to `::timestamptz`, and numbers to `::numeric`).
|
||||
* **Polymorphic SQL Generation (`$family`)**: Compiles `$family` properties by analyzing the **Physical Database Variations**, *not* the schema descendants.
|
||||
* **The Dot Convention**: When a schema requests `$family: "target.schema"`, the compiler extracts the base type (e.g. `schema`) and looks up its Physical Table definition.
|
||||
* **Polymorphic SQL Generation (`family`)**: Compiles `family` properties by analyzing the **Physical Database Variations**, *not* the schema descendants.
|
||||
* **The Dot Convention**: When a schema requests `family: "target.schema"`, the compiler extracts the base type (e.g. `schema`) and looks up its Physical Table definition.
|
||||
* **Multi-Table Branching**: If the Physical Table is a parent to other tables (e.g. `organization` has variations `["organization", "bot", "person"]`), the compiler generates a dynamic `CASE WHEN type = '...' THEN ...` query, expanding into sub-queries for each variation. To ensure safe resolution, the compiler dynamically evaluates correlation boundaries: it attempts standard Relational Edge discovery first. If no explicit relational edge exists (indicating pure Table Inheritance rather than a standard foreign-key graph relationship), it safely invokes a **Table Parity Fallback**. This generates an explicit ID correlation constraint (`AND inner.id = outer.id`), perfectly binding the structural variations back to the parent row to eliminate Cartesian products.
|
||||
* **Single-Table Bypass**: If the Physical Table is a leaf node with only one variation (e.g. `person` has variations `["person"]`), the compiler cleanly bypasses `CASE` generation and compiles a simple `SELECT` across the base table, as all schema extensions (e.g. `light.person`, `full.person`) are guaranteed to reside in the exact same physical row.
|
||||
|
||||
|
||||
104
add_test.py
104
add_test.py
@ -1,104 +0,0 @@
|
||||
import json
|
||||
|
||||
def load_json(path):
|
||||
with open(path, 'r') as f:
|
||||
return json.load(f)
|
||||
|
||||
def save_json(path, data):
|
||||
with open(path, 'w') as f:
|
||||
json.dump(data, f, indent=2)
|
||||
|
||||
def add_invoice(data):
|
||||
# Add 'invoice' type
|
||||
types = data[0]['database']['types']
|
||||
|
||||
# Check if invoice already exists
|
||||
if any(t.get('name') == 'invoice' for t in types):
|
||||
return
|
||||
|
||||
types.append({
|
||||
"name": "invoice",
|
||||
"hierarchy": ["invoice", "entity"],
|
||||
"primary_key": ["id"],
|
||||
"field_types": {
|
||||
"id": "uuid",
|
||||
"number": "text",
|
||||
"metadata": "jsonb"
|
||||
},
|
||||
"schemas": {
|
||||
"invoice": {
|
||||
"type": "entity",
|
||||
"properties": {
|
||||
"id": { "type": "string" },
|
||||
"number": { "type": "string" },
|
||||
"metadata": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"internal_note": { "type": "string" },
|
||||
"customer_snapshot": { "type": "entity" },
|
||||
"related_rules": {
|
||||
"type": "array",
|
||||
"items": { "type": "governance_rule" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
def process_merger():
|
||||
data = load_json('fixtures/merger.json')
|
||||
add_invoice(data)
|
||||
|
||||
# Add test
|
||||
data[0]['tests'].append({
|
||||
"name": "Insert invoice with deep jsonb metadata",
|
||||
"schema": "invoice",
|
||||
"payload": {
|
||||
"number": "INV-1001",
|
||||
"metadata": {
|
||||
"internal_note": "Confidential",
|
||||
"customer_snapshot": {
|
||||
"id": "00000000-0000-0000-0000-000000000000",
|
||||
"type": "person",
|
||||
"first_name": "John"
|
||||
},
|
||||
"related_rules": [
|
||||
{
|
||||
"id": "11111111-1111-1111-1111-111111111111"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"expect": {
|
||||
"sql": [
|
||||
[
|
||||
"INSERT INTO agreego.invoice (metadata, number, id) VALUES ($1, $2, gen_random_uuid()) ON CONFLICT (id) DO UPDATE SET metadata = EXCLUDED.metadata, number = EXCLUDED.number RETURNING id, type",
|
||||
{"metadata": {"customer_snapshot": {"first_name": "John", "id": "00000000-0000-0000-0000-000000000000", "type": "person"}, "internal_note": "Confidential", "related_rules": [{"id": "11111111-1111-1111-1111-111111111111"}]}, "number": "INV-1001"}
|
||||
]
|
||||
]
|
||||
}
|
||||
})
|
||||
save_json('fixtures/merger.json', data)
|
||||
|
||||
def process_queryer():
|
||||
data = load_json('fixtures/queryer.json')
|
||||
add_invoice(data)
|
||||
|
||||
data[0]['tests'].append({
|
||||
"name": "Query invoice with complex JSONB metadata field extraction",
|
||||
"schema": "invoice",
|
||||
"query": {
|
||||
"extract": ["id", "number", "metadata"],
|
||||
"conditions": []
|
||||
},
|
||||
"expect": {
|
||||
"sql": "SELECT jsonb_build_object('id', t1.id, 'metadata', t1.metadata, 'number', t1.number) FROM agreego.invoice t1 WHERE (t1.id IS NOT NULL)",
|
||||
"params": {}
|
||||
}
|
||||
})
|
||||
save_json('fixtures/queryer.json', data)
|
||||
|
||||
process_merger()
|
||||
process_queryer()
|
||||
152
append_test.py
152
append_test.py
@ -1,152 +0,0 @@
|
||||
import json
|
||||
|
||||
path = "fixtures/database.json"
|
||||
|
||||
with open(path, "r") as f:
|
||||
data = json.load(f)
|
||||
|
||||
new_test = {
|
||||
"description": "Schema Promotion Accuracy Test - -- One Database to Rule Them All --",
|
||||
"database": {
|
||||
"puncs": [],
|
||||
"enums": [],
|
||||
"relations": [],
|
||||
"types": [
|
||||
{
|
||||
"id": "t1",
|
||||
"type": "type",
|
||||
"name": "person",
|
||||
"module": "core",
|
||||
"source": "person",
|
||||
"hierarchy": ["person"],
|
||||
"variations": ["person", "student"],
|
||||
"schemas": {
|
||||
"full.person": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"type": {"type": "string"},
|
||||
"name": {"type": "string"},
|
||||
"email": {
|
||||
"$family": "email_address"
|
||||
},
|
||||
"generic_bubble": {
|
||||
"type": "some_bubble"
|
||||
},
|
||||
"ad_hoc_bubble": {
|
||||
"type": "some_bubble",
|
||||
"properties": {
|
||||
"extra_inline_feature": {"type": "string"}
|
||||
}
|
||||
},
|
||||
"tags": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"}
|
||||
},
|
||||
"standard_relations": {
|
||||
"type": "array",
|
||||
"items": {"type": "contact"}
|
||||
},
|
||||
"extended_relations": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "contact",
|
||||
"properties": {
|
||||
"target": {"type": "email_address"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"student.person": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"type": {"type": "string"},
|
||||
"kind": {"type": "string"},
|
||||
"school": {"type": "string"}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "t2",
|
||||
"type": "type",
|
||||
"name": "email_address",
|
||||
"module": "core",
|
||||
"source": "email_address",
|
||||
"hierarchy": ["email_address"],
|
||||
"variations": ["email_address"],
|
||||
"schemas": {
|
||||
"light.email_address": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"address": {"type": "string"}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "t3",
|
||||
"type": "type",
|
||||
"name": "contact",
|
||||
"module": "core",
|
||||
"source": "contact",
|
||||
"hierarchy": ["contact"],
|
||||
"variations": ["contact"],
|
||||
"schemas": {
|
||||
"full.contact": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"id": {"type": "string"}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "t4",
|
||||
"type": "type",
|
||||
"name": "some_bubble",
|
||||
"module": "core",
|
||||
"source": "some_bubble",
|
||||
"hierarchy": ["some_bubble"],
|
||||
"variations": ["some_bubble"],
|
||||
"schemas": {
|
||||
"some_bubble": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"bubble_prop": {"type": "string"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"tests": [
|
||||
{
|
||||
"description": "Assert exact topological schema promotion paths",
|
||||
"action": "compile",
|
||||
"expect": {
|
||||
"success": True,
|
||||
"schemas": [
|
||||
"ad_hoc_bubble",
|
||||
"email_address",
|
||||
"extended_relations",
|
||||
"extended_relations/target",
|
||||
"full.contact",
|
||||
"full.person",
|
||||
"full.person/ad_hoc_bubble",
|
||||
"full.person/extended_relations",
|
||||
"full.person/extended_relations/target",
|
||||
"light.email_address",
|
||||
"person",
|
||||
"some_bubble",
|
||||
"student.person"
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
data.append(new_test)
|
||||
with open(path, "w") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
|
||||
@ -1,34 +0,0 @@
|
||||
import json
|
||||
|
||||
path = "fixtures/database.json"
|
||||
|
||||
with open(path, "r") as f:
|
||||
data = json.load(f)
|
||||
|
||||
test_case = data[-1]
|
||||
# Get full.person object properties
|
||||
props = test_case["database"]["types"][0]["schemas"]["full.person"]["properties"]
|
||||
|
||||
# Find extended_relations target and add properties!
|
||||
target_ref = props["extended_relations"]["items"]["properties"]["target"]
|
||||
target_ref["properties"] = {
|
||||
"extra_3rd_level": {"type": "string"}
|
||||
}
|
||||
|
||||
# The target is now an ad-hoc composition itself!
|
||||
# We expect `full.person/extended_relations/target` to be globally promoted.
|
||||
|
||||
test_case["tests"][0]["expect"]["schemas"] = [
|
||||
"full.contact",
|
||||
"full.person",
|
||||
"full.person/ad_hoc_bubble",
|
||||
"full.person/extended_relations",
|
||||
"full.person/extended_relations/target", # BOOM! Right here, 3 levels deep!
|
||||
"light.email_address",
|
||||
"some_bubble",
|
||||
"student.person"
|
||||
]
|
||||
|
||||
with open(path, "w") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
|
||||
@ -1,22 +0,0 @@
|
||||
import json
|
||||
|
||||
path = "fixtures/database.json"
|
||||
|
||||
with open(path, "r") as f:
|
||||
data = json.load(f)
|
||||
|
||||
test_case = data[-1]
|
||||
test_case["tests"][0]["expect"]["schemas"] = [
|
||||
"full.contact",
|
||||
"full.person",
|
||||
"full.person/ad_hoc_bubble",
|
||||
"full.person/extended_relations",
|
||||
"full.person/extended_relations/items",
|
||||
"light.email_address",
|
||||
"some_bubble",
|
||||
"student.person"
|
||||
]
|
||||
|
||||
with open(path, "w") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
|
||||
87
fix_test.py
87
fix_test.py
@ -1,87 +0,0 @@
|
||||
import json
|
||||
|
||||
def load_json(path):
|
||||
with open(path, 'r') as f:
|
||||
return json.load(f)
|
||||
|
||||
def save_json(path, data):
|
||||
with open(path, 'w') as f:
|
||||
json.dump(data, f, indent=4)
|
||||
|
||||
def fix_merger():
|
||||
data = load_json('fixtures/merger.json')
|
||||
last_test = data[0]['tests'][-1]
|
||||
|
||||
# Check if the last test is our bad one
|
||||
if "name" in last_test and last_test["name"] == "Insert invoice with deep jsonb metadata":
|
||||
new_test = {
|
||||
"description": last_test["name"],
|
||||
"action": "merge",
|
||||
"schema_id": last_test["schema"],
|
||||
"data": last_test["payload"],
|
||||
"expect": {
|
||||
"success": True,
|
||||
"sql": [
|
||||
[
|
||||
"INSERT INTO agreego.invoice (",
|
||||
" \"metadata\",",
|
||||
" \"number\",",
|
||||
" entity_id,",
|
||||
" id,",
|
||||
" type",
|
||||
")",
|
||||
"VALUES (",
|
||||
" '{",
|
||||
" \"customer_snapshot\":{",
|
||||
" \"first_name\":\"John\",",
|
||||
" \"id\":\"00000000-0000-0000-0000-000000000000\",",
|
||||
" \"type\":\"person\"",
|
||||
" },",
|
||||
" \"internal_note\":\"Confidential\",",
|
||||
" \"related_rules\":[",
|
||||
" {",
|
||||
" \"id\":\"11111111-1111-1111-1111-111111111111\"",
|
||||
" }",
|
||||
" ]",
|
||||
" }',",
|
||||
" 'INV-1001',",
|
||||
" NULL,",
|
||||
" '{{uuid}}',",
|
||||
" 'invoice'",
|
||||
")"
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
data[0]['tests'][-1] = new_test
|
||||
save_json('fixtures/merger.json', data)
|
||||
|
||||
def fix_queryer():
|
||||
data = load_json('fixtures/queryer.json')
|
||||
last_test = data[0]['tests'][-1]
|
||||
|
||||
if "name" in last_test and last_test["name"] == "Query invoice with complex JSONB metadata field extraction":
|
||||
new_test = {
|
||||
"description": last_test["name"],
|
||||
"action": "query",
|
||||
"schema_id": last_test["schema"],
|
||||
"expect": {
|
||||
"success": True,
|
||||
"sql": [
|
||||
[
|
||||
"(SELECT jsonb_strip_nulls(jsonb_build_object(",
|
||||
" 'id', invoice_1.id,",
|
||||
" 'metadata', invoice_1.metadata,",
|
||||
" 'number', invoice_1.number,",
|
||||
" 'type', invoice_1.type",
|
||||
"))",
|
||||
"FROM agreego.invoice invoice_1)"
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
data[0]['tests'][-1] = new_test
|
||||
save_json('fixtures/queryer.json', data)
|
||||
|
||||
fix_merger()
|
||||
fix_queryer()
|
||||
@ -515,7 +515,7 @@
|
||||
"type": "string"
|
||||
},
|
||||
"email": {
|
||||
"$family": "email_address"
|
||||
"family": "email_address"
|
||||
},
|
||||
"generic_bubble": {
|
||||
"type": "some_bubble"
|
||||
@ -664,5 +664,268 @@
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "JSONB boundaries",
|
||||
"database": {
|
||||
"relations": [
|
||||
{
|
||||
"id": "33333333-3333-3333-3333-333333333333",
|
||||
"type": "relation",
|
||||
"constraint": "fk_invoice_line_invoice",
|
||||
"source_type": "invoice_line",
|
||||
"source_columns": [
|
||||
"invoice_id"
|
||||
],
|
||||
"destination_type": "invoice",
|
||||
"destination_columns": [
|
||||
"id"
|
||||
]
|
||||
}
|
||||
],
|
||||
"types": [
|
||||
{
|
||||
"name": "entity",
|
||||
"hierarchy": [
|
||||
"entity"
|
||||
],
|
||||
"grouped_fields": {
|
||||
"entity": [
|
||||
"id",
|
||||
"type",
|
||||
"archived",
|
||||
"created_at"
|
||||
]
|
||||
},
|
||||
"field_types": {
|
||||
"id": "uuid",
|
||||
"archived": "boolean",
|
||||
"created_at": "timestamptz",
|
||||
"type": "text"
|
||||
},
|
||||
"schemas": {
|
||||
"entity": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"format": "uuid"
|
||||
},
|
||||
"type": {
|
||||
"type": "string"
|
||||
},
|
||||
"archived": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"created_at": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"created": {
|
||||
"type": "boolean"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"fields": [
|
||||
"id",
|
||||
"type",
|
||||
"archived",
|
||||
"created_at"
|
||||
],
|
||||
"variations": [
|
||||
"entity",
|
||||
"invoice",
|
||||
"invoice_line"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "invoice",
|
||||
"schemas": {
|
||||
"invoice": {
|
||||
"type": "entity",
|
||||
"properties": {
|
||||
"total": {
|
||||
"type": "number"
|
||||
},
|
||||
"lines": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "invoice_line"
|
||||
}
|
||||
},
|
||||
"metadata_line": {
|
||||
"type": "invoice_line"
|
||||
},
|
||||
"metadata_lines": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "invoice_line"
|
||||
}
|
||||
},
|
||||
"metadata_nested_line": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"line": {
|
||||
"type": "invoice_line"
|
||||
}
|
||||
}
|
||||
},
|
||||
"metadata_nested_lines": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"lines": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "invoice_line"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"hierarchy": [
|
||||
"invoice",
|
||||
"entity"
|
||||
],
|
||||
"fields": [
|
||||
"id",
|
||||
"type",
|
||||
"total",
|
||||
"metadata_line",
|
||||
"metadata_lines",
|
||||
"metadata_nested_line",
|
||||
"metadata_nested_lines",
|
||||
"created_at",
|
||||
"created_by",
|
||||
"modified_at",
|
||||
"modified_by",
|
||||
"archived"
|
||||
],
|
||||
"grouped_fields": {
|
||||
"invoice": [
|
||||
"id",
|
||||
"type",
|
||||
"total",
|
||||
"metadata_line",
|
||||
"metadata_lines",
|
||||
"metadata_nested_line",
|
||||
"metadata_nested_lines"
|
||||
],
|
||||
"entity": [
|
||||
"id",
|
||||
"type",
|
||||
"created_at",
|
||||
"created_by",
|
||||
"modified_at",
|
||||
"modified_by",
|
||||
"archived"
|
||||
]
|
||||
},
|
||||
"lookup_fields": [
|
||||
"id"
|
||||
],
|
||||
"historical": true,
|
||||
"relationship": false,
|
||||
"field_types": {
|
||||
"id": "uuid",
|
||||
"type": "text",
|
||||
"archived": "boolean",
|
||||
"total": "numeric",
|
||||
"metadata_line": "jsonb",
|
||||
"metadata_lines": "jsonb",
|
||||
"metadata_nested_line": "jsonb",
|
||||
"metadata_nested_lines": "jsonb",
|
||||
"created_at": "timestamptz",
|
||||
"created_by": "uuid",
|
||||
"modified_at": "timestamptz",
|
||||
"modified_by": "uuid"
|
||||
},
|
||||
"variations": [
|
||||
"invoice"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "invoice_line",
|
||||
"schemas": {
|
||||
"invoice_line": {
|
||||
"type": "entity",
|
||||
"properties": {
|
||||
"invoice_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"price": {
|
||||
"type": "number"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"hierarchy": [
|
||||
"invoice_line",
|
||||
"entity"
|
||||
],
|
||||
"fields": [
|
||||
"id",
|
||||
"type",
|
||||
"invoice_id",
|
||||
"price",
|
||||
"created_at",
|
||||
"created_by",
|
||||
"modified_at",
|
||||
"modified_by",
|
||||
"archived"
|
||||
],
|
||||
"grouped_fields": {
|
||||
"invoice_line": [
|
||||
"id",
|
||||
"type",
|
||||
"invoice_id",
|
||||
"price"
|
||||
],
|
||||
"entity": [
|
||||
"id",
|
||||
"type",
|
||||
"created_at",
|
||||
"created_by",
|
||||
"modified_at",
|
||||
"modified_by",
|
||||
"archived"
|
||||
]
|
||||
},
|
||||
"lookup_fields": [],
|
||||
"historical": true,
|
||||
"relationship": false,
|
||||
"field_types": {
|
||||
"id": "uuid",
|
||||
"type": "text",
|
||||
"archived": "boolean",
|
||||
"invoice_id": "uuid",
|
||||
"price": "numeric",
|
||||
"created_at": "timestamptz",
|
||||
"created_by": "uuid",
|
||||
"modified_at": "timestamptz",
|
||||
"modified_by": "uuid"
|
||||
},
|
||||
"variations": [
|
||||
"invoice_line"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"tests": [
|
||||
{
|
||||
"description": "Assert no JSONB paths promoted",
|
||||
"action": "compile",
|
||||
"expect": {
|
||||
"success": true,
|
||||
"schemas": [
|
||||
"entity",
|
||||
"invoice",
|
||||
"invoice_line"
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
2794
fixtures/merger.json
2794
fixtures/merger.json
File diff suppressed because it is too large
Load Diff
@ -331,7 +331,7 @@
|
||||
"table_families": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$family": "widget"
|
||||
"family": "widget"
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -339,7 +339,6 @@
|
||||
}
|
||||
},
|
||||
"tests": [
|
||||
|
||||
{
|
||||
"description": "families mechanically map physical variants directly onto topological uuid array paths",
|
||||
"data": {
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[
|
||||
{
|
||||
"description": "Vertical $family Routing (Across Tables)",
|
||||
"description": "Vertical family Routing (Across Tables)",
|
||||
"database": {
|
||||
"types": [
|
||||
{
|
||||
@ -77,7 +77,7 @@
|
||||
],
|
||||
"schemas": {
|
||||
"family_entity": {
|
||||
"$family": "entity"
|
||||
"family": "entity"
|
||||
}
|
||||
}
|
||||
},
|
||||
@ -150,7 +150,7 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "Matrix $family Routing (Vertical + Horizontal Intersections)",
|
||||
"description": "Matrix family Routing (Vertical + Horizontal Intersections)",
|
||||
"database": {
|
||||
"types": [
|
||||
{
|
||||
@ -226,7 +226,7 @@
|
||||
],
|
||||
"schemas": {
|
||||
"family_light_org": {
|
||||
"$family": "light.organization"
|
||||
"family": "light.organization"
|
||||
}
|
||||
}
|
||||
},
|
||||
@ -278,7 +278,7 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "Horizontal $family Routing (Virtual Variations)",
|
||||
"description": "Horizontal family Routing (Virtual Variations)",
|
||||
"database": {
|
||||
"types": [
|
||||
{
|
||||
@ -319,10 +319,10 @@
|
||||
],
|
||||
"schemas": {
|
||||
"family_widget": {
|
||||
"$family": "widget"
|
||||
"family": "widget"
|
||||
},
|
||||
"family_stock_widget": {
|
||||
"$family": "stock.widget"
|
||||
"family": "stock.widget"
|
||||
}
|
||||
}
|
||||
},
|
||||
@ -636,5 +636,110 @@
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"description": "STI Projections (Lacking Kind Discriminator Definitions)",
|
||||
"database": {
|
||||
"types": [
|
||||
{
|
||||
"name": "widget",
|
||||
"variations": [
|
||||
"widget"
|
||||
],
|
||||
"schemas": {
|
||||
"widget": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"type": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"stock.widget": {
|
||||
"type": "widget",
|
||||
"properties": {
|
||||
"kind": {
|
||||
"type": "string"
|
||||
},
|
||||
"amount": {
|
||||
"type": "integer"
|
||||
}
|
||||
}
|
||||
},
|
||||
"projected.widget": {
|
||||
"type": "widget",
|
||||
"properties": {
|
||||
"alias": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"schemas": {
|
||||
"stock_widget_validation": {
|
||||
"type": "stock.widget"
|
||||
},
|
||||
"projected_widget_validation": {
|
||||
"type": "projected.widget"
|
||||
}
|
||||
}
|
||||
},
|
||||
"tests": [
|
||||
{
|
||||
"description": "stock.widget securely expects kind when configured",
|
||||
"schema_id": "stock_widget_validation",
|
||||
"data": {
|
||||
"type": "widget",
|
||||
"amount": 5
|
||||
},
|
||||
"action": "validate",
|
||||
"expect": {
|
||||
"success": false,
|
||||
"errors": [
|
||||
{
|
||||
"code": "MISSING_KIND",
|
||||
"details": {
|
||||
"path": ""
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "projected.widget seamlessly bypasses kind expectation when excluded from schema",
|
||||
"schema_id": "projected_widget_validation",
|
||||
"data": {
|
||||
"type": "widget",
|
||||
"alias": "Test Projection"
|
||||
},
|
||||
"action": "validate",
|
||||
"expect": {
|
||||
"success": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "projected.widget securely fails if user erroneously provides extra kind property",
|
||||
"schema_id": "projected_widget_validation",
|
||||
"data": {
|
||||
"type": "widget",
|
||||
"alias": "Test Projection",
|
||||
"kind": "projected"
|
||||
},
|
||||
"action": "validate",
|
||||
"expect": {
|
||||
"success": false,
|
||||
"errors": [
|
||||
{
|
||||
"code": "STRICT_PROPERTY_VIOLATION",
|
||||
"details": {
|
||||
"path": "kind"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
File diff suppressed because it is too large
Load Diff
19
scratch.rs
19
scratch.rs
@ -1,19 +0,0 @@
|
||||
use cellular_jspg::database::{Database, object::SchemaTypeOrArray};
|
||||
use cellular_jspg::tests::fixtures::get_queryer_db;
|
||||
|
||||
fn main() {
|
||||
let db_json = get_queryer_db();
|
||||
let db = Database::from_json(&db_json).unwrap();
|
||||
let keys: Vec<_> = db.schemas.keys().collect();
|
||||
println!("Found schemas: {}", keys.len());
|
||||
let mut found = false;
|
||||
for k in keys {
|
||||
if k.contains("email_addresses") {
|
||||
println!("Contains email_addresses: {}", k);
|
||||
found = true;
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
println!("No email_addresses found at all!");
|
||||
}
|
||||
}
|
||||
@ -2,9 +2,9 @@ pub mod edge;
|
||||
pub mod r#enum;
|
||||
pub mod executors;
|
||||
pub mod formats;
|
||||
pub mod object;
|
||||
pub mod page;
|
||||
pub mod punc;
|
||||
pub mod object;
|
||||
pub mod relation;
|
||||
pub mod schema;
|
||||
pub mod r#type;
|
||||
@ -60,7 +60,10 @@ impl Database {
|
||||
db.enums.insert(def.name.clone(), def);
|
||||
}
|
||||
Err(e) => {
|
||||
let name = item.get("name").and_then(|v| v.as_str()).unwrap_or("unknown");
|
||||
let name = item
|
||||
.get("name")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("unknown");
|
||||
errors.push(crate::drop::Error {
|
||||
code: "DATABASE_ENUM_PARSE_FAILED".to_string(),
|
||||
message: format!("Failed to parse database enum '{}': {}", name, e),
|
||||
@ -81,7 +84,10 @@ impl Database {
|
||||
db.types.insert(def.name.clone(), def);
|
||||
}
|
||||
Err(e) => {
|
||||
let name = item.get("name").and_then(|v| v.as_str()).unwrap_or("unknown");
|
||||
let name = item
|
||||
.get("name")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("unknown");
|
||||
errors.push(crate::drop::Error {
|
||||
code: "DATABASE_TYPE_PARSE_FAILED".to_string(),
|
||||
message: format!("Failed to parse database type '{}': {}", name, e),
|
||||
@ -106,7 +112,10 @@ impl Database {
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
let constraint = item.get("constraint").and_then(|v| v.as_str()).unwrap_or("unknown");
|
||||
let constraint = item
|
||||
.get("constraint")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("unknown");
|
||||
errors.push(crate::drop::Error {
|
||||
code: "DATABASE_RELATION_PARSE_FAILED".to_string(),
|
||||
message: format!("Failed to parse database relation '{}': {}", constraint, e),
|
||||
@ -127,7 +136,10 @@ impl Database {
|
||||
db.puncs.insert(def.name.clone(), def);
|
||||
}
|
||||
Err(e) => {
|
||||
let name = item.get("name").and_then(|v| v.as_str()).unwrap_or("unknown");
|
||||
let name = item
|
||||
.get("name")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("unknown");
|
||||
errors.push(crate::drop::Error {
|
||||
code: "DATABASE_PUNC_PARSE_FAILED".to_string(),
|
||||
message: format!("Failed to parse database punc '{}': {}", name, e),
|
||||
@ -199,7 +211,13 @@ impl Database {
|
||||
pub fn compile(&mut self, errors: &mut Vec<crate::drop::Error>) {
|
||||
let mut harvested = Vec::new();
|
||||
for (id, schema_arc) in &self.schemas {
|
||||
crate::database::schema::Schema::collect_schemas(schema_arc, id, id.clone(), &mut harvested, errors);
|
||||
crate::database::schema::Schema::collect_schemas(
|
||||
schema_arc,
|
||||
id,
|
||||
id.clone(),
|
||||
&mut harvested,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
for (id, schema_arc) in harvested {
|
||||
self.schemas.insert(id, schema_arc);
|
||||
@ -208,11 +226,12 @@ impl Database {
|
||||
self.collect_schemas(errors);
|
||||
|
||||
// Mathematically evaluate all property inheritances, formats, schemas, and foreign key edges topographically over OnceLocks
|
||||
let mut visited = std::collections::HashSet::new();
|
||||
for (id, schema_arc) in &self.schemas {
|
||||
// First compile pass initializes exact structural root_id mapping to resolve DB constraints
|
||||
let root_id = id.split('/').next().unwrap_or(id);
|
||||
schema_arc.as_ref().compile(self, root_id, id.clone(), &mut visited, errors);
|
||||
schema_arc
|
||||
.as_ref()
|
||||
.compile(self, root_id, id.clone(), errors);
|
||||
}
|
||||
}
|
||||
|
||||
@ -224,19 +243,37 @@ impl Database {
|
||||
for type_def in self.types.values() {
|
||||
for (id, schema_arc) in &type_def.schemas {
|
||||
to_insert.push((id.clone(), Arc::clone(schema_arc)));
|
||||
crate::database::schema::Schema::collect_schemas(schema_arc, id, id.clone(), &mut to_insert, errors);
|
||||
crate::database::schema::Schema::collect_schemas(
|
||||
schema_arc,
|
||||
id,
|
||||
id.clone(),
|
||||
&mut to_insert,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
}
|
||||
for punc_def in self.puncs.values() {
|
||||
for (id, schema_arc) in &punc_def.schemas {
|
||||
to_insert.push((id.clone(), Arc::clone(schema_arc)));
|
||||
crate::database::schema::Schema::collect_schemas(schema_arc, id, id.clone(), &mut to_insert, errors);
|
||||
crate::database::schema::Schema::collect_schemas(
|
||||
schema_arc,
|
||||
id,
|
||||
id.clone(),
|
||||
&mut to_insert,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
}
|
||||
for enum_def in self.enums.values() {
|
||||
for (id, schema_arc) in &enum_def.schemas {
|
||||
to_insert.push((id.clone(), Arc::clone(schema_arc)));
|
||||
crate::database::schema::Schema::collect_schemas(schema_arc, id, id.clone(), &mut to_insert, errors);
|
||||
crate::database::schema::Schema::collect_schemas(
|
||||
schema_arc,
|
||||
id,
|
||||
id.clone(),
|
||||
&mut to_insert,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@ -276,10 +313,10 @@ impl Database {
|
||||
all_rels.sort_by(|a, b| a.constraint.cmp(&b.constraint));
|
||||
|
||||
for rel in all_rels {
|
||||
let mut is_forward =
|
||||
p_def.hierarchy.contains(&rel.source_type) && c_def.hierarchy.contains(&rel.destination_type);
|
||||
let is_reverse =
|
||||
p_def.hierarchy.contains(&rel.destination_type) && c_def.hierarchy.contains(&rel.source_type);
|
||||
let mut is_forward = p_def.hierarchy.contains(&rel.source_type)
|
||||
&& c_def.hierarchy.contains(&rel.destination_type);
|
||||
let is_reverse = p_def.hierarchy.contains(&rel.destination_type)
|
||||
&& c_def.hierarchy.contains(&rel.source_type);
|
||||
|
||||
// Structural Cardinality Filtration:
|
||||
// If the schema requires a collection (Array), it is mathematically impossible for a pure
|
||||
|
||||
@ -37,7 +37,7 @@ pub struct SchemaObject {
|
||||
#[serde(rename = "additionalProperties")]
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub additional_properties: Option<Arc<Schema>>,
|
||||
#[serde(rename = "$family")]
|
||||
#[serde(rename = "family")]
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub family: Option<String>,
|
||||
|
||||
@ -154,12 +154,15 @@ pub struct SchemaObject {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub extensible: Option<bool>,
|
||||
|
||||
#[serde(rename = "compiledProperties")]
|
||||
// Contains ALL structural fields perfectly flattened from the ENTIRE Database inheritance tree (e.g. `entity` fields like `id`) as well as local fields hidden inside conditional `cases` blocks.
|
||||
// This JSON exported array gives clients absolute deterministic visibility to O(1) validation and masking bounds without duplicating structural memory.
|
||||
#[serde(rename = "compiledPropertyNames")]
|
||||
#[serde(skip_deserializing)]
|
||||
#[serde(skip_serializing_if = "crate::database::object::is_once_lock_vec_empty")]
|
||||
#[serde(serialize_with = "crate::database::object::serialize_once_lock")]
|
||||
pub compiled_property_names: OnceLock<Vec<String>>,
|
||||
|
||||
// Internal structural representation caching active AST Node maps. Unlike the Go framework counterpart, the JSPG implementation DOES natively include ALL ancestral inheritance boundary schemas because it compiles locally against the raw database graph.
|
||||
#[serde(skip)]
|
||||
pub compiled_properties: OnceLock<BTreeMap<String, Arc<Schema>>>,
|
||||
|
||||
@ -307,7 +310,7 @@ impl SchemaObject {
|
||||
return true;
|
||||
}
|
||||
|
||||
// 2. Implicit table-backed rule: Does its $family boundary map directly to the global database catalog?
|
||||
// 2. Implicit table-backed rule: Does its family boundary map directly to the global database catalog?
|
||||
if let Some(family) = &self.family {
|
||||
let base = family.split('.').next_back().unwrap_or(family);
|
||||
if db.types.contains_key(base) {
|
||||
|
||||
@ -28,21 +28,12 @@ impl Schema {
|
||||
db: &crate::database::Database,
|
||||
root_id: &str,
|
||||
path: String,
|
||||
visited: &mut std::collections::HashSet<String>,
|
||||
errors: &mut Vec<crate::drop::Error>,
|
||||
) {
|
||||
if self.obj.compiled_properties.get().is_some() {
|
||||
return;
|
||||
}
|
||||
|
||||
if let Some(crate::database::object::SchemaTypeOrArray::Single(t)) = &self.obj.type_ {
|
||||
if !crate::database::object::is_primitive_type(t) {
|
||||
if !visited.insert(t.clone()) {
|
||||
return; // Break cyclical resolution
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(format_str) = &self.obj.format {
|
||||
if let Some(fmt) = crate::database::formats::FORMATS.get(format_str.as_str()) {
|
||||
let _ = self
|
||||
@ -79,7 +70,7 @@ impl Schema {
|
||||
if let Some(crate::database::object::SchemaTypeOrArray::Single(t)) = &self.obj.type_ {
|
||||
if !crate::database::object::is_primitive_type(t) {
|
||||
if let Some(parent) = db.schemas.get(t) {
|
||||
parent.as_ref().compile(db, t, t.clone(), visited, errors);
|
||||
parent.as_ref().compile(db, t, t.clone(), errors);
|
||||
if let Some(p_props) = parent.obj.compiled_properties.get() {
|
||||
props.extend(p_props.clone());
|
||||
}
|
||||
@ -113,7 +104,7 @@ impl Schema {
|
||||
for t in types {
|
||||
if !crate::database::object::is_primitive_type(t) {
|
||||
if let Some(parent) = db.schemas.get(t) {
|
||||
parent.as_ref().compile(db, t, t.clone(), visited, errors);
|
||||
parent.as_ref().compile(db, t, t.clone(), errors);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -126,28 +117,49 @@ impl Schema {
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Set the OnceLock!
|
||||
// 3. Add cases conditionally-defined properties recursively
|
||||
if let Some(cases) = &self.obj.cases {
|
||||
for (i, c) in cases.iter().enumerate() {
|
||||
if let Some(child) = &c.when {
|
||||
child.compile(db, root_id, format!("{}/cases/{}/when", path, i), errors);
|
||||
}
|
||||
if let Some(child) = &c.then {
|
||||
child.compile(db, root_id, format!("{}/cases/{}/then", path, i), errors);
|
||||
if let Some(t_props) = child.obj.compiled_properties.get() {
|
||||
props.extend(t_props.clone());
|
||||
}
|
||||
}
|
||||
if let Some(child) = &c.else_ {
|
||||
child.compile(db, root_id, format!("{}/cases/{}/else", path, i), errors);
|
||||
if let Some(e_props) = child.obj.compiled_properties.get() {
|
||||
props.extend(e_props.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Set the OnceLock!
|
||||
let _ = self.obj.compiled_properties.set(props.clone());
|
||||
let mut names: Vec<String> = props.keys().cloned().collect();
|
||||
names.sort();
|
||||
let _ = self.obj.compiled_property_names.set(names);
|
||||
|
||||
// 4. Compute Edges natively
|
||||
let schema_edges = self.compile_edges(db, root_id, &path, visited, &props, errors);
|
||||
// 5. Compute Edges natively
|
||||
let schema_edges = self.compile_edges(db, root_id, &path, &props, errors);
|
||||
let _ = self.obj.compiled_edges.set(schema_edges);
|
||||
|
||||
// 5. Build our inline children properties recursively NOW! (Depth-first search)
|
||||
if let Some(local_props) = &self.obj.properties {
|
||||
for (k, child) in local_props {
|
||||
child.compile(db, root_id, format!("{}/{}", path, k), visited, errors);
|
||||
child.compile(db, root_id, format!("{}/{}", path, k), errors);
|
||||
}
|
||||
}
|
||||
if let Some(items) = &self.obj.items {
|
||||
items.compile(db, root_id, format!("{}/items", path), visited, errors);
|
||||
items.compile(db, root_id, format!("{}/items", path), errors);
|
||||
}
|
||||
if let Some(pattern_props) = &self.obj.pattern_properties {
|
||||
for (k, child) in pattern_props {
|
||||
child.compile(db, root_id, format!("{}/{}", path, k), visited, errors);
|
||||
child.compile(db, root_id, format!("{}/{}", path, k), errors);
|
||||
}
|
||||
}
|
||||
if let Some(additional_props) = &self.obj.additional_properties {
|
||||
@ -155,77 +167,27 @@ impl Schema {
|
||||
db,
|
||||
root_id,
|
||||
format!("{}/additionalProperties", path),
|
||||
visited,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
if let Some(one_of) = &self.obj.one_of {
|
||||
for (i, child) in one_of.iter().enumerate() {
|
||||
child.compile(
|
||||
db,
|
||||
root_id,
|
||||
format!("{}/oneOf/{}", path, i),
|
||||
visited,
|
||||
errors,
|
||||
);
|
||||
child.compile(db, root_id, format!("{}/oneOf/{}", path, i), errors);
|
||||
}
|
||||
}
|
||||
if let Some(arr) = &self.obj.prefix_items {
|
||||
for (i, child) in arr.iter().enumerate() {
|
||||
child.compile(
|
||||
db,
|
||||
root_id,
|
||||
format!("{}/prefixItems/{}", path, i),
|
||||
visited,
|
||||
errors,
|
||||
);
|
||||
child.compile(db, root_id, format!("{}/prefixItems/{}", path, i), errors);
|
||||
}
|
||||
}
|
||||
if let Some(child) = &self.obj.not {
|
||||
child.compile(db, root_id, format!("{}/not", path), visited, errors);
|
||||
child.compile(db, root_id, format!("{}/not", path), errors);
|
||||
}
|
||||
if let Some(child) = &self.obj.contains {
|
||||
child.compile(db, root_id, format!("{}/contains", path), visited, errors);
|
||||
}
|
||||
if let Some(cases) = &self.obj.cases {
|
||||
for (i, c) in cases.iter().enumerate() {
|
||||
if let Some(child) = &c.when {
|
||||
child.compile(
|
||||
db,
|
||||
root_id,
|
||||
format!("{}/cases/{}/when", path, i),
|
||||
visited,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
if let Some(child) = &c.then {
|
||||
child.compile(
|
||||
db,
|
||||
root_id,
|
||||
format!("{}/cases/{}/then", path, i),
|
||||
visited,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
if let Some(child) = &c.else_ {
|
||||
child.compile(
|
||||
db,
|
||||
root_id,
|
||||
format!("{}/cases/{}/else", path, i),
|
||||
visited,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
}
|
||||
child.compile(db, root_id, format!("{}/contains", path), errors);
|
||||
}
|
||||
|
||||
self.compile_polymorphism(db, root_id, &path, errors);
|
||||
|
||||
if let Some(crate::database::object::SchemaTypeOrArray::Single(t)) = &self.obj.type_ {
|
||||
if !crate::database::object::is_primitive_type(t) {
|
||||
visited.remove(t);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Dynamically infers and compiles all structural database relationships between this Schema
|
||||
@ -237,7 +199,6 @@ impl Schema {
|
||||
db: &crate::database::Database,
|
||||
root_id: &str,
|
||||
path: &str,
|
||||
visited: &mut std::collections::HashSet<String>,
|
||||
props: &std::collections::BTreeMap<String, std::sync::Arc<Schema>>,
|
||||
errors: &mut Vec<crate::drop::Error>,
|
||||
) -> std::collections::BTreeMap<String, crate::database::edge::Edge> {
|
||||
@ -263,31 +224,11 @@ impl Schema {
|
||||
}
|
||||
}
|
||||
|
||||
if parent_type_name.is_none() {
|
||||
// 3. Absolute fallback for anonymous inline structures
|
||||
let base_type_name = root_id
|
||||
.split('.')
|
||||
.next_back()
|
||||
.unwrap_or(root_id)
|
||||
.to_string();
|
||||
if db.types.contains_key(&base_type_name) {
|
||||
parent_type_name = Some(base_type_name);
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(p_type) = parent_type_name {
|
||||
// Proceed only if the resolved table physically exists within the Postgres Type hierarchy
|
||||
if let Some(type_def) = db.types.get(&p_type) {
|
||||
// Iterate over all discovered schema boundaries mapped inside the object
|
||||
for (prop_name, prop_schema) in props {
|
||||
if let Some(field_types_map) = type_def.field_types.as_ref().and_then(|v| v.as_object()) {
|
||||
if let Some(pg_type) = field_types_map.get(prop_name).and_then(|v| v.as_str()) {
|
||||
if pg_type == "json" || pg_type == "jsonb" {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut child_type_name = None;
|
||||
let mut target_schema = prop_schema.clone();
|
||||
let mut is_array = false;
|
||||
@ -325,16 +266,23 @@ impl Schema {
|
||||
}
|
||||
|
||||
if let Some(c_type) = child_type_name {
|
||||
// Skip edge compilation for JSONB columns — they store data inline, not relationally.
|
||||
// The physical column type from field_types is the single source of truth.
|
||||
if let Some(ft) = type_def
|
||||
.field_types
|
||||
.as_ref()
|
||||
.and_then(|v| v.get(prop_name.as_str()))
|
||||
.and_then(|v| v.as_str())
|
||||
{
|
||||
if ft == "jsonb" {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if db.types.contains_key(&c_type) {
|
||||
// Ensure the child Schema's AST has accurately compiled its own physical property keys so we can
|
||||
// inject them securely for Many-to-Many Twin Deduction disambiguation matching.
|
||||
target_schema.compile(
|
||||
db,
|
||||
root_id,
|
||||
format!("{}/{}", path, prop_name),
|
||||
visited,
|
||||
errors,
|
||||
);
|
||||
target_schema.compile(db, root_id, format!("{}/{}", path, prop_name), errors);
|
||||
|
||||
if let Some(compiled_target_props) = target_schema.obj.compiled_properties.get() {
|
||||
let keys_for_ambiguity: Vec<String> =
|
||||
compiled_target_props.keys().cloned().collect();
|
||||
@ -574,7 +522,7 @@ impl Schema {
|
||||
}
|
||||
|
||||
if let Some(family) = &schema_arc.obj.family {
|
||||
Self::validate_identifier(family, "$family", root_id, &path, errors);
|
||||
Self::validate_identifier(family, "family", root_id, &path, errors);
|
||||
}
|
||||
|
||||
Self::collect_child_schemas(schema_arc, root_id, path, to_insert, errors);
|
||||
|
||||
@ -347,22 +347,23 @@ impl<'a> Compiler<'a> {
|
||||
child_node.schema = Arc::clone(target_schema);
|
||||
child_node.is_polymorphic_branch = true;
|
||||
|
||||
let val_sql = if disc == "kind" && node.parent_type.is_some() && node.parent_type_aliases.is_some() {
|
||||
let val_sql =
|
||||
if disc == "kind" && node.parent_type.is_some() && node.parent_type_aliases.is_some() {
|
||||
let aliases_arc = node.parent_type_aliases.as_ref().unwrap();
|
||||
let aliases = aliases_arc.as_ref();
|
||||
let p_type = node.parent_type.unwrap();
|
||||
|
||||
|
||||
let select_args = self.compile_select_clause(p_type, aliases, child_node.clone())?;
|
||||
|
||||
|
||||
if select_args.is_empty() {
|
||||
"jsonb_build_object()".to_string()
|
||||
"jsonb_build_object()".to_string()
|
||||
} else {
|
||||
format!("jsonb_build_object({})", select_args.join(", "))
|
||||
format!("jsonb_build_object({})", select_args.join(", "))
|
||||
}
|
||||
} else {
|
||||
} else {
|
||||
let (sql, _) = self.compile_node(child_node)?;
|
||||
sql
|
||||
};
|
||||
};
|
||||
|
||||
case_statements.push(format!(
|
||||
"WHEN {}.{} = '{}' THEN ({})",
|
||||
|
||||
@ -1457,6 +1457,12 @@ fn test_queryer_0_13() {
|
||||
crate::tests::runner::run_test_case(&path, 0, 13).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_queryer_0_14() {
|
||||
let path = format!("{}/fixtures/queryer.json", env!("CARGO_MANIFEST_DIR"));
|
||||
crate::tests::runner::run_test_case(&path, 0, 14).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_polymorphism_0_0() {
|
||||
let path = format!("{}/fixtures/polymorphism.json", env!("CARGO_MANIFEST_DIR"));
|
||||
@ -1559,6 +1565,24 @@ fn test_polymorphism_4_1() {
|
||||
crate::tests::runner::run_test_case(&path, 4, 1).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_polymorphism_5_0() {
|
||||
let path = format!("{}/fixtures/polymorphism.json", env!("CARGO_MANIFEST_DIR"));
|
||||
crate::tests::runner::run_test_case(&path, 5, 0).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_polymorphism_5_1() {
|
||||
let path = format!("{}/fixtures/polymorphism.json", env!("CARGO_MANIFEST_DIR"));
|
||||
crate::tests::runner::run_test_case(&path, 5, 1).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_polymorphism_5_2() {
|
||||
let path = format!("{}/fixtures/polymorphism.json", env!("CARGO_MANIFEST_DIR"));
|
||||
crate::tests::runner::run_test_case(&path, 5, 2).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_not_0_0() {
|
||||
let path = format!("{}/fixtures/not.json", env!("CARGO_MANIFEST_DIR"));
|
||||
@ -3695,6 +3719,12 @@ fn test_database_5_0() {
|
||||
crate::tests::runner::run_test_case(&path, 5, 0).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_database_6_0() {
|
||||
let path = format!("{}/fixtures/database.json", env!("CARGO_MANIFEST_DIR"));
|
||||
crate::tests::runner::run_test_case(&path, 6, 0).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_cases_0_0() {
|
||||
let path = format!("{}/fixtures/cases.json", env!("CARGO_MANIFEST_DIR"));
|
||||
@ -8098,3 +8128,9 @@ fn test_merger_0_13() {
|
||||
let path = format!("{}/fixtures/merger.json", env!("CARGO_MANIFEST_DIR"));
|
||||
crate::tests::runner::run_test_case(&path, 0, 13).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_merger_0_14() {
|
||||
let path = format!("{}/fixtures/merger.json", env!("CARGO_MANIFEST_DIR"));
|
||||
crate::tests::runner::run_test_case(&path, 0, 14).unwrap();
|
||||
}
|
||||
|
||||
@ -95,11 +95,11 @@ fn test_library_api() {
|
||||
"name": { "type": "string" },
|
||||
"target": {
|
||||
"type": "target_schema",
|
||||
"compiledProperties": ["value"]
|
||||
"compiledPropertyNames": ["value"]
|
||||
}
|
||||
},
|
||||
"required": ["name"],
|
||||
"compiledProperties": ["name", "target", "type"],
|
||||
"compiledPropertyNames": ["name", "target", "type"],
|
||||
"compiledEdges": {
|
||||
"target": {
|
||||
"constraint": "fk_test_target",
|
||||
@ -112,7 +112,7 @@ fn test_library_api() {
|
||||
"properties": {
|
||||
"value": { "type": "number" }
|
||||
},
|
||||
"compiledProperties": ["value"]
|
||||
"compiledPropertyNames": ["value"]
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
@ -24,9 +24,6 @@ impl<'a> ValidationContext<'a> {
|
||||
|
||||
if let Some(obj) = self.instance.as_object() {
|
||||
for key in obj.keys() {
|
||||
if key == "type" || key == "kind" {
|
||||
continue; // Reserved keywords implicitly allowed
|
||||
}
|
||||
if !result.evaluated_keys.contains(key) && !self.overrides.contains(key) {
|
||||
result.errors.push(ValidationError {
|
||||
code: "STRICT_PROPERTY_VIOLATION".to_string(),
|
||||
|
||||
@ -54,14 +54,19 @@ impl<'a> ValidationContext<'a> {
|
||||
|
||||
// If the target mathematically declares a horizontal structural STI variation natively
|
||||
if schema_identifier_str.contains('.') {
|
||||
if obj.get("kind").is_none() {
|
||||
result.errors.push(ValidationError {
|
||||
code: "MISSING_KIND".to_string(),
|
||||
message: "Schema mechanically requires horizontal kind discrimination".to_string(),
|
||||
path: self.path.clone(),
|
||||
});
|
||||
} else {
|
||||
result.evaluated_keys.insert("kind".to_string());
|
||||
let requires_kind = self.schema.compiled_properties.get()
|
||||
.map_or(false, |p| p.contains_key("kind"));
|
||||
|
||||
if requires_kind {
|
||||
if obj.get("kind").is_none() {
|
||||
result.errors.push(ValidationError {
|
||||
code: "MISSING_KIND".to_string(),
|
||||
message: "Schema mechanically requires horizontal kind discrimination".to_string(),
|
||||
path: self.path.clone(),
|
||||
});
|
||||
} else {
|
||||
result.evaluated_keys.insert("kind".to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
|
||||
@ -21,7 +21,7 @@ impl<'a> ValidationContext<'a> {
|
||||
if conflicts {
|
||||
result.errors.push(ValidationError {
|
||||
code: "INVALID_SCHEMA".to_string(),
|
||||
message: "$family must be used exclusively without other constraints".to_string(),
|
||||
message: "family must be used exclusively without other constraints".to_string(),
|
||||
path: self.path.to_string(),
|
||||
});
|
||||
return Ok(false);
|
||||
|
||||
24
wipe_test.py
24
wipe_test.py
@ -1,24 +0,0 @@
|
||||
import json
|
||||
|
||||
def load_json(path):
|
||||
with open(path, 'r') as f:
|
||||
return json.load(f)
|
||||
|
||||
def save_json(path, data):
|
||||
with open(path, 'w') as f:
|
||||
json.dump(data, f, indent=4)
|
||||
|
||||
def fix_merger():
|
||||
data = load_json('fixtures/merger.json')
|
||||
last_test = data[0]['tests'][-1]
|
||||
last_test["expect"]["sql"] = []
|
||||
save_json('fixtures/merger.json', data)
|
||||
|
||||
def fix_queryer():
|
||||
data = load_json('fixtures/queryer.json')
|
||||
last_test = data[0]['tests'][-1]
|
||||
last_test["expect"]["sql"] = []
|
||||
save_json('fixtures/queryer.json', data)
|
||||
|
||||
fix_merger()
|
||||
fix_queryer()
|
||||
Reference in New Issue
Block a user