Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 61d07b3c68 | |||
| d08181e1d7 | |||
| a232bd4727 | |||
| 8865123b1b | |||
| f8bdac3428 | |||
| 99ae2a7e89 | |||
| 53776fc696 | |||
| d43be1def4 |
21
GEMINI.md
21
GEMINI.md
@ -184,23 +184,27 @@ It evaluates as an **Independent Declarative Rules Engine**. Every `Case` block
|
||||
* **`array`**: Homogeneous collection of items matching the `items` schema. Validation is homogeneous, so strictness checking does not apply. `"extensible"` is not applicable at the `array` level (tuple-like `prefixItems` are removed).
|
||||
* **Inheritance Boundaries**: Strictness resets when crossing non-primitive `type` boundaries. A schema extending a strict parent remains strict unless it explicitly overrides with `"extensible": true`.
|
||||
|
||||
### Immutable Properties (`"immutable": true`)
|
||||
To distinguish read-only hydrated endpoint references, computed properties, system-managed timestamps (`created_at`, `modified_at`), or audit fields from writable properties, JSPG introduces the universal `"immutable": true` schema keyword.
|
||||
### Immutable Properties (`"immutable": "always" | "external"`)
|
||||
To distinguish read-only hydrated endpoint references, computed properties, system-managed timestamps (`created_at`, `modified_at`), or audit fields from writable properties, JSPG supports the `"immutable"` property schema attribute, which takes string enum values (`"always"`, `"external"`, or omitted).
|
||||
|
||||
* **Developer Perspective**: Annotate properties in database schemas or trait definitions with `"immutable": true` when the property should be visible on reads (`jspg_query` / `.response`), but rejected or ignored on writes (`jspg_merge` / `.request`).
|
||||
* **Enum Values**:
|
||||
* `"always"`: Property is permanently read-only across all boundaries (e.g. system-managed audit timestamps like `created_at`, `modified_at`, or computed fields).
|
||||
* `"external"`: Property is read-only from external client API requests (`.request` payloads), but can be populated or mutated by internal system operations.
|
||||
* Omitted: Property is fully writable.
|
||||
* **Developer Perspective**: Annotate properties in database schemas or trait definitions with `"immutable": "always"` or `"immutable": "external"` when the property should be visible on reads (`jspg_query` / `.response`), but rejected or ignored on writes (`jspg_merge` / `.request`).
|
||||
```json
|
||||
"properties": {
|
||||
"source": {
|
||||
"family": "lite.organization",
|
||||
"immutable": true,
|
||||
"immutable": "always",
|
||||
"description": "Read-only hydrated member entity summary on a membership edge."
|
||||
}
|
||||
}
|
||||
```
|
||||
* **Behavior Across Pillars**:
|
||||
* **Queryer (`jspg_query`)**: Hydrates and includes `immutable` properties in output read responses without restriction.
|
||||
* **Validator (`jspg_validate`)**: Context-aware. Returns `IMMUTABLE_PROPERTY_VIOLATION` if an `immutable` property is supplied in a write/request payload (schema IDs not ending in `.response`). Response schemas (`.response`) permit `immutable` fields.
|
||||
* **Merger (`jspg_merge`)**: Automatically skips `immutable` properties during object graph merging so client payloads can never mutate database columns or relationship edges.
|
||||
* **Queryer (`jspg_query`)**: Hydrates and includes `"immutable"` properties in output read responses without restriction.
|
||||
* **Validator (`jspg_validate`)**: Context-aware. Rejects specified `"immutable"` properties supplied in write/request payloads. Response schemas (`.response`) permit `"immutable"` fields.
|
||||
* **Merger (`jspg_merge`)**: Automatically skips `"immutable"` properties during object graph merging so client payloads can never mutate database columns or relationship edges.
|
||||
|
||||
### Format Leniency for Empty Strings
|
||||
To simplify frontend form validation, format validators specifically for `uuid`, `date-time`, and `email` explicitly allow empty strings (`""`), treating them as "present but unset".
|
||||
@ -339,7 +343,7 @@ The Merger provides an automated, high-performance graph synchronization engine.
|
||||
* **Factual Creation Dependency Validation**: When `stage_entity` confirms `kind == "create"` via database lookup, `jspg_merge` evaluates `schema.obj.dependencies.get("created")`. If required creation fields are missing from `entity_fields`, `entity_objects`, or `entity_arrays`, `jspg_merge` halts and returns a structured `REQUIRED_FIELD_MISSING` `Drop`.
|
||||
* **Structured Fail-Fast Error Propagation**: Internal merger operations return typed `Result<T, crate::drop::Error>` instances, allowing clean fail-fast `?` traversal and returning `Drop::with_errors(vec![err])` directly at the top-level API boundary on failure.
|
||||
* **Prefix Foreign Key Matching**: Handles scenario where multiple relations point to the same table by using database Foreign Key constraint prefixes (`fk_`). For example, if a schema has `shipping_address` and `billing_address`, the merger resolves against `fk_shipping_address_entity` vs `fk_billing_address_entity` automatically to correctly route object properties.
|
||||
* **Dynamic Deduplication & Lookups**: If a nested object is provided without an `id`, the Merger utilizes custom `lookup_fields` declared directly in the schema registry JSON comments. It validates at setup compile-time that a corresponding unique index exists in PostgreSQL for these fields. When merging, it dynamically builds query predicates for any satisfied `lookup_fields` sets in the entity's type hierarchy (checking child-to-parent hierarchies order-independently and combining satisfied keys with `UNION` queries) to discover the correct UUID to perform an UPDATE, preventing data duplication.
|
||||
* **Dynamic Deduplication & Lookups (`lookup_fields` & `field_defaults`)**: If an entity payload is provided without an `id`, the Merger utilizes custom `lookup_fields` declared directly in the schema metadata (`COMMENT ON TABLE`). When searching for an existing database record (`fetch_entity`), it iterates candidate `lookup_fields` sets across the entity's type hierarchy. For any column in a lookup set that is omitted from the input payload, `fetch_entity` checks for a default value in `field_defaults` (populated from Postgres table column `DEFAULT` expressions during setup). If a default is found (e.g. `start_date: "0001-01-01T00:00:00+00:00"`), `fetch_entity` uses that default value to satisfy the lookup query predicate (`WHERE ... AND start_date = ...`). Crucially, `entity_fields` itself is **never mutated** with defaults during `merge_entity`, ensuring input payloads remain pure, preventing false `modified_at` updates or spurious change notifications, and allowing PostgreSQL to handle column default generation natively server-side on `INSERT`.
|
||||
* **Hierarchical Table Inheritance**: The Punc system uses distributed table inheritance (e.g. `person` inherits `user` inherits `organization` inherits `entity`). The Merger splits the incoming JSON payload and performs atomic row updates across *all* relevant tables in the lineage map.
|
||||
* **The Archive Paradigm**: Data is never deleted in the Punc system. The Merger securely enforces referential integrity by toggling the `archived` Boolean flag on the base `entity` table rather than issuing SQL `DELETE` commands.
|
||||
* **Change Tracking & Reactivity**: The Merger diffs the incoming JSON against the existing database row (utilizing static, `DashMap`-cached `lookup` SELECT string templates). Every detected change is recorded into the `agreego.change` audit table, tracking the user mapping. It then natively uses `pg_notify` to broadcast a completely flat row-level diff out to the Go WebSocket server for O(1) routing.
|
||||
@ -370,6 +374,7 @@ The Queryer transforms Postgres into a pre-compiled Semantic Query Engine, desig
|
||||
* **Multi-Table Branching**: If the Physical Table is a parent to other tables (e.g. `organization` has variations `["organization", "bot", "person"]`), the compiler generates a dynamic `CASE WHEN type = '...' THEN ...` query, expanding into sub-queries for each variation. To ensure safe resolution, the compiler dynamically evaluates correlation boundaries: it attempts standard Relational Edge discovery first. If no explicit relational edge exists (indicating pure Table Inheritance rather than a standard foreign-key graph relationship), it safely invokes a **Table Parity Fallback**. This generates an explicit ID correlation constraint (`AND inner.id = outer.id`), perfectly binding the structural variations back to the parent row to eliminate Cartesian products.
|
||||
* **Single-Table Bypass**: If the Physical Table is a leaf node with only one variation (e.g. `person` has variations `["person"]`), the compiler cleanly bypasses `CASE` generation and compiles a simple `SELECT` across the base table, as all schema extensions (e.g. `light.person`, `full.person`) are guaranteed to reside in the exact same physical row.
|
||||
* **Polymorphic Relation Type Filtering**: When a relationship maps to a polymorphic target with variations, the Queryer compiles an `IN` clause containing all allowed table variations (e.g., `counterparty_type IN ('bot', 'organization', 'person')`) rather than matching the base type literal, ensuring all polymorphic types are loaded correctly.
|
||||
* **Tenant Scoping & ReBAC Filtering**: When `jspg_query` compiles root-level or relationship-level entity queries, the generated SQL queries evaluate within the calling PostgreSQL transaction's session configuration (e.g. `auth.organization_ids`, `auth.roles`). The compiled queries seamlessly interact with PostgreSQL Row Level Security (RLS) policies on base tables (such as `agreego.entity`) and pre-materialized Zanzibar graph edges (`agreego.relationship`), ensuring multi-tenant isolation and $O(1)$ query evaluation without requiring manual WHERE scoping logic across every application punc.
|
||||
* **Static Relation Constraints (Kind Constraints)**: When a relationship (such as a nested object or array) is defined with a schema that constrains a field value statically using a `const` or `enum` keyword (for example, `kind` constrained to `"cover"` in a `cover_attachment`), the Queryer automatically extracts these static assertions during AST compilation. It injects them directly as static filters into the SQL subquery's `WHERE` clause (e.g. `AND attachment.kind = 'cover'`), allowing developers to query pre-filtered subsets of related tables natively through the schema.
|
||||
* **Proxy Schema Dereferencing / Resolution**: To support punc endpoints that return non-polymorphic table-backed shapes (using `type: "full.X"` proxy schemas at the root response level), the Queryer compiler automatically dereferences non-table schema pointers to their target schemas prior to checking the types. This allows the Queryer to correctly resolve the table relationship edges pre-compiled on the full schema, while avoiding polluting the database registry with relations on ad-hoc punc response schemas during setup.
|
||||
|
||||
|
||||
@ -132,6 +132,15 @@
|
||||
"columns": [
|
||||
"name"
|
||||
]
|
||||
},
|
||||
{
|
||||
"table": "relationship",
|
||||
"columns": [
|
||||
"type",
|
||||
"source_id",
|
||||
"target_id",
|
||||
"start_date"
|
||||
]
|
||||
}
|
||||
],
|
||||
"types": [
|
||||
@ -573,6 +582,8 @@
|
||||
"source_type",
|
||||
"target_id",
|
||||
"target_type",
|
||||
"start_date",
|
||||
"end_date",
|
||||
"id",
|
||||
"type",
|
||||
"name",
|
||||
@ -597,7 +608,9 @@
|
||||
"source_id",
|
||||
"source_type",
|
||||
"target_id",
|
||||
"target_type"
|
||||
"target_type",
|
||||
"start_date",
|
||||
"end_date"
|
||||
]
|
||||
},
|
||||
"field_types": {
|
||||
@ -608,6 +621,8 @@
|
||||
"source_type": "text",
|
||||
"target_id": "uuid",
|
||||
"target_type": "text",
|
||||
"start_date": "timestamptz",
|
||||
"end_date": "timestamptz",
|
||||
"name": "text",
|
||||
"created_at": "timestamptz",
|
||||
"created_by": "uuid",
|
||||
@ -620,7 +635,16 @@
|
||||
"properties": {}
|
||||
}
|
||||
},
|
||||
"lookup_fields": [],
|
||||
"lookup_fields": [
|
||||
"type",
|
||||
"source_id",
|
||||
"target_id",
|
||||
"start_date"
|
||||
],
|
||||
"field_defaults": {
|
||||
"start_date": "0001-01-01T00:00:00Z",
|
||||
"end_date": "9999-12-31T23:59:59Z"
|
||||
},
|
||||
"historical": true,
|
||||
"notify": true
|
||||
},
|
||||
@ -1333,10 +1357,10 @@
|
||||
"LEFT JOIN agreego.\"user\" t3 ON t3.id = t1.id",
|
||||
"LEFT JOIN agreego.\"person\" t4 ON t4.id = t1.id",
|
||||
"WHERE",
|
||||
" (\"first_name\" = 'LookupFirst'",
|
||||
" AND \"last_name\" = 'LookupLast'",
|
||||
" AND \"date_of_birth\" = '{{timestamp}}'",
|
||||
" AND \"pronouns\" = 'they/them'))"
|
||||
" (t4.\"first_name\" = 'LookupFirst'",
|
||||
" AND t4.\"last_name\" = 'LookupLast'",
|
||||
" AND t4.\"date_of_birth\" = '{{timestamp}}'",
|
||||
" AND t4.\"pronouns\" = 'they/them'))"
|
||||
],
|
||||
[
|
||||
"INSERT INTO agreego.\"entity\" (",
|
||||
@ -1497,10 +1521,10 @@
|
||||
"LEFT JOIN agreego.\"user\" t3 ON t3.id = t1.id",
|
||||
"LEFT JOIN agreego.\"person\" t4 ON t4.id = t1.id",
|
||||
"WHERE",
|
||||
" (\"first_name\" = 'LookupFirst'",
|
||||
" AND \"last_name\" = 'LookupLast'",
|
||||
" AND \"date_of_birth\" = '{{timestamp}}'",
|
||||
" AND \"pronouns\" = 'they/them'))"
|
||||
" (t4.\"first_name\" = 'LookupFirst'",
|
||||
" AND t4.\"last_name\" = 'LookupLast'",
|
||||
" AND t4.\"date_of_birth\" = '{{timestamp}}'",
|
||||
" AND t4.\"pronouns\" = 'they/them'))"
|
||||
],
|
||||
[
|
||||
"INSERT INTO agreego.\"entity\" (",
|
||||
@ -1663,17 +1687,17 @@
|
||||
"LEFT JOIN agreego.\"user\" t3 ON t3.id = t1.id",
|
||||
"LEFT JOIN agreego.\"person\" t4 ON t4.id = t1.id",
|
||||
"WHERE",
|
||||
" (\"first_name\" = 'LookupFirst'",
|
||||
" AND \"last_name\" = 'LookupLast'",
|
||||
" AND \"date_of_birth\" = '{{timestamp}}'",
|
||||
" AND \"pronouns\" = 'they/them')",
|
||||
" (t4.\"first_name\" = 'LookupFirst'",
|
||||
" AND t4.\"last_name\" = 'LookupLast'",
|
||||
" AND t4.\"date_of_birth\" = '{{timestamp}}'",
|
||||
" AND t4.\"pronouns\" = 'they/them')",
|
||||
"UNION SELECT COALESCE(to_jsonb(t1.*), '{}') || COALESCE(to_jsonb(t2.*), '{}') || COALESCE(to_jsonb(t3.*), '{}') || COALESCE(to_jsonb(t4.*), '{}')",
|
||||
"FROM agreego.\"entity\" t1",
|
||||
"LEFT JOIN agreego.\"organization\" t2 ON t2.id = t1.id",
|
||||
"LEFT JOIN agreego.\"user\" t3 ON t3.id = t1.id",
|
||||
"LEFT JOIN agreego.\"person\" t4 ON t4.id = t1.id",
|
||||
"WHERE",
|
||||
" (\"name\" = 'LookupName'))"
|
||||
" (t3.\"name\" = 'LookupName'))"
|
||||
],
|
||||
[
|
||||
"INSERT INTO agreego.\"entity\" (",
|
||||
@ -1838,10 +1862,10 @@
|
||||
"LEFT JOIN agreego.\"user\" t3 ON t3.id = t1.id",
|
||||
"LEFT JOIN agreego.\"person\" t4 ON t4.id = t1.id",
|
||||
"WHERE",
|
||||
" (\"first_name\" = 'LookupFirst'",
|
||||
" AND \"last_name\" = 'LookupLast'",
|
||||
" AND \"date_of_birth\" = '{{timestamp}}'",
|
||||
" AND \"pronouns\" = 'they/them'))"
|
||||
" (t4.\"first_name\" = 'LookupFirst'",
|
||||
" AND t4.\"last_name\" = 'LookupLast'",
|
||||
" AND t4.\"date_of_birth\" = '{{timestamp}}'",
|
||||
" AND t4.\"pronouns\" = 'they/them'))"
|
||||
],
|
||||
[
|
||||
"INSERT INTO agreego.\"entity\" (",
|
||||
@ -2766,6 +2790,17 @@
|
||||
" '00000000-0000-0000-0000-000000000000'",
|
||||
")"
|
||||
],
|
||||
[
|
||||
"(SELECT COALESCE(to_jsonb(t1.*), '{}') || COALESCE(to_jsonb(t2.*), '{}') || COALESCE(to_jsonb(t3.*), '{}')",
|
||||
"FROM agreego.\"entity\" t1",
|
||||
"LEFT JOIN agreego.\"relationship\" t2 ON t2.id = t1.id",
|
||||
"LEFT JOIN agreego.\"contact\" t3 ON t3.id = t1.id",
|
||||
"WHERE",
|
||||
" (t2.\"type\" = 'contact'",
|
||||
" AND t2.\"source_id\" = '{{uuid:generated_0}}'",
|
||||
" AND t2.\"target_id\" = '{{uuid:generated_1}}'",
|
||||
" AND t2.\"start_date\" = '{{timestamp}}'))"
|
||||
],
|
||||
[
|
||||
"INSERT INTO agreego.\"entity\" (",
|
||||
" \"created_at\",",
|
||||
@ -2887,6 +2922,17 @@
|
||||
" '00000000-0000-0000-0000-000000000000'",
|
||||
")"
|
||||
],
|
||||
[
|
||||
"(SELECT COALESCE(to_jsonb(t1.*), '{}') || COALESCE(to_jsonb(t2.*), '{}') || COALESCE(to_jsonb(t3.*), '{}')",
|
||||
"FROM agreego.\"entity\" t1",
|
||||
"LEFT JOIN agreego.\"relationship\" t2 ON t2.id = t1.id",
|
||||
"LEFT JOIN agreego.\"contact\" t3 ON t3.id = t1.id",
|
||||
"WHERE",
|
||||
" (t2.\"type\" = 'contact'",
|
||||
" AND t2.\"source_id\" = '{{uuid:generated_0}}'",
|
||||
" AND t2.\"target_id\" = '{{uuid:generated_5}}'",
|
||||
" AND t2.\"start_date\" = '{{timestamp}}'))"
|
||||
],
|
||||
[
|
||||
"INSERT INTO agreego.\"entity\" (",
|
||||
" \"created_at\",",
|
||||
@ -3008,6 +3054,17 @@
|
||||
" '00000000-0000-0000-0000-000000000000'",
|
||||
")"
|
||||
],
|
||||
[
|
||||
"(SELECT COALESCE(to_jsonb(t1.*), '{}') || COALESCE(to_jsonb(t2.*), '{}') || COALESCE(to_jsonb(t3.*), '{}')",
|
||||
"FROM agreego.\"entity\" t1",
|
||||
"LEFT JOIN agreego.\"relationship\" t2 ON t2.id = t1.id",
|
||||
"LEFT JOIN agreego.\"contact\" t3 ON t3.id = t1.id",
|
||||
"WHERE",
|
||||
" (t2.\"type\" = 'contact'",
|
||||
" AND t2.\"source_id\" = '{{uuid:generated_0}}'",
|
||||
" AND t2.\"target_id\" = '{{uuid:generated_9}}'",
|
||||
" AND t2.\"start_date\" = '{{timestamp}}'))"
|
||||
],
|
||||
[
|
||||
"INSERT INTO agreego.\"entity\" (",
|
||||
" \"created_at\",",
|
||||
|
||||
@ -260,12 +260,9 @@
|
||||
},
|
||||
"lookup_fields": [],
|
||||
"null_fields": [],
|
||||
"default_fields": [
|
||||
"id",
|
||||
"type",
|
||||
"created_at",
|
||||
"archived"
|
||||
],
|
||||
"field_defaults": {
|
||||
"archived": false
|
||||
},
|
||||
"variations": [
|
||||
"bot",
|
||||
"organization",
|
||||
|
||||
@ -36,7 +36,7 @@ pub struct Type {
|
||||
#[serde(default)]
|
||||
pub null_fields: Vec<String>,
|
||||
#[serde(default)]
|
||||
pub default_fields: Vec<String>,
|
||||
pub field_defaults: IndexMap<String, Value>,
|
||||
pub field_types: Option<Value>,
|
||||
#[serde(default)]
|
||||
pub schemas: IndexMap<String, Arc<Schema>>,
|
||||
|
||||
@ -335,6 +335,8 @@ impl Merger {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
let mut current_org_id = None;
|
||||
if let Some(compiled_props) = schema.obj.compiled_properties.get() {
|
||||
if let Some(org_schema) = compiled_props.get("organization_id") {
|
||||
@ -761,7 +763,10 @@ impl Merger {
|
||||
if !parent_type.lookup_fields.is_empty() {
|
||||
let mut lookup_complete = true;
|
||||
for column in &parent_type.lookup_fields {
|
||||
match entity_fields.get(column) {
|
||||
let val = entity_fields.get(column).or_else(|| {
|
||||
parent_type.field_defaults.get(column)
|
||||
});
|
||||
match val {
|
||||
Some(Value::Null) | None => {
|
||||
lookup_complete = false;
|
||||
break;
|
||||
@ -774,7 +779,7 @@ impl Merger {
|
||||
}
|
||||
}
|
||||
if lookup_complete {
|
||||
lookup_satisfied_keys.push(&parent_type.lookup_fields);
|
||||
lookup_satisfied_keys.push((&parent_type.lookup_fields, parent_type));
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -812,15 +817,21 @@ impl Merger {
|
||||
where_parts.push(format!("t1.id = {}", Self::quote_literal(id)));
|
||||
}
|
||||
|
||||
for lookup_fields in lookup_satisfied_keys {
|
||||
for (lookup_fields, parent_type) in lookup_satisfied_keys {
|
||||
let t_alias = entity_type
|
||||
.hierarchy
|
||||
.iter()
|
||||
.position(|name| name == &parent_type.name)
|
||||
.map(|idx| format!("t{}", idx + 1))
|
||||
.unwrap_or_else(|| "t1".to_string());
|
||||
|
||||
let mut lookup_predicates = Vec::new();
|
||||
for column in lookup_fields {
|
||||
let val = entity_fields.get(column).unwrap_or(&Value::Null);
|
||||
if column == "type" {
|
||||
lookup_predicates.push(format!("t1.\"{}\" = {}", column, Self::quote_literal(val)));
|
||||
} else {
|
||||
lookup_predicates.push(format!("\"{}\" = {}", column, Self::quote_literal(val)));
|
||||
}
|
||||
let val = entity_fields
|
||||
.get(column)
|
||||
.or_else(|| parent_type.field_defaults.get(column))
|
||||
.unwrap_or(&Value::Null);
|
||||
lookup_predicates.push(format!("{}.\"{}\" = {}", t_alias, column, Self::quote_literal(val)));
|
||||
}
|
||||
where_parts.push(format!("({})", lookup_predicates.join(" AND ")));
|
||||
}
|
||||
|
||||
@ -104,7 +104,7 @@ fn test_library_api() {
|
||||
},
|
||||
"types": {
|
||||
"source_schema": {
|
||||
"default_fields": [],
|
||||
"field_defaults": {},
|
||||
"field_types": null,
|
||||
"fields": [],
|
||||
"grouped_fields": null,
|
||||
@ -169,7 +169,7 @@ fn test_library_api() {
|
||||
"variations": ["source_schema"]
|
||||
},
|
||||
"target_schema": {
|
||||
"default_fields": [],
|
||||
"field_defaults": {},
|
||||
"field_types": null,
|
||||
"fields": [],
|
||||
"grouped_fields": null,
|
||||
|
||||
Reference in New Issue
Block a user