Entity OGM Hydration Pattern

Open Mission uses a strict Entity OGM pattern:

Entity class
  + InputSchema
  + StorageSchema
  + hydrated Schema
  + zod-surreal table/field metadata
  -> EntityModelCatalogue.compileModel(...)
  -> SurrealEntityStore read/write
  -> hydrated Entity data with almost no per-read glue code

The important idea is that hydration is not an ad hoc repository trick. It is a structural consequence of how each Entity family is defined.

The Terminal family is a good example, but this document is about the general pattern, not only terminals.

Why This Pattern Exists

Every non-trivial Entity in Open Mission has at least two distinct shapes:

  1. the persisted row shape that belongs in the database
  2. the hydrated Entity shape that callers actually want to work with

If those shapes are not made explicit, the codebase drifts into one of two failures:

  • bloated storage rows that try to store every child and every derived field
  • manual hydration glue scattered across repositories, daemons, and surfaces

The current pattern avoids both failures by freezing three canonical roles:

Layer Canonical owner Purpose
Input <Entity>InputSchema or method payload schemas Validate creation or command input.
Storage <Entity>StorageSchema Define the persisted row and the database-facing metadata.
Hydrated read <Entity>Schema Define the complete Entity boundary shape returned to callers.

The Entity class then becomes the owner of normalization between those shapes.

The Canonical Pieces

Every first-class persisted Entity should line up like this:

Piece What it must do
<Entity>.ts Own behavior, normalization, toStorage(), create/read/find logic, and lifecycle.
<Entity>Schema.ts Own InputSchema, StorageSchema, hydrated Schema, and zod-surreal metadata.
<Entity>Contract.ts Bind methods and events to the canonical schemas.
EntityModelCatalogue Compile one normalized storage model from the Entity’s canonical schemas.
SurrealEntityStore Use the compiled model to read and write without hand-written per-Entity SurrealQL drift.

That split is the point. The class owns behavior, the storage schema owns persistence shape, and the hydrated schema owns the read boundary.

How Hydration Works

The general hydration flow is:

flowchart TD
  A[Caller invokes Entity read or find] --> B[EntityFactory resolves Entity contract]
  B --> C[EntityModelCatalogue provides compiled model]
  C --> D[SurrealEntityStore reads canonical storage row]
  D --> E[Modeled references or computed fields are returned by the store]
  E --> F[Entity class or constructor normalizes storage or hydrated data]
  F --> G[Hydrated <Entity>Schema is returned]

The important detail is that the store does not need custom per-Entity hydration logic when the schema family is configured correctly.

It already knows:

  • which table backs the Entity
  • which fields are storage-backed
  • which fields are references or computed database fields
  • which schema validates the hydrated boundary

That information comes from the Entity schemas plus zod-surreal metadata, not from a second hand-maintained mapping layer.

Schema Responsibilities

1. <Entity>StorageSchema

<Entity>StorageSchema is the persisted row contract.

It should contain:

  • canonical id
  • persisted scalar fields
  • persisted value objects
  • direct reference fields
  • computed relation fields when the database should hydrate them
  • zod-surreal table and field metadata

It should not contain:

  • UI-only view state
  • procedural command descriptors
  • transport-only runtime fields that do not belong in storage

In practice, this means the storage schema is where you put:

  • table(...)
  • persisted field(...) metadata
  • reference metadata such as reference and onDelete
  • computed database hydration metadata such as compute
  • index and analyzer metadata

2. <Entity>Schema

<Entity>Schema is the hydrated read contract.

It should contain the shape that callers want after hydration, not only the raw row. That can include:

  • persisted fields
  • hydrated child Entities
  • computed read fields
  • bounded live runtime state when that state is canonically Entity-owned

The hydrated schema is allowed to be wider than storage. That is the normal case, not a smell.

When the same conceptual field is present in storage and hydrated schemas but the hydrated schema uses a related Entity’s data schema, generic Entity hydration must return the related Entity data shape. For example, a storage-computed recordingEntries field may use TerminalRecordingEntryStorageSchema to describe the database-facing rows, while hydrated TerminalSchema.recordingEntries uses TerminalRecordingEntrySchema. The Entity factory and model catalogue materialize that field through the related Entity data schema. Individual Entity constructors should not need relationship-specific glue to convert storage-shaped related rows into data-shaped related Entities.

Entity instance behavior may also use typed related Entity objects for those hydrated fields. Generic Entity hydration applies schema-declared fields to the Entity instance and may expose same-concept properties such as recordingEntries: TerminalRecordingEntry[]. Those properties are object materialization for behavior and object-oriented navigation; they are not second data shapes and not persisted storage. toData() serializes the current Entity instance back through <Entity>Schema, converting related Entity instance properties to related Entity data. A stored data object, if retained by an implementation, is a rebuildable serialization snapshot/cache rather than canonical live state.

This generic rule applies only to related Entity objects whose data is modeled on the hydrated Entity schema. Runtime capabilities that are not Entity data, such as an AgentAdapter handle, are separate runtime capability properties. They are instantiated by Entity behavior or daemon runtime composition, not by OGM data hydration.

3. <Entity>InputSchema

Input schemas validate creation and method payloads. They are not storage aliases and they are not the hydrated Entity contract.

They exist so the Entity class can:

  • accept narrow input from callers
  • generate the canonical id
  • fill timestamps and defaults
  • persist a validated storage row

Entity Class Responsibilities

The Entity class is what makes the schema split ergonomic instead of verbose.

At minimum, a persisted Entity class should expose:

Class responsibility Why it matters
static entityName Binds the Entity to the contract and factory.
static storageSchema Tells the store which row shape is canonical.
static dataSchema Tells the store which hydrated shape callers expect.
create/read/find methods Keep lifecycle behavior on the owning Entity.
generic instance hydration Accept storage or hydrated data, parse through <Entity>Schema, and apply schema fields to the Entity instance.
related Entity object materialization Expose typed related Entity instance properties for behavior when the hydrated data schema includes related Entity data.
toData() Serialize current Entity instance properties back through <Entity>Schema.
toStorage() Strip hydrated-only fields before persistence.

This is why Open Mission Entity classes can stay object-oriented without becoming storage-heavy. The class owns the behavioral seam, and the schemas own the structural seam.

Read Versus Resolve

The general Entity pattern distinguishes between two closely related but different operations:

Operation Returns Purpose
read(...) <Entity>Type Public contract-facing read that returns serialized hydrated Entity data.
resolve(...) <Entity> instance Internal class helper that loads the hydrated Entity object so behavior can continue on the live Entity instance.

This distinction is architectural, not decorative.

read(...) is the boundary method. It should accept the exact canonical payload already selected by the contract, validate it at ingress, resolve the Entity instance if needed, and return toData().

resolve(...) is an internal object-loading seam. It exists so other class or instance behavior can reuse one authoritative hydration path instead of reimplementing:

  • EntityFactory lookup
  • hydrated schema parsing
  • runtime capability attachment such as registries
  • local not-found handling

resolve(...) must not become a second public read model or a second locator vocabulary. If the public read path is addressed by EntityIdSchema, resolve(...) should accept that same canonical identity shape as its input. The difference is only the return type and the caller intent: read(...) returns boundary data, while resolve(...) returns the live Entity object needed for behavior.

This also means concrete Entity method signatures should usually not default to unknown. Once the contract freezes a precise payload shape, the concrete class should accept the schema-inferred type directly and parse it immediately. unknown is appropriate in generic dispatcher or transport code that has not yet resolved the target contract. It is usually the wrong signature for concrete Entity methods after the payload has already been made canonical.

Terminal As The Example

The Terminal family shows the pattern cleanly.

Terminal storage stays narrow

TerminalStorageSchema persists the row that actually belongs in the terminal table:

  • id
  • name
  • owner
  • workingDirectory
  • command
  • args

That is the durable row.

Terminal hydration is wider than storage

TerminalSchema adds fields callers actually need:

  • recordingEntries
  • connected
  • dead
  • exitCode
  • cols
  • rows
  • screen
  • chunk
  • truncated

Those do not all belong in storage, but they do belong in the hydrated Entity.

Terminal storage schema tells Surreal how to hydrate child rows

The general pattern is visible here:

recordingEntries: z.array(TerminalRecordingEntryStorageSchema).optional().register(field, {
  compute: '<~terminal_recording_entry.*',
  description: 'Ordered Terminal-owned raw PTY recording entries hydrated from TerminalRecordingEntry storage records.'
})

This is the critical seam.

The storage schema says:

  • this field belongs to the database-facing model
  • it is computed rather than directly written
  • it should hydrate from the reverse terminal_recording_entry relation

Then TerminalSchema validates the caller-facing hydrated version of that same field as z.array(TerminalRecordingEntrySchema).

The Terminal class normalizes the two shapes

Terminal.ts then does the class-level work:

  • create a narrow persisted row in createPersisted(...)
  • accept storage or hydrated input in the constructor
  • normalize both through normalizeTerminalData(...)
  • strip hydrated-only fields in toStorage()
  • overlay live TerminalRegistry state onto the hydrated Entity

That is the reusable Entity pattern:

caller input
  -> InputSchema
  -> StorageSchema
  -> persisted row
  -> hydrated Schema
  -> Entity class normalization

What zod-surreal Contributes

@flying-pillow/zod-surreal is the bridge that keeps the storage schema authoritative.

It gives Open Mission these capabilities:

  • register table metadata directly on the canonical storage schema
  • register field, reference, compute, and index metadata on storage fields
  • compile the Entity family into one normalized model
  • generate SurrealQL DDL from that model
  • compile consistent SELECT queries from that same model

This is what removes most of the usual OGM boilerplate. We do not maintain:

  • one Zod schema for application validation
  • another schema for persistence
  • a third mapping file for the database
  • hand-written DDL as the real source of truth

The storage schema does that work once.

The Store-Side Contract

The daemon side follows the same pattern for every Entity family.

EntityModelCatalogue

EntityModelCatalogue registers each contract and compiles one model from:

  • the Entity name
  • the storage schema
  • the hydrated schema

That compiled model becomes the store-facing structural description for the Entity.

SurrealEntityStore

SurrealEntityStore then uses that compiled model to:

  • coerce canonical string ids to Surreal record ids
  • compile select queries
  • know which fields belong to storage
  • know which fields are computed or hydrated
  • return records that the Entity constructor can normalize safely

The result is that most Entity families do not need custom repository hydration code. They need correct schemas and a correct class boundary.

Configuration Rules

When defining a new persisted Entity family, follow these rules.

  1. Put table(...) metadata on <Entity>StorageSchema, not on <Entity>Schema.
  2. Put persisted field metadata on storage fields, even when the hydrated schema also reuses the same concept.
  3. Put relation/reference metadata on storage fields, not in the Entity class.
  4. Use <Entity>Schema for the full caller-facing shape, even when it is wider than storage.
  5. Make the Entity class expose both storageSchema and dataSchema statically.
  6. Make toStorage() return only the persisted row, never the full hydrated view.
  7. Use the constructor or a normalization helper to accept either storage or hydrated data and converge to one in-memory shape.

What This Pattern Does Not Mean

This pattern does not mean every field is hydrated magically from arbitrary runtime state.

Hydration is seamless only when one of these is true:

  • the field is persisted directly in storage
  • the field is modeled as a reference or computed field in the storage schema
  • the Entity class owns the extra normalization step cleanly

Live runtime overlays such as Terminal screen state still need an owning runtime collaborator. The point is that this is additive hydration over a correct storage model, not an excuse to hide ownership.

Practical Checklist

Use this checklist when adding or refactoring an Entity family.

Check Expected answer
Does the Entity have a canonical <Entity>StorageSchema? Yes.
Does the storage schema carry all zod-surreal table and field metadata? Yes.
Does the Entity have a wider <Entity>Schema when callers need hydrated reads? Yes.
Does the class expose static storageSchema and static dataSchema? Yes.
Does toStorage() remove hydrated-only fields? Yes.
Are references and computed relations modeled in schema metadata instead of hand-built in adapters? Yes.
Can the store compile the model without a second mapping layer? Yes.