Gå til hovedinnhold

F-02: The content model

Summary​

Documents are a stream of Part values, each carrying a PartType discriminator and a Resource. Modifiable content is a Block, and a Block's content is a flat []Run per edition: a discriminated union of Text, Ph, PcOpen, PcClose, Sub, Plural, and Select. Each inline-code run carries its own metadata (native markup, abstract identity, semantic type, display text, text equivalent, canonical attributes, editing constraints), so tools process content semantically while writers replay native markup exactly.

Interpretations of that content (sentence segmentation, terms, entities, findings, alignment) are stand-off overlays: typed, run-anchored span sets layered over the runs, never rewriting them. There is no Segment type; a segment is a span in a segmentation overlay. Where a Block is anchored in its source follows the medium: a run range in text, a box on a rendered page, a time span in timed media. Content that was recognized rather than parsed records the extracting engine and a confidence in the same Origin a translation carries. A block's content is a set of peer editions, each a first-class record keyed by a language plus an optional tone or channel.

Context​

The content model has to represent documents in a way that is format-independent, type-safe, extensible, and able to express recursive embedded content naturally. Go's composition and interface system shapes the design toward discriminated unions and explicit resource types rather than deep type hierarchies. Both the Part stream and the inline-content model are discriminated unions: one keyed by PartType, the other by which Run field is set.

Beyond structural representation, real workflows demand stable content identity across extraction cycles, extensible metadata, display hints that guide rendering without coupling the model to a frontend, and a format-independent inline-code model that serves memory matching, model-backed translation, and editor rendering alike.

The inline-code challenge​

Documents contain inline formatting (bold, links, images, variables, placeholders) embedded in the text, and every source format spells these differently:

ConceptHTMLMarkdownDOCX (OpenXML)XLIFF 2.0
Bold<b>**<w:b/><pc type="fmt" subType="xlf:b">
Link<a href="…">[text](url)<w:hyperlink><pc type="link">
Line break<br/>two spaces + newline<w:br/><ph type="fmt" subType="xlf:lb"/>
Placeholdernonenonenone<ph>

Memory matching, translation, verification, and term lookup must not need to know whether the bold came from HTML or Markdown. At the same time round-trip fidelity is non-negotiable: <b class="emphasis"> must round-trip as exactly that, not as a generic bold tag.

The embedded-content challenge​

Documents frequently contain content in a different format: HTML inside JSON values, HTML in XML CDATA, Markdown in spreadsheet columns. A reader that sees only "<p>Hello <b>world</b></p>" as a flat string misses the inline formatting and produces worse results downstream.

Decision​

Part and Resource​

One Part struct carries a PartType enum and a Resource:

type Part struct {
    Type     PartType
    Resource Resource // Block, Layer, Group, Data, or Media
}

type Resource interface {
    ResourceID() string
}

PartType values are PartUnknown (the zero value), PartLayerStart, PartLayerEnd, PartGroupStart, PartGroupEnd, PartBlock, PartData, PartMedia, PartRawDocument, and PartCustom. Constants carry explicit integer values for wire compatibility: numbers are never renumbered, and retired slots stay reserved.

containsone flat sequence per editionanchored to runsnests, embedded contentLayerFormatstringParentIDstringBlockEditionsmap[EditionKey]*EditionNative[]EditionKeyOverlays[]OverlayAnnotationsmap[string]PayloadRunText | Ph | PcOpen | …Overlaytyped span setSpans[]Span

A document is a stream of Parts. Layers nest; a Block holds a flat Run sequence per edition plus stand-off overlays.

Resource types:

  • Layer: structural grouping (document, section, embedded content), delimited by PartLayerStart / PartLayerEnd.
  • Group: a nested structural group within a layer, such as a table, delimited by PartGroupStart / PartGroupEnd.
  • Block: modifiable content: one run sequence per edition (the edition the document is written in, each translation, each tone or channel) and optional stand-off overlays.
  • Data: non-content structure (skeleton, metadata).
  • Media: binary content. A Media is a binary reference, never inlined bytes: resolution precedence is BlobKey (a content-addressed key in a blob store) > URI (an external reference) > Data (inline, for small pipeline-internal assets only). A large raster or media track therefore never streams through the Part channel or the plugin boundary; a single helper at the consuming boundary materializes the bytes. This is the one binary idiom shared by the content model, path-based plugin IO, and model-provider parts (E-07).

PartResult{Part, Error} carries content and errors on the same channel, so a tool's author can choose how to handle an error (skip, retry, fail) without a second channel.

Block​

type Block struct {
    ID           string
    Name         string
    Key          string                  // durable identity, resolved by reconciliation (F-03)
    Type         string
    MimeType     string
    Translatable bool                    // eligible for modification or extraction
    SourceLocale LocaleID
    Editions     map[EditionKey]*Edition // every edition, as peers
    Native       []EditionKey            // the editions the document's bytes hold
    Overlays     []Overlay               // positional stand-off layers, each on one edition
    Annotations  map[string]Payload      // block-scoped typed metadata, keyed by type
    Identity     *BlockIdentity          // content-addressable hashes
    Properties   map[string]string       // opaque pass-through metadata only
    // …skeleton link, content ref, display hint, whitespace flag
}

A Block holds its content as peer editions: the edition the document is written in, and one more for every translation, tone or channel a reader or a tool attached. Each edition is one run sequence: the whole content, unsegmented. There is no Segment container: most blocks are a single string and its translations, and the model says exactly that. When a workflow needs sentence boundaries (a review surface, exact-match memory keys, XLIFF or TMX export), a tool computes them and attaches a segmentation overlay. The overlay layers over the runs; the runs are never repartitioned, so segmentation is reversible by construction.

Translatable is a parse-time classification, not a policy: a reader marks which Parts are authored content and which are surrounding structure. Blocks left unmarked stay in the skeleton, untouched by tools that edit, check, or translate.

Editions and their keys​

An edition is a first-class record carrying status, provenance and the edition it was derived from, keyed by an edition key rather than by locale alone:

// Locale is the only required dimension; tone and channel are optional and
// zero-valued by default, so the common case carries no extra ceremony. The
// zero key names the edition the block was read in.
type EditionKey struct {
    Locale  LocaleID
    Tone    string
    Channel string
}

// Edition is one edition's content with its lifecycle, provenance and
// derivation. Candidate translations (memory hits, machine output, model
// proposals) stay as alt-translation annotations.
type Edition struct {
    Runs    []Run
    Status  Status      // its role's ladder; "" (new) is no committed status
    Origin  Origin      // how the content was produced, and what governed it
    Score   float64
    Derived *Derivation // the edition it was made from, and that edition's revision then
}

Each edition climbs the ladder of its role. The authoritative edition goes from written to established (SourceStatus): a source edit resets it, a clean source check stamps written, and a person's approval stamps established. A derived edition goes from draft through translated to established (TargetStatus).

The edition a block was read in is filed under the zero key, whatever language SourceLocale names, and every other edition under its canonical key, so nb_NO and nb-NO name one. Native lists the editions the document's bytes hold: the edition the block was read in first, under the zero key, then each edition a bilingual reader files from the document, such as the target of an XLIFF unit or each language of an xcstrings entry. An empty list means the document holds the edition it was read in alone. A translation a reader files under no language (a KBF bundle's unlabelled edition) sits apart from the editions: TargetEdition("") reads it and EachTargetEdition yields it under the zero key. Its overlays sit apart with it, because an overlay naming the zero key is on the edition the block was read in: TargetSegmentation(locale) and SetTargetSegmentation(locale, spans) reach a translation's segmentation by the locale the target accessors take, the empty locale included.

One set of accessors reaches every edition by its key: Edition(k), SetEdition(k, e), SetEditionStatus(k, s), RemoveEdition(k), EditionKeys(), EachEdition, NativeEditions() and Authoritative(policy), the edition every other one is derived from. With no policy that is the first native edition, the one the block was read in; a project's policy names the source language its recipe declares. The zero key and the source language name the edition the block was read in, and a same-language edition with a tone or a channel (en;channel=short) is an edition of its own. A bilingual file whose two languages are one (an XLIFF file from en-US to en-US) holds an edition under the source language; that key then reaches it, and the zero key alone reaches the source. SetEdition on the edition a block was read in is an edit, and the block keeps the source as read (SourceAsRead); a status stamp goes through SetEditionStatus, which changes the status alone. EditionKeys() lists the keys in a stable order, and EachEdition visits every edition once without sorting, the one the block was read in first, for loops over many blocks. TargetEdition(locale), SetTargetEdition(k, e) and the locale accessors (TargetText, SetTargetRuns, …) address the editions other than the one the block was read in; there the source language reaches an edition only when the block holds one of its own, and EachTargetEdition visits each under the key it is filed under.

A derived edition's derivation is read and graded from the content: Derivation(k) and SetDerivation(k, d) reach it, and DerivationStanding(k) reads it current while the edition it names still has the revision it records, stale once that edition moves (BasisStanding(d) grades a derivation held elsewhere, such as the basis of a recorded write). A revision covers the key its edition is filed under, and readers of one document file its source under different keys: none, the language the file declares, or a project's source language. SourceRevisions(source) lists the revision of the source under each, the property PropReadSourceLocale keeping a declared language on a block a project read files under the project's, so a basis any reader took is matched by content. BasisStanding matches a derivation from the source the same way: where the edition it names is the one the block was read in, or a language the block holds no edition in, it reads current while any of those revisions is the basis. TranslateAfterLevel.AdmitsDerivation(b, from) is the derivation gate: an edition is derived from from only once from reaches the level on the ladder of its role.

Code outside core/model reads and writes editions through these accessors, so the storage can change inside core/model alone; make fieldguard fails on a use of Editions or Native anywhere else outside a test.

This separates two things a bare map[LocaleID][]Run conflates: the committed edition, with status and provenance, from candidate proposals, of which there may be many per edition, each scored. Accumulated history across runs and review trails are a persistence concern, outside the content model.

CopyEditionSet() copies a block with a set of editions of its own, its own entry for the edition it was read in, and each derived edition shared, so adding or removing an edition on either block leaves the other's set as it was. CopyEditions(copyRuns) also copies the runs of every edition and the source as read.

model.EditionRevision(block, k) names an edition's content: r: and 16 hex digits of the SHA-256 of the edition key and its runs as canonical JSON. Status, origin and the other editions are left out, so a review decision moves no revision. A change made against a read names the revision it read and lands only while the edition still has it.

Content-addressable identity​

type BlockIdentity struct {
    ContentHash string // SHA-256 of the whitespace-trimmed source text
    ContextHash string // SHA-256 of name, type, and identifying properties
}

The same content always produces the same identity, so only blocks whose identity has changed need reprocessing, and identical blocks across documents share a content hash. Key is the durable key that grading the pair against the previous read produces: a decision, a translation and a history entry are filed under it, and a venue stores it as the block's source id. A reader leaves it empty; reconciliation fills it, and Name stands in until it does. How the pair is graded, and how the store key is derived, is F-03: Identity.

Properties​

Properties carries arbitrary key-value metadata that readers, tools, and connectors attach as blocks flow through the pipeline: a source path in an upstream system, a format hint, a structural coordinate. It is opaque pass-through metadata only. Analytic and interpretive results a tool produces (match scores, word counts, findings) are overlays or annotations, not properties, and the tool IO contract (E-03) declares which a tool consumes and produces.

A property whose key begins with @ is advisory: carried on the block but skipped when hashing. Readers use it for derived locators (the line a block starts on, a byte offset): anything that says where to find this rather than what this is. A locator moves whenever anything above it moves, so folding one into the context hash would make an untouched block report as changed every time a blank line was added earlier in the file.

Overlays and annotations: two stand-off carriers​

Typed stand-off interpretations of a Block come in two kinds, kept in two fields because they differ in shape and lifecycle:

  • Overlays are positional: run-anchored span sets that point into the content: segmentation, terms, entities, term candidates, findings, source-to-target alignment. Each Span carries a run Range (its position), optional string Props, and an optional typed payload Value. Because ranges anchor to runs, a rewrite of an edition shifts the spans on that edition, the source's and each target's alike. Every content write goes through the one applier (change.ApplyBlock, E-03), which rebases the spans on the edition it rewrites with model.RemapOverlays. A span follows the text it covers: an edit inside it grows or shrinks it, an edit that replaces all of its text leaves it over the replacement, an edit across one of its boundaries leaves it over the text the edit kept, and it is dropped only when the edits delete everything it covered. A finding states something about the exact text under it, so an edit there drops it. Every span left must resolve in the new runs. A segmentation layer is the edition's whole segment list, which bilingual writers read as such, so it is kept or dropped whole: the segment holding an edit grows or shrinks with it, and an edit across a segment boundary drops the layer. An inline code has no width in the text, so a carried boundary keeps its place among the codes beside it, and a code that ended a segment still ends it. A transforming tool's edit plan supplies the edits; without them the applier uses the one region the old and new text differ in, and an opaque whole-block rewrite drops the edition's overlays.
  • Annotations are block-scoped: a keyed map of typed payloads describing the block as a whole: alt-translations, notes, analysis results, format round-trip state, and the block's anchor and role facets. A source rewrite does not invalidate them.
type Overlay struct {
    Type    OverlayType // segmentation | term | entity | qa | alignment | term-candidate | editor-anchor
    Edition EditionKey  // the edition its spans anchor to; the zero key is the one the block was read in
    Layer   string      // segmentation granularity; LayerPrimary ("") = primary
    Spans   []Span      // each with a run Range, string Props, and a typed Value
}

Whether an interpretation is positional is structural (it is an Overlay) rather than a runtime flag. Annotations are reached through the Anno/SetAnno/DelAnno/AnnoMap helpers, keyed by type; overlays through OverlayOf/AddOverlaySpan/OverlaySpan/RemoveOverlay.

InterpretationCarrierPurpose
Segmentationoverlay segmentationSentence or chunk boundaries over runs
Termsoverlay termMatched terms with their target forms
Term candidatesoverlay term-candidateExtraction candidates awaiting a decision
Entitiesoverlay entityNamed entities (people, places, dates)
Findingsoverlay qaVerification findings, each failing or reporting
Alignmentoverlay alignmentSource-span to target-span links
Alt-translationsannotation alt-translationCandidate translations with scores
Editor anchorsoverlay editor-anchorAn integration binding into the native editor that owns the format (a Word content control, a Figma node), set by a connector rather than a tool

Both overlay span values and annotation values are typed payloads registered with one payload registry (RegisterPayload / NewPayload), so the wire and store layers rehydrate the typed value from its type name.

Tools communicate by reading the overlays and annotations produced upstream and writing their own downstream, staying loosely coupled through the shared data model.

Two annotations, structure and geometry, are held in private Block fields rather than in the map, while every accessor presents them exactly as if they sat in Annotations, so the wire, the stores, and every consumer see no difference. They are set on nearly every structured block: a role on any block a format gives structure to, a position on any block that comes from a grid or a page. A map entry costs around 285 bytes, so a spreadsheet of a million cells was paying a quarter of a gigabyte to hold one enum and four small integers per cell. Nothing else in the vocabulary is common enough to be worth taking out of the map, and the map stays for all of it.

Anchoring across media, and source provenance​

Two things about a Block depend on where its content came from: where it sits in its source, and how its source text was produced. Both are block-scoped typed annotations, riding the same carrier as the analytic results above.

Anchor. A Block's position in its source is a coordinate, and the coordinate system follows the medium:

  • Text anchors by run position: the Anchor every overlay, finding and stand-off annotation already uses. Run anchoring is the text facet of one general idea.
  • Rendered media (an image, a page) anchors spatially: a geometry annotation holding a page and a bounding box.
  • Timed media (audio, video) anchors temporally: a timing annotation recording the time span in milliseconds.

A Block carries whichever facets its medium defines, and they compose: text burned into a video frame carries both a geometry box and a timing span. Alongside anchor, a structure annotation records the block's logical role and plane independent of medium. The role vocabulary (core/model/structure.go) is broad and aligned with document-structure standards: paragraph, title, heading, caption, table cell, code, formula, picture, list, footnote, section roles, and the form-field family. Where a block's name embeds the text of its ancestors, the structure annotation also carries Address, the same structural path with each such segment replaced by that ancestor's own identity, so the block reads the same in every language (F-03). A tool reads only the facet it needs; a Block with no media facet is plain anchored-by-runs text.

The role also disambiguates non-modifiable content surfaced for context. A block is not always modifiable: a reader may emit Block{Translatable: false} carrying a role such as code or formula so a listing or an equation is visible to model and retrieval ingestion while translation skips it. That surfacing is a default-on, per-format opt-out described in E-02: The format system.

Source provenance. A derived edition records its Origin: how the translation was produced (human, agent, memory, mt, ai), the engine, a reference, a timestamp. A Block's source carries the same Origin when it was recognized rather than parsed: an OCR or speech engine, or a model, produced the text, so the Origin names that engine (ocr, asr, or llm-refined for a recognition a multimodal model re-read) and carries a confidence in [0,1]. A block read losslessly from a text format has no source Origin; one produced by recognition does, and a confidence-gated refinement step reads it to decide what to re-examine (M-03). Every edition carries the same Origin record; the source's is held in the block's source-origin annotation (SourceOrigin), which every wire and store carries.

What governed it. Origin records how a target was produced and what governed it: the Profile and ProfileVersion in force, and a ContextFingerprint, a hash of the rendered voice guidance and the terms as they actually reached the producer. That second half cannot be reconstructed after the fact, because a profile is edited in place and terms carry no version of their own, so it is stamped at production time or lost. The fingerprint is a change detector, not a snapshot: it answers "have the governing inputs moved since this was produced?" and nothing more. It is narrower than an engine's config fingerprint, which also moves on a model or prompt change and would make a model swap read as a governance change.

Every non-pseudo producer (model, machine translation, and recycled content memory) stamps it, computed one way (profile.GovernanceContext), so targets written under one context share a fingerprint whichever engine produced them and fall stale together when the context moves. A recycled memory hit resolves its own fingerprint from the context governing the collection at fill time rather than inheriting the matched entry's approving one: a fill asserts the entry as valid under the governance in force now, so it must fall stale when that governance moves. Pseudo-translation takes no governing context and stamps none. See C-05: Freshness and the composite ref.

The Run sequence​

Each edition's content is a flat sequence of Run values (Edition.Runs), with no embedded marker characters:

type Run struct {
    Text    *TextRun        // plain text chunk
    Ph      *PlaceholderRun // self-closing: variable, icon, <br>, redaction
    PcOpen  *PcOpenRun      // opening half of a paired code
    PcClose *PcCloseRun     // closing half of a paired code
    Sub     *SubRun         // reference to a nested Block (subfilter output)
    Plural  *PluralRun      // ICU plural with per-form Runs
    Select  *SelectRun      // ICU select with per-case Runs
}

Exactly one pointer field is set, which is the run's kind; Run.Kind() returns the discriminator and Run.RunID() the id for the kinds that carry one. Intention-revealing constructors (TextR, PhR, and siblings) build each kind without hand-writing the union. Text and inline codes interleave positionally; there is no parallel side-table and no marker characters.

A TextRun marked NoTranslate is text the reader classified as code rather than prose: the contents of a code span, a <kbd>, a <samp>. It stays a run, so positions and round-trip are unaffected, and every translation path leaves it as it is (E-02).

Stand-off overlays are the one interpretation mechanism​

The runs are the content. Everything that interprets the content (where the sentence boundaries fall, which spans are terms or entities, what a check flagged, how a source span aligns to a target span) is a stand-off overlay layered over the run sequence without rewriting it.

The split is by origin. A run records what the source document structurally contained; an overlay records what something concluded about it. So a conclusion never becomes a run, however a format happens to spell it: an XLIFF <mrk type="term"> arrives as a term overlay span rather than as a run kind, and a term this engine locates leaves as a <mrk> the writer draws around runs it does not alter. Because marking is a projection at write time and a reading at parse time, the same content round-trips through a format that carries marks and one that carries none. What a writer will draw is a declared capability (E-02).

type Span struct {
    ID    string            // overlay-local id (e.g. a segment id "s1")
    Range Anchor            // run-anchored, never a flattened-string offset
    Props map[string]string // type-specific, e.g. the "ignorable" marker
    Value Payload           // typed payload from the payload registry
}

Anchor is the one type that says where inside a block something is, and every producer records positions with it: overlays, check findings, and the stand-off annotations an overlay sidecar carries. It addresses one of four things, discriminated by Kind: the whole block, one run by id, a half-open span of characters, or one branch of a plural or select run.

type Anchor struct {
    Kind  AnchorKind // block | run | range | form
    Path  RunPath    // the run sequence addressed; empty is the block's own runs
    RunID string     // for kind run
    Start RunPos     // for kind range, half-open [Start, End)
    End   RunPos
    Key   string     // for kind form: a plural form or a select case
}

// RunPos is a character boundary: an index into a run sequence and a rune
// offset into that run's text.
type RunPos struct{ Run, Offset int }

Two properties make it hold up. Positions are run-relative, so a boundary stays where it was put when a neighbouring run is rewritten and can sit either side of a placeholder, which a flat character offset can do neither of. And they are pathed: a block's content is a tree, so Path walks to the sequence being addressed and a term inside the other form of a plural is addressable rather than approximated by a position in the flattening.

Four properties follow from anchoring interpretations to runs rather than baking them into structure:

  • Segmentation is opt-in and dynamic. Nothing segments by default; a tool computes boundaries and writes a segmentation overlay. Whole-block content is the norm, which is also what document-level translation wants for coherence. The engine is pluggable and selected per run; see M-02: Segmentation.
  • It is reversible by construction. Desegmentation is "drop the overlay." There is no inverse operation to get wrong and no inter-segment material to lose: the gaps between segment spans are runs that no span covers.
  • It is uniform. Terms, entities, and findings are the same kind of overlay, anchored the same run-aware way, rather than each re-detecting boundaries at render time.
  • Several layers coexist. A primary sentence segmentation (LayerPrimary, the zero value, the one bilingual formats project to and from) can sit alongside a token-budgeted chunking layer, each its own overlay distinguished by Layer.

A span marked ignorable in its Props is non-content structural material: an XLIFF <ignorable>, inter-segment whitespace, an ICU plural selector. Tools preserve such a span's target verbatim instead of translating it, and the span still occupies its run range so neighbouring segment positions stay aligned. It is the format-agnostic marker shared by native readers and by plugin-backed ones.

Inline-code runs carry their own metadata​

PlaceholderRun and PcOpenRun carry the full set of inline-code concerns directly on their fields:

type PcOpenRun struct {
    ID          string            // abstract identity; shared with the matching PcClose
    Type        string            // semantic type from the vocabulary ("fmt:bold")
    SubType     string            // format-specific refinement ("html:b", "xlf:b")
    Data        string            // native markup, replayed verbatim by writers
    Equiv       string            // plain-text equivalent ("" for bold, "\n" for <br>)
    Disp        string            // translator-facing label ("[B]")
    Attrs       map[string]string // canonical, format-neutral attributes
    Constraints *RunConstraints   // Deletable, Cloneable, Reorderable
}

PlaceholderRun has the same shape; it is self-closing, so it has no pairing partner. PcCloseRun is leaner: it shares ID with its opener and replays its own Data, and it carries no Constraints, because the closing half inherits its opener's behavior. SubRun references a subblock produced by a subfilter; PluralRun and SelectRun are structured ICU constructs whose branches are themselves []Run.

Three fields double as portable cross-format carriers, so a writer for a format that knows nothing about the source markup can still re-synthesize the construct natively:

  • Attrs holds canonical, format-neutral attributes: href, src, alt, title. A reader populates them from the source markup; a foreign writer emits a Markdown [text](href "title") or an HTML <a href title> from the same values without parsing the source format's literal Data. The set is open; those four are the canonical keys.
  • Equiv and Disp carry a plain-text equivalent and a translator label, and beyond that a format-independent rendering of an opaque payload. Equations use this: the OpenXML reader keeps the OMML on Data, replayed verbatim for a byte-exact round-trip, and places portable LaTeX on Equiv and Disp, so the Markdown writer renders the formula from the carrier (M-04).

Equiv and Disp are excluded from the parity canonical projection, so carrying them never disturbs parity with the reference implementation (A-02).

Semantic types and format-specific refinement​

The Type field draws on a vocabulary: a registered, data-defined type system that maps format-specific markup to format-independent semantic types grouped by namespace prefix: fmt: for formatting, link: for linking, media: for media, struct: for structural constructs, code: for non-modifiable inline tokens, entity: for named entities. A vocabulary entry says what a type means, how it renders and is labeled, and what a translator may do with it. The authoritative catalog of types, and the file format for defining new ones, are the Vocabularies concept page and the Authoring vocabularies guide.

SubType refines a type for one format using a prefix convention: xlf: for XLIFF 2.0 subtypes, html: for element names, md: for Markdown constructs, docx: for OpenXML run properties. Custom subtypes use a reverse-domain prefix (com.acme:custom-tag).

Run ID assignment and pairing​

Readers assign sequential numeric IDs to inline-code runs within each run sequence. A PcOpenRun and the PcCloseRun that closes it share the same ID; a PlaceholderRun gets its own. IDs start at "1", pairs nest last-in-first-out, and runs inside a Plural or Select branch form their own scope. From Click <b>here</b> for <a href="x">info</a>. a reader produces the text run Click , a PcOpen{ID:"1", Type:"fmt:bold"} around here, its matching PcClose{ID:"1"}, then for, a PcOpen{ID:"2", Type:"link:hyperlink"} around info, its PcClose{ID:"2"}, and a final ..

This produces stable structural keys for memory matching: HTML <b>Click</b> and Markdown **Click** both yield {1}Click{/1}, so an entry created from HTML matches a Markdown source at the structural tier.

Run text projections​

A Run sequence is the single source of truth; every textual form that crosses a boundary is a projection computed from []Run on demand. The framework provides, in core/model:

// Plain flattening: TextRun content verbatim, placeholders contribute {equiv},
// paired codes contribute their inner content, plural/select take the 'other'
// branch. Use: word count, search, text comparison.
func FlattenRuns(runs []Run) string

// Structural text: inline-code runs become numbered placeholders ({1}, {/1},
// {2/}). Use: memory exact matching at the structural tier.
func RunsStructuralText(runs []Run) string

// Generalized text: entity placeholder runs become typed placeholders
// ({PERSON}), other inline codes numbered. Use: generalized memory matching.
func RunsGeneralizedText(runs []Run) string

// Markup-preserving render: re-emits each run's captured Data verbatim.
// Use: writers splicing opaque markup back into a string.
func RenderRunsWithData(runs []Run) string

// Placeholder and semantic-HTML forms for prompt and provider boundaries.
func RunsPlaceholderText(runs []Run) string
func RunsSemanticHTML(runs []Run, reg *VocabularyRegistry) string

// Edit text: the placeholder form with each character reference (&amp;,
// &rsquo;) written as its character. ParseRunsEditText reads an edit back and
// restores a reference where the edit kept its character.
func RunsEditText(runs []Run) string

For the run sequence Click , PcOpen{ID:"1", Type:"fmt:bold", Data:"<b class='x'>"}, here, PcClose{ID:"1", Data:"</b>"}, for info:

FlattenRuns():        Click here for info
RunsStructuralText(): Click {1}here{/1} for info
RenderRunsWithData(): Click <b class='x'>here</b> for info

Boundaries: structural inside, textual at the edges​

The inline-code model is structural-canonical. []Run is the single source of truth for a Block's content. Every representation that crosses a boundary (to a translator, a model, a translation provider, an external editing tool, a runtime, a memory index) is a projection computed on demand.

ProjectionSurfaceConsumer
[]Run (no projection)each edition's Runs, KBF wirePipeline tools, stores, readers and writers
RenderRunsWithDatanative source markupFormat writers; replays Data verbatim
RunsStructuralTextClick {1}here{/1} for infoMemory matching, structural tier
RunsGeneralizedTextstructural + entity placeholdersMemory matching, generalized tier
RunsPlaceholderText<x id="1"/>here<x id="/1"/>Prompts where tag preservation is critical
RunsEditTextFish & <x id="1"/>chips<x id="/1"/>The edit loop: kapi inspect, kapi apply, MCP read_blocks and apply_edits
RunsSemanticHTML<a href="…">here</a>Providers and prompts that expect HTML
flattenRuns (TypeScript)Click {=m1}here{/=m1}ICU runtime and the React i18n re-attach
runsToCoded (TypeScript)private-use-area marker text + span infoThe visual editor's styled chips

Two consequences fall out of the convention:

  1. There is no single "translator format." A user editing in the visual editor sees nested chips with semantic formatting; the same Block in an external editing tool arrives as XLIFF <pc>; the same Block sent to a model goes as a placeholder or semantic-HTML projection. The structural Block is identical.
  2. Extensions follow the same rule. A new reader, a new extractor, a new translator surface: each emits []Run and lets the existing projections serve every consumer. A new textual convention is introduced only when an existing projection is genuinely insufficient.

Reader and writer contracts​

Readers populate every field on each inline-code run they emit: a sequential ID shared by a pair; Type and SubType from the vocabulary plus a format-specific refinement; Data as verbatim native markup; Disp as a short human-readable label; Equiv as a plain-text equivalent where one applies; Attrs for the canonical attributes the construct carries; and Constraints derived from format semantics.

Writers reconstruct output from Run.Data, the native markup, not from the semantic type. That is what makes round-trip fidelity a property of the model rather than of each format's implementation: the writer replays exactly what the reader captured. A writer for a different format than the source instead synthesizes from Type plus Attrs.

Layers and embedded content​

Embedded content is modeled as nested Layers. A Layer carries its own format identifier and a ParentID linking it to the enclosing layer. When a reader encounters embedded content (an HTML string inside a JSON value), it emits a child Layer with that format containing the parsed Blocks, nested between the parent Layer's Parts:

PartLayerStart (format="json", id="doc1")
  PartBlock (key: "title", text: "Hello")
  PartLayerStart (format="html", id="sf1", parentID="doc1")
    PartBlock ("Welcome to <b>our site</b>")
  PartLayerEnd (id="sf1")
  PartData (structural JSON)
PartLayerEnd (id="doc1")

Each Layer is independently processable. Layers nest recursively: HTML in JSON in YAML is three levels deep with no special cases.

Subfilter resolution​

Format-to-format embedding is coordinated by a small interface:

type SubfilterResolver interface {
    ResolveReader(formatName string) (DataFormatReader, error)
    ResolveWriter(formatName string) (DataFormatWriter, error)
}

FormatRegistry implements it through its NewReader and NewWriter methods. The interface decouples readers from the registry, prevents circular imports, and allows test doubles. Readers and writers that support subfiltering implement a marker interface, SubfilterAware, and the resolver is injected before Open or Write is called. Any registered format (native, plugin, or bridged) can serve as a subfilter.

Format configs declare subfilter mappings that bind content locations to a format:

subfilters:
  - pattern: "*.body"
    format: html
  - pattern: "*.description"
    format: markdown

Patterns use filepath.Match semantics with . as the separator; JSON readers use key-path globs, XML readers use element-path patterns. When a reader matches a pattern it emits PartLayerStart, delegates to the resolved sub-reader, and emits PartLayerEnd when the sub-reader finishes. Writers buffer the parts between matching Layer boundaries, delegate to the sub-writer, and splice the rendered string into the parent format.

Consequences​

  • Type dispatch via switch part.Type replaces run-time type inspection, and linters provide exhaustiveness checking.
  • Adding a resource type needs only a new PartType constant and a struct implementing Resource.
  • Tools that only handle Blocks ignore every other Part type through the base tool pass-through behavior (E-03).
  • The Part stream stays a single ordered channel; no fan-out complexity in the base pipeline.
  • Content-addressable identity enables incremental extraction and deduplication across documents.
  • Properties, overlays, and annotations let tools carry metadata without content-model changes.
  • The semantic-type abstraction lets memory match across formats and lets prompts receive a consistent inline-code representation.
  • Writers replay Run.Data verbatim, so round-trip fidelity is a property of the model rather than of each format.
  • Layers nest recursively with no special cases; embedded content is a first-class pipeline citizen.
  • The Run union (paired PcOpen/PcClose, self-closing Ph, structured Plural/Select) maps naturally onto XLIFF 2.0's <pc>/<ph> model, making XLIFF serialization a mapping rather than a lossy conversion.
  • Segmentation, terms, entities, and findings share one run-anchored stand-off mechanism; segmentation never mutates content, so it is opt-in, multi-layered, and losslessly reversible.
  • Anchoring is one idea with per-medium facets (run range, geometry, timing), so image, audio, and video content sits in the same Block model as text, and a recognized source carries the same Origin a translation does. Multimodal work adds no new positional primitive.
  • Bilingual interchange formats that carry sentence segments project to and from segmentation plus alignment overlays at the reader and writer boundary, without forcing segment structure into the content model.
  • Editions are first-class records keyed by language, tone and channel, so a block holds a same-language channel edition beside its source without adding ceremony at locale-only call sites.

See also​