Content Model

The content model is the vocabulary every part of neokapi shares. Whatever the input format (JSON, HTML, Markdown, a Word document), a reader turns it into the same handful of types, so tools, flows, content memory, and editors all work against one representation, whatever the format's quirks. The model covers the content inside a document: the blocks you read, check, edit and write back.

By analogy: read it as a streaming DOM (Parts flow past instead of sitting in one tree), where each content node is a record with editions (the one the document is written in, and one per translation, tone or channel) and annotations are margin notes pinned to spans of an unchanged text. Part, Layer, Block, Run, Edition, Overlay and EditionKey are each defined once in Concepts; this page develops them.

Try it: the content model, every way

Pick a lesson and a sample (or drop in your own file) and watch the real kapi reader decompose it in your browser via WebAssembly. Anatomy shows the Layers, Groups, Blocks and Runs, so an HTML <strong> shows up as a paired inline code while a JSON {name} stays literal text. The other lessons show what a Block carries beside its text: segmentation boundaries, terms & findings overlays, the source ↔ target editions, the document structure, and a round-trip that shows only the text changed.

Loading the interactive lab…

The Part is the streaming unit

A document flows through the pipeline as a stream of Parts, the indivisible unit that travels over the channels between stages. Each Part carries a type discriminator and a resource payload: a layer start or end, a block, non-translatable structural data, or media. A reader emits Parts as it parses; tools transform the Parts they care about and relay the rest; a writer reconstructs the document from the stream.

A typical small JSON document with one embedded HTML value produces a stream like this:

The model is built around the Part, not a whole-document tree, because a stream keeps memory bounded and lets stages run concurrently. The mechanics are in Pipeline.

The resource types

The payload a Part carries is one of a few resource types. Together they describe both the content you read, edit, or translate and the structure that surrounds it.

A Block is the centre: each of its editions is a flat run sequence under its key, and its Overlays sit beside the runs.

• Layer is a structural grouping: a whole document, a section, or embedded content. Layers nest. Embedded content (HTML inside a JSON value, CDATA inside XML) becomes a child layer with its own format, so the right reader handles it and inline markup survives at every level.
• Block is the primary modifiable content. Its content is a set of peer editions, each a single flat []Run: the edition the document is written in, the content you read, check, and edit, and one more for each translation, tone or channel. Each edition is a first-class Edition record with its status and provenance, keyed by an EditionKey (a language plus an optional tone and channel); a monolingual pass leaves the block with the one edition it was read in. Native lists the editions the document itself holds: one for a Markdown paragraph, two for a bilingual XLIFF entry. It carries a Translatable flag (a parse-time classification marking content the reader extracted versus inert skeleton), opaque pass-through Properties, and the two stand-off carriers described in Two ways to annotate a block: positional Overlays and block-scoped Annotations.
• Overlay is a typed, run-anchored interpretation of a block's runs: sentence segmentation, terminology, entities, check findings, source↔target alignment. Each overlay is a positional stand-off layer over one edition of the block, kept apart from the runs. A segment is a span in the segmentation overlay, and there is no structural Segment type. Segmentation is therefore opt-in, multi-layer and reversible: drop the overlay and the unsegmented content is back. The segmentation tool writes that overlay from a pluggable engine chosen with --engine: srx (the default SRX 2.0 rule engine), uax29 (the ICU Unicode baseline), llm (semantic chunks), or sat (the wtpsplit ML model, run via the kapi-sat plugin). See Segmentation and F-02.
• Run is one element of a block's inline content: a chunk of text, an opening or closing inline tag, a self-closing placeholder, or a structured plural/select construct (see below).
• Data and Media carry non-translatable document structure and binary content, which flow through so the writer can reconstruct the original byte-for-byte.

Two ways to annotate a block

A block's content is its editions, each a []Run under its key. Every typed interpretation of that content is stand-off, kept separate from the runs, so the same content can carry segmentation, terminology, check findings, notes, and analysis results at once without rewriting it. A block holds stand-off interpretations in two carriers, chosen by whether the interpretation has a position:

• Overlays (Block.Overlays) are positional: each overlay anchors to run ranges. An overlay has a Type, the Edition it sits on (the zero key for the edition the block was read in), an optional Layer (segmentation granularity; "" = the primary sentence segmentation), and a list of Spans. A Span carries a run Range (its position), an ID, optional Props, and a typed payload Value. Because spans anchor to runs, a source rewrite moves them. When a transformer rewrites the runs, the framework applier rebases surviving spans onto the new runs and drops any span that overlaps a rewritten range.
• Annotations (Block.Annotations) are block-scoped: typed metadata keyed by type name, with no position. A source rewrite does not invalidate them. Multiplicity lives inside the value, never in numbered keys: every alternative translation is one AltTranslations collection under the single alt-translation key, not alt-translation-1, -2, and so on.

The built-in stand-off types:

Carrier	Type	Anchored to	Description
Overlay	segmentation	run ranges	sentence / chunk boundaries (per Layer)
Overlay	term	run ranges	matched terminology spans
Overlay	term-candidate	run ranges	proposed terminology awaiting review
Overlay	entity	run ranges	recognized named-entity spans
Overlay	qa	run ranges	quality-check findings
Overlay	alignment	run ranges	links source spans to target spans
Annotation	structure	whole block	logical role (heading, table cell, form field, …), layout layer, level, table-cell spans
Annotation	geometry	whole block	page + bounding box + (per-axis) resolution for content from a rendered medium
Annotation	timing	whole block	time span for content from timed media (audio, video)
Annotation	relations	whole block	typed cross-block edges (caption-of, footnote-of, continues, …)
Annotation	note	whole block	translator / reviewer note
Annotation	alt-translation	whole block	alternative-translation candidates
Annotation	tm-match	whole block	content-memory match metadata
Annotation	word-count	whole block	word-count analysis result
Annotation	char-count	whole block	character-count analysis result
Annotation	seg-count	whole block	segment-count analysis result
Annotation	comparison	whole block	source/target comparison result
Annotation	repetition	whole block	repetition / leverage analysis
Annotation	voice	whole block	voice check result

Both overlay span Values and annotation values are typed payloads registered with one payload registry (RegisterPayload / NewPayload) keyed by type name, so the plugin gRPC bridge and store layers can rehydrate the concrete type on the far side of the wire.

Properties is a separate map for opaque pass-through metadata only, such as connector keys and format round-trip hints. Analytic or interpretive results are overlays or annotations, never properties. A few round-trip hints follow a normalized convention so writers and the editor read them the same way across formats: code.language (a code block's language key), picture.subclass (a chart kind), table.header-kind (an OTSL header's column/row/corner/section role), and the checkbox.checked / field.fillable form-state flags. These are fine structural subtypes that have no typed home on the structure annotation; the canonical keys live in core/model/structure.go.

Runs keep inline markup out of the way

The Run sequence lets a tool, a translation engine or content memory operate on the words while inline markup like <b>, ** or {count} stays intact. Each edition of a block, source and target alike, is a flat []Run, a discriminated union where each run is exactly one of:

Run kind	Field	Represents
Text	Text	a plain text chunk
Placeholder	Ph	a self-closing token (<br/>, <img>, {n})
Paired open	PcOpen	the opening half of a paired code (<b>, <a>)
Paired close	PcClose	the closing half of a paired code (</b>, </a>)
Sub	Sub	a reference to a nested sub-block (subfilter output)
Plural / Select	Plural / Select	a structured ICU construct with per-form runs

Bold text becomes a PcOpen / text / PcClose triple; a <br/> or a variable becomes a single Ph. The original markup is carried in the run's Data field, so the writer can replay it verbatim:

Source HTML: Click <b>here</b> for info

Source runs:
  - {Text: "Click "}
  - {PcOpen:  {ID: "1", Type: "fmt:bold", Data: "<b>"}}
  - {Text: "here"}
  - {PcClose: {ID: "1", Type: "fmt:bold", Data: "</b>"}}
  - {Text: " for info"}

A tool can project the runs to plain text (block.SourceText() returns "Click here for info"); a translation engine sees text with opaque tokens it must preserve; and the writer re-emits each run's Data at its position to reconstruct the source faithfully, attributes and all. Because the same <b>, Markdown **, and DOCX <w:b/> all reduce to a PcOpen/PcClose pair of the same semantic Type, the representation is format-independent. Inline Formatting and Vocabularies cover how runs are classified and what metadata they carry.

See it on a real file

Here kapi parses a small JSON message catalog into blocks, each with an identifier and its source text:

The same parser run against an HTML page shows runs with inline codes (the chips mark the PcOpen/PcClose/Ph runs lifted out of the text):

Reconstruction with skeletons

Blocks are only part of a document; the rest is structure: surrounding tags, whitespace, keys, attributes. A skeleton captures that non-translatable structure interleaved with references to block content, so the writer can rebuild the document exactly, substituting translated content where a target exists and falling back to source where it does not. Round-trip fidelity rests on the skeleton: a file read and written back comes out unchanged, or differs only in the text that changed.

Monolingual content: source only

A block can hold only the edition it was read in for the whole run, and the model works the same way when the source is the only locale in play. A voice or terminology pass takes this path: read a file, check the source content, edit it in place, and write the original back with only the edited text changed.

Take a Markdown file with one off-brand sentence. The reader produces blocks that hold one edition, the one the document is written in:

Block "intro"
  Editions: {en: "Our solution is a game-changing, world-class platform."}
                     // no translation, monolingual

A check reads each block's SourceText(), compares it against a voice profile or the project terms store, and records each problem as a stand-off qa overlay anchored to the offending runs. A check annotates and leaves the text alone (see the immutability model):

Block "intro"
  Editions: {en: "Our solution is a game-changing, world-class platform."}
  Overlays: [{Type: "qa", Range: runs[…], note: "off-voice: 'game-changing, world-class'"}]

An edit then settles the source. A Transform tool (or ksed, or an AI rewrite) returns an edit plan; the framework applier rewrites the runs of the edition the block was read in, in place, and rebases the surviving overlays onto the new runs:

Block "intro"
  Editions: {en: "Our product is a content engine."}
                     // still monolingual

Finally the writer reconstructs the file from the Part stream. Because every untouched block is replayed from its skeleton byte-for-byte, only the one sentence that changed differs in the output. A translation run relies on the same round-trip guarantee.

Related reading

• Formats: the readers and writers that produce and consume the model.
• Inline Formatting and Vocabularies: how inline-code runs are represented and classified.
• Pipeline: how Parts stream through the executor.
• Interface Reference: the concrete Go types and method signatures.
• F-02: Content Model: the design rationale.