Content Model

The content model is the vocabulary every part of neokapi shares. Whatever the input format (JSON, HTML, Markdown, a Word document), a reader turns it into the same handful of types, so tools, flows, content memory, and editors all work against one representation rather than against each format's quirks. It is a deliberate, format-independent abstraction over the content inside a document: the unit you read, check, edit, and write back.

By analogy: read it as a streaming DOM (Parts flow past instead of sitting in one tree), where each translatable node is a record with variants (one source, many keyed targets) and annotations are margin notes pinned to spans rather than edits to the text. Part, Layer, Block, Run, Target, Overlay and VariantKey are each defined once in Concepts; this page develops them.

Try it: the content model, every way

Pick a lesson and a sample (or drop in your own file) and watch the real kapi reader decompose it in your browser via WebAssembly. Anatomy shows the Layers, Groups, Blocks and Runs, so an HTML <strong> shows up as a paired inline code while a JSON {name} stays literal text. The other lessons reveal what rides on a Block without touching its text: segmentation boundaries, terms & findings overlays, a variant-keyed source ↔ target, the document structure, and a round-trip that proves only the text changed.

Loading the interactive lab…

The Part is the streaming unit

A document is not loaded as a tree and handed around whole. It flows through the pipeline as a stream of Parts, the indivisible unit that travels over the channels between stages. Each Part carries a type discriminator and a resource payload: a layer start or end, a translatable block, non-translatable structural data, or media. A reader emits Parts as it parses; tools transform the Parts they care about and relay the rest; a writer reconstructs the document from the stream.

A typical small JSON document with one embedded HTML value produces a stream like this:

Streaming is why the model is shaped around a Part rather than a document tree: it keeps memory bounded and lets stages run concurrently. The mechanics are covered in Pipeline.

The resource types

The payload a Part carries is one of a few resource types. Together they describe both the content you read, edit, or translate and the structure that surrounds it.

A Block is the centre: its Source is a flat run sequence, its Targets are keyed by variant, and its Overlays sit beside the runs rather than in them.

• Layer is a structural grouping: a whole document, a section, or embedded content. Layers nest. Embedded content (HTML inside a JSON value, CDATA inside XML) becomes a child layer with its own format, so the right reader handles it and inline markup is preserved at every level rather than being flattened.
• Block is the primary modifiable content unit. Its Source is a single flat []Run, the content you read, check, and edit. When a workflow translates, the results are first-class Target records keyed by a VariantKey (locale plus optional tone and channel); a monolingual pass leaves Targets empty. It carries a Translatable flag (a parse-time classification marking content the reader extracted versus inert skeleton), opaque pass-through Properties, and the two stand-off carriers described in Two ways to annotate a block: positional Overlays and block-scoped Annotations.
• Overlay is a typed, run-anchored interpretation of a block's runs: sentence segmentation, terminology, entities, check findings, source↔target alignment. Each overlay is a positional stand-off layer over one side of the block, layered over the runs rather than baked into the structure. There is no structural Segment type: a segment is just a span in the segmentation overlay, so segmentation is opt-in, multi-layer, and reversible (drop the overlay to get the unsegmented content back). The segmentation tool writes that overlay from a pluggable engine chosen with --engine: srx (the default SRX 2.0 rule engine), uax29 (the ICU Unicode baseline), llm (semantic chunks), or sat (the wtpsplit ML model, run via the kapi-sat plugin). See Segmentation and F-02.
• Run is one element of a block's inline content: a chunk of text, an opening or closing inline tag, a self-closing placeholder, or a structured plural/select construct (see below).
• Data and Media carry non-translatable document structure and binary content, which flow through so the writer can reconstruct the original byte-for-byte.

Two ways to annotate a block

A block's content is just its Source []Run and its variant-keyed Targets. Every typed interpretation of that content is stand-off, kept separate from the runs, so the same content can carry segmentation, terminology, check findings, notes, and analysis results at once without rewriting it. A block holds stand-off interpretations in two carriers, chosen by whether the interpretation has a position:

• Overlays (Block.Overlays) are positional: each overlay anchors to run ranges. An overlay has a Type, an optional Variant (nil = the source side; set = a target variant), an optional Layer (segmentation granularity; "" = the primary sentence segmentation), and a list of Spans. A Span carries a run Range (its position), an ID, optional Props, and a typed payload Value. Because spans anchor to runs, a source rewrite moves them. When a transformer rewrites the runs, the framework applier rebases surviving spans onto the new runs and drops any span that overlaps a rewritten range.
• Annotations (Block.Annotations) are block-scoped: typed metadata keyed by type name, with no position. A source rewrite does not invalidate them. Multiplicity lives inside the value, never in numbered keys: every alternative translation is one AltTranslations collection under the single alt-translation key, not alt-translation-1, -2, and so on.

The built-in stand-off types:

Carrier	Type	Anchored to	Description
Overlay	segmentation	run ranges	sentence / chunk boundaries (per Layer)
Overlay	term	run ranges	matched terminology spans
Overlay	term-candidate	run ranges	proposed terminology awaiting review
Overlay	entity	run ranges	recognized named-entity spans
Overlay	qa	run ranges	quality-check findings
Overlay	alignment	run ranges	links source spans to target spans
Annotation	structure	whole block	logical role (heading, table cell, form field, …), layout layer, level, table-cell spans
Annotation	geometry	whole block	page + bounding box + (per-axis) resolution for content from a rendered medium
Annotation	timing	whole block	time span for content from timed media (audio, video)
Annotation	relations	whole block	typed cross-block edges (caption-of, footnote-of, continues, …)
Annotation	note	whole block	translator / reviewer note
Annotation	alt-translation	whole block	alternative-translation candidates
Annotation	tm-match	whole block	content-memory match metadata
Annotation	word-count	whole block	word-count analysis result
Annotation	char-count	whole block	character-count analysis result
Annotation	seg-count	whole block	segment-count analysis result
Annotation	comparison	whole block	source/target comparison result
Annotation	repetition	whole block	repetition / leverage analysis
Annotation	voice	whole block	voice check result

Both overlay span Values and annotation values are typed payloads registered with one payload registry (RegisterPayload / NewPayload) keyed by type name, so the plugin gRPC bridge and store layers can rehydrate the concrete type on the far side of the wire.

Properties is a separate map for opaque pass-through metadata only, such as connector keys and format round-trip hints. Analytic or interpretive results are overlays or annotations, never properties. A few round-trip hints follow a normalized convention so writers and the editor read them the same way across formats: code.language (a code block's language key), picture.subclass (a chart kind), table.header-kind (an OTSL header's column/row/corner/section role), and the checkbox.checked / field.fillable form-state flags. These are fine structural subtypes that have no typed home on the structure annotation; the canonical keys live in core/model/structure.go.

Runs keep inline markup out of the way

The Run sequence is where neokapi solves a hard problem: how to let a tool, a translation engine, or content memory operate on the words while keeping inline markup like <b>, **, or {count} intact. A block's source (and each target) is a flat []Run, a discriminated union where each run is exactly one of:

Run kind	Field	Represents
Text	Text	a plain text chunk
Placeholder	Ph	a self-closing token (<br/>, <img>, {n})
Paired open	PcOpen	the opening half of a paired code (<b>, <a>)
Paired close	PcClose	the closing half of a paired code (</b>, </a>)
Sub	Sub	a reference to a nested sub-block (subfilter output)
Plural / Select	Plural / Select	a structured ICU construct with per-form runs

Bold text becomes a PcOpen / text / PcClose triple; a <br/> or a variable becomes a single Ph. The original markup is carried in the run's Data field, so the writer can replay it verbatim:

Source HTML: Click <b>here</b> for info

Source runs:
  - {Text: "Click "}
  - {PcOpen:  {ID: "1", Type: "fmt:bold", Data: "<b>"}}
  - {Text: "here"}
  - {PcClose: {ID: "1", Type: "fmt:bold", Data: "</b>"}}
  - {Text: " for info"}

A tool can project the runs to plain text (block.SourceText() returns "Click here for info"); a translation engine sees text with opaque tokens it must preserve; and the writer re-emits each run's Data at its position to reconstruct the source faithfully, attributes and all. Because the same <b>, Markdown **, and DOCX <w:b/> all reduce to a PcOpen/PcClose pair of the same semantic Type, the representation is format-independent. Inline Formatting and Vocabularies cover how runs are classified and what metadata they carry.

See it on a real file

The clearest way to understand the content model is to watch a reader produce it. Below, kapi parses a small JSON message catalog into blocks, each with an identifier and its source text:

The same parser run against an HTML page shows runs with inline codes (the chips mark the PcOpen/PcClose/Ph runs lifted out of the text):

Reconstruction with skeletons

Translatable blocks are only part of a document; the rest is structure: surrounding tags, whitespace, keys, attributes. A skeleton captures that non-translatable structure interleaved with references to block content, so the writer can rebuild the document exactly, substituting translated content where a target exists and falling back to source where it does not. This is what gives neokapi roundtrip fidelity: read a file and write it back unchanged, or write it back with only the changed text differing.

A monolingual path: no targets at all

Translation is the most visible thing the content model carries, but it is not a requirement. A block's Targets map can stay empty for the whole run, and the model works the same way when the only locale in play is the source. This is the path a voice or terminology pass takes: read a file, check the source content, edit it in place, and write the original back with only the edited text changed.

Take a Markdown file with one off-brand sentence. The reader produces blocks whose Source is populated and whose Targets is empty:

Block "intro"
  Source:  "Our solution is a game-changing, world-class platform."
  Targets: {}        // no translation — monolingual

A check reads each block's SourceText(), compares it against a voice profile or the project terms store, and records each problem as a stand-off qa overlay anchored to the offending runs. A check annotates and leaves the text alone (see the immutability model):

Block "intro"
  Source:   "Our solution is a game-changing, world-class platform."
  Overlays: [{Type: "qa", Range: runs[…], note: "off-voice: 'game-changing, world-class'"}]

An edit then settles the source. A Transform tool (or ksed, or an AI rewrite) returns an edit plan; the framework applier rewrites the block's Source runs in place and rebases the surviving overlays onto the new runs:

Block "intro"
  Source:  "Our product is a content engine."
  Targets: {}        // still monolingual

Finally the writer reconstructs the file from the Part stream. Because every untouched block is replayed from its skeleton byte-for-byte, only the one sentence that changed differs in the output. It is the same round-trip guarantee a translation run relies on, with zero targets in sight.

Related reading

• Formats: the readers and writers that produce and consume the model.
• Inline Formatting and Vocabularies: how inline-code runs are represented and classified.
• Pipeline: how Parts stream through the executor.
• Interface Reference: the concrete Go types and method signatures.
• F-02: Content Model: the design rationale.