Skip to main content

kapi serialization formats

These are the formats kapi writes for itself: the on-disk shapes it uses to move its own content, memory, terminology, annotations and decisions between commands, machines and people.

They are distinct from the formats kapi reads and writes on your behalf (JSON catalogs, DOCX, Markdown, strings.xml and the rest), which are covered in the format reference. This section is about kapi's own file family.

Three kinds of artifact​

The most useful distinction in this family is between three roles a file can play.

A store is working state. One SQLite database at .kapi/work/store.db, whose tables hold the content memory, the terms, the voice store, the block cache and the unit-state working set, plus the skeleton and parse caches beside it. It is fast, indexed, mutable, and rebuildable: a projection of what the project commits, never its truth. See the project store.

A bundle is a portable serialization. One reviewable, hashable, diff-able file carrying the same content losslessly: a content bundle, a content-memory bundle, a terms bundle, or a whole project archive. Bundles move: they are what you commit, hand over, or ship.

Committed state is an authored or decided record. A terms bundle kept as the project terminology, a voice profile, a project unit-state record. These are the review surface, the files a person reads in a pull request.

The rule this produces​

Artifacts that must be reviewed are committed, and never the database. They share one home: .kapi/ holds the terms bundle, the memory seeds, the voice profile and the unit-state record (state/*.jsonl), and the whole of .kapi/ is committed. Only .kapi/work/, where the store and its caches are derived, is ignored. Nothing in the review surface would survive a fresh clone, or ever appear in a diff, if it were part of the derived store.

If a person needs to review it, it does not belong in a rebuildable index.

The formats​

FormatSuffixkindWhat it carries
Content bundle.kbf.jsonkapi-bundleExtracted blocks and their per-locale targets
Content memory.memory.jsonkapi-memoryMultilingual entries, entities, provenance
Terms.terms.jsonkapi-termsConcepts and their relations
Annotation sidecar.overlays.jsonlheader annotationTypeStand-off annotations anchored into runs
Overlay set.overlays.jsonkapi-overlay-setIn-progress workspace overlay layers
Project archive.kpzkapi-project, kapi-interchangeA whole project, or one interchange task
Project state.jsonl (under .kapi/state/)kapi-stateReview and workflow decisions
Voice profile.yaml(none)Tone, style, vocabulary, examples

Not sure which you want? Choosing a format.

The compound suffix convention​

Every one of these files except the project archive and the voice profile is JSON or JSON Lines, and each portable one keeps a marker segment ahead of the serialization suffix: .kbf.json, .terms.json, .overlays.jsonl. The unit shards are the exception, and the exception proves the rule: they never travel alone, their names are derived from document keys, and the directory holding them (.kapi/state/) already says what they are.

Both halves earn their place. The .json tail keeps jq, GitHub, editors and syntax highlighting treating the file as what it is. The marker segment says which JSON it is, so a directory listing stays readable and a format can be recognised before parsing.

One cost follows: the standard library's filepath.Ext reports .json for every one of them. Extension-driven code goes through the helpers in core/format (Ext, TrimExt, Stem, HasExt), which return the most specific match. A naive filepath.Ext on en-US.kbf.json yields .json, and trimming that leaves a stray .kbf behind.

One version contract, applied per format​

Every format in the family carries a schemaVersion as MAJOR.MINOR and follows the same forward-compatibility rule:

Reject an unknown major. Accept unknown minors of a known major.

Strict on major, best-effort on minor. A file from a future major version is refused with a message naming the version the build speaks, rather than parsed into something half-understood.

The rule is uniform; the code is not. Each format implements the check separately, and the overlay set inside a project archive carries its own schemaVersion, versioned independently of the enclosing package.

Two ways a file says what it is​

Identification is deliberately split, and the two mechanisms answer different questions:

  • kind: the magic string on the document root. Answers what schema is this?, works on content alone, and survives a rename. It is what sniffing uses.
  • The extension: answers what should I expect? before the file is opened, which is what lets a command route a path without reading it.

The two are checked independently, so a file that is not what it claims to be is caught by whichever mechanism notices first: a wrong kind by the reader that owns the envelope, a wrong extension by the command that had to choose a reader before opening anything.

Import identifies the file, or says that it cannot​

kapi memory import and kapi terms import resolve --format auto from the extension, and they refuse to guess, because a wrong guess is silent. The TMX reader handed a file with no translation units returns zero entries, and the CSV reader skips every row it cannot turn into a concept, so a file the importer never actually read reports Imported 0 entries and exits 0, which is indistinguishable from an empty content memory.

An extension the command does not recognise is therefore an error, and one that names the file, the extensions that are supported, and the --format flag that settles the question:

cannot identify seed.dat: ".dat" is not a content-memory extension
(expected .memory.json, .tmx or .tmx.gz); pass --format (bundle, tmx) to say what it is

Export works the other way round. An output path expresses what the caller wants written rather than what an unread file already is, and stdout carries no extension at all, so there auto falls back to the interchange default.

See also​