The Kapi format family & the .kpz package

KBF is the native serialization of one content atom, the translatable block. A project owns more than blocks, though: a content memory and a terms store, plus stand-off annotations and media. The Kapi format family gives each of those atoms a native, deterministic, lossless format, and the .kpz package bundles them into one portable artifact.

Two tiers: native vs interchange

For every atom there are two serializations, playing the same two roles KBF and XLIFF play for blocks (see KBF vs XLIFF):

Atom	Native (lossless: pack, cache, hash)	Interchange (lossy: industry handoff)
blocks + targets	KBF, Kapi Bundle Format, .kbf.json	XLIFF / PO
stand-off annotations	KBF annotations, .overlays.jsonl	(none)
content memory	content-memory bundle, .memory.json	TMX
terms store	terms bundle, .terms.json	TBX
whole project	KPZ, Kapi Project Archive, .kpz	(none)

Every native form except .kpz is JSON, and the suffix says so. The marker segment ahead of .json keeps the file self-describing; the .json tail means jq, GitHub, and an editor's syntax highlighting all recognise it without configuration. .kpz keeps a dedicated extension because it is a binary zip.

The native forms round-trip every field of the internal model. The interchange forms map only the standard, industry-portable subset, so they cannot pack a project losslessly:

• TMX drops the entity mappings (including the conceptId cross-link from a content-memory entity to a terms concept), provenance origins and import sessions, per-entry properties, and notes.
• TBX drops the term source (terminology vs brand_vocabulary), the competitorTerm flag, and the extensible properties.

The bundles preserve all of these. They share the KBF discipline: a kind magic string, a MAJOR.MINOR schemaVersion with the same forward-compatibility contract as KBF, and a deterministic serializer (sorted records, no HTML escaping, trailing newline) so the bytes are stable for content hashing and git diffing. The content-memory bundle reuses the same Run model as KBF blocks for its variant content, so inline codes, placeholders, and plural/select constructs serialize identically.

One bundle per store

Each store has a native bundle format:

kapi memory export -o .kapi/memory/cli-nb.memory.json
kapi terms import seeds/terms.json    # format detected from the extension
kapi terms export --format bundle -o brand.terms.json   # or name it explicitly

The explicit spelling is --format bundle for both stores. The spelling json already means something else: kapi terms --format json is a separate, lossy JSON export that does not preserve concept identity. Reach for bundle when you want the lossless form.

A bundle diffs line by line in git, which is what makes a reviewed content memory or terms store reviewable at all. There is no separate compressed spelling: a whole project that needs to travel is a .kpz, and a single bundle that needs to travel small is what zip and gzip are for.

The suffix identifies a bundle wherever it sits: a project puts one wherever it likes and may keep several. An import reads the bundles in the layout it is given (Context files), and an explicit -o path names one outright.

Where a bundle sits when nothing binds it

kapi store import reads .kapi/terms.json by default. A snapshot writes terms.json under the required --out directory. You can commit that snapshot for review; when using .kapi/, adjust its default ignore rules first. See Terms for import locations.

Content memory has no matching single name: a project has one terms bundle but many content-memory bundles, typically one per content surface. This repository's own dogfood keeps a bundle per surface under .kapi/memory/ against a single .kapi/terms.json, and an import reads every one of them.

The .kpz package

A .kpz is a deterministic zip bundling a project's authoritative content, one member per content type: the same content-type set the sync protocol moves over the wire:

project.kpz
├── manifest.json          inventory: per-member sha256 + Merkle rootHash
├── blocks/<id>.kbf.json   KBF              (blocks + targets)
├── annotations/<id>.overlays.jsonl  KBF annotations  (stand-off overlays)
├── memory.json            content memory
├── terms.json             terminology
├── media/<name>           opaque blobs
│
│   ── working-state members (M-06 §5) ──
├── source/<name>          ingested source document(s)
├── overlays.json          in-progress overlays  (targets, annotations, …)
└── history.jsonl          advisory provenance log (opt-in; excluded from rootHash)

The memory and terms members use the conventional bare names, not the compound suffix, so unzipping a package by hand gives you the spelling the rest of the tooling teaches. Member names are cosmetic: unpacking routes each member by the manifest's contentType, never by its filename.

(A .kpz also carries the project recipe in manifest.json (flows, plugins, defaults, content), kept out of the rootHash, so the package is a runnable project in a file. Side-effecting recipe (the venue block, automations:) travels inert.)

The same container serves two profiles, set by the manifest kind:

• kind: kapi-project: the whole project (all locales, full recipe, content memory, terms, overlays, source identity + skeletons). The snapshot / transport parcel, moved by pack / unpack.
• kind: kapi-interchange: a task-scoped bilingual slice for one locale pair (blocks, inline codes, segmentation, skeleton, memory and term context). This is neokapi's interchange format, sent to a translator/reviewer by extract and ingested by merge (see KBF vs XLIFF).

Both are parcels; day-to-day work happens in the ambient .kapi project (M-06 §7).

Because every member is a native lossless format, the whole package is lossless: unpacking can seed a fresh project store (content memory, terms and blocks alike), and the regenerable caches (parse cache, sync hashes) rebuild faithfully from it. The manifest's per-member SHA-256 and Merkle rootHash give the package, and each member, a stable content identity; unpacking verifies both before trusting the contents.

What a package leaves out

A .kpz packs the source of truth and leaves the regenerable index behind. It excludes .kapi/work/store.db and everything under .kapi/work/cache/, which rebuild on demand, and it excludes secrets (the sync claim token). A part with nothing in it is absent. The package is therefore the at-rest twin of the sync wire format: packing is the sync converters writing files instead of protobuf, and unpacking is the inverse.

Raw source bytes are also excluded by default: a .kpz carries each source's identity (path, format, content hash) + round-trip skeleton (enough to merge) but not the originals, so it doesn't duplicate git-tracked source. Pass --with-source to kapi extract / kapi pack to embed the raw bytes too (needed to re-extract offline). And kapi pack refuses to write a content-less .kpz (a project with nothing extracted/translated yet), like git bundle refusing an empty bundle; share the kapi.yaml recipe via git instead.

Redacted packages

A package written under a redaction policy holds a visible placeholder wherever the policy withholds a value, in every member. Its manifest records that, and the ID of the vault that restores it:

"redaction": { "vaultId": "3f9c0a6e1d2b4c5a8e7f6d5c4b3a2918" }

The vault file records the same ID. It never travels inside the package: it stays in the extracting project's .kapi/vault/batches/<vault-id>.json, or beside the package as <name>.kpz.vault for a package extracted from source files. kapi merge restores the values only from the vault whose ID matches, and stops before it writes a file when that vault is absent. The field is metadata, kept out of the rootHash, and a package with no withheld values has none.

Working state: hand-off and resume

A .kpz is an at-rest snapshot of finished content, and it can also carry in-progress working state, so work can stop, move between machines, and resume where it left off. There are two routes, and neither needs per-step CLI verbs:

• .kpz as an ad-hoc workspace (extract / transform / merge). No project required. The .kpz is a portable bundle; the runtime is a persistent shadow cache keyed to the file, so transforms are fast and incremental and the .kpz is rewritten only when you pack (or pass --pack). extract ingests sources (+ a recipe), running a tool/flow on the .kpz transforms it in place, and merge emits the finished files. info shows whether the cache is dirty.

  kapi extract src/*.json -o work.kpz --target-lang fr,qps   # ingest
  kapi translate work.kpz --target-lang fr                # transform (cache)
  kapi info  work.kpz                                         # dirty?
  kapi pack  work.kpz                                         # eject for sharing
  kapi merge work.kpz -o locales/                             # emit → locales/<lang>/<name>

  A small recipe (target locales + output layout) travels with the file, so merge needs no flags. Transforming reuses work already done instead of recomputing it, and each document's work stays isolated.

• Project snapshot (pack / unpack). Inside a .kapi project the working state also includes the project's content memory and terms store. kapi pack snapshots all of it; kapi unpack rehydrates it into another machine's .kapi/ directory. Within a project, re-running a flow also reuses the block tables already in .kapi/work/store.db, so resume there is running again.

  kapi pack   -o snapshot.kpz   # snapshot the project's working state
  kapi unpack snapshot.kpz      # rehydrate elsewhere, then run to resume

The working-state members are source/<name> (the ingested documents) and overlays.json (the in-progress overlays: targets/<locale>, annotations/<name>, segmentation, … keyed by (kind, blockHash)). Overlays in a multi-document package are scoped per source, since block ids are only unique within one document. A recipe block in the manifest (not a content member, so out of the rootHash) carries the workspace's target locales and output layout.

Progress is derived from content

Because the block store is append-only and content-addressed, "has step X run?" is a pure function of the content: does X's overlay exist for the current block hashes? That is what makes cached resume correct: re-running is a no-op where the overlay is already present, and a source change re-hashes its block so only the affected work recomputes. There is no authoritative progress journal; it would be a second source of truth that could drift from the content.

The one optional log is advisory provenance: kapi pack --log stamps a hash-chained line into history.jsonl recording the pack (tamper-evident custody for hand-off; unpack verifies the chain and warns if broken). It is strictly subordinate to content: excluded from the package rootHash, never read to decide anything, and safe to delete with no loss of work. A default pack (no --log) is byte-deterministic.

When to use it

Use a .kpz to move a whole project losslessly: backup and archival, seeding a fresh server or an offline desktop working copy, transferring a project between machines without a server, or building a deterministic test fixture. Use the interchange formats (XLIFF/PO, TMX, TBX) instead when content crosses into the wider translation industry (a translation vendor, an external CAT tool, or a TMS) and the lossy, standards-based subset is what the other side expects.

See also

• M-06: KBF Family and the .kpz Package: the decision and rationale.
• Specification: the KBF block format the family is built around.
• KBF vs XLIFF: the native-vs-interchange split for blocks.
• Reference implementations: memory/kmb (the content-memory bundle), terms/ktb (the terms bundle), kpz (the package container).