Skip to main content

Quality Assurance Checks

Quality assurance in neokapi is a kind of tool: a QA check reads translated blocks, inspects each one against a set of rules, and reports findings without modifying the content. Findings are attached to the block — recorded in its properties and surfaced to the CLI, an editor, or a downstream tool — so a QA pass slots into any flow as an ordinary stage. neokapi offers two complementary approaches: fast, deterministic rule-based checks, and LLM-assisted review.

Run as a gate, these checks behave like tests for AI output: deterministic and repeatable, they read translated content against its source and report exactly what broke — a dropped placeholder, a translated do-not-translate term, a term that drifts from the project's terms store. kapi check runs them over a file or a source/target pair and exits non-zero when the gate fails, so a regression is caught in CI, or in an assistant's fix-loop, the same way a failing test is.

Rule-based checks​

By default — with no --provider — the qa tool runs a battery of deterministic rules over each block, comparing source and target. It needs no API key. It records each finding as a structured issue with a type and a severity (error or warning) and marks whether the block passed. The checks span several concerns:

ConcernExamples of what it catches
WhitespaceLeading/trailing whitespace mismatch, double spaces
CompletenessEmpty target where the source has content, target identical to source
Inline codesMissing or extra inline codes, code order, non-deletable code dropped, non-cloneable code duplicated
PatternsSource patterns (placeholders, numbers) without the expected target counterpart
CharactersCorruption patterns (for example, UTF-8 text decoded as ISO-8859-1)
LengthTarget length outside an allowed ratio of the source, or over an absolute limit
RepetitionConsecutive doubled words in the target

Each check is individually configurable — every rule has a flag, and length checks have thresholds. Because the schema is declared on the tool's config struct, the available options and their defaults are generated into the Tool Reference rather than listed by hand here.

Run it from the CLI against a bilingual file. The command below parses an XLIFF file and reports its findings as JSON:

kapi exec qa app.xliff --target-lang fr --json

In a flow, qa is just another step after translation:

steps:
  - tool: translate
    config: { provider: anthropic }
  - tool: qa
    label: Quality checks

These rule families are all options of the one qa tool: length ratios and absolute character/word limits (--check-max-char-length, --check-absolute-max-char-length, --check-max-words), forbidden or corrupted characters and charset conformance (--forbidden-chars, --required-chars, --check-charset), regex patterns such as printf placeholders (pattern rules in flow config), and the same source translated differently across a batch (--check-target-inconsistency). Terminology has its own validator, term-check. The full set is in the Tool Reference.

LLM-assisted review​

Where rule-based checks catch the mechanical errors, running qa with an LLM --provider uses that LLM provider to assess qualities a rule cannot easily express — fluency, accuracy against the source, and terminology appropriateness — and attaches its assessment to each block. It is the natural companion to translate: the built-in translate-qa flow runs translation and then this review in one pass.

kapi run translate-qa -i app.xliff --target-lang fr

Findings travel with the block​

Both kinds of check use the Block annotation system rather than rewriting text. Rule-based findings are recorded in block properties; LLM findings are attached as an annotation. This is the same shared channel that memory matches, terminology, and brand-voice results use, so a single downstream consumer — a report, an editor view, a CI gate — can read every kind of finding from one place.