Checks

A check reads content, inspects it against a set of rules, and reports findings without modifying it. neokapi runs every kind of verification through one engine: deterministic rule-based checks, terminology enforcement, placeholder and do-not-translate integrity, and voice profile are check families that share one model.

In the CLI, kapi check <path> checks authored content and kapi check --ship enforces project release gates. Checks also run inside kapi up's loop: each pass runs the project's bound checks over what was produced. kapi exec runs a single check tool (qa, term-check, voice-check) on its own. See Understanding the CLI layers.

Checks are tests for AI output

Exact rule checks behave like tests. For fixed input and context they are repeatable, and each reports a specific violation: an over-long string, a forbidden phrase, an off-voice term, a doubled word.

kapi check runs a content-first checkset over any file, with no translation needed, and returns one stable, machine-readable kapi.check/v2 Report: pass, a 0–100 score, and a finding per stable rule id (length.max-chars-exceeded, hygiene.doubled-word, …) anchored to the exact block. The Check reference documents every rule id the source checks report and what fixes each. Each finding either fails or reports, as the rule that raised it says (see What fails). The command exits non-zero when a finding fails, so CI or an AI assistant's fix-loop catches a regression the way it catches a failing test. The fix-loop:

The assistant re-checks after each fix until no finding fails.

Meaning, and guidance that no rule encodes, remain for review.

Bilingual checks (do-not-translate and placeholder integrity, which compare a translated target against its source) are an opt-in: pass kapi check src.json --target src.de.json --target-lang de.

The Report

A completed kapi check run produces a core/check.Report (versioned kapi.check/v2): a summary (findings, failing, reporting and score) and a list of diagnostics, failing ones first and then by rule. Each diagnostic carries a stable rule id, fails (true or false), a human message, an optional suggestion, a location (the block, plus a run-range when the checker pinpointed one), and suggested: true when a rule nobody has confirmed raised it. A finding whose rule names a replacement, such as a term rule's replacement, carries a fix: the replace_text operation of a kapi.change/v1 change set that puts the replacement in place of the words the finding objects to, addressed to the block and naming the revision the check read. Each fix names its words by their place in the text the check read, so the fixes for one block travel as one operation, their edits together; a second operation on the same block would see the text the first one left. The command below applies every fix in a report. kapi refuses a fix whose block changed since the check as stale:

kapi check docs/guide.md --json \
  | jq '{ops: [.findings[] | select(.fix) | .fix] | group_by(.at) | map(.[0] + {edits: [.[].edits[]]})}' \
  | kapi apply -

A fix names its document by its path from the project root, so kapi apply finds it from any directory of the project. Two findings over the same words, such as a term and a longer term that contains it, make one operation with overlapping edits, which is refused as a whole; apply one of them and check again.

Some findings carry no fix. A replacement is plain text, so a finding whose words have an inline code among them (a link, a bold span, a placeholder) has none: replacing the words would delete the code with them. A finding on the file of a translation has none, since the check held that file's words to the source language's rules, and nor has a finding on a code comment or on a draft sent to check_text. Edit these by hand.

The human table's first column is the outcome, FAILS or REPORTS, and the roll-up line reads N failing, M reported · score S/100. The stable rule id is the loop's primary key: an assistant tracks it across iterations to confirm a fix and avoid regressions. --json emits the Report verbatim; over MCP, the check_file and check_text tools return the same Report, the verifier counterpart to the read_blocks/apply_edits editing tools, so an assistant can author → check → revise → re-check without leaving the conversation.

For a draft intended for a project file, pass that destination as context_path to check_text. kapi applies the destination's voice channel and terms even before the file exists. For example:

{"text": "Your draft wording", "context_path": "content/en/page.json"}

The report describes the supplied text and records its intended destination in target.context_path. After saving, use check_file to check the file itself. An unscoped snippet can use profile_file or profile_pack; these explicit profile options cannot be combined with context_path.

The optional execution object records which analyzers ran for each input, which were not requested, and why a capability was unsupported. Each entry has an analyzer ID, execution status, finding count and, when it ran, a duration. The host also reports context, extraction, analysis, report-construction and total time in milliseconds; serialization and process startup are excluded. Reports without execution have unreported coverage.

The report's verdict is passed, failed or did_not_run, and pass is true only for passed. Beside the content, each analyzer is given a known-bad sample, its canary, through the same configured checker, and records what it made of it in execution.analyzers[].canary: caught, missed or impossible. An analyzer that misses its canary has status invalid, and the run did not run, whatever else it reported. An analyzer whose configuration gives it nothing to catch, such as a voice profile with no deterministic rules, has status did_not_run; when the invocation asked for it by name, the run did not run either. A voice profile that holds comment limits and no term or pattern gives voice.rules no rule for a file's comments, and the comment analyzers check them against those limits. voice.rules then has status not_applicable, with the reason, and never counts as a pass, so the comment analyzers decide the verdict. The same profile over content holding no comment did not run. A run that checked no blocks did not run. The top-level did_not_run field lists the reasons. kapi check exits 4 for this verdict, and --no-fail leaves that unchanged.

Every did_not_run verdict carries a did_not_run_cause, and the causes share exit code 4, so read the cause before acting on it:

• checker_invalid: an analyzer reported nothing on its canary. A checker is broken, and nothing this run reported can be trusted, including its findings.
• nothing_to_check: no content was in scope, such as an empty file or a diff that touches no content block.
• content_not_checked: content in scope was not checked, such as a changed file whose blocks could not be located, an analyzer the invocation asked for with nothing to catch, or declared content in a format with no installed reader when the check read nothing else.

execution.contexts records the effective guidance for each checked input. Each entry identifies the file or draft destination, the voice selection (project, override or none), the loaded profile name and source, and the project profile and channel when they governed the check. voice.applied distinguishes a loaded profile from a project location with no voice binding. terms_applied states whether terminology was supplied to the checks; it does not imply that a term matched or produced a finding. Project terminology checks also run when no voice profile is bound. An omitted contexts field means the producer did not report context selection.

When a project resolved the guidance, each entry carries the point it was resolved at: the profile and channel, and comments for the point a file's comments sit at. A file whose comments sit apart from its other content has one entry for each point. Each finding carries the point its block was checked at, and an analyzer that ran once for each point carries its point too.

The optional warnings array names problems in the configuration a check ran under, apart from the findings about content. Each entry has a stable code, a message, the source it came from (a profile file's path, or pack:<name> or store:<name>) and, when the warning concerns one key, its dotted key. A run reports the warnings of each voice profile it loads once, however many files the profile governs:

• voice.unknown_key: a key the voice profile model does not define. The profile loads without it, so whatever the key was meant to state applies nowhere. kapi voice validate refuses the same key.
• voice.unfamiliar_value: a tone value outside the usual set, kept and rendered into the voice guide as written.
• voice.override_drops_pattern: a base style pattern that stops applying where a channel or persona supplies its own style.
• format.no_reader: a file the project declares as content, in a format no installed reader handles, usually one a plugin supplies. source is the file, and the message names the plugin to install. A check over the project's content leaves the file out, checks the rest and prints the same warning on stderr. A check of a directory or a glob does the same for each file it finds, including a file no format claims, such as go.mod. A file named on the command line in such a format fails the check instead.

Warnings never change the summary, the score, the verdict or the exit code. Fix the configuration they name. kapi check prints them after the verdict, kapi check --ship carries the same array, and the MCP check tools return it in their report.

What the run was evaluated against

Check results depend on workspace context as well as repository content. The optional evaluation object records that context and the checker version so results can be interpreted later from logs and reports:

• at: when the run was evaluated, RFC 3339 in UTC, read once per run so every field of the record shares one instant.
• context: the project the run read its governance from. project is the project's stable identity, name the label a person recognises, and revision the position the workspace's operation log had reached. Two results carrying one revision were checked against one state of the context. stale and stale_reason report that the blocks kapi holds were read from files that have since changed. A check of files outside any project carries no context at all, and neither does a check_text draft with no context_path, which no project governed.
• tool: the name, version and commit of the build that ran the check.
• plugins: each plugin that served the run, with the version its manifest declares and a sorted serves list naming what it did, such as format:pdf, comments:python or analyzer:voice.similarity. A run every part of which kapi handled itself carries no plugins.
• analyzers: coverage, one entry per analyzer, sorted by id. ran counts the inputs the analyzer evaluated, and not_run groups the rest by the status and reason already recorded in execution.analyzers, with inputs counting each group. An analyzer that missed its canary is grouped under status invalid, because what it reported cannot be read as a result.

Evaluation metadata does not affect scores, verdicts or gates, and is omitted from human-readable output. Every producer of a kapi.check/v2 Report carries the record. kapi check --ship reports gates rather than a Report and carries the same record once, at the top level of its output, for the whole run: analyzers covers what every gate recorded, and plugins names every plugin any gate reached. The ship gate is the form of the check that runs in CI, where a verdict is read later against a context that has moved on, so the record says which project, which revision and which build the gates were evaluated against.

For a project file, omit MCP profile_file and profile_pack to retain its applicable profile and channel. An explicit profile replaces that voice selection, while project terms still apply. Check the reported scope before using a passing result to assess the task.

A passing verdict means the configured checks raised no failing finding. Factual accuracy, and rules nobody encoded, are outside it. Exact voice rules, advisory example similarity (--voice), and optional LLM review are distinct analyses. kapi check does not run LLM review and records voice.llm as not_requested. Applicable profile guidance that exact rules cannot assess is recorded as voice.guidance with status unsupported; it fails no check.

A requested analyzer that cannot run, or governing context that cannot resolve, returns an operational error instead of a successful report. --no-fail affects only the exit for failing findings. MCP uses its existing error response for these failures. Embedded applications share checker primitives but can retain their own response shapes and operation error views.

Checking a change

kapi check --diff-against <rev> checks only the content a change touched. kapi runs git diff against the revision, read-only, treats untracked files as wholly new, and reads each changed file once. kapi check --staged checks what a commit made now would record: the staged changes, with each file read from the index. An unstaged edit or an untracked file leaves the result unchanged, and a finding's lines are the lines the index holds. Inside a project, a staged file is content when the recipe declares its path, whether or not the working tree still holds it. An index holding an unresolved merge conflict is refused. --diff-file <path> reads a unified diff you already hold, or standard input with -. Named files narrow the scope.

kapi check --diff-range A..B checks the change between two commits, and A...B the change B made since its merge base with A, which is the form a pull request check uses (origin/main...HEAD). An empty side names HEAD, as it does to git. Each file is read from B and nothing from the working tree, so untracked files and uncommitted edits never count, a finding's lines are the lines B holds, and the range checks the same from any checkout, including one at another commit.

Every diff check is governed by the recipe, voice profiles and terms on disk where kapi runs. For a range, those are the rules in force now, never the versions B holds, so each commit in a range is held to today's rules. A pull request check needs that, and so does a search of history for the commit where a file first broke a rule. Inside a project, a file B holds is content when that recipe declares its path. Check changes in git has a pre-commit hook, a pull request check, a sweep for the commits that added findings and a git bisect run wrapper built on these flags.

A diff names lines, and kapi widens each changed line to the content block it belongs to: a one-line edit inside a seven-line paragraph checks the whole paragraph. A finding's location.lines gives the lines of its block, and a finding on a comment also gives location.comment_sha256, the SHA-256 of the comment's bytes, which a comment edit carries back to kapi apply. Rules that hold over a whole document, such as a voice profile's required patterns, read the whole changed file.

In a source file read for its comments, such as a Go file, or a TypeScript, Python or CSS file the sourcecode plugin reads, the unit is the comment. A changed line inside a comment checks that comment whole, and a change to code or to directives alone leaves the file untouched. Where the language has a formatter, it compares only the comments the change touched, and it catches its canary as it does when kapi checks the whole file. When a recipe declares the comments of a file a reader parses, such as a YAML, Markdown or PO file with comments: true, each comment is a block beside the reader's blocks, and a changed line takes whichever block it sits in.

Removed lines leave nothing to point at, so a deletion first takes the blocks on both sides of where the lines were. A block that lost its first or last lines is then in scope, and so is one the deletion merged with its neighbour. For a block the deletion only borders, kapi rebuilds the file as it was before the change from the diff, and drops the block from the scope when the earlier file holds the same block on the same lines: removing one key of a JSON catalog checks neither the key before it nor the key after it.

The report's scope lists every file the diff names with a status:

• checked, with the blocks checked and their lines;
• untouched, when the change touched no content block, as with markup, a rename or a mode change;
• no_content, no_reader or deleted, where the reason of a no_reader file the project declares in a plugin's format names the plugin to install;
• out_of_scope, for a file a project's recipe does not declare as content, or one outside the files named;
• did_not_run, with the reason, for a changed file whose touched content cannot be located: a format that keeps no record of where its content sits, a binary change, or a change on lines where the file's structure leaves a block's position ambiguous. The reason names that block and the lines it could sit on. A change elsewhere in the same file is checked as usual, and the touched blocks kapi can place are checked and listed either way.

A did_not_run file leaves the whole check did_not_run, and so does a diff that touches no content block. Over MCP, check_file takes diff (unified diff text), diff_against (a revision), staged or diff_range for the same scope, and file then narrows it.

One model: findings

Every check emits the same structured finding (the core/check.Finding type): a category, whether it fails, the run-index range it points at, and an optional suggested replacement. A check is a read-only tool: it uses the annotate capability, so it may attach findings but never rewrite content (see the immutability model). Findings are recorded as stand-off overlays anchored to the offending runs, so a check pass slots into any flow as an ordinary stage and its results surface uniformly to the CLI, an editor, the MCP tools, or a downstream gate.

What fails

A finding's fails field is determined by the rule and its status:

• An established word rule fails: a forbidden or competitor term in the terms store, or a term rule. A rule marked advisory: true, or a terms-store concept marked advisory, reports instead. A retired term in the terms store reports.
• A voice profile's prohibited or required pattern reports, and so does its rate ceiling. A pattern marked fails: true fails, unless it is also marked advisory: true.
• A suggested rule, one recorded by kapi context note and not yet kept, reports and never fails. Its finding carries suggested: true and reads Suggested rule about "X", not yet established.
• Style measures report: the voice-similarity check (--voice), an AI voice judge, and comment limits, unless the profile sets style.comments.fails: true.
• Empty content fails; whitespace and doubled-word findings report.
• Length and pattern findings from --max-chars, --max-words, --forbid and --require fail.
• A dropped or extra placeholder fails, and so does a comment the language's formatter would rewrite.

The score (0–100) is reported beside the findings and gates nothing. A failing finding weighs 25 against it, a reported one 1, and a suggested one 0.

The shared finding model lets the CLI, Kapi Desktop and downstream editors consume the checks each surface invokes. Matching findings does not imply that every surface requested the same analysis.

The check families

Generic content checks (source-side, no translation needed; the default checkset):

• Text hygiene: empty content, doubled spaces and words, stray leading/ trailing whitespace, control characters. Always on.

  In a comment, hygiene.double-spaces reports a run of exactly two spaces between two characters on one line. A comment lays text out in columns and quotes literal text, so a run of three or more spaces, a run that ends where a word starts on the line above or below, and a run inside backticks or double quotes are layout and are not reported. A double space reports, in a comment and in other content.

  Hygiene is judged against the block's content boundaries, where an inline code counts as content. A leading or trailing placeholder is the edge of the content, so the space beside it is a separator, not stray whitespace: {price} each has no leading whitespace, Hello {name} world has no double space, and a block that is only a placeholder is not empty. Real whitespace ( {price} each) still reports.

  An inline code is likewise a real boundary for the adjacency rules: it separates what sits either side of it, so the {name} the cat holds no doubled word. It is a token of its own, so it cannot hide a doubled word either: {name}the the cat reports. Nothing separates two adjacent text runs, so a defect spanning their join (the + the cat) reports. The editor's highlights come from these same rules, so a preview and a kapi check finding agree.

• Length: flag content over a character or word budget (--max-chars/ --max-words).

• Patterns: regex that must not appear (--forbid) or must appear (--require) in the content.

• Word rules: the forbidden, competitor and retired terms that govern the content, from the terms store, the rules established across the workspace, and a starter pack bound as the voice. They are one analyzer, terms, with rule id terms.vocabulary, and a finding from a pack or a workspace rule names its source in metadata.from.

• Voice patterns: the prohibited and required patterns of a bound voice profile, the voice.rules analyzer. Optional --voice adds an advisory similarity comparison with profile examples. LLM voice review is a separate tool or an explicit AI voice command.

• Formatter agreement: for the comments in source code, a comment the language's formatter would rewrite is a formatter.<formatter> finding, such as formatter.gofmt, and it fails. The comment reader and the formatter each catch a canary on every run, as every other analyzer does. See Comments in source code.

• Comment limits: where the voice profile at a comment's point sets comment limits, a sentence over the word limit is a comment.sentence-length finding, and a comment over the limit for what it documents is a comment.length finding. These report; with style.comments.fails: true a sentence over the limit, a comment over its length limit and a too-dense change fail. Sentences are split with the UAX #29 sentence break over the comment's prose: the lines a comment wraps over are joined first, a blank line, a divider line, a list item, a heading or a documentation tag starts a new sentence, and code spans, references and links are not counted as words. A build without the sentence break reports the sentence check as not run. In a check scoped to a diff, a change that adds at least the minimum number of comment lines to a file, and more comment lines for each code line than the ratio allows, is a comment.density finding. Lines are counted from where the comment reader places each comment, and the package doc comment is not counted. A check over whole files reports density as unsupported.

Bilingual checks (opt-in, with --target: a translated target against its source):

• Placeholder integrity: catch a dropped {count} or a corrupted <b> in the translation.
• Do-not-translate: terms that must survive verbatim into the target.
• Target terms: the terms.target analyzer holds a translation to the renderings the project terms store requires, through the term-check tool.

The full rule-based family (whitespace, inline-code integrity, cross-block consistency, optional LLM review) is documented under Rule-based checks.

Document structure & encoding validity belongs to the format readers, which extract leniently by default. Surface it on demand with kapi check --validate report: the reader emits located structure.* / encoding.* findings (malformed XML/YAML, invalid UTF-8, charset mismatch, and the JSON faults the parser rejects) into the same Report. They report, and a critical one fails; --validate strict also makes a major one fail. Coverage tracks each reader's own strictness. Validate source and target files separately; reader validation cannot be combined with --target.

A file the reader cannot parse at all is a structure.unreadable finding when the check found the file through the project, a directory, a glob or a diff. The finding fails the check and names the file and the parse error, and the check goes on to the other files. A file named on the command line that cannot be parsed ends the check with the error.

Composing and gating

Checks are tools, so they compose in a flow like translation or transform stages, usually as the last stage after translation. In CI, gate on the exit code. In an editor or assistant, surface the findings and apply the fixes they carry through the change service. A check changes no content itself.

In a project, kapi check --ship (and each pass of kapi up) runs the bound gates over what was produced. The project-gate response groups findings by gate:

Gate	What it checks
voice	The bound profile's rules; fails on a failing finding, and reports the compliance score
terminology	The terms the project requires
qa	The rule-based quality checks
ship	Target status
source	The source-side checks
staleness	Content produced under a context that has since changed, such as a new voice profile version or new term rules; it must be re-run before its locale ships

Source-only collections receive authored-content checks, including in a project that also has translated targets. Naming a source-only file checks that content directly; translation checks apply to target content with its source. Each gate carries the same three verdicts as a report: a gate with no content in scope, or whose checks did not catch their canaries, did not run, and the command exits 4. See the agent surface design for the edit loop and its release checks.

Related

• Ship gates & CI: gate a pull request on a project's bound checks with GitHub Actions.
• Rule-based checks: the full rule-based family.
• Check reference: every rule id and its fix.