Skip to main content

Check content like tests

Goal: verify content before it ships (over-long strings, forbidden phrases, voice drift, structural faults) with a repeatable gate: structured findings, a pass/fail, and an exit code. kapi check never modifies content; fixing flagged blocks is kapi apply's job. For the Report model and every check family, see Checks.

Open your project and switch to the Checks view.

  1. Run the checkset. Kapi runs the project's bound rules (hygiene, length, patterns, and voice vocabulary when a profile is bound) over the tracked content.
  2. Read the findings. Each finding names its rule, severity, and the exact block; click through to see the flagged text in context.
  3. Fix and re-run. Edit the source, re-run, and watch the findings clear. The same gate result the CLI reports is what the panel shows.
ScreenshotPending capture

The Checks view: a findings list grouped by rule and severity, with a block-level detail pane and the pass/fail gate summary.

Check changes in git​

kapi check can take a change from git and check only the content blocks the change touched, each block whole. See Checking a change for how a changed line becomes a block and what the report's scope lists.

FlagWhat kapi checks
--diff-against <rev>The working tree against a revision, with untracked files as new
--stagedWhat the next commit records, each file read from the index
--diff-range A..BThe change between two commits, each file read from B
--diff-range A...BThe change B made since it left A

The exit code carries the outcome: 0 the check passed, 3 the gate failed, 4 the check did not run, and 1 kapi could not run it. A check that did not run is never a pass. The recipes below map each code to what the tool around them expects.

Before each commit​

A pre-commit hook checks what the commit records:

.git/hooks/pre-commit
#!/bin/sh
# Stops a commit whose staged content fails the check. A commit that touches
# no content kapi reads goes through.
cause=$(kapi check --staged --strict --jq '.did_not_run_cause' 2> /dev/null)
status=$?
if [ "$status" -eq 0 ] || { [ "$status" -eq 4 ] && [ "$cause" = '"nothing_to_check"' ]; }; then
	exit 0
fi
kapi check --staged --strict
exit "$status"

Make it executable with chmod +x .git/hooks/pre-commit. Inside a project, the hook applies the rules the recipe binds. An unstaged edit and an untracked file leave the result unchanged. A commit that touches only files kapi does not check, such as a binary or content the recipe does not declare, exits 4 with nothing_to_check, and the hook lets that commit through. Any other outcome stops the commit and prints the findings. --strict stops it on a critical or major finding.

On every pull request​

In GitHub Actions, check the change a pull request makes with kapi-action:

.github/workflows/content-checks.yml
name: Content checks
on: pull_request

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - uses: neokapi/setup-kapi@v1
        with:
          version: "1.2.0"
          plugins: ""
      - uses: neokapi/kapi-action@v1
        with:
          command: check
          args: --diff-range origin/${{ github.base_ref }}...HEAD --strict

fetch-depth: 0 fetches the base branch and the history kapi needs to find the merge base. With A...B, the job checks only what the pull request changed, however far the base branch has moved since. A failed gate exits 3, which the action reports as an unmet gate. A check that did not run fails the job too.

Which commits added findings​

A range check reads each file from the commit it names, so it can walk history without checking anything out. This script lists the commits since a base whose change adds a finding in the Go files it touches:

kapi-sweep.sh
#!/bin/bash
# kapi-sweep.sh <base> <voice-profile>
# Lists the commits since base whose change adds a finding in the Go files it
# touches, held to the rules in the voice profile. Run it from the top of the
# repository. A commit that kapi could not check is named on standard error.
set -u
base="$1"
profile="$2"
for commit in $(git rev-list --reverse "$base..HEAD"); do
	files=()
	while IFS= read -r -d '' f; do files+=("$f"); done < <(git diff -z --name-only "$commit^" "$commit" -- '*.go')
	[ ${#files[@]} -eq 0 ] && continue
	KAPI_NO_PROJECT=1 kapi check --diff-range "$commit^..$commit" --profile-file "$profile" \
		--max-major 0 --max-minor 0 "${files[@]}" > /dev/null 2>&1
	case $? in
	0) ;;
	3) git log -1 --format='%h %s' "$commit" ;;
	4) echo "$(git log -1 --format=%h "$commit"): not checked" >&2 ;;
	*) echo "$(git log -1 --format=%h "$commit"): kapi check failed" >&2 ;;
	esac
done
./kapi-sweep.sh origin/main comments.yaml

KAPI_NO_PROJECT=1 and --profile-file hold every commit to the rules in the profile you name, whatever the project's recipe said when the commit was made. --max-major 0 --max-minor 0 turns any finding into a failed gate. The script names the changed Go files, so a commit that changes only other files is left out, and a change to Markdown with no comment to check cannot stop the check from running.

When a file started breaking a rule​

git bisect run finds the first commit where a file breaks one rule. It checks out each commit it tests, so keep kapi and the profile outside the repository:

kapi-bisect.sh
#!/bin/bash
# kapi-bisect.sh <file> <rule>, for git bisect run.
# Exits 1 when the commit checked out breaks rule in file, 0 when it does not,
# and 125, which skips the commit, when kapi could not check it. KAPI names a
# copy of kapi and PROFILE the voice profile, both kept outside the repository
# so that every commit is checked by the same binary against today's rules.
set -u
file="$1"
rule="$2"
[ -e "$file" ] || exit 0
count=$(KAPI_NO_PROJECT=1 "$KAPI" check --profile-file "$PROFILE" "$file" \
	--jq "[.findings[] | select(.rule == \"$rule\")] | length" 2> /dev/null)
case $? in
0 | 3) ;;
*) exit 125 ;;
esac
case "$count" in
0) exit 0 ;;
'' | *[!0-9]*) exit 125 ;;
*) exit 1 ;;
esac
cp "$(command -v kapi)" /tmp/kapi
cp comments.yaml /tmp/comments.yaml
git bisect start HEAD v1.0
KAPI=/tmp/kapi PROFILE=/tmp/comments.yaml git bisect run ./kapi-bisect.sh internal/parse/parse.go comment.length
git bisect reset

Exit 0 and 3 both mean the check ran, so the wrapper reads the findings: a finding for the rule marks the commit bad, and none marks it good. A check that did not run, exit 4, and a kapi error, exit 1, skip the commit with 125, because that commit can say nothing about the rule. A commit where the file does not exist yet counts as good.

In a project: the ship gate​

Inside a kapi project, kapi check with no file arguments checks the project's declared content with the recipe's bound rules. kapi check --ship is the project gate mode: it runs the bound quality gates (voice, terminology, rule-based checks, staleness) plus the ship/source coverage gates over the project's content, and exits non-zero when any gate is unmet. It is the pre-release bar. Ordinary builds never fail on target drift; --ship is the explicit enforcement point. See Ship gates & CI.

The gate reads the project's own record where a rule can only guess. A target identical to its source is normally a unit nobody translated, so it fails the gate, unless the project has settled it: an approval bound to that exact pairing (a person read it and said the wording is right), or a terms entry whose target is its source. Both are the same rule wherever the gate is evaluated, so kapi check --ship and the checks kapi up runs inside the loop cannot reach opposite verdicts on one unit. Nothing else is settled by it: a dropped placeholder on an approved unit still fails.

The staleness gate is the one that reads provenance rather than content. Every producer stamps the governing context it was given onto what it writes, so a target can be compared against the context in force at the point that governs its file. Move the voice profile or the terminology and the targets written under the old one fail the gate, naming what moved; kapi up reproduces them under the context now in force. Content that carries no stamp (written before the stamp existed, or by hand) is reported and never failed.

The two are different bars: a bare kapi check <files> gates the files you name (a checkset over content: the test runner), while --ship gates the project against its ship_gates: coverage thresholds across every target language (the release bar).

Check-only content in other assets​

Product copy is not confined to the documentation. A package manager prints a description before anything installs. Windows shows a summary in file properties. Homebrew prints cask notes after an install finishes. Readers often meet these lines before they open the docs.

kapi can read this copy and check it. It cannot write these files back, because the formats that parse them supply a reader and no writer.

Declare such a collection with source_only: true:

kapi.yaml
collections:
  - name: desktop-cask
    channel: acme/desktop
    base: deploy/homebrew
    source_only: true
    content:
      - path: "*.rb"
        format:
          name: sourcecode
          config:
            language: ruby
            # The calls whose arguments hold sentences. Everything else in a
            # cask is a path, a version or an identifier.
            nodePathPatterns: [desc, caveats]

source_only: true says the collection has no target language. kapi up leaves it alone, and the ship gates that measure target coverage exclude it. The rules that only read still apply: the voice profile bound to this point, the project's terms, and the hygiene rules.

Check it the way you check anything else:

kapi check 'deploy/homebrew/*.rb'

kapi rejects a recipe that sets source_only: true on a collection that also carries a target. Without that check, a deliberate omission and a forgotten target look the same in the file.

Comments in source code​

The comments in a source file are prose, and an assistant that writes code writes them too. kapi checks the comments in Go source files with the same rules as any other content.

Name a Go file and kapi checks its comments:

kapi check internal/parse/parse.go

Each comment is one block, named for what it documents, such as func/Parse or type/Block/ID. Directives such as //go:embed and //nolint, the comments in generated files, and the code blocks and references inside a doc comment stay out of the prose, so a rule reads sentences and nothing else.

kapi also compares each comment with what gofmt writes. A comment gofmt would rewrite is a formatter.gofmt finding, and it fails the gate, so formatting work is caught before the commit rather than after it. --lenient reports it without failing.

Beside your file, each run gives the comment reader and gofmt a small Go file with a known fault. If either reports nothing on it, the check did not run and exits 4, whatever it found in your file. A check over a Go file that holds no comment prose did not run either.

To check the comments as project content, declare the files with comments: true in a source-only collection:

kapi.yaml
collections:
  - name: engine-comments
    channel: acme/engineering
    source_only: true
    content:
      - path: "internal/**/*.go"
        comments: true

A bare kapi check and kapi check --ship then check those comments under the voice profile and terms bound to acme/engineering. kapi up, flow runs and source coverage leave the files alone.

Some comment lines are written for your own tools, such as the marker a test audit collects. Declare those markers as directives and kapi sets their lines aside, so a rule reads them as neither prose nor content:

kapi.yaml
defaults:
  comments:
    directives: ["okapi-skip:", "okapi-unmapped:"]
collections:
  - name: engine-comments
    channel: acme/engineering
    source_only: true
    content:
      - path: "internal/**/*.go"
        comments:
          directives: ["audit-note:"]

A comment line is set aside when its text, after the comment marker and any leading spaces, starts with a declared marker. The match is exact and case-sensitive, so a sentence that mentions okapi-skip: later in the line stays prose. A marker in the middle of a comment splits it into two comments, each checked on its own. The markers under defaults.comments apply wherever kapi reads comments in the project, and an item's own directives add to them for its files. A marker that is empty, starts with whitespace, or is declared twice makes the recipe fail to load, with an error naming the key.

Comments can also sit at a governance point of their own, so a style written for code comments applies to the comments in a file and the values a YAML reader extracts keep the voice they ship under:

kapi.yaml
defaults:
  comments:
    channel: acme/comments
collections:
  - name: deploy-config
    channel: acme/engineering
    content:
      - path: "deploy/*.yaml"
        comments: true

Each comment is then checked under the voice profile and terms bound to acme/comments, and each value under those bound to acme/engineering. An item's own comments: {channel: ...} places its comments at another point. A bare kapi check, a check of named files, kapi check --ship, a check scoped to a diff and the MCP check_file tool all hold each block to its point, and each finding reports the point it was checked at.

The values in some files belong to another tool, such as the steps of a CI workflow or the settings of a build. Declare such files for their comments alone:

kapi.yaml
collections:
  - name: workflow-comments
    channel: acme/engineering
    source_only: true
    content:
      - path: ".github/workflows/*.yaml"
        comments:
          only: true

kapi then checks each comment in those files under the voice and terms of its point, as it checks Go comments, and a check of the project reads none of the values. The item governs only the comments. When another item also matches those files, that item claims their values wherever either item is listed, so kapi up converges the values and the comments stay at the comments-only item's point. A check that names a file, such as kapi check .github/workflows/release.yaml, checks its values too, at the next item that claims the file or at the project's default point, and kapi voice guide and kapi context answer for that point. kapi up, kapi merge, kapi extract, flow runs, kapi stats, kapi status coverage and the --ship gates leave the values alone. only works for every format that supplies comments: YAML, the XML-based formats, HTML, Markdown, MDX, PO and properties. An item declared this way has no values to deliver or shape, so a target, target_languages, redaction, format.config or format.preset beside it makes the recipe fail to load, with an error naming the item. A format: that names the format alone stays allowed, and says which format's comments the files hold.

The voice profile at the comments' point can hold them to word limits of their own under style.comments:

.kapi/profiles/acme/voice.yaml
name: Acme comments
style:
  sentence_length: short
  comments:
    sentence_words: { minor: 50, major: 70 }
    comment_words: 100
    doc_words: 150
    package_doc_words: 300
    density: { ratio: 1, min_comment_lines: 8 }

A sentence over 50 words is then a minor comment.sentence-length finding and one over 70 a major one, and a comment over the limit for what it documents is a major comment.length finding. A change that adds 8 or more comment lines to a file and more comment lines than code lines is a major comment.density finding. The package doc comment does not count toward it, and only a check scoped to a diff measures it. comments: {} applies these numbers, which are the defaults. A profile that holds only these limits can be named on its own, as in kapi check --profile-file comments.yaml parse.go, and the comment checks then decide the verdict. In a check scoped to a diff, the limits apply to the comments the change touched. See Comment limits for how sentences and words are counted.

Prohibited patterns in the same profile hold comments to a writing style. These report phrasings that read as machine-written or that narrate a change instead of describing the code:

.kapi/profiles/acme/voice.yaml
style:
  prohibited_patterns:
    - regex: "\u2014"
      description: An em dash; use a comma, a full stop or two sentences.
      severity: major
      scope: prose
    - regex: '(?i)\bused to\b'
      not_after: '(?i)(?:(?:\b(?:is|are|was|were|be|been|being|get|gets|got|isn''t|aren''t|wasn''t|weren''t)|[''’]s)\s+(?:\w+\s+)?|[^\w\s)\]"''\x60’”]\s*|(?:^|\s)["“‘''\x60]|^\s*)$'
      description: The past-habitual "used to"; describe what the code does.
      severity: major
      scope: prose
    - regex: "(?i)\\b(?:crucially|importantly|load[- ]bearing|worth noting)\\b"
      description: A significance label; state the claim.
      severity: major
      scope: prose

not_after keeps "is used to" and "an id, used to flag" out of the rule, which a regular expression alone cannot say.

kapi voice guide --comments <file> prints the voice and the limits in force for the comments of a file, such as a Go file, so an assistant about to write a comment there reads them first. Without --comments, the guide answers for the file's own content.

Repair a comment finding​

A comment finding names the file, the comment's id and its lines, and the JSON report gives it a location.comment_sha256: the SHA-256 of the comment's bytes. Rewrite the comment with kapi apply instead of editing the file around it, so every other byte of the file stays as it is.

Check the change first:

kapi check --diff-against HEAD
  severity  rule         location                                 message
  CRITICAL  voice.style  internal/parse/parse.go:func/Parse L5-7  Prohibited pattern: Say use rather than utilize.

Write the new prose for that comment as a comment entry, with the id, lines and comment_sha256 from the finding in kapi check --diff-against HEAD --json:

edits.jsonl
{"kind":"comment","file":"internal/parse/parse.go","id":"func/Parse","lines":{"first":5,"last":7},"comment_sha256":"8d65ce744d4fe688bd8c64ca408fa32ccde8d0ae979b33ce9936f61993b8f26b","text":"Parse reads the input from an [io.Reader] and helps callers use the result.\n\nIt stops at the end."}
kapi apply edits.jsonl

text is the comment's prose without the // markers. Keep the comment's code blocks, references such as [io.Reader] and list items, and keep a Deprecated: paragraph if the comment has one. kapi writes the comment at its own indentation, wraps a paragraph that is too wide, and writes a doc comment as gofmt does.

kapi writes the edit only when the file still parses and gofmt agrees with it. It refuses the edit with a reason when the comment's bytes no longer match comment_sha256, because someone changed it after the check, when the text drops or adds a code block, reference or list item, or when the id names a directive, a generated file's comment or a /* */ comment. A refused edit leaves the file untouched, and kapi apply exits 3. A comment that only moved, because code above it changed, keeps its fingerprint, and kapi writes the edit at its new lines. Without a fingerprint, pass the prose you read in current_text. An entry with neither is rejected before anything is written.

After writing, kapi apply checks what changed and reports it beside the edit:

comment internal/parse/parse.go func/Parse: written (lines 5-7)
check internal/parse/parse.go: passed, 0 finding(s)

A finding in that check is one the new prose still holds. Send another edit for it, then run kapi check --diff-against HEAD again before you finish. The MCP apply_edits tool takes the same entries and returns the same check.

The same key checks the comments in YAML files, in XML-based files, such as Android string resources, .NET RESX and XLIFF, and in HTML, Markdown, MDX, PO and Java properties files, beside the values their readers extract:

kapi.yaml
collections:
  - name: deploy-config
    channel: acme/engineering
    content:
      - path: "deploy/*.yaml"
        comments: true

A bare kapi check, kapi check --ship and a check scoped to a diff read those comments as well. If the item also names a target:, kapi up converges the YAML file as it would without the key, each comment stays where it is in every file it writes, and the ship gate checks the comments once, in the source language. Set on a format whose comments kapi cannot read, the key makes the check report that it did not run.

In an XML-based or HTML file each <!-- --> comment is one block, named for the element it sits on, such as comment/resources/string[greeting]. A commented-out element and a tool's suppression, such as <!--suppress UnusedResources -->, stay out of the prose. In HTML, so do conditional comments, server-side includes and the markers a framework such as React writes for its own use. In Markdown, a comment is an HTML comment outside code, named for the section it sits in, such as comment/install/from-homebrew, and the <!-- truncate --> marker a blog uses stays out of the prose. In MDX, a comment is a {/* */} expression on lines of its own, named the same way, and the comments of a page marked DO NOT EDIT stay out of the prose. In a PO file, translator comments are checked, and the lines gettext writes and reads, such as #: references and #. extracted comments, stay out of the prose. In a properties file, a # or ! line is a comment unless it continues a value. When kapi cannot place a file's comments exactly, as in an XML file with -- inside a comment, which XML does not allow, the check reports that it did not run. kapi has no comment formatter for any of these formats, so a report lists the formatter check for those files as unsupported.

The key also checks the comments in TypeScript, TSX, JavaScript, Python, Bash, CSS, Rust, Java, C#, C, C++ and Ruby files. The sourcecode plugin reads them, so install it first:

kapi plugins install sourcecode
kapi.yaml
collections:
  - name: app-comments
    channel: acme/engineering
    source_only: true
    content:
      - path: "src/**/*.ts"
        comments: true
      - path: "src/**/*.tsx"
        comments: true
      - path: "scripts/**/*.py"
        comments: true

Each comment is one block named for what it documents, such as func/parse, interface/Options/keep or rule/.header. Directives a tool reads, such as eslint-disable-next-line, @ts-expect-error, # noqa, # shellcheck, stylelint-disable and an SPDX licence tag, the shebang, the comments in generated files, and the tags, links and code spans inside a TypeScript, JavaScript or Java /** */ doc comment, and the XML tags of a C# /// comment, stay out of the prose. These languages have no formatter comparison. A file whose comments kapi cannot place exactly is reported as not checked, never read in part. Without the plugin, a bare kapi check and kapi check --ship warn that no reader for those comments is installed, name the plugin, and report the files as not checked.

Next​