Skip to main content

Check content like tests

Goal: verify content before it ships (over-long strings, forbidden phrases, voice drift, structural faults) with a repeatable gate: structured findings, a pass/fail, and an exit code. kapi check reads content and changes nothing; kapi fix applies the fixes it computed, and kapi apply lands the other edits. For the Report model and every check family, see Checks.

Open your project. It opens on its Map, whose findings count reports what the checks found.

  1. Run the checkset. Run on everything on the Map runs the project's bound rules (hygiene, length, patterns, the word rules held as terms, and the voice's patterns when a profile is bound) over the tracked content.
  2. Read the findings. Click the findings count. Each finding names its rule, whether it fails or only reports, and the exact block; click through to see the flagged text in context.
  3. Fix and re-run. Edit the source and re-run; the fixed findings clear. The panel shows the same gate result as the CLI.
A file in the Reader with a finding marked on its words, the rule's wording under the block and Fix beside itA file in the Reader with a finding marked on its words, the rule's wording under the block and Fix beside it
ScreenshotPending capture

The findings the Map's count opens: a list grouped by rule, failing findings first, with a block-level detail pane and the pass/fail summary.

Fix what kapi can​

A finding of a word rule that names a replacement carries its fix: the replace_text operation that puts the replacement in place of the words. kapi fix runs the same check and applies those fixes, one change set per file, through the change service kapi apply uses, so each file keeps its formatting.

kapi fix                       # the project's declared content
kapi fix docs/guide.md         # one file
kapi fix 'docs/**/*.md' --dry-run
4 fixed in 2 files
  docs/guide.md    3 fixed: utilize → use, utilize → use, e-mail → email
  strings/en.json  1 fixed: utilize → use
1 finding needs a rewrite, with no automatic fix (voice.style). Edit by hand.
1 finding not fixed. Edit by hand:
  docs/guide.md guide/p#2 "mail address": another fix changes the same words
kapi check: 1 failing finding remains (voice)

kapi fix changes failing findings with a replacement and nothing else. A finding about tone or style, a rule with no replacement, words inside a code span, and words that hold a link, a formatting span or a placeholder have no automatic fix, and kapi lists them for you to edit. A finding that only reports stays as it is. A fix the change service refuses is listed with the reason, and the file's other fixes land. After writing, kapi fix checks the files it wrote again and prints how many failing findings remain.

kapi fix previews each file's change set before it writes. When the preview shows a change outside the replaced words, such as an escape the format's writer spells another way, kapi leaves the file as it is and lists its fixes as not fixed.

--dry-run prints the diff each file would get and writes nothing. --json prints the kapi.fix-result/v1 result, with each file's change set result. The exit code is 0 when every fix landed, 1 when a fix did not, and 4 when the check did not run.

Check changes in git​

kapi check can take a change from git and check only the content blocks the change touched, each block whole. See Checking a change for how a changed line becomes a block and what the report's scope lists.

FlagWhat kapi checks
--diff-against <rev>The working tree against a revision, with untracked files as new
--stagedWhat the next commit records, each file read from the index
--diff-range A..BThe change between two commits, each file read from B
--diff-range A...BThe change B made since it left A

The exit code carries the outcome: 0 the check passed, 3 a finding failed, 4 the check did not run, and 1 kapi could not run it. A check that did not run is never a pass. The recipes below map each code to what the tool around them expects.

Before each commit​

A pre-commit hook checks what the commit records:

.git/hooks/pre-commit
#!/bin/sh
# Stops a commit whose staged content fails the check. A commit that touches
# no content kapi reads goes through.
cause=$(kapi check --staged --jq '.did_not_run_cause' 2> /dev/null)
status=$?
if [ "$status" -eq 0 ] || { [ "$status" -eq 4 ] && [ "$cause" = '"nothing_to_check"' ]; }; then
	exit 0
fi
kapi check --staged
exit "$status"

Make it executable with chmod +x .git/hooks/pre-commit. Inside a project, the hook applies the rules the recipe binds. An unstaged edit and an untracked file leave the result unchanged. A commit that touches only files kapi does not check, such as a binary or content the recipe does not declare, exits 4 with nothing_to_check, and the hook lets that commit through. Any other outcome stops the commit and prints the findings. A finding that only reports lets the commit through.

On every pull request​

In GitHub Actions, check the change a pull request makes with kapi-action:

.github/workflows/content-checks.yml
name: Content checks
on: pull_request

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - uses: neokapi/setup-kapi@v1
        with:
          version: "1.2.0"
          plugins: ""
      - uses: neokapi/kapi-action@v1
        with:
          command: check
          args: --diff-range origin/${{ github.base_ref }}...HEAD

fetch-depth: 0 fetches the base branch and the history kapi needs to find the merge base. With A...B, the job checks only what the pull request changed, however far the base branch has moved since. A failing finding exits 3, which the action reports as an unmet gate. A check that did not run fails the job too.

Which commits added findings​

A range check reads each file from the commit it names, so it can walk history without checking anything out. This script lists the commits since a base whose change adds a finding in the Go files it touches:

kapi-sweep.sh
#!/bin/bash
# kapi-sweep.sh <base> <voice-profile>
# Lists the commits since base whose change adds a finding in the Go files it
# touches, held to the rules in the voice profile. Run it from the top of the
# repository. A commit that kapi could not check is named on standard error.
set -u
base="$1"
profile="$2"
for commit in $(git rev-list --reverse "$base..HEAD"); do
	files=()
	while IFS= read -r -d '' f; do files+=("$f"); done < <(git diff -z --name-only "$commit^" "$commit" -- '*.go')
	[ ${#files[@]} -eq 0 ] && continue
	count=$(KAPI_NO_PROJECT=1 kapi check --diff-range "$commit^..$commit" --profile-file "$profile" \
		--jq '.summary.findings' "${files[@]}" 2> /dev/null)
	case $? in
	0 | 3) [ "${count:-0}" -gt 0 ] && git log -1 --format='%h %s' "$commit" ;;
	4) echo "$(git log -1 --format=%h "$commit"): not checked" >&2 ;;
	*) echo "$(git log -1 --format=%h "$commit"): kapi check failed" >&2 ;;
	esac
done
./kapi-sweep.sh origin/main comments.yaml

KAPI_NO_PROJECT=1 and --profile-file hold every commit to the rules in the profile you name, whatever the project's recipe said when the commit was made. .summary.findings counts every finding, failing or reported. The script names the changed Go files, so a commit that changes only other files is left out, and a change to Markdown with no comment to check cannot stop the check from running.

When a file started breaking a rule​

git bisect run finds the first commit where a file breaks one rule. It checks out each commit it tests, so keep kapi and the profile outside the repository:

kapi-bisect.sh
#!/bin/bash
# kapi-bisect.sh <file> <rule>, for git bisect run.
# Exits 1 when the commit checked out breaks rule in file, 0 when it does not,
# and 125, which skips the commit, when kapi could not check it. KAPI names a
# copy of kapi and PROFILE the voice profile, both kept outside the repository
# so that every commit is checked by the same binary against today's rules.
set -u
file="$1"
rule="$2"
[ -e "$file" ] || exit 0
count=$(KAPI_NO_PROJECT=1 "$KAPI" check --profile-file "$PROFILE" "$file" \
	--jq "[.findings[] | select(.rule == \"$rule\")] | length" 2> /dev/null)
case $? in
0 | 3) ;;
*) exit 125 ;;
esac
case "$count" in
0) exit 0 ;;
'' | *[!0-9]*) exit 125 ;;
*) exit 1 ;;
esac
cp "$(command -v kapi)" /tmp/kapi
cp comments.yaml /tmp/comments.yaml
git bisect start HEAD v1.0
KAPI=/tmp/kapi PROFILE=/tmp/comments.yaml git bisect run ./kapi-bisect.sh internal/parse/parse.go comment.length
git bisect reset

Exit 0 and 3 both mean the check ran, so the wrapper reads the findings: a finding for the rule marks the commit bad, and none marks it good. A check that did not run, exit 4, and a kapi error, exit 1, skip the commit with 125, because that commit can say nothing about the rule. A commit where the file does not exist yet counts as good.

In a project: the ship gate​

Inside a kapi project, kapi check with no file arguments checks the project's declared content with the recipe's bound rules. kapi check --ship is the project gate mode: it runs the bound quality gates (voice, terminology, rule-based checks, staleness) plus the ship/source coverage gates over the project's content, and exits non-zero when any gate is unmet. Run it before a release. Ordinary builds never fail on target drift; --ship is where you opt in to enforcement. See Ship gates & CI.

A target identical to its source fails the gate unless the project records that the unchanged wording is intentional. This evidence can be an approval bound to that exact source–target pair or a terms entry with identical source and target text. kapi check --ship and kapi up apply the same rule. Other checks still apply; an approved block with a missing placeholder fails.

The staleness gate reads provenance: the record of the context each target was written under. Every producer stamps the governing context it was given onto what it writes, so kapi can compare a target with the context in force at the point that governs its file. Change the voice profile or the terminology, and the targets written under the old one fail the gate, naming what changed; kapi up reproduces them under the current context. Content that carries no stamp, such as a target written by hand, is reported and never failed.

The two are different bars: a bare kapi check <files> gates the files you name (a checkset over content: the test runner), while --ship gates the project against its ship_gates: coverage thresholds across every target language (the release bar).

Check-only content in other assets​

Product copy is not confined to the documentation. A package manager prints a description before anything installs. Windows shows a summary in file properties. Homebrew prints cask notes after an install finishes. Readers often meet these lines before they open the docs.

kapi can read this copy and check it. It cannot write these files back, because the formats that parse them supply a reader and no writer.

Declare such a collection with source_only: true:

kapi.yaml
collections:
  - name: desktop-cask
    channel: acme/desktop
    base: deploy/homebrew
    source_only: true
    content:
      - path: "*.rb"
        format:
          name: sourcecode
          config:
            language: ruby
            # The calls whose arguments hold sentences. Everything else in a
            # cask is a path, a version or an identifier.
            nodePathPatterns: [desc, caveats]

source_only: true says the collection has no target language. kapi up leaves it alone, and the ship gates that measure target coverage exclude it. The rules that only read still apply: the voice profile bound to this point, the project's terms, and the hygiene rules.

Check it the way you check anything else:

kapi check 'deploy/homebrew/*.rb'

kapi rejects a recipe that sets source_only: true on a collection that also carries a target. Without that check, a deliberate omission and a forgotten target look the same in the file.

Comments in source code​

The comments in a source file are prose, and an assistant that writes code writes them too. kapi checks the comments in Go source files with the same rules as any other content.

Name a Go file and kapi checks its comments:

kapi check internal/parse/parse.go

Each comment is one block, named for what it documents, such as func/Parse or type/Block/ID. Directives such as //go:embed and //nolint, the comments in generated files, and the code blocks and references inside a doc comment stay out of the prose, so a rule reads sentences and nothing else.

kapi also compares each comment with what gofmt writes. A comment gofmt would rewrite is a formatter.gofmt finding, and it fails, so a formatting problem surfaces before the commit.

Beside your file, each run gives the comment reader and gofmt a small Go file with a known fault. If either reports nothing on it, the check did not run and exits 4, whatever it found in your file. A check over a Go file that holds no comment prose did not run either.

To check the comments as project content, declare the files with comments: true in a source-only collection:

kapi.yaml
collections:
  - name: engine-comments
    channel: acme/engineering
    source_only: true
    content:
      - path: "internal/**/*.go"
        comments: true

A bare kapi check and kapi check --ship then check those comments under the voice profile and terms bound to acme/engineering. kapi up, flow runs and source coverage leave the files alone.

Some comment lines are written for your own tools, such as the marker a test audit collects. Declare those markers as directives and kapi sets their lines aside, so a rule reads them as neither prose nor content:

kapi.yaml
defaults:
  comments:
    directives: ["okapi-skip:", "okapi-unmapped:"]
collections:
  - name: engine-comments
    channel: acme/engineering
    source_only: true
    content:
      - path: "internal/**/*.go"
        comments:
          directives: ["audit-note:"]

A comment line is set aside when its text, after the comment marker and any leading spaces, starts with a declared marker. The match is exact and case-sensitive, so a sentence that mentions okapi-skip: later in the line stays prose. A marker in the middle of a comment splits it into two comments, each checked on its own. The markers under defaults.comments apply wherever kapi reads comments in the project, and an item's own directives add to them for its files. A marker that is empty, starts with whitespace, or is declared twice makes the recipe fail to load, with an error naming the key.

Comments can also sit at a governance point of their own, so a style written for code comments applies to the comments in a file and the values a YAML reader extracts keep the voice they ship under:

kapi.yaml
defaults:
  comments:
    channel: acme/comments
collections:
  - name: deploy-config
    channel: acme/engineering
    content:
      - path: "deploy/*.yaml"
        comments: true

Each comment is then checked under the voice profile and terms bound to acme/comments, and each value under those bound to acme/engineering. An item's own comments: {channel: ...} places its comments at another point. A bare kapi check, a check of named files, kapi check --ship, a check scoped to a diff and the MCP check_file tool all hold each block to its point, and each finding reports the point it was checked at.

The values in some files belong to another tool, such as the steps of a CI workflow or the settings of a build. Declare such files for their comments alone:

kapi.yaml
collections:
  - name: workflow-comments
    channel: acme/engineering
    source_only: true
    content:
      - path: ".github/workflows/*.yaml"
        comments:
          only: true

kapi then checks each comment in those files under the voice and terms of its point, as it checks Go comments, and a check of the project reads none of the values. The item governs only the comments. When another item also matches those files, that item claims their values wherever either item is listed, so kapi up converges the values and the comments stay at the comments-only item's point. A check that names a file, such as kapi check .github/workflows/release.yaml, checks its values too, at the next item that claims the file or at the project's default point, and kapi context <file> answers for the same item. kapi up, kapi merge, kapi extract, flow runs, kapi stats, kapi status coverage and the --ship gates leave the values alone. only works for every format that supplies comments: YAML, the XML-based formats, HTML, Markdown, MDX, PO and properties. An item declared this way has no values to deliver or shape, so a target, target_languages, redaction, format.config or format.preset beside it makes the recipe fail to load, with an error naming the item. A format: that names the format alone stays allowed, and says which format's comments the files hold.

The voice profile at the comments' point can hold them to word limits of their own under style.comments:

the acme profile
name: Acme comments
style:
  sentence_length: short
  comments:
    sentence_words: 50
    comment_words: 100
    doc_words: 150
    package_doc_words: 300
    density: { ratio: 1, min_comment_lines: 8 }

A sentence over 50 words is then a comment.sentence-length finding, and a comment over the limit for what it documents is a comment.length finding. A change that adds 8 or more comment lines to a file and more comment lines than code lines is a comment.density finding. These limits are style measures, so their findings report. Add fails: true under style.comments to make a sentence over the limit, a comment over its length limit and a too-dense change fail. The package doc comment does not count toward it, and only a check scoped to a diff measures it. comments: {} applies these numbers, which are the defaults. A profile that holds only these limits can be named on its own, as in kapi check --profile-file comments.yaml parse.go, and the comment checks then decide the verdict. In a check scoped to a diff, the limits apply to the comments the change touched. See Comment limits for how sentences and words are counted.

Prohibited patterns in the same profile hold comments to a writing style. These report phrasings that read as machine-written or that narrate a change instead of describing the code:

the acme profile
style:
  prohibited_patterns:
    - regex: "\u2014"
      description: An em dash; use a comma, a full stop or two sentences.
      scope: prose
    - regex: '(?i)\bused to\b'
      not_after: '(?i)(?:(?:\b(?:is|are|was|were|be|been|being|get|gets|got|isn''t|aren''t|wasn''t|weren''t)|[''’]s)\s+(?:\w+\s+)?|[^\w\s)\]"''\x60’”]\s*|(?:^|\s)["“‘''\x60]|^\s*)$'
      description: The past-habitual "used to"; describe what the code does.
      scope: prose
    - regex: "(?i)\\b(?:crucially|importantly|load[- ]bearing|worth noting)\\b"
      description: A significance label; state the claim.
      scope: prose

Each pattern reports where it matches. Add fails: true to a pattern to make a match fail the check.

not_after keeps "is used to" and "an id, used to flag" out of the rule, which a regular expression alone cannot say.

kapi context <file> --comments prints the voice and the limits that apply to the comments of a file, such as a Go file, so an assistant about to write a comment there reads them first. Without --comments, the answer is for the file's own content.

Repair a comment finding​

A comment finding names the file, the comment's id and its lines, and the JSON report gives it a location.comment_sha256: the SHA-256 of the comment's bytes. Rewrite the comment with kapi apply instead of editing the file around it, so every other byte of the file stays as it is.

Check the change first:

kapi check --diff-against HEAD
  outcome  rule         location                                 message
  FAILS    voice.style  internal/parse/parse.go:func/Parse L5-7  Prohibited pattern: Say use rather than utilize.

A comment is a block of its source file. kapi inspect lists each comment of the file with its prose and the revision an edit sends as if_match, which is r: and the first sixteen hex digits of the finding's comment_sha256:

kapi inspect internal/parse/parse.go --jsonl
{"ref":{"doc":"internal/parse/parse.go","block":"func/Parse"},"rev":"r:8d65ce744d4fe688","text":"Parse reads the input from an [io.Reader] and helps callers utilize the result.\n\nIt stops at the end.","ops":["set_content"],"role":"comment"}

Write the new prose as a set_content operation on that block:

edits.json
{"ops": [{"op": "set_content", "at": {"doc": "internal/parse/parse.go", "block": "func/Parse"}, "if_match": "r:8d65ce744d4fe688",
          "text": "Parse reads the input from an [io.Reader] and helps callers use the result.\n\nIt stops at the end."}]}
kapi apply edits.json

text is the comment's prose without the // markers. Keep the comment's code blocks, references such as [io.Reader] and list items, and keep a Deprecated: paragraph if the comment has one. kapi writes the comment at its own indentation, wraps a paragraph that is too wide, and writes a doc comment as gofmt does.

kapi writes the edit only when the file still parses and gofmt agrees with it. It refuses the edit with a reason when the comment's revision differs from the one you read, because someone changed it since, when the text drops or adds a code block, reference or list item, or when the id names a directive, a generated file's comment or a /* */ comment. A refused edit leaves the file untouched, and kapi apply exits 3. A comment that only moved, because code above it changed, keeps its revision, and kapi writes the edit at its new lines. A change set edits code comments or documents, never both.

After writing, kapi apply checks what changed and reports the check's findings beside the edit, in its JSON result under the document:

wrote internal/parse/parse.go
change set applied: 1 applied

A finding in that check is one the new prose still holds. Send another edit for it, then run kapi check --diff-against HEAD again before you finish.

The same key checks the comments in YAML files, in XML-based files, such as Android string resources, .NET RESX and XLIFF, and in HTML, Markdown, MDX, PO and Java properties files, beside the values their readers extract:

kapi.yaml
collections:
  - name: deploy-config
    channel: acme/engineering
    content:
      - path: "deploy/*.yaml"
        comments: true

A bare kapi check, kapi check --ship and a check scoped to a diff read those comments as well. If the item also names a target:, kapi up converges the YAML file as it would without the key, each comment stays where it is in every file it writes, and the ship gate checks the comments once, in the source language. Set on a format whose comments kapi cannot read, the key makes the check report that it did not run.

In an XML-based or HTML file each <!-- --> comment is one block, named for the element it sits on, such as comment/resources/string[greeting]. A commented-out element and a tool's suppression, such as <!--suppress UnusedResources -->, stay out of the prose. In HTML, so do conditional comments, server-side includes and the markers a framework such as React writes for its own use. In Markdown, a comment is an HTML comment outside code, named for the section it sits in, such as comment/install/from-homebrew, and the <!-- truncate --> marker a blog uses stays out of the prose. In MDX, a comment is a {/* */} expression on lines of its own, named the same way, and the comments of a page marked DO NOT EDIT stay out of the prose. In a PO file, translator comments are checked, and the lines gettext writes and reads, such as #: references and #. extracted comments, stay out of the prose. In a properties file, a # or ! line is a comment unless it continues a value. When kapi cannot place a file's comments exactly, as in an XML file with -- inside a comment, which XML does not allow, the check reports that it did not run. kapi has no comment formatter for any of these formats, so a report lists the formatter check for those files as unsupported.

The key also checks the comments in TypeScript, TSX, JavaScript, Python, Bash, CSS, Rust, Java, C#, C, C++ and Ruby files. The sourcecode plugin reads them, so install it first:

kapi plugins install sourcecode
kapi.yaml
collections:
  - name: app-comments
    channel: acme/engineering
    source_only: true
    content:
      - path: "src/**/*.ts"
        comments: true
      - path: "src/**/*.tsx"
        comments: true
      - path: "scripts/**/*.py"
        comments: true

Each comment is one block named for what it documents, such as func/parse, interface/Options/keep or rule/.header. Directives a tool reads, such as eslint-disable-next-line, @ts-expect-error, # noqa, # shellcheck, stylelint-disable and an SPDX licence tag, the shebang, the comments in generated files, and the tags, links and code spans inside a TypeScript, JavaScript or Java /** */ doc comment, and the XML tags of a C# /// comment, stay out of the prose. These languages have no formatter comparison. A file whose comments kapi cannot place exactly is reported as not checked, never read in part. Without the plugin, a bare kapi check and kapi check --ship warn that no reader for those comments is installed, name the plugin, and report the files as not checked.

Next​