Ţéšţš àñđ Éṽàļš
Ţĥŕéé ƃàñđš, ƃý ŵĥàţ îš üñđéŕ ţéšţ. kapi'š öŵñ çöđé çàñ ƃé àššéŕţéđ. À ḿöđéļ'š öüţþüţ çàñ öñļý ƃé éšţîḿàţéđ. Àñ àĝéñţ'š ƃéĥàṽîöüŕ îš šçöŕéđ šçéñàŕîö ƃý šçéñàŕîö. Îñšîđé éàçĥ ƃàñđ ţĥé šţŕüçţüŕé îš ţĥé àŕçĥîţéçţüŕé.
7measured
9partial
2unvalidated
0 šţàļé öŕ üñđàţéđ
0/6 ļàýéŕš ŵîţĥ ñöţĥîñĝ ƃéĥîñđ ţĥéḿ
Engine and formats3 measured3 partial
Šüƃĵéçţ kapi's own code: what it does to a document it reads, holds and writes back. Éṽîđéñçé Deterministic. The same document produces the same content model every run, so these evals assert rather than estimate, and a regression is a fact rather than a shift in a distribution. Ĝàţéš ÇÎ Yes. A failure here stops a build.
6 éṽàļš àçŕöšš F, E · öþéñ →
AI and context3 measured4 partial2 unvalidated
Šüƃĵéçţ What a model writes under kapi's governance, and whether the governance reached it. Éṽîđéñçé Sampled. A score is an estimate over a corpus, and where a model does the scoring it is an estimate of an estimate. Corpus size is the ceiling on what any of it can conclude, so every card states it. Ĝàţéš ÇÎ No. These are tracked over time, and a drop is a signal to look rather than a build failure.
9 éṽàļš àçŕöšš C, M, A · öþéñ →
Agent skills1 measured2 partial
Šüƃĵéçţ An agent driving kapi: the shipped Agent Skill, and the MCP tools an assistant calls. Éṽîđéñçé Scenario-scored. A prompt and a workspace go in, and what the agent did comes out: which tool it reached for, whether it finished at a green gate. No model grades anything. Triggering is stochastic, so a scenario means little until it is repeated. Ĝàţéš ÇÎ No, and these never run in CI. They drive an interactive agent on a maintainer's machine, so the committed dataset is all CI sees and the date on it is the real currency of the numbers.
3 éṽàļš àçŕöšš S · öþéñ →
Ŵĥàţ ţĥé šţàţüšéš ḿéàñ
measuredruns, data committed, numbers can be read as they standpartialruns, and covers less than the layer needsunvalidatedproduces numbers that should not yet be relied onblockedthe harness runs; the surface it measures returns nothingnot measurednothing measures this