E-03: The tool system

Summary

A tool is a single stage in a processing pipeline. It reads Parts from an input channel and writes Parts to an output channel. Tools compose into flows; flows are executed by the pipeline engine (E-01). The BaseTool struct with optional handler fields (a capability-typed block handler, Annotate / Produce / Transform, plus untyped handlers for other Part types) lets most tools implement only the handler for the Part type they care about; everything else passes through unchanged. The block handler a tool sets also declares what it may write. Tools declare parameter schemas via SchemaProvider, which drives CLI flag generation, flow-editor config panels, and validation. An IO contract on ToolMeta declares locale cardinality, the stand-off layers a tool produces, and its side effects, so the runner can infer locale iteration and the flow editor can show data flow. A tool whose behaviour comes from one of several interchangeable backends is a tool group: it registers self-describing members with a default, and the user selects and configures one member at a time.

Context

Most tools only care about one or two Part types. A translation tool processes Blocks; a word counter reads Blocks; a binary extractor handles Media. Requiring every tool to implement the full Process(ctx, in, out) method with a type switch over all Part types produces repetitive boilerplate and creates a risk of accidentally dropping Parts.

Beyond structural dispatch, a tool system needs to answer several questions uniformly for the CLI, the flow editor, and plugin consumers:

• What parameters does this tool accept, and what are their types?
• How many locales does it operate on? Which ones?
• What stand-off layers does it produce? Which does it consume?
• What external systems does it touch (content memory, terms, remote APIs)?

Decision

Tool interface and BaseTool dispatch

The core interface is minimal:

type Tool interface {
    Process(ctx context.Context, in <-chan *Part, out chan<- *Part) error
}

BaseTool provides a standard dispatch shell. The block handler is one of three capability-typed fields; the tool sets exactly one, and the parameter type bounds what it may write:

type BaseTool struct {
    Annotate  func(BlockView) error             // read-only: overlays/annotations/properties
    Produce   func(VariantView) error           // writes target
    Transform func(BlockView) (EditPlan, error) // read-only producer; the applier rewrites source

    HandleDataFn       PartHandler
    HandleMediaFn      PartHandler
    HandleLayerStartFn PartHandler

    SchemaFn func() *schema.ComponentSchema
}

BaseTool.Process reads Parts from the input channel, dispatches Blocks to whichever capability-typed handler is set (and other Part types to their Handle*Fn), and passes unhandled Part types through unchanged. Concrete tools embed BaseTool and set only the handlers they need. A tool that needs the full stream (batching, 1→N fan-out, cross-block state) overrides Process directly; it writes a held block through tool.WriteAs, which hands a typed handler a view of the block and returns any write the view could not apply.

The caller owns both channels. A tool observes ctx on every receive and send, returns once its input is exhausted or the context is cancelled, joins any goroutine it started before returning, and leaves closing the output to the caller. A block handler that calls Drop() on its view, or a Handle*Fn that returns nil, removes that Part from the stream, and nothing is emitted in its place.

tool.RetryTool retries a BaseTool's capability-typed block handler, with backoff, on the errors its configuration names as retryable (every error when it names none). It wraps the handler on a private copy of the BaseTool, so the caller's tool keeps its own handlers, and a retried handler must be idempotent: a retry keeps the block writes and external effects of the attempt before it. A tool that overrides Process, or sets no typed block handler, runs once, since its input channel cannot be rewound and its earlier output may already be downstream. Cancellation and deadline errors return at once, and SessionProcess passes to a wrapped SessionTool, which owns any retry inside its session.

SessionTool extension

The channel-based Tool.Process is a forward-only transform. Some tools need random access to the project's block state: lookup by content hash, reading prior overlays (memory matches, check findings, targets an earlier run produced) to skip work that is already done, or writing annotations that downstream tools in the same or a later run will consult. Those tools opt into the SessionTool interface alongside Tool:

type SessionTool interface {
    Tool

    SessionProcess(
        ctx context.Context,
        sess blockstore.Session,
        in <-chan *Part,
        out chan<- *Part,
    ) error
}

Lifecycle is owned by the executor, not the tool:

1. At flow start the executor opens a blockstore.Session against the project's declared store backend (C-01).
2. For each tool the executor calls SessionProcess when the tool implements SessionTool and the store reports itself persistent, otherwise the plain streaming Process, which is what every tool gets over an ephemeral store. Hybrid implementations are allowed: SessionProcess can read from in, enrich via the session, and emit to out.
3. The executor commits the session on success or rolls back on error. Tools must not call Commit or Rollback themselves.

Every SessionTool also implements Tool, so flow composition keeps working whether or not a step uses the session. See the SessionTool authoring guide for idiomatic patterns.

Tool categories

Every tool declares one of six canonical categories, which set expectations for idempotency and ordering and drive the grouping in the reference and the config UI:

Category	Constant	Responsibility
translation	schema.CategoryTranslation	produces target content
quality	schema.CategoryQuality	validates a target; produces check or term findings
analysis	schema.CategoryAnalysis	read-only metrics and reports
text-processing	schema.CategoryTextProcessing	rewrites, segments, or redacts source
convert	schema.CategoryConvert	format conversion
pipeline	schema.CategoryPipeline	composite or sub-pipeline

IO model

Each tool declares an IO contract in its ToolMeta (package core/schema). The contract is expressed over IOPorts (typed stand-off layers of a Block, F-02) rather than over coarse part-type names: Consumes lists the ports a tool reads upstream and Produces the ports it writes. An IOPort's Type names an overlay type (term, qa, …), a block-annotation type (voice, …), or a pseudo-port (PortTarget / PortSource); its Side says which side it pertains to; and Optional marks a consumed port as degradable rather than required.

// core/schema/schema.go
type IOPort struct {
    Type     string     // overlay type, annotation type, or "target"/"source"
    Side     model.Side // source | target
    Optional bool       // consumed: degrades without it, does more with it
    Layer    string     // segmentation granularity; "" = primary
}

// PortTarget is the committed Target; PortSource is a rewritten source.
const (
    PortTarget = "target"
    PortSource = "source"
)

type ToolMeta struct {
    ID          string
    Category    string // one of the six canonical categories above
    DisplayName string
    Description string
    Tags        []string

    // Requires declares external resources the tool needs at runtime:
    // "target-language", "source-language", "memory", "terms", "credentials",
    // "retryable". A tool that declares one and does not get it cannot run.
    Requires []string

    // Accepts declares optional resources the tool uses when the run has them
    // ("memory", "target-language", "source-language"). A tool that declares
    // one and does not get it still runs.
    Accepts []string

    // Cardinality declares how many locales the tool operates on per execution.
    Cardinality LocaleCardinality

    // DefaultLocale is an optional default for monolingual and bilingual tools.
    DefaultLocale model.LocaleID

    // Consumes / Produces are the IO contract. Non-Optional consumed
    // ports are hard requirements the flow validator enforces.
    Consumes []IOPort
    Produces []IOPort

    // SideEffects lists external systems this tool reads from or writes to.
    SideEffects []SideEffect

    // Recoverable marks a transformer that vaults the originals it removes
    // and restores them later (redaction); the placement pass holds it to
    // the remote-egress rule.
    Recoverable bool

    WritesOutput          bool     // CLI adds -o/--output when true
    DefaultParallelBlocks int      // concurrency for IO-bound tools
    Aliases               []string // alternative CLI command names
    Internal              bool     // withheld from kapi exec, kapi tools and MCP; still a flow step
}

For example recycle optionally consumes source segmentation and produces a memory-match annotation, an alternative translation, and target; qa requires a target and produces the qa overlay. The flow loader uses these contracts for data-flow validation: a flow whose tool needs a port that no upstream tool or the source binding supplies is rejected at build (E-04).

Requires is also a construction hook. When a tool declares terms, the runner opens the project's terms store and brackets the tool with the term stages: term-lookup (package terms) runs terms.Locate, the one pass over the terms store and the recipe's term_rules:, and records where declared terms are used without grading them; enforcement then grades the uses by the gate's own policy. When a tool declares memory, a content-memory provider is injected from --memory or the project's own store; Accepts: memory asks for the same provider without failing when there is none. These stages are built by the runner rather than registered, so they do not appear in kapi tools; they are how a declaration is honoured.

Every governed step takes its terminology under one recipe key, term_rules:, as a list of profile.TermRule: one term, what to use instead, how hard it bites, and an optional concept id that ties it to the terms store and the graph. profile.TermRuleMap is the single projection into the map a prompt or a check consumes, and that map feeds the context fingerprint a target's Origin records (F-02).

Locale cardinality

Tools declare how many locales they operate on per execution:

type LocaleCardinality string

const (
    // Monolingual: operates on a single locale.
    Monolingual LocaleCardinality = "monolingual"

    // Bilingual: operates on exactly two locales, provided as a pair.
    Bilingual LocaleCardinality = "bilingual"

    // Multilingual: operates on N locales simultaneously.
    Multilingual LocaleCardinality = "multilingual"
)

Cardinality describes how many locales a tool needs. Which locales are provided at runtime by the runner or flow configuration, never hardcoded in the tool.

Cardinality, together with the target-language entry in Requires and Accepts, also decides whether a run can name a target language for the tool. ToolMeta.TakesTargetLanguage holds for a bilingual or multilingual tool, and for a monolingual one that requires or accepts target-language or names a DefaultLocale. A tool with no declared cardinality takes one. search-replace and xml-validation accept it, because they work on the source and, when the run names one, on that target as well. That target is one the input carries, as a bilingual file does. A monolingual input carries none, so kapi exec case-transform app.json --target-lang fr writes the transformed source to fr/app.json, over any file already there. kapi exec <tool> declares --target-lang from this answer, so a tool that works on the source alone, such as redact or encoding-detect, offers none. TestToolsDeclareTheTargetLanguageTheyRead in core/tools holds the declarations to what each tool's config factory reads.

Every kapi exec command offers --source-lang beside --format and --encoding, because every run passes the source language to the format reader. The reader labels each block with it, and picks by it the source variant of a multilingual file that names none. A monolingual tool whose result also depends on the source language, as segmentation picks its rules by it, declares source-language in Accepts.

Uniform locale access

A block holds its content as peer editions (F-02). The edition the block was read in, in the language SourceLocale names, anchors the document skeleton and inline code positions, but tools should not need to know whether a locale names that edition or a translation; they just need text for a given locale:

// Text returns the plain text for a locale: the source text if the
// locale matches the Block's source locale, otherwise the target text.
func (b *Block) Text(locale LocaleID) string

// SetText writes text for a locale (source if it matches the source
// locale, otherwise a target).
func (b *Block) SetText(locale LocaleID, text string)

// HasLocale reports whether the Block has content for the locale.
func (b *Block) HasLocale(locale LocaleID) bool

A bilingual tool comparing [fr, de] calls block.Text("fr") and block.Text("de"): identical code whether fr is source or target. SourceText() and TargetText(locale) are available when a tool specifically needs the edition the block was read in or one translation, and the view's SetEdition and RemoveEdition write any derived edition by its full key (model.EditionKey: locale, tone and channel).

Segments: one iterator over segmented and unsegmented blocks

A segment is a span in the segmentation overlay rather than a structural type (F-02), so a tool that works per segment would otherwise repeat the same branch: iterate the spans when a segmentation overlay exists, treat the whole block as one segment when none does, and map each per-segment write back into the right run range. The tool views (core/tool/view.go) carry that branch once:

type BlockView interface {
    // SourceSegments yields the source segments of the given segmentation
    // layer (LayerPrimary = primary), or one whole-block segment when none is
    // present.
    SourceSegments(layer string) iter.Seq[Segment]
}

type VariantView interface {
    BlockView
    // TargetSegments yields writable per-segment target production over the
    // source segmentation of the given layer, splicing each segment's runs back
    // into the block at the segment's range and preserving ignorable spans
    // verbatim.
    TargetSegments(loc model.LocaleID, layer string) iter.Seq[WritableSegment]
}

Reads reuse Anchor.ExtractRuns; writes use the inverse splice, which respects half-open ranges and Span.Ignorable(). A segment reads a translation's text through that translation's own segmentation (Block.TargetSegmentation), never the source's. A tool that wants the whole block keeps using SourceRuns(); a tool that wants segments opts into SourceSegments(""), on any side and any named layer, and pairs naturally with the alignment overlay for source-to-target segment correspondence.

Stand-off types and the payload registry

The stand-off types a tool consumes and produces are typed string constants (F-02). Positional, run-anchored layers use the OverlayType constants; block-scoped metadata uses the annotation-key constants. Both an overlay span's Value and an annotation value are typed payloads; the framework registers the well-known content payloads, and formats and plugins register additional types and their constructors via one payload registry (model.RegisterPayload / model.NewPayload):

// Positional layers (Block.Overlays): core/model/overlay.go
const (
    OverlaySegmentation  OverlayType = "segmentation"
    OverlayTerm          OverlayType = "term"
    OverlayEntity        OverlayType = "entity"
    OverlayCheck            OverlayType = "qa"
    OverlayAlignment     OverlayType = "alignment"
    OverlayTermCandidate OverlayType = "term-candidate"
)

// Block-scoped metadata (Block.Annotations): core/model/annotation_access.go
const (
    AnnoNote           = "note"
    AnnoAltTranslation = "alt-translation"
    AnnoMemoryMatch    = "tm-match"
    AnnoWordCount      = "word-count"
    AnnoVoice          = "voice"
    // …char-count, seg-count, comparison, repetition, entity-mapping,
    //   term-enforcement, scoping-report
)

A handful of these string values ("tm-match" above, and the memory and terms side effects below) keep spellings the prose has retired. Annotation keys and side-effect ids are a wire and storage boundary, where renaming a value splits a record rather than moving it. The Go identifiers carry the current vocabulary; the strings behind them do not change.

Every checker (terminology, do-not-translate, placeholder, quality, voice) writes the same qa overlay (a core/check.FindingsAnnotation payload carrying a []check.Finding plus a rolled-up score), so one scoring, annotation, and governance path serves them all.

A tool's Consumes/Produces name these overlay and annotation types, or a pseudo-port, so the same registry that discriminates a payload's concrete type on the wire is the vocabulary the flow validator checks the IO contract against.

Three consequences follow from validating on the declared contract. A wrong Consumes/Produces on a built-in tool breaks a real flow, so each tool's declaration is audited against its actual reads and writes, with an end-to-end test per built-in flow as the guardrail. The overlay and annotation vocabulary is open to plugins (E-05) through model.RegisterPayload: a plugin-defined type crosses the plugin wire by type name and JSON, and rehydrates to its concrete type wherever the payload constructor is registered. Alignment is the one relational overlay: it links a source span to a target span, so its payload carries the counterpart range while the Overlay shape stays single-sided.

Side effects

Side effects are a closed set of known external interactions:

type SideEffect string

const (
    SideEffectMemoryRead  SideEffect = "tm-read"
    SideEffectMemoryWrite SideEffect = "tm-write"
    SideEffectTermsRead   SideEffect = "termbase-read"
    SideEffectTermsWrite  SideEffect = "termbase-write"
    SideEffectAPICall     SideEffect = "api-call"
    SideEffectAnalytics   SideEffect = "analytics"

    // RemoteSourceEgress marks a tool that sends source content to a remote
    // system, distinct from APICall: a local detector, terms, or
    // content-memory lookup must not carry it, every cloud-provider call must.
    SideEffectRemoteSourceEgress SideEffect = "remote-source-egress"

    // ProcessExec marks a tool that hands block content to a program
    // outside kapi (external-command), whatever that program does with it.
    SideEffectProcessExec SideEffect = "process-exec"
)

Most side-effect declarations are informational metadata for the flow editor and the documentation. They are not enforced at runtime: a tool declaring a memory write still runs normally with no content memory configured; it skips the write. This keeps the tool interface simple while giving the UI enough information to warn meaningfully. The exceptions are RemoteSourceEgress and ProcessExec. The transformer placement pass and the redaction coverage check key a hard build error off each, and a tool whose remoteness depends on configuration (a model tool pointed at a local runtime or the offline demo provider) refines RemoteSourceEgress away through its contract resolver. A single-tool run under a redaction policy places redact ahead of a tool that carries ProcessExec or APICall.

Flow locale inference

The runner inspects the tool chain's cardinality declarations to determine which locales to process. This is the single applicability-based answer to "which locales does this flow run for": the CLI's project flow-run and the desktop runner both resolve their locale passes through it, so the two surfaces cannot disagree. Convergence (kapi up) intentionally answers a different question with a need-based selection: only the locales still short of their ship gate run a pass.

func ResolveFlowLocales(
    spec *StepsSpec,
    toolInfos map[registry.ToolID]registry.ToolInfo,
    sourceLocale string,
    projectTargets []string,
) [][]string

The runner passes the flow's *StepsSpec plus a map from registry.ToolID to registry.ToolInfo, which carries each tool's cardinality and default-locale metadata.

Resolution returns a slice of locale sets, one set per execution pass:

Flow	Tools	Passes
case-transform	[case-transform(mono)]	[[en]]
pseudo-translate	[pseudo-translate(bi, default:qps)]	[[en, qps]]
translate	[translate(bi)]	[[en, de], [en, fr], [en, ja], ...]
translate+qa	[translate(bi), qa(bi)]	[[en, de], [en, fr], ...]
compare de vs fr	[comparison(bi)] with config [de, fr]	[[de, fr]]
cross-locale check	[consistency-check(multi)]	[[en, de, fr, ja, nb, ar]]
translate + pseudo	[translate(bi), pseudo(bi, default:qps)]	[[en, de], [en, fr], ..., [en, qps]]

Mixed flows resolve to the union of all needed passes.

Parameter schemas

Tools declare parameter schemas via the tool.SchemaProvider interface with ComponentSchema in the core/schema/ package:

type SchemaProvider interface {
    Schema() *schema.ComponentSchema
}

type ComponentSchema struct {
    ID          string                    // "$id"
    Version     string                    // "$version"
    Title       string
    Description string
    Type        string                    // "object"
    ToolMeta    *ToolMeta                 // tool identity (see above)
    Groups      []ParameterGroup          // UI groupings ("ui:groups")
    StepMeta    *StepMeta                 // bridge step metadata, when applicable
    Properties  map[string]PropertySchema // parameter definitions
    RawJSON     json.RawMessage           // full schema access
}

schema.FromStruct(cfg, meta) generates a ComponentSchema by reflecting on a Go struct. It supports struct tags for additional metadata:

type PseudoConfig struct {
    ExpansionPercent int    `schema:"description=Text expansion percentage,min=0,max=200"`
    Prefix           string `schema:"description=Prefix for pseudo text"`
    Suffix           string `schema:"description=Suffix for pseudo text"`
    InternalField    string `schema:"-"` // excluded from schema
}

schema.ApplyConfig() bridges map[string]any configuration (from flow YAML) to a typed struct via a JSON round-trip.

The ToolRegistry stores schemas alongside factories via RegisterWithSchema(name, factory, schema). All built-in tools register auto-generated schemas. Schema-driven features:

• CLI flags: cli.RegisterSchemaFlags() auto-generates Cobra flags from the schema, mapping camelCase properties to kebab-case flags.
• Flow editor: schema-driven config panels for tool nodes, reusing the same editor component that drives format configuration.
• Validation: ComponentSchema.Validate() checks parameter values against the schema.
• JSON export: kapi tools schema <name> prints the schema for any tool.
• MCP exposure: host/mcp_tools.go registers every CLI-visible tool on the kapi mcp stdio server, projecting the tool's schema plus a text input into the MCP input schema and running the tool over the supplied text. The exposed set is scoped by mode, mirroring the desktop's project/ad-hoc split: inside a kapi project only the tools the project declares are advertised, with the project's target language as the default; ad-hoc, the full set is exposed. Resource-wrapping helpers (voice profile, terms, content memory) stay hand-authored in host/mcp_voice.go.

Model-backed tool schemas include provider fields (provider, API key, model, with enum support for provider selection), so their CLI flags are generated the same way as any other tool's.

Registration

Tools register into a registry.ToolRegistry with a name, factory function, and optional parameter schema:

reg.RegisterWithSchema("pseudo-translate", func() tool.Tool {
    return NewPseudoTranslateTool(&PseudoConfig{Prefix: "▒ ", Suffix: " ▒", TargetLocale: "qps"})
}, toolSchema(&PseudoConfig{Prefix: "▒ ", Suffix: " ▒"},
   toolMeta("pseudo-translate", "Pseudo Translate", schema.CategoryTranslation, ...)))

The factory is a zero-argument func() tool.Tool (registry.ToolFactory); it returns a tool built from a default config, with no error return. A separate config factory (SetConfigFactory) builds the tool from a config map when flow YAML overrides the defaults.

RegisterAll(reg) in core/tools/register.go auto-registers all built-in tools. The model-backed tools are auto-registered separately by aitools.RegisterAll and mttools.RegisterAll (core/ai/tools, core/mt/tools), called alongside the built-ins during app init. Each registers with a default offline factory plus a config factory; the real provider is resolved from the credential-bearing config map at tool-creation time, not at registration time (E-07).

A plugin manifest can declare flow tools (capabilities.tools), and the registry can list a tool it cannot build (ToolRegistry.RegisterMetadata), but the host registers no plugin tool, so a flow runs the built-in tools. A plugin reaches a flow through its formats and its segmentation engines (E-05).

Tool groups (pluggable backends)

Some tools are a family of interchangeable backends rather than one implementation: segmentation runs on SRX rules, UAX-29, the browser Intl.Segmenter, an LLM, or an ML segmenter; qa runs deterministic rules or an LLM judge; translate runs any model provider; entity-extract runs a local NER model, an LLM, or both. Each backend carries its own configuration and the family has a sensible default.

A tool group models this: a tool whose behaviour is provided by one of several self-describing members, selected by a discriminator field, with a default member and common config. The user picks a member and configures only that one; the group never merges members' parameters together.

reg.RegisterGroup(registry.ToolGroupDef{
    Name:          "segmentation",
    Discriminator: "engine",                  // the config key that selects the member
    Default:       "srx",
    Common:        commonSchema,              // discriminator + shared options + ToolMeta
    Members:       members,                   // each: Name, Label, Description, own Schema
    ConfigFactory: NewSegmentationFromConfig, // dispatches on the discriminator
})

Members are not separately-registered tools; they belong to the group, and the candidate members already exist as domain sub-registries: segmentation engines (M-02) and model providers (E-07).

A member may carry its own factory (ToolGroupMember.Factory). The group's registered ConfigFactory dispatches on the discriminator: a member with a factory is built directly, the rest fall through to the group's factory. This is the seam for runtime-contributed members: RegisterGroup defines the built-in members, and AddGroupMember(group, member) appends one later (recomposing the flat schema, MemberSchema, the ToolInfo.Group metadata, and the dispatcher), so a source outside the group's own package can extend it without that package knowing. The discriminator value selects the right backend, built-in or contributed.

For plugin-contributed members, segmentation wires its engines from the manifest segmenters[] capability into the segmentation engine sub-registry over a dedicated Segment RPC (E-05); segmentation members are narrow sub-components (text → boundaries), not full tools. The other groups' members are full block-processing tools; a manifest and daemon transport for contributing those is not yet built.

A group is a single registry entry, so flat consumers are unaffected; it serves two renderings from one definition:

• Master-detail (the config UI, the docs reference) reads ToolInfo.Group (members, default, discriminator) and MemberSchema(group, member), and renders the common fields plus a member selector plus only the selected member's own schema. Nothing is merged or hidden.
• Flat projection (CLI flags, MCP input, and the registry's Schema(id)) is inherently flat (cobra flags and a single input schema cannot be selected into at parse time), so the group projects to one schema via schema.ComposeVariants: a discriminator select plus each member's fields, grouped and gated to their member. ComposeVariants is therefore a projection of a group, not the model.

The discriminator carries a default, so the bare tool works with no configuration. A qa config selects its backend with mode alone; a provider selects nothing.

Annotation-based communication

Tools communicate through overlays and annotations on Blocks. A typical pipeline:

• entity-extract adds OverlayEntity with named entities and term candidates.
• recycle reads entity annotations for generalized matching and adds an alternative translation.
• translate reads term and entity annotations for context-aware translation.
• term-check verifies terminology consistency in the target.
• qa validates translation quality.

Each tool reads the annotations it cares about and adds its own, keeping tools loosely coupled through a shared data model rather than direct dependencies.

Built-in tool inventory

All built-in tools register via RegisterAll() in core/tools/register.go; the model-backed ones register from core/ai/tools and core/mt/tools. The authoritative list with every parameter is the generated Tool Reference; the categories below are the shape of it.

Translation (produce target content): translate (a model provider), recycle (content memory, exact and fuzzy), diff-leverage (reuse from a previous document version), pseudo-translate (deterministic translation-readiness testing), translate-after.

Quality (validate without rewriting content): qa (rule-based, or an LLM judge with mode: ai), review (per-translation score, assessment and suggestion), term-check, dnt-check (alias dnt), placeholder-check, voice-check, voice-vocab-check, xml-validation.

The qa checkset also carries length constraints (ratio and absolute character and word limits), invalid or forbidden characters and charset conformance, regex pattern rules (required and forbidden), and cross-block translation consistency, rather than splitting each rule family into its own tool.

Analysis (read-only metrics and reports): entity-extract, term-extract, voice-infer, encoding-detect.

Text processing (rewrite, segment, or redact): case-transform, search-replace, whitespace-correct, inline-codes-remove, span-classify, tag-protect, properties-set, create-target, remove-target, segmentation, media-refine, redact, unredact, plus the stream-shaping batch, layer-processor, external-command, and script.

Byte-level output style (BOM, newlines, charset) is writer configuration, not a pipeline stage: set output.bom / output.newline / output.encoding under any format's config and every writer applies them as a shared post-encode step (E-02).

external-command and script are the exec class: their job is to run code the configuration names, rather than to transform content with code kapi ships. They are ordinary tools on the command line (kapi exec runs both), but a recipe cannot arm one silently. Which surfaces may name them, and how a recipe asks, is E-06.

Model-backed tools

Model-backed tools are registered at startup like the other built-ins, so they appear in kapi tools and resolve in flows. Their distinguishing trait is provider injection: the registry holds a default offline-provider factory, and the real provider, with credentials, is supplied on demand via the config factory when the tool is instantiated. They use the same Tool interface and work identically in flows. Translation is one translate tool across every backend, with the backend chosen by --provider, and quality one qa tool, with the backend chosen by --mode (E-07).

Flow steps format

Flows are authored as a YAML step list. The parser compiles the list to the definition graph, and the host builds the ordered tool chain that runs (E-01):

A flow's source and sink are context-resolved bindings (E-04), not fields of the flow document; the steps carry only the composition.

apiVersion: v1
kind: FlowDefinition
metadata:
  name: Production Pipeline
spec:
  steps:
    - tool: recycle
      config:
        fuzzyThreshold: 75
    - tool: translate
      config:
        provider: anthropic
    - tool: qa
    - tool: term-check
    - tool: xml-validation

Steps run in the order listed. The definition parser also accepts a parallel: block, which no host path runs as concurrent branches. The script step lets authors drop in custom JavaScript when no existing tool fits.

Mutable streaming model

Tools modify Blocks in place as they flow through channels. This is a deliberate trade-off:

• Performance: no copying or delta accumulation for high-volume streaming; zero allocation per tool for pass-through Part types.
• Simplicity: tools read and write fields on the same Block. No immutable builders, lenses, or patch application.
• Proven pattern: the Okapi Framework, the Java content and translation framework neokapi reimagines, uses the same mutable-event model across thousands of production workflows.

Document-level immutability is achieved by external storage layers that version entire Block states. Within a single pipeline execution, mutable streaming is the right trade-off.

Stages pass the same Block from one to the next, so sending a Part on a channel hands responsibility for its Block to the receiver: a handler finishes its writes before the Part is emitted and leaves the Block alone afterwards. ParallelBlockTool relies on this, since each of its workers holds a distinct Block. A caller that sends one Block to several concurrent receivers, as tool.Tee does, has to add its own coordination.

Content immutability by capability

A tool's write surface is a compile-time property: it declares what it may write by which block handler it sets on BaseTool, and the handler's parameter type makes the wrong writes unrepresentable.

Handler	View	May write
Annotate(BlockView)	source + target read-only	overlays, annotations, properties
Produce(VariantView)	source read-only	derived editions (translations, and tone or channel editions by key), plus the above
Transform(BlockView)	source + target read-only	an edit plan the framework applies to source

• Analysis and check tools (qa, term-check, entity-extract, the segmenter) set Annotate. BlockView exposes no source or target setter, so they cannot mutate content; they emit overlays, annotations, and properties.
• Target-producing tools (translate, recycle, create-target) set Produce and write targets; source stays read-only.
• Transformers (redaction, normalization, case and encoding conversion) are the only tools that rewrite the edition a block was read in, and they never do so directly. A transformer is a read-only edit producer: it inspects the block and returns an edit plan: a set of structured model.RunEdits (a span → replacement map), in-place text edits that keep the inline codes, structure and run flags they leave alone (a substitution, a case conversion), any originals to vault (recoverable transformers such as redaction), or an opaque whole-block replacement for rewrites with no derivable mapping. A single framework-owned applier vaults any secrets first, then compiles the plan into change operations (EditPlan.Ops) and applies them through change.ApplyBlock, which applies the edits, rebases the surviving run-anchored overlays of the rewritten edition once (model.RemapOverlays) so segmentation, terms, and entities follow the rewrite, and bounds-checks the result, all atomically. Because tool code holds no source setter, a transformer cannot corrupt run-anchoring or leak a secret; an opaque whole-block replacement drops the overlays it cannot rebase. Recoverable transformers keep the original in a block annotation or a sidecar vault and restore it on the way out. The applier replaces a source through Block.EditSourceRuns, which keeps the source the reader produced, so a writer can tell a rewritten block from an untouched one (Block.SourceAsRead): it writes an untouched block as it was read and encodes the new wording of a rewritten one for its format.

The read views hand back the block's live run slices, which Go cannot make deeply immutable without copying. So a backstop in BaseTool's block dispatch content-hashes source and targets around each handler and errors if a handler edited a surface its capability forbids, catching in-place edits made through those aliased slices. Hashing every block twice per handler is dev and test tooling, not production work, so it is off unless a caller asks for it: an executor opts a whole run in with flow.WithImmutabilityCheck, any dispatch path opts in per context with tool.WithImmutabilityCheck, and the tool, tools, and flow test suites turn it on for every test. change.ApplyBlock likewise asserts that every surviving overlay span on an edition it rewrites still resolves in bounds against the new runs, on the source and on every target, so a rebase that left an overlay dangling is rejected; that check is unconditional.

A handler's view applies its writes the same way. Each target write, provenance stamp and overlay write is an operation applied at once through change.ApplyBlock, as the tool, so a handler reads what it wrote: a new target it stamps keeps the stamp. A tool's write that changes a target's wording makes the target a draft, since nobody has read what the tool wrote, and keeps the origin that names who produced it; a tool records what it produced itself with its stamp. A target write in the block's source language reaches a target the block holds in that language, and is refused when the block holds none, so it never replaces the source. A write the applier refuses is the handler's error. The contract the operations follow is in the change applier note.

What a run's tools leave in each block reaches a file through the format writer and the file home a change set commits through (E-01), and inside a project the kapi host records each document the run wrote as the flow's content.edit (E-09): the revisions of the editions the run moved, each translation's basis, and the stamp the producing tool left, which a file of strings has nowhere to keep.

A tool that genuinely needs the maximal surface (script, which runs arbitrary JavaScript) overrides Process instead and self-gates source mutation behind its own flag (allowSourceMutation). The script hands back a whole block, and change.Diff(before, after) turns the pair into operations, each guarded by the revision of the edition it changes: set_content for an edition added or rewritten, replace_text where one text edit with every inline code kept describes the change, and remove_edition. They apply through change.ApplyBlock as the script tool, so a whole-block write meets the same rules as a typed one.

The same function lets a run show what it would do. Under --print-ops (kapi exec <tool>, kapi run, kapi up, and the flow porcelains such as kapi translate and kapi pseudo-translate), the run commits through a home that writes nothing, and the difference between each block as the reader gave it and as the writer received it becomes the change set the run prints, which kapi apply applies (E-09). ksed --print-ops prints the operations its substitutions compile to the same way.

Transformer placement

Transformers and analyzers are ordinary steps in one ordered tool list; there is no separate structural stage. Because the applier mutates inline and in order, each transformer settles the source before later steps observe it, so analysis that depends on a transform (segmentation over normalized text, or an annotator feeding a redactor such as entity-extract → redact) sees the applied result.

Ordering safety is a placement pass that runs beside the data-flow contract, using the capability and SideEffects a tool already declares:

Severity	Rule	Rationale
Error	a transformer must not follow a step that produces a committed target, unless it produces the target port itself (unredact rewrites both sides coherently)	rewriting source orphans the targets, which anchor to it
Error	a recoverable (redacting) transformer must run before any step that egresses source to a remote sink, except the step or steps producing an input its config-resolved contract requires	otherwise unprotected source leaks before redaction applies; a cloud NER feeding entity-driven redaction is the documented detection trade-off (C-10)
Warning	a transformer placed later than its earliest valid slot (after its last required input)	every overlay present at apply time must be rebased; an earlier slot avoids the work

The remote-egress rule keys off a remote source egress side effect, distinct from a plain API call, so a local detector or term lookup does not trip it while a cloud-provider call does. The effect itself is config-refined: a model tool pointed at a local provider carries no remote egress. Tools, including plugins, contribute their own placement diagnostics through the same config-derived contract hook that resolves a tool's required inputs from its configuration. For example redaction requires an upstream entity overlay only when entity detection is enabled, and only a required input exempts its producer from the egress rule, so a rules-only redact placed after a cloud NER step is still rejected.

Consequences

• Implementing a new tool requires only embedding BaseTool and setting one handler function field.
• Unhandled Part types pass through automatically; there is no risk of accidentally dropping Parts.
• Every content write a tool makes, through a view, an edit plan or change.Diff, is an operation change.ApplyBlock applies, so one set of rules guards every write (E-09).
• Schema-driven CLI flags, flow-editor config panels, and validation all share one schema representation, so a change to a tool's config propagates automatically.
• IO contracts enable flow-level locale inference: the runner works out whether to iterate project targets, run once on source, or run for a specific locale set from the declared cardinality.
• Annotation-based inter-tool communication keeps tools loosely coupled through shared data rather than direct dependencies.
• Typed constants for overlay types, side effects, and locale cardinality catch typos at compile time and enable IDE autocomplete.
• Mixed-cardinality flows resolve cleanly through pass union; tool authors do not coordinate locale iteration.
• The capability-typed handler makes the wrong write unrepresentable rather than merely discouraged, and the single applier means overlay rebasing and secret vaulting are written once rather than per transformer.
• Tool groups let a family of interchangeable backends share one tool while each backend keeps its own config; members come from domain registries or plugins, and the same definition drives the master-detail config UI and the flat CLI/docs/MCP projection.

Related

• F-02: The content model: Blocks, overlays, annotations, and Fragment projections
• E-01: The processing engine: how tools compose into flows
• E-02: The format system: readers and writers bracket the tool chain
• E-04: Flows and I/O binding: a flow is composition only; tool = unit, binding = the ends
• E-05: The plugin system: plugin formats and segmentation engines
• E-09: The change contract: the operations every tool write becomes, the file home a flow commits through, and --print-ops
• E-06: Execution trust: the exec class and how a recipe arms it
• E-07: Model and translation providers: provider injection into model-backed tools