Skip to main content

AD-006: Tool System

Summary

A Tool is a single stage in a processing pipeline. It reads Parts from an input channel and writes Parts to an output channel. Tools compose into Flows; Flows are executed by the pipeline engine (AD-004: Processing Engine). The BaseTool struct with optional handler fields — a capability-typed block handler (Annotate / Produce / Transform) plus untyped HandleDataFn, HandleMediaFn for other Part types — lets most tools implement only the handler for the Part type they care about; everything else passes through unchanged. The block handler a tool sets also declares what it may write (see "Content immutability by capability" below). Tools declare parameter schemas via SchemaProvider, which drives CLI flag generation, flow-editor config panels, and validation. An IO contract on ToolMeta declares locale cardinality, the stand-off layers a tool produces, and side effects so the runner can infer locale iteration and the flow editor can show data flow. A tool whose behaviour comes from one of several interchangeable backends (segmentation engines, AI/MT providers, QA modes) is a tool group — it registers self-describing members with a default, and the user selects and configures one member at a time.

Context

Most tools only care about one or two Part types. A translation tool processes Blocks; a word counter reads Blocks; a binary extractor handles Media. Requiring every tool to implement the full Process(ctx, in, out) method with a type switch over all Part types produces repetitive boilerplate and creates risk of accidentally dropping Parts.

Beyond structural dispatch, a tool system needs to answer several questions uniformly for CLI, flow editor, and plugin consumers:

  • What parameters does this tool accept, and what are their types?
  • How many locales does it operate on? Which ones?
  • What stand-off layers does it produce? Which does it consume?
  • What external systems does it touch (content memory, terms, APIs)?

Decision

Tool interface and BaseTool dispatch

The core interface is minimal:

type Tool interface {
Process(ctx context.Context, in <-chan *Part, out chan<- *Part) error
}

BaseTool provides a standard dispatch shell. The block handler is one of three capability-typed fields — the tool sets exactly one, and the parameter type bounds what it may write:

type BaseTool struct {
Annotate func(BlockView) error // read-only: overlays/annotations/properties
Produce func(VariantView) error // writes target
Transform func(BlockView) (EditPlan, error) // read-only producer; the applier rewrites source

HandleDataFn func(ctx context.Context, data *Data) (*Data, error)
HandleMediaFn func(ctx context.Context, media *Media) (*Media, error)

SchemaFn func() *schema.ComponentSchema
}

BaseTool.Process reads Parts from the input channel, dispatches Blocks to whichever capability-typed handler is set (and other Part types to their Handle*Fn), and passes unhandled Part types through unchanged. Concrete tools embed BaseTool and set only the handlers they need. A tool that needs the full stream — batching, 1→N fan-out, cross-block state (e.g. the batch collector, the concurrent translate path) — overrides Process directly; it may reuse a typed handler over a held block via tool.NewBlockView/NewVariantView.

SessionTool extension

The channel-based Tool.Process is a forward-only transform. Some tools need random access to the project's block state — lookup by content hash, reading prior overlays (memory matches, QA findings, previously-produced targets) to skip work that's already done, or writing annotations that downstream tools in the same or a later run will consult. Those tools opt into the SessionTool interface alongside Tool:

type SessionTool interface {
Tool

SessionProcess(
ctx context.Context,
sess blockstore.Session,
in <-chan *Part,
out chan<- *Part,
) error
}

Lifecycle (owned by the executor, not the tool):

  1. At flow start the executor opens a blockstore.Session against the project's declared store backend (memory, cache, remote — see AD-008: Kapi Project Model).
  2. For each tool the executor calls SessionProcess when the tool implements SessionTool, otherwise the plain streaming Process. Hybrid implementations are allowed: SessionProcess can read from in, enrich via the session, and emit to out.
  3. The executor commits the session on success or rolls back on error. Tools MUST NOT call Commit / Rollback themselves.

SessionTool is additive — every SessionTool also implements Tool so flow composition (chaining steps that may or may not use the session) keeps working. See the SessionTool authoring guide for idiomatic patterns (skip-if-cached, overlay conventions, provider selection).

Tool categories

Tools fall into four categories that set expectations for idempotency and ordering:

CategoryResponsibilityExamples
TransformModify content in placecase change, search/replace, redaction
EnrichAdd metadata or overlayssegmentation, memory leveraging, AI translation, terminology lookup
ValidateCheck quality without modifyingQA checks, word count, character count
ConvertTransform representationsEncoding conversion, line-break normalization

IO model

Each tool declares an IO contract in its ToolMeta (package core/schema). The contract is expressed over IOPorts — typed stand-off layers of a Block (AD-002) — not over coarse part-type names: Consumes lists the ports a tool reads upstream and Produces the ports it writes. An IOPort's Type names an overlay type (term, qa, …), a block-annotation type (brand-voice, …), or a pseudo-port (PortTarget / PortSource); its Side says which side it pertains to; and Optional marks a consumed port as degradable (graceful degradation) rather than required.

// core/schema/schema.go
type IOPort struct {
Type string // overlay type, annotation type, or "target"/"source"
Side model.Side // source | target
Optional bool // consumed: degrades without it, does more with it
Layer string // segmentation granularity; LayerPrimary = primary
}

// PortTarget is the committed Target; PortSource is a rewritten source.
const (
PortTarget = "target"
PortSource = "source"
)

type ToolMeta struct {
ID string
Category string // "translate","validate","enrich","convert","transform","pipeline"
DisplayName string
Description string
Tags []string

// Requires declares external resources the tool needs at runtime.
Requires []string // "target-language","tm","termbase","credentials",…

// Cardinality declares how many locales the tool operates on per execution.
Cardinality LocaleCardinality

// DefaultLocale is an optional default for monolingual and bilingual tools.
DefaultLocale model.LocaleID

// Consumes / Produces are the IO contract. Non-Optional consumed
// ports are hard requirements the flow validator enforces.
Consumes []IOPort
Produces []IOPort

// SideEffects lists external systems this tool reads from or writes to.
SideEffects []SideEffect

// Recoverable marks a transformer that vaults the originals it removes
// and restores them later (redaction); the placement pass holds it to
// the remote-egress rule.
Recoverable bool

WritesOutput bool // CLI adds -o/--output when true
DefaultParallelBlocks int // concurrency for IO-bound tools
Aliases []string // alternative CLI command names
}

For example recycle optionally consumes source segmentation and produces tm-match, alt-translation and target; qa requires a target and produces qa. The flow loader uses these contracts for data-flow validation — a flow whose tool needs a port that no upstream tool or the source binding supplies is rejected at build (AD-026).

Locale cardinality

Tools declare how many locales they operate on per execution:

type LocaleCardinality string

const (
// Monolingual — operates on a single locale.
// Examples: case-transform (source), pseudo-translate (target),
// encoding-detect (source).
Monolingual LocaleCardinality = "monolingual"

// Bilingual — operates on exactly two locales, provided as a pair.
// Examples: translate (source→target), qa (source vs target).
Bilingual LocaleCardinality = "bilingual"

// Multilingual — operates on N locales simultaneously.
// Examples: cross-locale comparison, cross-locale QA.
Multilingual LocaleCardinality = "multilingual"
)

Cardinality describes how many locales a tool needs. Which locales are provided at runtime by the runner or flow configuration — never hardcoded in the tool.

Uniform locale access

Blocks carry one source locale and N target locales. The source locale is structurally distinct because it anchors the document skeleton and inline code positions, but tools should not need to know whether a locale is "source" or "target" — they just need text for a given locale:

// Text returns the plain text for a locale: the source text if the
// locale matches the Block's source locale, otherwise the target text.
func (b *Block) Text(locale LocaleID) string

// SetText writes text for a locale (source if it matches the source
// locale, otherwise a target).
func (b *Block) SetText(locale LocaleID, text string)

// HasLocale reports whether the Block has content for the locale.
func (b *Block) HasLocale(locale LocaleID) bool

A bilingual tool comparing [fr, de] calls block.Text("fr") and block.Text("de") — identical code whether fr is source or target. SourceText() and TargetText(locale) remain available when a tool specifically needs the source-anchored skeleton.

Stand-off types and the payload registry

The stand-off types a tool consumes and produces are typed string constants (AD-002). Positional, run-anchored layers use the OverlayType constants (OverlaySegmentation, OverlayTerm, OverlayEntity, OverlayQA, OverlayAlignment, OverlayTermCandidate); block-scoped metadata uses the annotation-key constants (AnnoNote, AnnoAltTranslation, AnnoMemoryMatch, AnnoWordCount, …). Both an overlay span's Value and an annotation value are typed payloads; the framework registers the well-known content payloads, and formats and plugins register additional types and their constructors via one payload registry (model.RegisterPayload / NewPayload):

// Positional layers (Block.Overlays) — core/model/overlay.go
const (
OverlaySegmentation OverlayType = "segmentation"
OverlayTerm OverlayType = "term"
OverlayEntity OverlayType = "entity"
OverlayQA OverlayType = "qa"
OverlayAlignment OverlayType = "alignment"
OverlayTermCandidate OverlayType = "term-candidate"
)

// Block-scoped metadata (Block.Annotations) — core/model/annotation_access.go
const (
AnnoNote = "note"
AnnoAltTranslation = "alt-translation"
AnnoMemoryMatch = "tm-match"
AnnoWordCount = "word-count"
// …char-count, seg-count, comparison, repetition, brand-voice, …
)

The IO contract also uses two pseudo-ports — PortTarget ("target", the committed Target) and PortSource ("source", a rewritten source) — which name produced/consumed outputs that participate in data-flow validation but are not stored as stand-off layers.

Every checker — terminology, do-not-translate, placeholder, QA, brand voice — writes the same qa overlay (a core/check.FindingsAnnotation payload carrying a []check.Finding plus a rolled-up score), so one scoring, annotation, and governance path serves them all.

A tool's Consumes/Produces name these overlay and annotation types (or a pseudo-port), so the same registry that discriminates a payload's concrete type on the wire is the vocabulary the flow validator checks the IO contract against.

Side effects

Side effects are a closed set of known external interactions:

type SideEffect string

const (
SideEffectMemoryRead SideEffect = "tm-read"
SideEffectMemoryWrite SideEffect = "tm-write"
SideEffectTermsRead SideEffect = "termbase-read"
SideEffectTermsWrite SideEffect = "termbase-write"
SideEffectAPICall SideEffect = "api-call"
SideEffectAnalytics SideEffect = "analytics"

// RemoteSourceEgress marks a tool that sends source content to a remote
// system — deliberately distinct from APICall: a local detector, terms, or
// content-memory lookup must not carry it, every cloud-provider call must.
SideEffectRemoteSourceEgress SideEffect = "remote-source-egress"
)

Most side-effect declarations are informational metadata for the flow editor and documentation. They are not enforced at runtime — a tool with SideEffects: [SideEffectMemoryWrite] still runs normally even if no content memory is configured (it simply skips the write). This keeps the tool interface simple while giving the UI enough information to warn meaningfully. The one exception is RemoteSourceEgress: the transformer placement pass (below) keys a hard build/load error off it, and a tool whose remoteness depends on configuration (an AI tool pointed at a local Ollama or the offline demo provider) refines it away through its contract resolver.

Flow locale inference

The runner inspects the tool chain's cardinality declarations to determine which locales to process. This is the single applicability-based answer to "which locales does this flow run for": the CLI's project flow-run and the Kapi Desktop runner both resolve their locale passes through it (via the cli module's shared multi-locale orchestrator), so the two surfaces cannot disagree. Convergence (kapi up) intentionally answers a different question with a need-based selection — only the locales still short of their ship gate run a pass.

func ResolveFlowLocales(
spec *StepsSpec,
toolInfos map[registry.ToolID]registry.ToolInfo,
sourceLocale string,
projectTargets []string,
) [][]string

The runner passes the flow's *StepsSpec plus a map from registry.ToolID to registry.ToolInfo (which carries each tool's cardinality and default-locale metadata), not a []ToolMeta slice.

Resolution returns a slice of locale sets — one set per execution pass. Examples:

FlowToolsPasses
case-transform[case-transform(mono)][[en]]
pseudo-translate[pseudo-translate(bi, default:qps)][[en, qps]]
translate[translate(bi)][[en, de], [en, fr], [en, ja], ...]
translate+qa[translate(bi), qa(bi)][[en, de], [en, fr], ...]
compare de vs fr[comparison(bi)] with config [de, fr][[de, fr]]
cross-locale QA[consistency-check(multi)][[en, de, fr, ja, nb, ar]]
translate + pseudo[translate(bi), pseudo(bi, default:qps)][[en, de], [en, fr], ..., [en, qps]]

Mixed flows resolve to the union of all needed passes.

Parameter schemas

Tools declare parameter schemas via the tool.SchemaProvider interface with ComponentSchema in the core/schema/ package:

type SchemaProvider interface {
Schema() *schema.ComponentSchema
}

type ComponentSchema struct {
ID string // "$id"
Version string // "$version"
Title string
Description string
Type string // "object"
ToolMeta *ToolMeta // tool identity (see above)
Groups []ParameterGroup // UI groupings ("ui:groups")
StepMeta *StepMeta // Okapi-bridge step metadata, when applicable
Properties map[string]PropertySchema // parameter definitions
RawJSON json.RawMessage // full schema access
}

schema.FromStruct(cfg, meta) generates a ComponentSchema by reflecting on a Go struct. It supports struct tags for additional metadata:

type PseudoConfig struct {
ExpansionPercent int `schema:"description=Text expansion percentage,min=0,max=200"`
Prefix string `schema:"description=Prefix for pseudo text"`
Suffix string `schema:"description=Suffix for pseudo text"`
InternalField string `schema:"-"` // excluded from schema
}

schema.ApplyConfig() bridges map[string]any configuration (from flow YAML) to a typed struct via JSON round-trip.

The ToolRegistry stores schemas alongside factories via RegisterWithSchema(name, factory, schema). All built-in tools register auto-generated schemas.

Schema-driven features:

  • CLI flagscli.RegisterSchemaFlags() auto-generates Cobra flags from the schema, mapping camelCase properties to kebab-case flags.
  • Flow editor — schema-driven config panels for tool nodes, reusing the same FilterConfigEditor component that drives format filter configuration.
  • ValidationComponentSchema.Validate() checks parameter values against the schema.
  • JSON exportkapi tools schema <name> prints the schema for any tool.
  • MCP exposurehost/mcp_tools.go registers every CLI-visible tool on the kapi mcp stdio server, projecting the tool's schema (plus a text input) into the MCP input schema and running the tool over the supplied text. The exposed set is scoped by mode, mirroring the desktop's ListTools vs ListProjectTools split: inside a kapi project only the tools the project declares are advertised (with the project's target language as the default); ad-hoc (no project resolved), the full set is exposed. Resource-wrapping helpers (brand profile, terms, content memory) stay hand-authored in host/mcp_brand.go.

AI tool schemas include provider fields (Provider, APIKey, Model with enum support for provider selection), so AI-tool CLI flags are generated the same way as any other tool's.

Registration

Tools register into a ToolRegistry with a name, factory function, and optional parameter schema:

reg.RegisterWithSchema("pseudo-translate", func() tool.Tool {
return NewPseudoTranslateTool(&PseudoConfig{Prefix: "▒ ", Suffix: " ▒", TargetLocale: "qps"})
}, toolSchema(&PseudoConfig{Prefix: "▒ ", Suffix: " ▒"}, toolMeta("pseudo-translate", "Pseudo Translate", schema.CategoryTranslation, ...)))

The factory is a zero-argument func() tool.Tool (registry.ToolFactory); it returns a tool built from a default config, with no error return. A separate config factory (SetConfigFactory) builds the tool from a config map when flow YAML overrides the defaults.

RegisterAll(reg) in core/tools/register.go auto-registers all built-in tools. AI and MT tools are auto-registered separately by aitools.RegisterAll and mttools.RegisterAll (core/ai/tools, core/mt/tools), called alongside the built-ins during App init (cli/app.go). Each registers with a default offline factory (the mock LLM provider for AI tools, the demo MT provider for the <provider>-translate tools) plus a config factory (SetConfigFactory); the real provider is resolved from the credential-bearing config map at tool-creation time, not at registration time.

Plugin tools (AD-007: Plugin System and Okapi Bridge) use the same Tool interface via gRPC translation, so plugin-provided tools and built-in tools are interchangeable from the pipeline's perspective.

Tool groups (pluggable backends)

Some tools are a family of interchangeable backends rather than one implementation: segmentation runs on SRX, UAX-29, Intl.Segmenter, an LLM, or the SaT ML model; qa runs deterministic rules or an LLM judge; translate runs any LLM or MT provider; entity-extract runs a local NER model, an LLM, or both. Each backend carries its own configuration and the family has a sensible default.

A ToolGroup models this: a tool whose behaviour is provided by one of several self-describing members, selected by a discriminator field, with a default member and common config. The user picks a member and configures only that one — the group never merges members' parameters together.

reg.RegisterGroup(registry.ToolGroupDef{
Name: "segmentation",
Discriminator: "engine", // the config key that selects the member
Default: "srx",
Common: commonSchema, // discriminator + shared options + ToolMeta
Members: members, // each: Name, Label, Description, own Schema
ConfigFactory: NewSegmentationFromConfig, // dispatches on the discriminator
})

Members are not separately-registered tools — they belong to the group, and the candidate members already exist as domain sub-registries: segmentation engines (AD-002), AI and MT providers (AD-011, AD-012).

A member may carry its own factory (ToolGroupMember.Factory). The group's registered ConfigFactory dispatches on the discriminator: a member with a factory is built directly, the rest fall through to the group's factory. This is the seam for runtime-contributed membersRegisterGroup defines the built-in members, and AddGroupMember(group, member) appends one later (recomposing the flat schema, MemberSchema, the ToolInfo.Group metadata, and the dispatcher), so a source outside the group's own package can extend it without that package knowing. The discriminator value selects the right backend whether it is built-in or contributed.

For plugin-contributed members, segmentation wires its engines from the manifest segmenters[] capability into the segmentation engine sub-registry over a dedicated Segment RPC (AD-007) — segmentation members are narrow sub-components (text → boundaries), not full tools. The other groups' members are full block-processing tools; a manifest/daemon transport for contributing those is not yet wired (the AddGroupMember seam above is the registry-side foundation it will build on).

A group is a single registry entry, so flat consumers are unaffected; it serves two renderings from one definition:

  • Master-detail (the config UI, the docs reference) reads ToolInfo.Group (members, default, discriminator) and MemberSchema(group, member), and renders the common fields + a member selector + only the selected member's own schema. Nothing is merged or hidden.
  • Flat projection (CLI flags, MCP input, and the registry's Schema(id)) is inherently flat — cobra flags and a single input schema cannot be selected into at parse time — so the group projects to one schema via schema.ComposeVariants: a discriminator select plus each member's fields, grouped and gated to their member. ComposeVariants is therefore a projection of a group, not the model.

The discriminator carries a default, so the bare tool works with no configuration. Where a tool can infer its backend from another field (e.g. qa from whether a provider is set), that inference is the fallback when the discriminator is unset.

Annotation-based communication

Tools communicate through annotations on Blocks. A typical pipeline:

sourcechanentity-extractchanterm-lookupchanrecyclechantranslatechanterm-enforcechanqachansink
  • entity-extract adds OverlayEntity with named entities.
  • term-lookup adds OverlayTerm with matched terminology.
  • recycle reads entity annotations for generalized matching, adds AltTranslation.
  • translate reads term and entity annotations for context-aware translation.
  • term-enforce validates terminology consistency in targets.
  • qa validates translation quality.

Each tool reads the annotations it cares about and adds its own, keeping tools loosely coupled through a shared data model rather than direct dependencies.

Built-in tool inventory

All built-in tools register via RegisterAll() in core/tools/register.go.

Transform tools — modify content in place:

ToolDescription
pseudo-translateGenerate pseudo-translations with accent marks and prefix/suffix wrapping
search-replaceRegex-based search and replace in content
case-transformTransform case of source and/or target text
create-targetCreate a target for blocks, optionally copying the source runs
remove-targetRemove a locale's target (or all targets) from blocks
inline-codes-removeStrip inline-code runs to produce clean plain text
properties-setSet or modify block properties programmatically
whitespace-correctNormalize and fix whitespace issues in translations
span-classifyReclassify code:markup spans into semantic vocabulary types
tag-protectIdentify and mark tags and placeholders for protection
redactReplace sensitive spans with placeholders pre-translation (recoverable transformer)
unredactRestore redacted spans from the vault post-translation

Enrich tools — add metadata or overlays via annotations:

ToolDescription
segmentationAnnotate blocks with a sentence-segmentation overlay (SRX-like rules)
recyclePre-fill translations from content memory
diff-leverageCompare against previous version, preserve translations for unchanged text

Validate tools — check quality without modifying:

ToolDescription
qaRule-based quality checks (missing translations, whitespace, numbers, span constraints)
dnt-checkFlag do-not-translate spans that were translated in the target (alias dnt)
placeholder-checkVerify placeholders/variables are preserved between source and target
brand-vocab-checkCheck target text against brand vocabulary / preferred-term rules
term-checkVerify terminology usage in translations against the terms store
xml-validationValidate XML well-formedness of block text

The qa checkset also carries length constraints (ratio and absolute character/word limits), invalid or forbidden characters and charset conformance, regex pattern rules (required and forbidden), and cross-block translation consistency, rather than splitting each rule family into its own tool.

Analyze tools — inspect byte-level characteristics:

ToolDescription
encoding-detectDetect encoding characteristics of block text

Byte-level output style (BOM, newlines, charset) is writer configuration, not a pipeline stage: set output.bom / output.newline / output.encoding under any format's config (defaults.formats[<id>].config) and every writer applies them as a shared post-encode step (see AD-005).

Pipeline tools — operate on the part stream:

ToolDescription
layer-processorApply format-specific tool chains to child layers
external-commandExecute an external command on block text
scriptRun user-provided JavaScript (ES5 via goja) on each part
batchCollect blocks into configurable batches for downstream batch processing

external-command and script are the exec class: their job is to run code the configuration names, rather than to transform content with code kapi ships. They are ordinary tools on the command line — kapi exec runs both — but a recipe cannot arm one silently. Which surfaces may name them, and how a recipe asks, is AD-038.

AI, MT, and terminology tools

AI and MT tools are registered at startup like the other built-ins, so they appear in kapi tools and resolve in flows. Their distinguishing trait is provider injection: the registry holds a default offline-provider factory, and the real LLM/MT provider (with credentials) is supplied on demand via the config factory when the tool is instantiated. They use the same Tool interface and work identically in flows.

AI tools (core/ai/tools/):

ToolDescription
translateTranslate blocks using an LLM (or MT) provider (batch + concurrent)
qa --providerCheck translation quality using an LLM provider
reviewReview translations with explanations using an LLM
term-extractExtract terminology from blocks using an LLM
entity-extractExtract named entities and term candidates using AI + optional NER

MT tools (core/mt/tools/):

ToolDescription
{provider}-translateTranslate blocks using an MT provider (DeepL, Google, Microsoft, ModernMT, MyMemory)

Terminology tools (terms/):

ToolDescription
term-lookupAnnotate blocks with matching terms from a Terminology
term-enforceVerify correct terminology usage in translations

Content-memory tools (memory/):

ToolDescription
recycleContent-aware memory leverage with generalized, structural, and plain matching

Flow steps format

Flows are authored as a YAML step list (compiled to the internal graph by the executor, see AD-004: Processing Engine):

A flow's source and sink are context-resolved bindings (AD-026: Flow I/O Binding), not fields of the flow document; the steps carry only the composition.

apiVersion: v1
kind: FlowDefinition
metadata:
name: Production Pipeline
spec:
steps:
- tool: recycle
config:
fuzzyThreshold: 75
- tool: translate
config:
provider: anthropic
- tool: qa
- parallel:
- tool: term-check
- tool: xml-validation

Steps are sequential by default; parallel: blocks provide fan-out. The script step lets authors drop in custom JavaScript when no existing tool fits.

Mutable streaming model

Tools modify Blocks in place as they flow through channels. This is a deliberate trade-off:

  • Performance — no copying or delta accumulation for high-volume streaming; zero allocation per tool for pass-through Part types.
  • Simplicity — tools read and write fields on the same Block. No immutable builders, lenses, or patch application.
  • Proven pattern — the Okapi Framework, a Java localization framework, uses the same mutable-event model across thousands of production workflows.

Document-level immutability is achieved by external storage layers that version entire Block states. Within a single pipeline execution, mutable streaming is the right trade-off.

Content immutability by capability

Mutable-in-place does not mean anything goes. A tool's write surface is a compile-time property: it declares what it may write by which process-named block handler it sets on BaseTool, and the handler's parameter type makes the wrong writes unrepresentable.

HandlerViewMay write
Annotate(BlockView)source + target read-onlyoverlays, annotations, properties
Produce(VariantView)source read-onlytarget content (+ the above)
Transform (edit producer)source + target read-onlyan edit plan the framework applies to source
  • Analysis / annotation tools (qa, term-check, term-lookup, entity-extract, the segmenter) set Annotate. BlockView exposes no source/target setter, so they cannot mutate content — they emit overlays, annotations, and properties.
  • Translation tools (translate, the MT tools, recycle, create-target) set Translate and write Block.Targets; source stays read-only.
  • Transformers (redaction, normalization, case/encoding conversion) are the only tools that rewrite Block.Source, and they never do so directly. A transformer is a read-only edit producer: it inspects the block and returns an edit plan — a set of structured model.RunEdits (a span→replacement map), any originals to vault (recoverable transformers such as redaction), or an opaque whole-block replacement for rewrites with no derivable mapping (LLM simplification). A single framework-owned applier is the one place that mutates the block: it applies the edits, rebases the surviving run-anchored overlays once (model.RemapOverlays) so segmentation, terms, and entities (see AD-002) follow the rewrite, vaults any secrets, and bounds-checks the result — atomically. Because tool code holds no source setter, a transformer cannot corrupt run-anchoring or leak a secret; an opaque whole-block replacement drops the overlays it cannot rebase. Recoverable transformers (redaction) keep the original in a block annotation or a sidecar vault and restore it on the way out.

The read views hand back the block's live run slices, which Go cannot make deeply immutable without copying. So a dev/test backstop in BaseTool.handleBlock content-hashes source and targets around each handler and errors if a handler edited a surface its tier forbids (catching in-place edits through those aliased slices). The applier likewise asserts that every surviving source overlay span still anchors in-bounds against the rewritten runs (Block.SourceOverlaysInBounds), so a rebase that left an overlay dangling is rejected. The backstop is gated by tool.EnforceImmutability (on by default). A tool that genuinely needs the maximal surface — script, which runs arbitrary JavaScript — overrides Process instead and self-gates source mutation behind its allowSourceMutation flag.

Transformer placement

Transformers and analyzers are ordinary steps in one ordered tool list; there is no separate structural stage. Because the applier mutates inline and in order, each transformer settles the source before later steps observe it, so analysis that depends on a transform — segmentation over normalized text, an annotator feeding a redactor (entity-extractredact) — sees the applied result.

Ordering safety is a placement pass that runs beside the data-flow contract, using the Capability and SideEffects a tool already declares:

SeverityRuleRationale
Errora transformer must not follow a step that produces a committed target — unless it produces the target port itself (unredact rewrites both sides coherently)rewriting source orphans the targets, which anchor to it
Errora recoverable (redacting) transformer must run before any step that egresses source to a remote sink — except the step(s) producing an input its config-resolved contract requiresotherwise unprotected source leaks before redaction applies; a cloud NER feeding entity-driven redaction is the documented detection trade-off (AD-020)
Warninga transformer placed later than its earliest valid slot (after its last required input)every overlay present at apply time must be rebased; an earlier slot avoids the work

The remote-egress rule keys off a remote source egress side-effect (schema.SideEffectRemoteSourceEgress), distinct from a plain API call, so a local detector or term lookup does not trip it while a cloud-provider call does. The effect itself is config-refined: an AI tool pointed at a local provider (Ollama, the offline demo) carries no remote egress. Tools — including plugins — contribute their own placement diagnostics through the same config-derived contract hook that resolves a tool's required inputs from its configuration (e.g. redaction requires an upstream entity overlay only when entity detection is enabled — and only a required input exempts its producer from the egress rule, so a rules-only redact placed after a cloud NER step is still rejected).

Consequences

  • Implementing a new tool requires only embedding BaseTool and setting one handler function field.
  • Unhandled Part types pass through automatically; no risk of accidentally dropping Parts.
  • Plugin tools use the same interface via gRPC translation, so the pipeline treats all tools uniformly (AD-007: Plugin System and Okapi Bridge).
  • Schema-driven CLI flags, flow editor config panels, and validation all share one schema representation — changes to a tool's config propagate automatically.
  • IO contracts enable flow-level locale inference: the runner figures out whether to iterate project targets, run once on source, or run for a specific locale set based on declared cardinality.
  • Annotation-based inter-tool communication keeps tools loosely coupled through shared data, not direct dependencies.
  • Typed constants for AnnotationType, SideEffect, and LocaleCardinality catch typos at compile time and enable IDE autocomplete.
  • Mixed-cardinality flows resolve cleanly through pass union; tool authors do not coordinate locale iteration.
  • Tool groups let a family of interchangeable backends share one tool while each backend keeps its own config; members come from domain registries or plugins, and the same definition drives master-detail config UI and the flat CLI/docs/MCP projection.