Tools

A tool is the unit of processing in neokapi. A format reader turns a document into a stream of Parts, and a writer turns the stream back into a document. A tool sits in between: it reads Parts from an input channel, transforms them, and writes them to an output channel.

Because every tool speaks the same channel contract, tools compose freely. A translation run is one chain: leverage from memory, look up terminology, translate the remainder, check quality. A monolingual voice pass is another: check the source, rewrite the off-voice runs, check again. Each tool handles the Parts it cares about and passes the rest through untouched. The same interface backs analysis, transformation, enrichment and validation. The generated Tool Reference lists the tools in the current build.

Try a tool on a file

Pick a tool, edit its configuration in the live form, and run it on a sample file to see how each Block changes, source before and tool output after. The same form that drives the configuration here is the one the visual editors and the Tool Reference render from the tool's schema. This runs the real kapi engine in your browser via WebAssembly.

Loading the interactive lab…

The Tool interface

A tool is anything that satisfies one small interface:

type Tool interface {
    Name() string
    Description() string
    Process(ctx context.Context, in <-chan *model.Part, out chan<- *model.Part) error
    Config() ToolConfig
    SetConfig(cfg ToolConfig) error
}

Process does the work. A tool consumes Parts from in, does its work, and emits Parts on out. It blocks until the input channel is closed (input exhausted) or the context is cancelled. Flow composition relies on this method alone: chaining tools means wiring one tool's out channel to the next tool's in channel.

The remaining methods carry the tool's identity (Name, Description) and its configuration. Configuration is a small interface of its own. A tool config knows which tool it belongs to, how to reset to defaults, and how to validate itself:

type ToolConfig interface {
    ToolName() string
    Reset()
    Validate() error
}

Part-type dispatch with BaseTool

Most tools only care about one or two kinds of Part, usually Blocks (content blocks). Writing the full channel loop for every tool would be repetitive and error-prone, so the framework provides BaseTool, an embeddable type that implements Process once and dispatches each Part to a per-type handler:

type PartHandler func(part *model.Part) (*model.Part, error)

type BaseTool struct {
    ToolName        string
    ToolDescription string
    Cfg             ToolConfig

    // Block handler: set exactly one. The view type bounds what it may write.
    Annotate  func(BlockView) error             // read-only: overlays / annotations / properties
    Produce func(VariantView) error            // writes target
    Transform func(BlockView) (EditPlan, error) // edit producer: the framework applier rewrites source

    // Other Part types stay untyped.
    HandleDataFn       PartHandler
    HandleMediaFn      PartHandler
    HandleLayerStartFn PartHandler
    HandleLayerEndFn   PartHandler
    HandleGroupStartFn PartHandler
    HandleGroupEndFn   PartHandler
}

A concrete tool embeds BaseTool and sets only the handlers it needs. BaseTool.Process reads each Part, switches on its Type, and calls the matching handler. Any handler left unset is a pass-through, so the Part flows to the output channel unchanged. For Blocks, a tool sets one of three capability-typed handlers, and the view it receives limits what it may write (the immutability model; see the tool-system AD): Annotate reads source and target but writes only overlays, annotations, and properties; Produce writes the target; Transform is a read-only edit producer that returns an edit plan, and a framework-owned applier performs the source rewrite, rebasing surviving overlays and vaulting any secrets. The view has no method for a forbidden write, so a quality check can't mutate the source by accident, and a transformer holds no source setter.

The case-transform tool is a representative example. It can rewrite the source, so it sets Transform and returns the rewrite as a plan:

func NewCaseTransformTool(cfg *CaseTransformConfig) *tool.BaseTool {
    t := &tool.BaseTool{
        ToolName:        "case-transform",
        ToolDescription: "Transforms the case of source and/or target text",
        Cfg:             cfg,
    }
    // Transform producer: returns the case rewrite as an edit plan; the
    // framework applier rewrites the block (E-03).
    t.Transform = func(v tool.BlockView) (tool.EditPlan, error) {
        if !v.Translatable() {
            return tool.EditPlan{}, nil // pass through
        }
        conf := t.Cfg.(*CaseTransformConfig)
        convert := unicode.ToUpper
        if conf.Mode == CaseLower {
            convert = unicode.ToLower
        }
        var plan tool.EditPlan
        if conf.ApplySource {
            // The zero key is the source.
            plan.AddTextEdits(model.EditionKey{}, caseEdits(v.SourceRuns(), convert))
        }
        return plan, nil
    }
    return t
}

// caseEdits returns one in-place edit for each run of characters convert
// changes. An edit never spans an inline code, so each code stays between the
// same two characters, and each edit keeps the length of the text it replaces,
// so every overlay span keeps the characters it covered.
func caseEdits(runs []model.Run, convert func(rune) rune) []change.TextEdit {
    var edits []change.TextEdit
    at := 0 // code points of text before the current character
    for _, r := range runs {
        if r.Text == nil {
            continue // a code, a plural or a select: no edit spans it
        }
        start, changed := -1, []rune(nil)
        flush := func() {
            if start >= 0 {
                s, e := start, at
                edits = append(edits, change.TextEdit{Start: &s, End: &e, Text: string(changed)})
                start, changed = -1, nil
            }
        }
        for _, c := range r.Text.Text {
            if u := convert(c); u != c {
                if start < 0 {
                    start = at
                }
                changed = append(changed, u)
            } else {
                flush()
            }
            at++
        }
        flush()
    }
    return edits
}

The case-transform tool in core/tools works the same way and also edits the text of each plural form and select case.

When a tool needs full control of the loop, for example to accumulate state across many Parts or to emit more Parts than it consumes, it can implement Process directly instead of using the handler fields.

How tools compose

Composition follows from the streaming contract. Three Parts (a layer start, a block, a layer end) flowing through a two-tool chain look like this:

Each tool runs in its own goroutine, connected by buffered channels. A tool that does not handle layer markers relays them, so structural context survives the whole chain even though only some stages act on it. Ordering is preserved: the segmentation tool's output for a block reaches the translation tool before the next block does. The mechanics of that concurrency (goroutines, buffered channels, backpressure, error propagation) are covered in Pipeline; how chains are described and built is covered in Flows.

Wrapping tools

Because a tool is an interface, one tool can wrap another to add behavior without the inner tool knowing. The framework uses this for intra-tool block parallelism: ParallelBlockTool wraps a block-handling tool and fans its block handler out across N goroutines while preserving Part order. This helps IO-bound tools such as AI or MT translation, where each block is an independent network call. The wrapper presents the same Tool interface, so the rest of the flow is unaffected.

Categories of work

The framework enforces no tool categories. As a guide, the built-in tools fall into five broad kinds:

Kind	What it does	Examples
Transform	Modify content in place	case change, search/replace, redact
Enrich	Attach matches or metadata to content	segmentation, memory leverage, terminology lookup, AI translation
Validate	Check content without modifying it	rule-based checks, length checks, terminology enforcement
Analyze	Accumulate statistics across the stream	word count, repetition analysis, character inventory
Convert	Adjust representation	encoding conversion, line-break normalization

Enrich and validate tools commonly write to the Block annotation system and leave the text alone: the recycle tool attaches candidate memory matches, a check tool attaches findings, and downstream tools or an editor read those annotations. Content memory, terminology and voice profile results reach the same consumer through this shared channel without colliding.

Configuration and schemas

Tools that expose configuration declare it as a struct with schema:"…" field tags. The framework derives a JSON-Schema-style descriptor from that struct by reflection. The descriptor drives the generated CLI flags, validation and the configuration forms in the visual editors. A tool that opts into this advertises its schema through an optional interface; the generated Tool Reference renders each tool's parameters from these schemas, so it matches the build.

Scripting: write a transform in JavaScript

For a transform too small to need its own Go tool, the built-in script tool runs a small JavaScript program against each Part. Define process(part), edit part.block.source or part.block.targets, and return the part to keep it (or null to drop it). Alternatively, omit the function and write top-level code against the global part, calling emit(part) / skip(). Use it to prototype a one-off rule. It runs anywhere the engine runs, the browser included, via the embedded interpreter.

Write a script, run it on your file

Edit the JavaScript below, where process(part) runs once per Part with full autocomplete for the part API, or load an example, then run it on a sample or your own file and read the per-Block before/after of source and target.

Loading the interactive lab…

Where tools come from

Built-in tools live in the framework and are registered into a ToolRegistry, which maps a tool name to a factory. Tools can also be supplied by plugins, discovered at runtime and dispatched as subprocesses over gRPC. A plugin adds tools to a binary without changing the interface they satisfy.

Related reading

• Tool Reference: the generated list of built-in tools and their parameters.
• Flows: composing tools into a named pipeline.
• Pipeline: the streaming executor that runs the chain.
• Implementing a Tool and Tool Authoring: writing your own.