ŠéššîöñŢööļ àüţĥöŕîñĝ ĝüîđé

À tool.SessionTool îš àñý ţööļ ţĥàţ ŵàñţš ŕàñđöḿ àççéšš ţö ţĥé þŕöĵéçţ'š ƃļöçķ šţàţé: ƃļöçķ ļööķüþš ƃý ĥàšĥ, öṽéŕļàý ŕéàđš ƒöŕ "šķîþ îƒ àļŕéàđý đöñé", öṽéŕļàý ŵŕîţéš ƒöŕ çŕöšš-ŕüñ àññöţàţîöñš. Ţĥé éẋîšţîñĝ tool.Tool šţŕéàḿîñĝ çöñţŕàçţ îš üñçĥàñĝéđ; SessionTool îš àđđîţîṽé.

Ţĥîš ñöţé ŵàļķš ţĥŕöüĝĥ ŵĥéñ ţö îḿþļéḿéñţ îţ àñđ ŵĥàţ ţĥé ŵîŕé çöñṽéñţîöñš àŕé. Šéé Ç-01 ƒöŕ ţĥé đéšîĝñ ŕàţîöñàļé.

Ŵĥéñ ţö îḿþļéḿéñţ

Îḿþļéḿéñţ SessionTool ŵĥéñ ýöüŕ ţööļ:

• Çàñ šķîþ éẋþéñšîṽé ŵöŕķ îƒ à þŕîöŕ ŕüñ àļŕéàđý þŕöđüçéđ ţĥé öüţþüţ ƒöŕ à ƃļöçķ. Çàñöñîçàļ çàšé: ÀÎ ţŕàñšļàţîöñ; ŕé-çàļļîñĝ ţĥé ĻĻḾ ƒöŕ à ƃļöçķ ŵĥöšé ţàŕĝéţ îš àļŕéàđý çàçĥéđ îš ŵàšţéđ ḿöñéý àñđ ļàţéñçý.
• Ŵŕîţéš àññöţàţîöñš ţĥàţ à đöŵñšţŕéàḿ ţööļ (šàḿé ƒļöŵ öŕ ñéẋţ ŕüñ) ŵàñţš ţö çöñšüļţ. Ḿéḿöŕý ƒüžžý ḿàţçĥéš, ţéŕḿ ĥîţš, çĥéçķ ƒîñđîñĝš.
• Ñééđš ƃļöçķ-ƃý-ĥàšĥ ļööķüþ ƒöŕ çŕöšš-ŕéƒéŕéñçé (ŕàŕé, ƃüţ é.ĝ. "îñļîñé-çöđé àļîĝñḿéñţ àĝàîñšţ ţĥé ļàšţ-ķñöŵñ ţàŕĝéţ").

Đö ñöţ îḿþļéḿéñţ îţ ŵĥéñ ýöüŕ ţööļ:

• Κ à þüŕé šţŕéàḿ ţŕàñšƒöŕḿ (ƒîļţéŕ, îđéñţîţý, éñçöđîñĝ çöñṽéŕţ, ƒöŕḿàţ ŕéàđ/ŵŕîţé). Ţĥé šţŕéàḿ çöñţŕàçţ àļŕéàđý ĝîṽéš ýöü ŵĥàţ ýöü ñééđ.
• Þŕöđüçéš öüţþüţ ţĥàţ îš çĥéàþ ţö ŕéçöḿþüţé, šö çàçĥîñĝ ĝàîñš ñöţĥîñĝ.
• Ŵŕîţéš öüţþüţ éẋçļüšîṽéļý ţö ţĥé îñ-ƒļîĝĥţ model.Block àñđ ĥàš ñö þéŕšîšţéñţ šţàţé šţöŕý.

Ḿîñîḿàļ îḿþļéḿéñţàţîöñ

import (
    "github.com/neokapi/neokapi/core/blockstore"
    "github.com/neokapi/neokapi/core/tool"
)

// Compile-time assertion catches accidental drift.
var _ tool.SessionTool = (*MyTool)(nil)

func (t *MyTool) SessionProcess(
    ctx context.Context,
    sess blockstore.Session,
    in <-chan *model.Part,
    out chan<- *model.Part,
) error {
    overlayKind := "targets/" + string(t.targetLocale)
    caps := sess.Capabilities()

    for {
        select {
        case <-ctx.Done():
            return ctx.Err()
        case part, ok := <-in:
            if !ok {
                return nil
            }
            // Skip logic, expensive work, overlay write...
            if err := t.handle(sess, caps.RandomAccess, overlayKind, part); err != nil {
                return err
            }
            select {
            case out <- part:
            case <-ctx.Done():
                return ctx.Err()
            }
        }
    }
}

Ţĥé þéŕ-ƃļöçķ ĥéļþéŕ çĥéçķš çàþàƃîļîţîéš, çöñšüļţš ţĥé öṽéŕļàý, ŕüñš ţĥé çöŕé ŵöŕķ, ŵŕîţéš ţĥé öṽéŕļàý ƃàçķ:

func (t *MyTool) handle(sess blockstore.Session, ra bool, kind string, part *model.Part) error {
    block, ok := part.Resource.(*model.Block)
    if !ok || !block.Translatable || block.ID == "" {
        _, err := t.doTheWork(part)
        return err
    }

    // Hydrate from cache when possible.
    if ra {
        if sc, err := sess.GetOverlay(kind, block.ID); err == nil && len(sc.Payload) > 0 {
            var cached myOverlay
            if json.Unmarshal(sc.Payload, &cached) == nil && cached.Text != "" {
                block.SetTargetText(t.targetLocale, cached.Text)
                return nil
            }
        }
    }

    // Do the expensive work.
    if _, err := t.doTheWork(part); err != nil {
        return err
    }

    // Cache the result for next time.
    if target := block.TargetText(t.targetLocale); target != "" {
        payload, _ := json.Marshal(myOverlay{Text: target})
        if err := sess.PutOverlay(blockstore.Overlay{
            Kind:      kind,
            BlockHash: block.ID,
            Payload:   payload,
        }); err != nil && !errors.Is(err, blockstore.ErrReadOnly) {
            return fmt.Errorf("my-tool: write overlay: %w", err)
        }
    }
    return nil
}

Öṽéŕļàý çöñṽéñţîöñš

Ķîñđ þŕéƒîẋ	Üšéđ ƃý	Þàýļöàđ šĥàþé
targets/<locale>	ţŕàñšļàţöŕš (ţŕàñšļàţé, þšéüđö-ţŕàñšļàţé, ĥüḿàñ éđîţöŕ)	{"runs": [...], "text": "...", "status": "...", "origin": {...}}
annotations/<name>	ţéŕḿ-ļööķüþ, ŕéçýçļé, ǫà çĥéçķš	ţööļ-šþéçîƒîç ĴŠÖÑ
skeletons/<format>	ƒöŕḿàţ ŵŕîţéŕš (ŕöüñđ-ţŕîþ šķéļéţöñš)	öþàǫüé þàýļöàđ

Ţĥé targets/<locale> šĥàþé îš çŕöšš-ţööļ: àñý ţŕàñšļàţöŕ ŵŕîţéš àñđ ŕéàđš ţĥé šàḿé ķéý, šö à šéššîöñ ĥýđŕàţéđ ƃý öñé çàñ ƃé çöñţîñüéđ ƃý àñöţĥéŕ. Ķééþ ţĥé þàýļöàđ šḿàļļ àñđ ĴŠÖÑ-çöḿþàţîƃļé.

origin çàŕŕîéš ţĥé þŕöṽéñàñçé ţĥé þŕöđüçéŕ šţàḿþéđ, îñçļüđîñĝ ţĥé ContextFingerprint öƒ ţĥé ĝöṽéŕñîñĝ çöñţéẋţ. Ḿöšţ ţàŕĝéţ ƒöŕḿàţš ĥàṽé ñöŵĥéŕé ţö ķééþ îţ, šö ƒöŕ ţĥöšé ţĥé öṽéŕļàý îš ţĥé öñļý đüŕàƃļé ŕéçöŕđ öƒ ŵĥàţ ĝöṽéŕñéđ ţĥé àñšŵéŕ, àñđ îţ îš ŵĥàţ à çöñṽéŕĝéñçé ŕéàđš ƃàçķ ŵĥéñ îţ ŕéçöŕđš ţĥé ƃàšîš ţĥé šţàļéñéšš ĝàţé çöḿþàŕéš (host/basisrecord.go).

annotations/<name> îš çŕöšš-ţööļ îñ ţĥé šàḿé ŵàý, öñé ļéṽéļ đöŵñ: <name> îš ţĥé ƃļöçķ-àññöţàţîöñ ķéý, šö à šţöŕé ţĥàţ ĥöļđš ƃļöçķš ŕàţĥéŕ ţĥàñ ļööšé öṽéŕļàýš ƒîļéš ţĥé þàýļöàđ àš ţĥàţ ƃļöçķ'š àññöţàţîöñ. À <name> ţĥé þàýļöàđ ŕéĝîšţŕý ķñöŵš (note, quality.findings, …) ḿéàñš ţĥé ƃöđý ĥàš ţö ḿàţçĥ ţĥàţ àññöţàţîöñ'š šçĥéḿà; àñýţĥîñĝ éļšé îš ķéþţ ṽéŕƃàţîḿ àñđ çöḿéš ƃàçķ ţö ýöüŕ ţööļ éẋàçţļý àš ŵŕîţţéñ. skeletons/<format> àñđ àñý öţĥéŕ þŕéƒîẋ šţàý öþàǫüé: ţĥéý ñàḿé ñö àññöţàţîöñ àñđ ñö šţöŕé ŕéàđš ţĥéḿ ƃüţ ţĥé ţööļ ţĥàţ ŵŕöţé ţĥéḿ.

Ŕéàđ-öñļý šţöŕéš

Ţĥé FormatReaderStore ŵŕàþš à ŕàŵ ẊĻÎƑƑ / ĴŠÖÑ / éţç. ƒîļé àš à ŕéàđ-öñļý blockstore.Store. Îţš PutOverlay ŕéţüŕñš blockstore.ErrReadOnly. Ţööļš šĥöüļđ îĝñöŕé ţĥîš éŕŕöŕ öñ ţĥé öṽéŕļàý-ŵŕîţé þàţĥ: ţĥé îñ-ƒļîĝĥţ *model.Block àļŕéàđý çàŕŕîéš ţĥé ŕéšüļţ, àñđ çàçĥîñĝ îš ƃéšţ-郃öŕţ ƒöŕ ţĥé ñéẋţ ŕüñ. Šéé ţĥé þàţţéŕñ îñ core/tools/pseudo.go àñđ core/ai/tools/translate.go.

Ƃàţçĥîñĝ + çöñçüŕŕéñçý

΃ ýöüŕ ţööļ ĥàš à çöñçüŕŕéñţ / ƃàţçĥéđ þàţĥ (ļîķé translate ŵîţĥ batchSize > 1 öŕ concurrency > 1), ŵŕàþ ţĥé ƃàţçĥéđ þàţĥ ŵîţĥ šéššîöñ ƒîļţéŕîñĝ àţ ţĥé îñþüţ (šķîþ çàçĥéđ) àñđ öṽéŕļàý-ŵŕîţé àţ ţĥé öüţþüţ. Éẋàḿþļé: core/ai/tools/translate.go::processBatchedWithSession.

Šţöŕé þŕöṽîđéŕš

Ţĥé þŕöṽîđéŕš àŕé þļàîñ çöñšţŕüçţöŕš îñ core/blockstore, ñöţ šţŕîñĝ-ķéýéđ éñţŕîéš đéçļàŕéđ îñ à ŕéçîþé. Ţĥé çàļļéŕ (ÇĻÎ, þŕöĵéçţ ŕüññéŕ, éẋéçüţöŕ) çöñšţŕüçţš ţĥé öñé îţ ŵàñţš àñđ ĥàñđš îţ ţö ţĥé éẋéçüţöŕ:

• NewMemoryStore(): ţĥé đéƒàüļţ ŵĥéñ ñö šţöŕé îš þàššéđ. Šñàþšĥöţ-þéŕ-šéššîöñ, ļàšţ-ŵŕîţéŕ-ŵîñš öñ çöḿḿîţ. Çàþàƃîļîţîéš: ŔàñđöḿÀççéšš + Çöñçüŕŕéñţ + Ŵŕîţàƃļé; ñöţ Þéŕšîšţéñţ.
• sqlitestore.New(path) / sqlitestore.NewFromDB(db, …) (core/blockstore/sqlitestore): ţĥé ŠǪĻîţé-ƃàçķéđ šţöŕé, îñ à kapi þŕöĵéçţ ţĥé ƃļöçķ-çàçĥé ţàƃļéš öƒ .kapi/work/store.db. Ţĥé đéƒàüļţ ƒöŕ kapi þŕöĵéçţš. Ƒüļļ ÀÇÎĐ, þéŕšîšţéñţ àçŕöšš ŕüñš.
• NewPersistentMemoryStore(): à ḿéḿöŕý šţöŕé ţĥàţ àđṽéŕţîšéš Persistent àñđ ŵĥöšé Close îš à ñö-öþ, šö îţ šüŕṽîṽéš ŕéþéàţéđ öþéñ àñđ çļöšé àçŕöšš çöḿḿàñđš îñ öñé þŕöçéšš. Îţ šţàñđš îñ ƒöŕ ţĥé ŠǪĻîţé çàçĥé öñ ţĥé ŵàšḿ ƃüîļđ, ŵĥéŕé ŠǪĻîţé îš üñàṽàîļàƃļé.
• NewFormatReaderStore(factory): ŵŕàþš à format.DataFormatReader ƒàçţöŕý àš à ŕéàđ-öñļý šţöŕé. Üšéƒüļ ƒöŕ àđ-ĥöç ÇĻÎ ƒļöŵš (kapi translate -i file.xliff): ŔàñđöḿÀççéšš=ţŕüé, Ŵŕîţàƃļé=ƒàļšé. Îţš PutOverlay ŕéţüŕñš blockstore.ErrReadOnly.

Ţĥé éẋéçüţöŕ ŕéçéîṽéš ţĥé šţöŕé ṽîà ţĥé flow.WithBlockStore(s) öþţîöñ (đéƒàüļţ NewMemoryStore()); ţööļš ñéṽéŕ öþéñ ţĥé šţöŕé đîŕéçţļý.