Šķîþ ţö ḿàîñ çöñţéñţ

Ļéàŕñ ķàþî › Explore the engine

Audio and video

Speech and on-screen text as timed content.

Subtitles are timed blocks: a cue is a block with a start and an end, and on-screen text is a block with a box and a time. The first three chapters show the finished shape with recorded results, animated over a playhead. The last two run the real models on a sample clip: speech recognition, and the full demux, transcribe and frame-OCR pipeline.

  1. An image, translatedRecorded results, no model: an invoice's lines recognised and translated, drawn over the image at their boxes. Toggle the language to see the translation sit where the source was.
  2. Audio as cuesSpeech becomes subtitle cues with a start and an end. The playhead advances and the active cue highlights, exactly as it would over real playback.
  3. Video: speech and framesA video carries both: the speech track as cues, and the on-screen text as boxes with a time. Both are translated, and both are drawn at their moment.
  4. Transcribe for realWhisper runs in the browser on the sample clip's audio. Press Transcribe: the model downloads once, and the cues appear in the player as they would from the native plugin.
  5. The whole pipelinePress Process: ffmpeg demuxes the clip into audio and frames, Whisper transcribes the speech, and recognition reads the frame text. The result is one timed document with both tracks.