CrisperWhisper 2.0
The most accurate verbatim speech recognition you can run in production: controllable, multilingual, and timed to the word.
- #1
- verbatim accuracy, English & German
- 30 ms
- word-timing error on read speech
- 96.1%
- rare-word recall with Verbatimize
Most speech-to-text systems never explicitly decide between two kinds of transcript: a verbatim one that captures speech word-for-word, every filler, repetition, false start and vocal sound included, or a clean, readable one that captures only what the speaker meant to say. Trained on data that mixes both conventions, they internalize competing transcription policies by accident and lack a controllable output policy to stabilize their behaviour: the same hesitation is kept, dropped, or reworded depending on context, and word error rate on conversational speech conflates style mismatch with actual recognition failures. CrisperWhisper 2.0 turns that hidden policy into an explicit, controllable switch, and sets a new state of the art in verbatim speech transcription.
It's the successor to CrisperWhisper 1.0, our model for verbatim transcription. 1.0 had real limits: it was English and German only, verbatim-only with no way to switch to clean output, couldn't recognize vocal sound events, and its verbatim accuracy, word-level timing, and longform handling all left room to improve. CrisperWhisper 2.0 addresses all of them: controllable verbatim or intended output, multilingual out of the box, vocal sounds and stutters alongside fillers, markedly sharper timing, seamless longform, and, with Verbatimize, a scalable path to upgrading legacy speech corpora with faithful verbatim annotations in retrospect, recovering the fillers, stutters, and vocal sound events that many downstream applications, from expressive speech synthesis to clinical analysis, depend on, without touching the transcripts you already trust.
The Nyra Verbatim Speech Benchmark
WER hides everything that makes speech real, whether a filler was kept, a stutter preserved, a laugh written down, inside one number. So we built the Nyra Verbatim Speech Benchmark, an open evaluation suite that scores fillers, repetitions, cut-offs, and vocal sounds as separate, typed metrics instead of collapsing them into a single figure. Rather than one opaque number, you can see exactly how a model fails: which phenomena it silently drops, which it hallucinates, and which it mislabels.
The headline metric is disfluency F1: how reliably a system writes down the disfluencies that were actually spoken, without inventing ones that weren’t. We ran the benchmark against every serious verbatim system we could reach, open and closed source, across ten languages. CrisperWhisper 2.0 tops the leaderboard, and CrisperWhisper 2.0 Pro ranks #1 in every single language we tested, beating every closed-source alternative, including ElevenLabs Scribe v2, Microsoft, and AssemblyAI, often by 10 to 30 points.
Controllable, multilingual transcription
The same recording can now be transcribed two ways, depending on what you need. Verbatim writes down exactly what was said: every filled pause, repetition, stutter, false start, and vocal sound like laughter, in one consistent format. Intended gives you the clean, readable version the speaker meant, dropping the disfluencies and formatting numbers, dates, and emails the way you’d actually write them. Same audio, your choice of output:
[um] So we we need to, to reschedule the Th- Thursday meeting to [uh] March third at nine thirty, [laughter] and loop in Sarah at sarah dot lee at acme dot com.
So we need to reschedule the Thursday meeting to March 3 at 9:30, and loop in Sarah at sarah.lee@acme.com.
And this isn’t English-only. We’ve extended both verbatim and intended transcription, with the same fillers, vocal sounds, and clean formatting, to most of the languages Whisper supports.
Read the researchHow we unlocked multilingual style-controlled ASR at scalePrecise word-level timing
Whisper’s cross-attention heads were already known to develop alignment-like behavior on their own: a faint, emergent signal of when each word occurs that nobody had trained for. CrisperWhisper 2.0 turns that signal into a dependable capability: by supervising a select set of those heads with ground-truth word timings during training, precise alignment becomes something the model learns rather than something we hope it stumbles into. Left to their own devices, those heads are effectively unusable for most applications, with boundary errors running into the hundreds of milliseconds. Supervising them, and then carefully extracting the timings from their cross-attention, cuts that error by roughly an order of magnitude.
The result is 30 ms mean boundary error on clean read speech (TIMIT) and 40 ms on spontaneous, conversational speech (Buckeye): the lowest boundary error of any transcription system we tested, on both. Scored on exactly the words each system gets right, CrisperWhisper 2.0 places word boundaries more precisely than every comparable commercial speech-to-text service we benchmarked.
| System | Read · TIMIT | Conversational · Buckeye |
|---|---|---|
| CrisperWhisper 2.0 | 29.6 #1 | 40.6 #1 |
| xAI Grok Speech-to-Text | 37.1 | 47.1 |
| ElevenLabs Scribe v2 | 51.3 | 59.6 |
| Deepgram Nova-3 | 63.3 | 88.4 |
| Cartesia Ink-Whisper | 69.4 | 125.7 |
Seamless longform
Whisper only ever sees 30 seconds at a time, so longer audio has to be handled at the seams, and the usual approaches each carry a cost. Chunking the audio and stitching the transcripts back together produces duplicated words, lost words, and hallucinations at the edges. Whisper’s own method instead predicts special timestamp tokens and shifts the next window to wherever it thinks the last one ended. That adds timestamp tokens to generate, slowing inference, and can turn timestamp-prediction errors into transcription errors at the boundaries.
CrisperWhisper 2.0 takes a different approach we call conditional continuation. As each window shifts forward through the audio, we hand the model the last few words it has already transcribed and ask it to pick up exactly where those words end, so the seam is resolved by what was actually said, not by a fragile predicted timestamp. It’s trained to cope with imperfect context, too: if some of those words fall outside the current window, or don’t match it at all, the model finds the right starting point on its own, or simply transcribes from the top. With no timestamp tokens to emit and no overlap to stitch, inference is faster, boundaries are more accurate, and training is simpler than the original Whisper implementation.
Read the researchLongform transcription with conditional continuationVerbatimize: towards better datasets
Verbatim-annotated speech, transcripts that mark every filler, repetition, stutter, and vocal sound, is scarce and costly, and almost nonexistent outside English. Yet clean, intended transcripts already exist in abundance across many languages. Verbatimize is our answer to that gap: give the model audio plus any existing clean transcript, and it reconstructs a faithful verbatim version, copying the prompted content word-for-word while inserting the disfluencies and sounds it actually hears. In effect, it turns the world’s abundant clean corpora into verbatim ones, at scale and across languages.
That matters in two places. For TTS, high speech-to-text coherence, a transcript that matches the audio exactly, down to every hesitation and laugh, is the precursor to high text-to-speech coherence, letting models learn to produce disfluent, natural, expressive speech rather than only smooth read-aloud text. For clinical and speech-science work, those same disfluencies are the signal rather than the noise: fillers, repetitions, and stutters are diagnostic markers for conditions like aphasia, stuttering, and cognitive decline. That is exactly what clean transcripts discard. Verbatimize makes the creation of large, faithful, multilingual datasets of this kind practical and affordable for the first time.
Read the researchClosing the verbatim data gapFast, production-ready inference
Accuracy only matters if the system can run where speech actually happens: at scale, under latency budgets, and without pathological failures. So CrisperWhisper 2.0 ships with a CTranslate2 inference path built specifically for production use. We extended CTranslate2 ourselves to support speculative decoding and our word-level timing-extraction algorithms, so speed never comes at the cost of quality or features. The result is the fastest Whisper-style inference stack we have found: accelerated decoding, precise word timings, and the same controllable verbatim/intended behavior available through a production-grade runtime.
We also address one of Whisper’s most dangerous failure modes: looping hallucinations. Standard Whisper decoding can get stuck repeating text that was never spoken, sometimes for long stretches, especially around silence, noise, music, or uncertain boundaries. CrisperWhisper 2.0 includes a dedicated hallucination-mitigation decoding scheme that detects and suppresses this looping behavior during generation, keeping the model anchored to the audio instead of drifting into self-reinforcing repetition. And looping is only the most visible case: thanks to the model’s improved output policy and a series of training refinements, CrisperWhisper 2.0 effectively mitigates the full range of Whisper’s hallucination failure modes. Faster inference is useful; faster inference you can trust is what makes it deployable.
Read the researchFaster inference and mitigating hallucinations in CrisperWhisper 2.0