The nyra forced aligner
- Phoneme
- granular alignment, down to the individual sound, not just words
- Robust
- built for spontaneous, disfluent, imperfect real-world speech
- Explicit
- fillers, laughs, breaths and vocal sounds modeled as first-class events
The nyra forced aligner is our upcoming phoneme-level aligner for real-world speech. It is built to answer, precisely, which sound happens when, and for how long, even when the speech is anything but clean.
Forced aligners take audio and a transcript and return where in time each unit of speech lives. Most of them fall short in one of three ways: they are not granular enough, stopping at word boundaries when deeper analysis needs phonemes; they are not robust enough, tuned to read-aloud studio recordings and brittle on spontaneous, accented, noisy speech; or they have no explicit model of fillers, laughs, breaths, and other vocal sounds, so when those occur, and in real speech they always do, the aligner smears them onto the neighboring words and the alignment around them quietly corrupts.
The nyra forced aligner tackles all three at once: phoneme-level granularity, robustness to speech as it is actually spoken, and first-class modeling of the events other aligners pretend do not exist.
It is “the when” of the measuring line of our research program: knowing which sound occurs, exactly when, and for how long is the substrate that lets us design losses and supervision that attach to the exact frames where a phenomenon lives.
The model and the full write-up are coming soon!
The deep dive into how the aligner is built, how it handles disfluent and imperfect speech, and how it performs against existing aligners is on its way, together with the model itself.
Related research
Turning emergent cross-attention into a precise aligner
How CrisperWhisper 2.0 reads millisecond-accurate word timings out of an ASR decoder's cross-attention: by making the output policy explicit, then supervising the heads that already lean toward alignment.
CrisperWhisper 2.0
The most accurate verbatim speech recognition you can run in production: controllable, multilingual, and timed to the word.
PINT: content is what remains
A parallel-invariant speech tokenizer: when many voices say the same words under different conditions, content is the only shared factor. PINT keeps it and discards the rest.