How much ASR training data do I need?

It depends more on coverage than raw hours. A focused model in one language and domain can reach usable accuracy with a few hundred to a few thousand hours, while broad multi-accent or multilingual systems need far more. Early on, adding new accents and recording conditions lowers word error faster than adding more of what you already have.

What is the difference between ASR and STT data?

They refer to the same thing. ASR (automatic speech recognition) and STT (speech to text) are interchangeable terms for converting spoken audio into text, so ASR training data and STT data describe the same kind of paired audio and transcript corpus.

Does training data need to be recorded in noisy conditions?

If your product runs in noisy environments, yes. A model trained only on clean studio audio degrades sharply in cars, offices, and over phone lines. Real recordings at varied signal-to-noise ratios teach the model to handle the conditions it will actually meet, and added-noise augmentation alone does not fully reproduce real room acoustics.

How accurate do transcriptions need to be?

Transcription error sets a hard floor on model accuracy, so it should sit comfortably below your target word error rate. Just as important is consistency: a documented style guide for numbers, disfluencies, casing, and non-speech events keeps annotators from injecting noise that later looks like model error.

Should I keep a separate evaluation set?

Always. Hold out data drawn from production conditions that the model never trains on, split by speaker so the same voice never appears in both train and test. Build per-slice eval sets for each accent, noise type, and domain so a single average cannot hide a failing subgroup.

ASR Training Data: A Practical Buyer's Guide

Most speech recognition models fail in predictable ways: a new accent, a noisy car cabin, a clipped phone call, a name the tokenizer never saw. The fix is rarely a cleverer architecture. It is better ASR training data, matched to the conditions the model will actually meet in production. If you are training or fine-tuning an STT system, the quality of your speech recognition dataset sets a ceiling no amount of hyperparameter tuning will lift.

This guide covers what separates a useful corpus from a generic one: acoustic and linguistic coverage, recording conditions and signal-to-noise ratio, transcription accuracy and conventions, domain match, and the held-out evaluation set that tells you whether any of it actually worked.

What counts as good ASR training data

Good ASR training data is not just hours of clean audio with a transcript attached. It is audio that resembles, in its messiness and variety, the speech your model will be asked to decode after you ship. A model learns the distribution you show it. Show it studio reads from twenty narrators and it will be excellent at studio reads from people who sound like those twenty narrators, and brittle everywhere else.

So the useful question is not how clean a corpus is but how well its distribution covers your deployment distribution. That breaks into a few things you can actually inspect: who is speaking, where and how it was recorded, how accurately it was transcribed, and whether the content matches your domain. Get those right and the rest of the pipeline has something to work with. Get them wrong and you are tuning a model that learned the wrong world.

Acoustic and linguistic coverage

Coverage is the part teams underestimate most. Two speakers reading the same script in the same room give you very little new information after the first one. A different accent, a different age, a different microphone, a different room: each is a genuinely new region of the space your model has to generalize over.

The dimensions worth tracking explicitly:

Accent and dialect. Regional variation inside a single language is often the largest source of word error. A Glasgow speaker and a London speaker are not interchangeable to an acoustic model, and neither are Rio and Lisbon Portuguese.
Age and gender. Pitch range, speaking rate, and articulation shift with both. Children and older speakers are routinely underrepresented and routinely the hardest cases.
Speaking style. Scripted reads, spontaneous conversation, dictation, and command-and-control each carry different prosody and disfluency. Spontaneous speech has the false starts, fillers, and overlaps that read-aloud corpora simply do not contain.
Language and code-switching. Multilingual users mix languages mid-sentence. If your speakers never do, your model never learns to.

This is where hard-to-source speakers matter. Anyone can record a hundred standard-accent adults. The value sits in the long tail: specific dialects, smaller languages, and the speakers who are scarce on every public dataset. Much of our work at Spirelight is exactly that recruitment, through a global crowd of contributors across more than 50 languages, because that tail is where most production word error actually lives.

Recording conditions and SNR

A clean transcript over the wrong audio is still the wrong training example. If your product runs in a moving car, an open-plan office, or over a low-bitrate phone connection, then clean studio recordings are teaching the model habits it cannot use.

Signal-to-noise ratio is the headline number, but it is not the whole story. Two recordings at the same SNR can behave very differently depending on the type of noise (steady road rumble versus a barking dog versus competing speech), the reverberation of the room, the distance from the microphone, and the codec the audio passed through. Far-field capture, where the speaker is across the room from a smart speaker, is its own difficulty class and deserves its own samples.

The practical move is to collect, on purpose, in the conditions you care about. We record in real car cabins with the engine and HVAC running, in noisy offices, and over phone channels, rather than synthesizing all of it with added noise afterward. Augmentation has its place and is cheap, but real room acoustics, real microphones, and real overlapping speech carry artifacts a noise-mixing script does not reproduce. Label each clip with its conditions so you can slice evaluation by environment later. That is the only way to catch a model that looks fine in the lab and falls apart in the car.

Transcription accuracy and conventions

The transcript is the supervision signal. Every systematic error in it becomes a systematic error the model is trained toward. Transcription error rate is a hard floor under the accuracy you can reach, so it should sit well below your target word error rate.

Accuracy is necessary but not sufficient. The conventions matter just as much, and they have to be consistent across the whole corpus and written down. Decide and document how you handle:

Numbers, dates, and currency: spoken form (twenty twenty five) versus written form (2025).
Disfluencies and fillers: whether um, false starts, and repetitions are transcribed verbatim or cleaned.
Casing, punctuation, and the spelling of named entities.
Non-speech events: laughter, coughs, music, and crosstalk, and how each is tagged.
Unintelligible audio: a consistent marker rather than a guess.

Two annotators who disagree on these rules will quietly inject noise that looks like model error later. A measured inter-annotator agreement and a documented style guide are worth more than a slightly larger pile of inconsistent labels. If you are weighing how much to verify by hand versus trust, our guide to what audio annotation involves walks through the layers in more detail.

Domain match

A general-purpose ASR dataset gets you a general-purpose model. If your users dictate medical notes, read out order numbers, or say product names and street addresses, the vocabulary and phrasing in your training data should reflect that. Rare words the model has barely seen are the words it will most often get wrong, and those are frequently the words that matter most to the task.

Domain match covers vocabulary, but also turn structure and acoustics. Contact-center audio is two-party, interruption-heavy, and telephone-band. A voice assistant gets short, isolated commands. In-car voice gets road noise plus a specific command grammar. Each is a different problem wearing the same speech recognition label, and the closer your corpus sits to the real one, the less you have to hope generalization saves you. If your deployment is narrow, a custom collection usually beats stretching an off-the-shelf corpus to fit.

Held-out evaluation sets

You cannot measure ASR honestly on data the model trained on. A held-out evaluation set, drawn from the same conditions as production but never seen during training, is what turns the loss went down into word error rate on real users dropped.

A few things make an eval set trustworthy. Split by speaker, not by clip, so the same voice never appears in both train and test, otherwise you are measuring memorization. Build separate slices for the cases you care about, each accent, each noise environment, each domain, so a single average cannot hide a subgroup that is failing. Keep it stable over time so results stay comparable across model versions, and resist the urge to tune against it until it stops representing anything real. Treat the eval set as a fixed contract with reality, and report per-slice word error rate rather than one comforting headline number.

Diversity versus volume

The honest answer is that it depends on where you are. Early on, when a model has thin coverage, diversity wins decisively. Adding an accent or a noise condition you had nothing of will move word error more than doubling the hours of something you already cover well. Most plateaus we see are coverage gaps, not volume gaps.

Once coverage is reasonable, volume within each slice starts to pay off, sharpening the model on conditions it already knows in outline. The trap is buying scale before coverage: a hundred thousand hours that all look alike leave the same blind spots a thousand well-chosen hours would have. For a structured way to size a collection against a target word error rate, see our guide on how much speech data you actually need. Spend the first budget on breadth, then deepen.

When you are ready to act on that, you can browse licensable speech corpora by language, domain, and recording condition, or scope a collection that matches your exact deployment, in our speech datasets catalogue.

ASR Training Data: What Makes Speech Recognition Accurate