Multilingual speech data contains recorded speech in more than one language, with the labels and metadata needed for a defined task. Planning useful coverage means deciding which languages, locales, speaker profiles and audio conditions the application needs, then checking what existing data actually contains.
This guide is for teams sourcing speech across markets. It explains how to compare language and accent coverage, investigate low-resource gaps, plan corpus allocation and specify code-switching when relevant. The practical outcome is a coverage brief that can be checked against a sample, an evaluation reference or a proposed collection before buying more hours.
What multilingual speech data has to cover
Start with the task and model stage. Supervised recognition can need audio paired with transcripts; self-supervised pretraining can use speech without transcripts. Translation, speaker tasks and evaluation have other output requirements. The XLS-R research paper (2021) describes self-supervised cross-lingual speech pretraining and transfer across languages. A multilingual model can share learned representations; test its suitability for the target task and conditions.
List the languages and relevant variation within them. Regional speech, domain vocabulary, first- or additional-language use, conversational style and capture conditions can all be useful planning dimensions. Include a dimension when the application or available evidence makes it relevant. A recording country or language name alone does not establish every accent, dialect or user profile represented in the files.
Our guide to what speech data is explains common source and label types. For recognition, the ASR training-data guide shows how to connect a measured evaluation gap to a next data decision. Here the focus is choosing and documenting coverage across languages and use profiles.
Sourcing across languages, dialects, and accents
Inventory what is available before planning recruitment. Ask whether each proposed source is recorded inventory, an accessible research release, customer-provided audio or a future collection. Check a sample, documentation, permitted uses, transcript or label policy, speaker metadata and recording conditions. Record missing evidence as an open question so it can be resolved before a larger commitment.
Define target speaker criteria against the use case. If regional dialect knowledge or a particular accent matters, specify the criterion, how it will be verified and which relevant metadata can be delivered under the agreed permissions. Assess language and dialect knowledge for annotation separately from the ability to record the audio. Recruitment location can be one field in that plan; it is insufficient evidence of the full speaker profile.
A supplier should explain its proposed recruitment and review method for each required coverage cell. Spirelight can assess language- and locale-specific recruitment against a written brief. Feasibility, speaker criteria, locations, supported deliverables, schedule and price are confirmed per project. Existing supplier reach is a starting point for that assessment, rather than proof that a particular target group is already recorded.
Why low-resource languages are hard
Describe the shortage precisely: usable audio for a task, a dialect, a domain, a transcript convention or an evaluation reference. A language may have public recordings while still lacking the particular permissions, labels or conditions the application needs. Reviewer availability, orthography decisions and recruitment access can each affect the proposed scope. Ask the supplier to identify which assumptions remain unverified.
The historical Common Voice research paper (2019/2020) describes a multilingual transcribed corpus with crowd collection and validation. It is an example of an existing source to investigate, rather than a current release inventory or proof of every required dialect. Verify the actual release, files, documentation and terms you intend to use.
- Agree spelling, script, normalization and uncertain-speech conventions for the required labels.
- Compare existing sources and a proposed collection against the same target task and coverage criteria.
- Specify reviewer qualifications, independent checks, escalation and acceptance before production.
- Choose read, prompted or spontaneous recording methods against the intended use and pilot evidence.
Where accessible sources leave a relevant gap, compare a focused custom collection with other documented options. Ask what a representative pilot can establish and what recruitment, permission or QA questions would remain. No universal number of hours resolves every low-resource setting.
Balancing a multilingual speech data corpus
Document the distribution of usable audio and speakers by the coverage dimensions you selected. That describes the corpus; it does not establish model performance. Equal hours per language, the largest possible total and a dominant-language mix each need assessment against the task, existing model, intended use and evaluation evidence.
Use the following original planning aid to write a coverage cell, attach available evidence and decide the next check. The rows are planning prompts with unfilled project decisions, not recorded Spirelight cohorts, benchmark results, required quotas or a recommended demographic mix. Replace them with the cells relevant to the application.
| Coverage cell | Evidence or gap | Candidate source and unknowns | Pilot and held-out check |
|---|---|---|---|
| Target language or locale, task and audio condition | Record the sample, documentation or evaluation available; mark unchecked fields | Compare an existing release or inventory with a proposed collection; confirm permissions and scope | Set task-specific output, label and acceptance checks before testing |
| Relevant regional or additional-language speaker profile | State the required profile and evidence of representation or an observed gap | Confirm eligibility verification, reviewer knowledge and metadata; leave feasibility open until assessed | Keep the relevant group visible in evaluation and inspect errors or uncertain cases |
| Language-switching pair and setting, if needed | Document evidence that the application needs this switching pattern | Confirm recording method, language/script policy and proposed speakers | Check the switching and transcription policy on representative examples |
Allocate the next collection or sourcing batch against the open cells and their priority, then evaluate the result. Keep a held-out check separate from training, with speaker, session or other separation suited to the task. Report relevant group results alongside the aggregate and document coverage that remains untested. State the metric and reference policy, the amount of evaluation data and the uncertainty or limits of each result. The guide to how much speech data to collect explains staged decisions; an added accent or additional hours have no guaranteed performance gain.
Transfer the target cells, evidence gaps, output requirements and acceptance questions into the speech dataset specification worksheet. Mark unknown values for feasibility review. A source comparison should distinguish verified contents, proposed work and unanswered questions.
Code-switching and bilingual speech
Include language switching when the application needs it. Define the language pair, direction, conversational setting and relevant speaker profile from product requirements or observed use. The SEAME corpus paper (2010) describes Mandarin-English switching recorded in interview and conversational settings. It supports that specific example; switching patterns and their relevance should be established for the intended application.
Specify the script and treatment of borrowed words, transliteration, uncertain boundaries and language tags where needed. Test the convention with appropriate speakers and reviewers on a pilot. Agree which cases receive correction or adjudication and how the export represents the result. The audio annotation guide explains label layers, review methods and acceptance questions that also apply to multilingual work.
Why representative speakers and dialect coverage matter
Choose source and recording methods against the target use. If the application serves first-language, additional-language or regional speakers, describe those users and their relevant audio conditions in the brief. Native status can be a justified project criterion when its meaning and verification are clear. It does not, by itself, document the required dialect, domain, recording condition or transcript quality.
Read scripts, translated prompts, conversational recordings and synthetic speech have different properties to assess. For example, Google's CVSS research repository for the 2022 corpus describes synthetic target speech for a defined speech-to-speech translation task. That design is one task-specific use, rather than evidence that a synthetic source matches a particular natural conversation. Inspect samples and evaluate the proposed method against the actual purpose and acceptance criteria.
Spirelight's dataset catalogue has 60 language and dialect listings, with published hours, speaker counts, recording country and price on individual pages. Check those listed facts, samples, source style, labels and licence against each required cell. Additional languages, accents or conditions can be assessed as a custom collection; feasibility, rights, schedule and price require a written proposal. A catalogue list and a proposed collection are different evidence states.
Existing catalogue starting points include Spanish from Mexico, French from France, German, Arabic MSA and Hindi. Each page publishes its listed hours, speaker count, recording country and price; confirm current availability and asset-specific contents. Where several sources are needed, compare speech data providers using the same coverage brief and evidence questions.