Custom-scoped evaluation set
Voice agent evaluation data for accents, noise, and real call conditions
Build a private test set around the callers, channels, languages, and failure modes your voice agent actually faces. Spirelight scopes the collection, reference transcripts, slices, and holdout rules with your team.
Example availability: Tell us your language and channel requirements. The team will check whether a relevant conversational or telephony example can be shared free of charge, then confirm the available options and timing.
Collection specification
- Audio
- Wideband, narrowband, mono, or separate channels by project
- Coverage
- Accents, languages, SNR bands, interruptions, overlap, and domain scenarios
- References
- Human-reviewed transcripts and agreed scoring fields
- Holdout
- Collection split and non-reuse terms documented in the project scope
- Delivery
- Audio, transcripts, metadata, split manifest, and evaluation brief
- Pricing
- Custom, scoped to your conditions
Test the conditions hidden by a single average score
A useful voice-agent evaluation set separates accent, language, channel, noise, interruption, and scenario effects. The result is not just one word error rate. It shows which caller groups and conditions cause the system to fail.
The collection is scoped around your deployment. If a hosted model cannot be fine-tuned, the same held-out set can still compare providers and detect regressions after each release.
Keep evaluation audio out of the training delivery
Training leakage makes a benchmark look better without making the production system better. We define the evaluation split, file manifest, access rules, and any non-reuse commitment in the statement of work before recording starts.
Define the first batch your team can evaluate
Specify which caller groups, scenarios and recording channels the first batch must represent, and the reference text and labels your evaluation pipeline expects. Agree the held-out speaker and session rules before collection.
The written proposal should separate collection and annotation from the evaluation your team runs. Confirm usable-audio accounting, acceptance checks, rework rules and any later production batches against that initial scope.
This offer is not the right fit when
- You need a public benchmark with unrestricted redistribution.
- You cannot describe the deployment channel, language, or main failure mode.
- You need an immediately downloadable fixed pack without a scoping step.
Request a free sample
- Your details
- Your project
- Verify
Three short steps. The team checks whether a permissioned example matches your use and can be shared, then follows up with the available options.
Would rather talk it through first? Request an evaluation project quote
Prepare the brief
Bring a specification your vendors can price consistently
Use the free worksheet to define coverage, rights, delivery fields, held-out rules, and acceptance tests before requesting a collection plan.
Buyer documentation
Specimen data card and provenance structure, evaluation brief template, and acceptance and held-out questions.
Guides for this decision
Need a scoped collection plan? Send the language, channel, volume, annotation, and timing you know. The quote form opens directly in project-brief mode.