Audio annotation adds structured, machine-readable labels to recorded sound for a defined task. A project might need a transcript, time segments, speaker turns, sound-event tags or task-specific classifications. Choose the layers the application needs and define how each will be checked.
If you are planning labeled audio for a speech model, the label definitions and reference policy belong in the specification before production starts. This guide explains the main types, manual and model-assisted workflow choices, and the review and acceptance questions to confirm with audio annotation services.
What audio annotation actually produces
The output can be one label layer or several related layers. A transcript records words under a chosen convention; intervals connect a label to a span of audio; speaker labels distinguish turns; other tasks classify a sound or utterance. Agree the file identifiers, time units and relationships between layers so the delivered export can be used by the intended pipeline.
For an invented two-second record, example fields might be start: 0.0, end: 2.0, speaker: S1 and text: "I, uh, need to cancel.". A separately defined intent task could add cancel_request. A perceived-emotion task needs its own rubric and an uncertain or unratable option. Agree the actual schema and time units in the project brief; these invented fields illustrate the separate decisions.
Voice annotation and non-speech audio
Voice annotation usually means labeling speech: the words spoken, when they occur, which speaker produced a turn, or a task-specific tag such as an intent. Audio annotation is broader and can also label non-speech sounds. The terms overlap, so ask which layers a brief actually includes. A voice annotation project does not automatically require speaker names, emotion ratings or every background noise to be labeled.
For example, an assistant that needs to detect a cancellation request may need an utterance transcript and an intent label. A speech-recognition evaluation set may instead need a reviewed transcript and a stated normalization policy. The required labels follow the use case, not the name of the annotation tool.
The main types of audio annotation
Verbatim and non-verbatim transcription
Transcription records the spoken words under an agreed text policy. Verbatim and clean-read are useful starting labels, but define the actual treatment of fillers, repetitions, false starts, contractions, punctuation, numbers and unintelligible speech. A clean transcript might remove selected disfluencies for readability; a supervised recognition target may retain them or use a documented normalization step.
Choose the convention against the required output, then apply it consistently to any evaluation reference and model hypothesis. The Hugging Face Audio Course explanation of ASR evaluation shows how text normalization changes recognition error measurements. The official Label Studio transcription template illustrates a transcription labeling job. State the project’s own transcript style explicitly.
Timestamps and segmentation
Segmentation marks intervals such as utterances, words or speaker turns. Specify the unit being labeled, time units, treatment of silence and overlap, and the boundary policy. An utterance-level timestamp and a word-level alignment are different deliverables. Import a small proposed export and inspect the intervals against the recording before accepting a larger batch.
A forced aligner attempts to align supplied text with audio. NVIDIA NeMo Forced Aligner documentation describes that tool's inputs and constraints. Test the chosen tool on the relevant recordings and inspect its output before committing to the workflow.
Speaker labels and diarization
Diarization concerns who spoke when. Specify whether labels such as S1 and S2 are consistent within a file, a session or a wider agreed scope, and how overlapping speakers are represented. Those labels do not verify a person's name. Role or other requested speaker attributes are separate tasks that need their own definitions and evidence. Label Studio's speaker-segmentation example illustrates assigning speaker labels to audio regions; our diarization guide explains the distinction from identification.
Events and non-speech sounds
A sound-event task might mark music, a door closing, traffic or competing speech. Choose the events relevant to the application, whether labels cover whole clips or intervals, and how simultaneous events, uncertain cases and unlabeled regions are handled. The official sound-event detection template provides an example of interval labels. Event annotation is optional; a transcription project need not include every background sound.
Intent, sentiment, and emotion tags
Define intent, sentiment and perceived emotion as separate tasks when they are needed. Intent might classify a cancellation request; a perceived-emotion label follows a separately agreed listening rubric. Specify the audio and conversational context available to annotators, label definitions, uncertain cases and how disagreements are resolved. Do not treat speaker IDs, emotion ratings and verified speaker attributes as interchangeable. Label Studio's audio-classification template illustrates configurable choices. Select the taxonomy against the actual task and available context.
Choose review depth against the acceptance test
Review depth should follow the use of the labels. A training batch, a held-out evaluation reference and a subjective classification task can need different checks. Names such as standard or premium do not establish how much work is reviewed. Ask for the actual review share, independence, rework triggers and batch-acceptance rules; Spirelight confirms these in the written scope.
| Layer needed | Define before annotation | Check on a representative pilot |
|---|---|---|
| Transcript | Text convention, normalization and unintelligible-speech policy | Compare with reviewed references under the same convention |
| Timing | Segment unit, time unit and boundary or overlap policy | Load the export and inspect intervals against the audio |
| Speaker turns | Identifier scope and simultaneous-speaker representation | Review speaker consistency and challenging turns |
| Sound events | Classes, interval or clip labels, and uncertain cases | Inspect examples for each required class and unlabeled regions |
| Intent or perceived emotion | Separate rubric, available context and disagreement process | Compare independent judgments and adjudicate disputed examples |
Use the relevant transcript, annotation and acceptance fields in the speech dataset specification worksheet. Leave collection fields open when they do not apply to existing audio. The table is an original planning aid. Set passing criteria in the project scope and request actual pilot evidence.
From an audio file to an accepted export
Consider an invented two-speaker clip in which one person says “I need to cancel” while another begins to speak. A usable pilot does more than transcribe the sentence:
- Record the file ID, language, channel, source permission and time unit in the brief. Decide whether the overlapping voice needs its own speaker turn.
- Write the agreed words and start/end times, assign anonymous S1 and S2 labels where supported by the audio, and apply the separately defined
cancel_requestintent only to the intended unit. - Export the fields in the requested schema. Load the file into the buyer’s pipeline and compare the intervals with the clip, including the overlap and any uncertain words.
- Have the agreed reviewer check the pilot against the label guide. Record discrepancies, corrections and the written rule for accepting or reworking the batch.
This is an illustrative workflow, not a Spirelight delivery or a universal schema. The actual brief determines which layers and checks are needed.
Human versus machine-assisted workflows
Manual, model-assisted and automated steps are workflow options to test against the task. A model may propose transcripts, speaker turns, timing or classifications; people may correct that draft or independently create a reference. Record which steps use a model, which outputs receive review, and how the final export passes acceptance rather than assuming one approach suits every recording.
Compare editing time, review effort, rework and accepted output on a representative pilot using the same criteria. Savings and quality depend on the chosen model, audio, conventions and workflow. The Whisper model card, for example, documents that model's capabilities and limitations. Evaluate the proposed model and version on the conditions that matter to the project. Keep difficult pilot examples for investigation and relate any new recording need to how the speech data is collected.
Choosing an audio annotation tool
An annotation tool can let a team open audio files, place boundaries on a waveform, assign labels and export the result. Some workflows import model suggestions for people to check. Before selecting a tool, try the proposed label guide on a small set of real project recordings: can it represent overlap, unknown speakers and uncertain words; preserve the required time units; and export the fields your pipeline reads? The Label Studio transcription template illustrates one configurable interface, not a required Spirelight platform.
Keep the annotation instructions and review decisions with the export. A tool can store labels, but the project still needs people or agreed checks to decide whether those labels meet the acceptance criteria.
How QA actually works
Start with representative pilot audio and a written label guide. Independent double annotation means people label without seeing each other's answers; a review pass inspects existing work. They answer different questions. Specify which is used, the share covered, escalation and correction rules, and who approves a batch. Record independent annotation and review as distinct checks.
Name the agreement method, unit of comparison and label layer. Disagreement can call for checking definitions, uncertain audio, task difficulty, tool behavior, expertise or mistakes; inspect examples before diagnosing a cause. Shared agreement alone does not establish that labels are correct. HumanSignal's agreement documentation distinguishes methods and comparison settings. For transcripts, an error metric such as WER requires an appropriate reviewed reference and declared normalization. For timing, define the boundary rule, units and error-measurement method.
Document the sampling plan. Stratified random sampling within relevant language, channel or recording-condition groups can cover important conditions; targeted risk checks can add scrutiny to known problems. Keep those purposes distinct when reporting results. Record the coverage and denominators, layer-specific checks, disputed examples and resulting actions so a good aggregate cannot substitute for an untested requirement.
Where annotation sits in the pipeline
The intended use shapes the labels and acceptance test. Check a baseline and a first batch before choosing more volume; our guide to how much speech data a model needs explains baseline-and-batch planning. The same reference, export and quality questions apply when buying finished data.
Spirelight can assess transcription and labeling through its audio annotation service. Share the language, audio conditions, label definitions and acceptance requirements; supported labels, reviewers, QA, permissions, timing and price are confirmed per project. For an existing conversational asset, compare the speech catalogue against the actual transcript, speech style, source and licence your application requires.