There are two established ways to put emotion labels on speech, and picking the wrong one costs you a relabeling pass. The categorical approach tags each utterance or speaker turn with a class: anger, disgust, fear, happiness, sadness, surprise, or neutral. The dimensional approach scores the same audio on continuous scales, usually valence, meaning how positive or negative the speaker sounds, and arousal, meaning how activated or subdued they sound.
Neither is more correct. They answer different questions, they suit different model heads, and they fail in different ways. This guide covers what each one trains well, where each one breaks, and why collecting both in a single annotation pass is often cheaper than choosing.
Categorical labels: one class per turn
The seven-class scheme above comes from Paul Ekman's work on basic emotions, the six he argued were recognized across cultures, plus a neutral class that in practice covers most of any real corpus. Its appeal is mechanical: the classes map directly onto a softmax output, the labels are cheap to collect, and accuracy is trivial to report.
The costs show up at the edges. Real speech is frequently blended, a caller can be simultaneously frightened and irritated, and a single-class scheme forces the annotator to discard one of those. Intensity vanishes too: mild annoyance and shouting rage collapse into the same anger bucket. And the neutral class quietly absorbs everything ambiguous, so a model can score well while having learned mostly to detect the absence of strong signal.
Dimensional labels: valence and arousal
Dimensional labeling scores each segment on continuous scales instead of naming it. Valence runs negative to positive, arousal runs calm to activated. Anger and fear sit close on arousal but differ elsewhere, which is precisely the distinction a categorical scheme makes an annotator guess at.
The gain is that intensity and blend survive: a rising arousal score across a call is exactly the trend an escalation model wants, and no annotator is forced to pick a single word for a mixed state. Dimensional ratings also tend to produce steadier agreement than categories do, because placing a point on a scale is an easier, more consistent task than choosing between near-synonyms like anger and disgust, where two careful annotators can land on different words for the same clip. The residual cost is calibration, not disagreement: one rater's 0.6 valence is another's 0.4 even when the two broadly agree on direction, so dimensional data still needs per-rater calibration or enough raters that the offset averages out.
How to choose
Use categorical labels when the downstream action is discrete: routing a call, triggering a handoff, tagging a recording for review. Use dimensional labels when the downstream question is about degree or direction: how tense this conversation became, whether it improved, how strongly a synthesized line reads as positive.
Where the decision is genuinely unclear, collect both. Because the annotator has already located and listened to the segment, adding valence and arousal to a pass that is producing classes costs far less than a second pass would, and it leaves the choice of model head open until you have trained something.
What neither approach fixes
Both schemes inherit the same underlying problem: emotion has no ground truth the way a transcript does. Two careful annotators will disagree on speech far more often than they disagree on words, and no labeling scheme removes that. What a scheme can do is make the disagreement visible, which is why inter-annotator agreement should be reported per class rather than folded into one headline number.
Treat any emotion dataset shipped without agreement figures as unmeasured. The labels may well be good, but nothing in the delivery tells you so.