Paralinguistic covers how something is said, as distinct from linguistic content, which is what is said. The words "I'm fine" carry one meaning as text and a completely different one depending on pitch, pace, and where the pauses land. Paralinguistic features are the measurable properties of the how: pitch, loudness, speaking rate, pausing, voice quality, and audible events like laughter and filled pauses.

This guide covers the concrete feature families, which ones a human annotator can reliably mark and which ones are better left to signal-processing extraction, and why paralinguistic labels tend to hold up better under scrutiny than emotion labels do. For the harder question of how to label the emotional content sitting on top of these features, see our guide on categorical and dimensional emotion labels.

What paralinguistic means

Linguistic content is the transcript: the sequence of words a speaker produced. Paralinguistic content is everything carried in how those words were delivered that is not itself a word. A rising pitch at the end of a sentence, a long pause before an answer, a sentence spoken quickly and quietly rather than slowly and loudly: none of that shows up in a transcript, and all of it changes how a listener interprets the sentence. Paralinguistic features are the layer a text-only model cannot see and an audio model can.

The feature families

Pitch, or fundamental frequency, tracks how high or low the voice sounds and how much it varies across an utterance. Energy, or loudness, tracks the intensity of the signal. Speaking rate tracks syllables or words per second. Pausing tracks the location, frequency, and duration of silence within and between utterances. Voice quality covers properties like breathiness, creak, and roughness in how the vocal folds are producing sound. Laughter and filled pauses, the "um" and "uh" of spontaneous speech, are discrete audible events rather than continuous measurements. Each family answers a different question about the audio, and a specification that just says "paralinguistic features" without naming which of these it wants is not a specification.

What humans annotate and what gets extracted

Pitch, energy, and speaking rate are almost never hand-labeled. They are computed directly from the waveform with standard signal-processing algorithms, and asking a human annotator to estimate a fundamental frequency by ear would be slower and less accurate than running the extraction. Voice quality sits in between: acoustic measures exist, but a trained annotator's judgment of breathiness or creak is still often used, because automated measures are noisier on real-world audio than they are on clean studio recordings.

Laughter, filled pauses, and pause boundaries are the opposite case. These are discrete events with a clear start and end that a human can mark reliably by listening, and in practice human annotation is how most datasets get these labels, sometimes with an automated pass used to flag candidate locations for a human to confirm or reject.

Why paralinguistic labels hold up better than emotion labels

An annotator marking "laughter occurred at 4.2 to 4.6 seconds" or "there was a 400 millisecond pause here" is describing an observable event with a start time, an end time, and a criterion two people can apply the same way. An annotator marking "this speaker sounds frustrated" is inferring an internal state from indirect evidence, and two careful listeners disagree on that far more often than they disagree on whether a pause happened. That is the practical reason paralinguistic annotation tends to show higher inter-annotator agreement than emotion annotation on the same audio: the target is closer to being directly observable. That does not make paralinguistic labels trivial to collect well, boundary placement on a pause or a laugh still needs a documented rule, but it is a different order of problem than judging emotion. For how to handle that harder judgment call, see the guide on categorical versus dimensional emotion labels.