Speakers and diarization
A transcript of a call or a meeting is only useful when you know who said what. This page explains what speaker labels tell you, how to turn them into real names, and how to show speakers who talk at the same time.
Who spoke when, not who is speaking
Diarization groups the parts of a recording that come from the same voice and gives each group a label, such as speaker_1 or speaker_2. A label tells you that two passages come from the same voice. It doesn't tell you whose voice it is.
Labels stay consistent within one transcript, so you can follow each voice from start to end. They mean nothing outside it: speaker_1 in one call and speaker_1 in the next aren't necessarily the same person. To follow a person from one recording to the next, name them with reference clips.
Diarization works in both modes, but the labels don't arrive in the same form:
| Offline | Realtime | |
|---|---|---|
| Where labels appear | On each segment of the finished transcript, with its start and end times | On each piece of text, as it arrives during the session |
| What you can build | Speaker-labeled records, subtitles, and per-speaker summaries | Live captions that show who is speaking |
For the request options and examples in each mode, see Diarize speakers.
Naming speakers with reference clips
When you know in advance who speaks, such as the agent and the customer on a support call, you can send a short clip of each voice under a name. The transcript then shows those names instead of generic labels, and the same name refers to the same person in every transcript.
A reference only helps if it matches the voice in the recording:
- Use the same person. A different voice that sounds similar isn't matched to the name.
- Record in similar conditions. A clip from the same microphone or phone line as the real audio is the closest match.
- Expect unnamed speakers. A speaker without a reference keeps a generic label, so a call with an unexpected third person still transcribes.
Reference clip guidelines
| Property | Recommendation |
|---|---|
| Duration | 3 to 15 seconds |
| Number of references | Up to 8. The total number of speakers across references and prediction must not exceed 10. |
| Speaking rate | At least 85% of the clip should be speech. Avoid long silences. |
| Conditions | The closer the acoustic and speaking conditions are to the audio being transcribed, the better the match. |
Reference clips work offline and in realtime. For how to send them, see Diarize speakers.
Overlapping speech
In real conversations, people talk at the same time: one person interrupts another, or someone says "yes" while someone else is speaking. Overlapping speech (two speakers talking at the same time) is not supported in this version.
A transcript with overlaps isn't a simple sequence of turns. If your interface shows one speaker at a time, such as a chat-style transcript or a timeline with a single track, plan how to show two active speakers. For example, order the segments by start time and let them overlap, so you don't cut one of them.