Custom pronunciation

Custom pronunciation tells a voice exactly how to say a word that it would otherwise guess from the spelling, such as a brand name, a surname, or an acronym read as a word. This page shows how to write a pronunciation, which phonemes the API accepts, and what to expect when a hint is wrong.

Information

For why voices mispronounce rare words, and how this relates to context biasing in transcription, see Custom vocabulary.

API reference — POST /v1/audio/speech

Define a pronunciation

In phoneme_hints, map a word of input to the list of phonemes to say it with. Each phoneme is one string of the list.

import base64
import os
from pathlib import Path

import httpx

response = httpx.post(
    "https://api.mistral.ai/v1/audio/speech",
    headers={"Authorization": f"Bearer {os.environ['MISTRAL_API_KEY']}"},
    json={
        "model": "voxtral-mini-tts-3-0",
        "input": "I like tomato soup.",
        "voice_id": "gb_jane_neutral",
        "phoneme_hints": {"tomato": ["t", "ə", "m", "ɑ", "t", "oʊ"]},
        "response_format": "wav",
    },
    timeout=60,
)
response.raise_for_status()

output = Path("tomato.wav")
output.write_bytes(base64.b64decode(response.json()["audio_data"]))
print(f"Saved {output.resolve()}")

import base64
import os
from pathlib import Path

import httpx

response = httpx.post(
    "https://api.mistral.ai/v1/audio/speech",
    headers={"Authorization": f"Bearer {os.environ['MISTRAL_API_KEY']}"},
    json={
        "model": "voxtral-mini-tts-3-0",
        "input": "I like tomato soup.",
        "voice_id": "gb_jane_neutral",
        "phoneme_hints": {"tomato": ["t", "ə", "m", "ɑ", "t", "oʊ"]},
        "response_format": "wav",
    },
    timeout=60,
)
response.raise_for_status()

output = Path("tomato.wav")
output.write_bytes(base64.b64decode(response.json()["audio_data"]))
print(f"Saved {output.resolve()}")

The voice says "tomato" with the six phonemes of the hint, and reads the rest of the sentence as usual. A few rules apply to the keys and the values:

One key per word to fix. Add as many words as you need to the same object.
Context biasing biases the model toward the provided terms; it doesn't require an exact match. For example, if you provide ChatGippiti in the context and the user says ChatGPT, the model is biased to transcribe ChatGippiti.
The hints apply to one request. Send them with every request whose text contains the words.

Accepted phonemes

The API validates each phoneme against its supported IPA inventory. Invalid phonemes are rejected when the request is sent.

Group	Phonemes
Vowels	`a` `e` `i` `o` `u` `æ` `ɑ` `ɔ` `ə` `ɚ` `ɛ` `ɜ` `ɪ` `ʊ` `ʌ`
Diphthongs	`aɪ` `aʊ` `eɪ` `oʊ` `ɔɪ`
Consonants	`b` `d` `dʒ` `f` `g` `h` `j` `k` `l` `m` `n` `p` `s` `t` `tʃ` `v` `w` `z` `ð` `ŋ` `ɖ` `ɱ` `ɹ` `ɾ` `ʃ` `ʈ` `ʋ` `ʒ` `ʔ` `θ`

Caution

A few symbols differ from what you might type out of habit:

For the English r, use ɹ. The plain letter r isn't in the list.
Type g as the regular keyboard letter, not the IPA character ɡ.
Write a diphthong or a combined sound such as oʊ, tʃ, or dʒ as one entry, not two.
Stress and length marks, such as ˈ or ː, aren't in the list.
Some sounds of other languages, such as the French ʁ or nasal vowels, aren't in the list either.

Limits and errors

The API checks every phoneme of every hint, and rejects the whole request if one of them is invalid:

Situation	What you get	Fix
A phoneme isn't in the list	A `422` error whose message lists the accepted phonemes	Replace the symbol with one from the list

Going further