Agent configuration

This page lists every option you can pass to client.audio.sessions.init(). Options come in two scopes. Agent options define the agent itself: its models, voice, instructions, tools, and turn-taking. You can save them as a saved agent. Session options only apply to the session you open, such as which saved agent to start from and which audio interface to use.

client.audio.sessions.init() is a method on the general-purpose Mistral client. Construct the client once, then start any number of sessions from it.

Mistral(
    api_key: str,
    server_url: str = "https://api.mistral.ai",
    timeout_ms: int = 60000,
)

Mistral(
    api_key: str,
    server_url: str = "https://api.mistral.ai",
    timeout_ms: int = 60000,
)

You can pass the agent options to init() every time, which suits quick experiments. Or you can save them once as an agent and start each session with agent_id. When you set agent_id, the saved configuration is the base, and any agent option you also pass to init() overrides it for that session only. The saved agent doesn't change. Session options always apply. For an example, see Create an agent and start a session.

These options are part of the agent configuration, and you can save them on an agent.

These options only apply to the session you open.

Set stt.turn_detection to let a Voxtral 3 realtime model decide when the user has finished speaking. Without it, the agent uses the model's default turn detection. Pass {} to use the server defaults, or set any of these fields:

For example, to let the agent answer after a shorter silence:

session = await client.audio.sessions.init(
    stt={
        "model": "voxtral-mini-transcribe-realtime-3",
        "turn_detection": {"min_silence_ms": 300},
    },
)

session = await client.audio.sessions.init(
    stt={
        "model": "voxtral-mini-transcribe-realtime-3",
        "turn_detection": {"min_silence_ms": 300},
    },
)

For what each threshold does and the faster Aggressive preset, see Turn detection.

tts accepts the following fields:

Use phoneme_hints to fix how the agent says names, brands, or acronyms. Each word maps to a list of IPA phonemes. Words are case-sensitive. Only English phonemes are supported, and a phoneme outside that set is rejected when you create the session or save the agent.

session = await client.audio.sessions.init(
    tts={
        "model": "voxtral-mini-tts-3",
        "voice_id": "voice_447",
        "phoneme_hints": {"Mistral": ["m", "ɪ", "s", "t", "ɹ", "ɑ", "l"]},
    },
)

session = await client.audio.sessions.init(
    tts={
        "model": "voxtral-mini-tts-3",
        "voice_id": "voice_447",
        "phoneme_hints": {"Mistral": ["m", "ɪ", "s", "t", "ɹ", "ɑ", "l"]},
    },
)

On top of Voxtral TTS 2 voices listed below, here are the additional actor voices supported by Voxtral TTS 3.

On top of these actor voices, Voxtral TTS 2 supports instant voice cloning to create a voice from an audio sample.

turn controls how the agent shares the conversation with the user:

inactivity accepts:

session = await client.audio.sessions.init(
    turn={
        "allow_interruptions": True,
        "inactivity": {
            "timeout_ms": 10000,
            "max_reminders": 2,
            "messages": ["Are you still there?"],
        },
    },
)

session = await client.audio.sessions.init(
    turn={
        "allow_interruptions": True,
        "inactivity": {
            "timeout_ms": 10000,
            "max_reminders": 2,
            "messages": ["Are you still there?"],
        },
    },
)

You can change instructions, tools, and turn on a running session. See Update a live session.