Quickstart: build a voice agent

This quickstart creates a complete voice agent. The Voice Agents API handles the whole pipeline. It captures your voice, transcribes it, generates an answer with a model, and speaks that answer.

To make this work, you need a Mistral API key, a microphone, speakers, and Python or Node.js. To run an agent in a web page instead, follow the Browser guide.

The Python SDK uses PyAudio. As PyAudio uses PortAudio, you'll need to install PortAudio. Include the voice-agents extra when you install the SDK, which pulls in the audio stack (PyAudio, LiveKit, NumPy) so the SDK can use your microphone and speaker.

For JavaScript or TypeScript, you'll need to install SoX and FFmpeg.

# macOS
brew install portaudio
# Linux
sudo apt install libportaudio2 libportaudiocpp0 portaudio19-dev libasound-dev libsndfile1-dev -y

pip install "mistralai[voice-agents]"

# macOS
brew install portaudio
# Linux
sudo apt install libportaudio2 libportaudiocpp0 portaudio19-dev libasound-dev libsndfile1-dev -y

pip install "mistralai[voice-agents]"

Store your API key in the MISTRAL_API_KEY environment variable. To create an API key, see Activate Studio and generate an API key.

export MISTRAL_API_KEY="your-api-key"

export MISTRAL_API_KEY="your-api-key"

Now save this code to a file, and run it with Python or Node.js.

import asyncio
import os

from mistralai.client import Mistral
from mistralai.extra.voice_agents import DefaultAudioInterface

MISTRAL_API_KEY = os.environ["MISTRAL_API_KEY"]

async def main():
    async with Mistral(api_key=MISTRAL_API_KEY) as client:
        session = await client.audio.sessions.init(
            audio_interface=DefaultAudioInterface(),
            instructions="Please respond to me in the most dramatic way possible."
        )

        def on_transcript(event):
            print(f"{event.role}: {event.content}")

        session.on("transcript", on_transcript)

        async with session:
            await session.wait()

asyncio.run(main())

import asyncio
import os

from mistralai.client import Mistral
from mistralai.extra.voice_agents import DefaultAudioInterface

MISTRAL_API_KEY = os.environ["MISTRAL_API_KEY"]

async def main():
    async with Mistral(api_key=MISTRAL_API_KEY) as client:
        session = await client.audio.sessions.init(
            audio_interface=DefaultAudioInterface(),
            instructions="Please respond to me in the most dramatic way possible."
        )

        def on_transcript(event):
            print(f"{event.role}: {event.content}")

        session.on("transcript", on_transcript)

        async with session:
            await session.wait()

asyncio.run(main())

Press Ctrl+C to stop.

This example only uses two parameters: "audio_interface" / audioInterface and "instructions". There are many more! For example, to change the agent's voice, use the "tts" parameter:

tts={"voice_id": "fr_marie_neutral"}

tts={"voice_id": "fr_marie_neutral"}