Regional inference

Mistral offers regional inference as an optional service through dedicated API endpoints. It supports processing inference requests by systems located within a chosen geography: currently the European Union or the United States.

Use regional inference when you have data location requirements for inference inputs and outputs, or when your users are concentrated in a region and want lower latency.

Inference is the execution step where a model processes your input and generates output. On Mistral-hosted endpoints, that execution runs on GPU infrastructure managed by Mistral.

Regional inference routes that model execution to infrastructure located within a selected geography. The request still uses the Mistral API, but the regional endpoint determines where inference processing happens.

Regional inference helps you:

• Control where inference input and output data is processed.
• Support workloads with specific data location requirements.
• Reduce latency when most users or systems are close to the selected geography.

Mistral distinguishes three endpoints:

Use regional endpoints only for workloads whose models, tools, and features are available in the target region. Regional endpoints only serve models hosted in that region.

Regional endpoints have feature limitations. If a workload depends on an unsupported feature, the regional processing guarantees for that workload do not apply.

• Not all tool calls are currently supported. Function calling is the only supported regional tool.
• Stateful features, including Agents, Batch, and the Files API, are not available on regional endpoints.
• Available models vary by region.

Regional endpoints only serve models hosted in that region. Use the same model IDs as the global API, but check availability against the regional endpoint before you send production traffic.

To list available models, call models.list against the regional base URL or the SDK server value for the region.

import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"], server="eu")
models = client.models.list()

for model in models.data:
    print(model.id)

import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"], server="eu")
models = client.models.list()

for model in models.data:
    print(model.id)

Select a regional endpoint by configuring the SDK client or by sending requests to the regional base URL directly.

Use the server parameter when initializing the client to specify the region:

import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"], server="eu")
response = client.chat.complete(
    model="mistral-medium-latest",
    messages=[{"role": "user", "content": "What is Mistral?"}],
)
print(response.choices[0].message.content)

import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"], server="eu")
response = client.chat.complete(
    model="mistral-medium-latest",
    messages=[{"role": "user", "content": "What is Mistral?"}],
)
print(response.choices[0].message.content)

If you use an older SDK, use the server_url parameter instead:

import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"], server_url="https://api.eu.mistral.ai/")
list_models_response = client.models.list()
print(list_models_response)

import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"], server_url="https://api.eu.mistral.ai/")
list_models_response = client.models.list()
print(list_models_response)

curl https://api.eu.mistral.ai/v1/chat/completions \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-large-latest",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

curl https://api.eu.mistral.ai/v1/chat/completions \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-large-latest",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

The regional endpoint you call is the source of truth for the requested processing geography. For auditability, log the endpoint hostname, SDK server value, model ID, timestamp, and response request identifier for each regional request.

If you route traffic through a proxy, gateway, or observability tool, keep request logs that show the target base URL (api.eu.mistral.ai or api.us.mistral.ai). These logs help you confirm that the request was sent to the expected regional endpoint.

Regional inference controls where eligible inference requests are processed. Zero data retention controls whether eligible request and response content is retained after processing. These are separate controls, and you may need both depending on your data handling requirements.

To learn how zero data retention works, see Zero data retention.