Document extractors

Document extractors turn a File into a Document that the rest of the pipeline processes. Each file type needs its own extraction logic: PDFs can go through OCR, while spreadsheets need tabular extraction.

Built-in extractors (except LegacyOfficeExtractor, which re-creates the file after conversion) stamp file.source_id onto the produced Document and its chunks, and the document ID is computed deterministically from it. See the Document model for details.

MistralOCRExtractor uses the Mistral OCR API to extract structured text and images from PDF, DOCX, PPTX, ODT, and EPUB files, and from images. We recommend it for scanned and image-heavy documents.

Requirements:

• A Mistral API key

Installation: Core library (no extra required)

Example:

import os

from mistralai.client import Mistral
from mistralai.search.toolkit.ingestion.extractors import MistralOCRExtractor

mistral_client = Mistral(
    api_key=os.environ.get("MISTRAL_API_KEY", "your-api-key"),
    server_url=os.environ.get("MISTRAL_SERVER_URL"),  # Optional
)
extractor = MistralOCRExtractor(
    client=mistral_client,
    include_image_base64=True,      # Include image data
    include_image_annotation=True,  # Add image annotations
)
document = await extractor.extract(file)

import os

from mistralai.client import Mistral
from mistralai.search.toolkit.ingestion.extractors import MistralOCRExtractor

mistral_client = Mistral(
    api_key=os.environ.get("MISTRAL_API_KEY", "your-api-key"),
    server_url=os.environ.get("MISTRAL_SERVER_URL"),  # Optional
)
extractor = MistralOCRExtractor(
    client=mistral_client,
    include_image_base64=True,      # Include image data
    include_image_annotation=True,  # Add image annotations
)
document = await extractor.extract(file)

Parameters:

Features:

• OCR for scanned documents and images
• Markdown output with structure preservation
• Image reference tracking and optional base64 encoding
• Configurable page extraction (specific pages or page ranges)
• Optional inline markdown tables (embed_tables)
• Large file handling with automatic splitting

PyMuPDFExtractor extracts text and metadata from a document's native text layer in-process via the PyMuPDF (fitz) C library. It needs no API call or network round trip, and has no per-page billing: the only cost is local CPU. Use it for PDFs and other files that already carry an extractable text layer.

It does not perform OCR: scanned or image-only documents produce empty output. Use MistralOCRExtractor for those, or SmartPDFExtractor to route between the two automatically. Embedded images and bounding-box annotations are not returned. Document metadata (title, author, subject, dates) is surfaced from the document's built-in metadata dict.

Installation:

uv add "mistralai-search-toolkit[extractor-pymupdf]"

uv add "mistralai-search-toolkit[extractor-pymupdf]"

A PyMuPDF Pro license unlocks DOCX, PPTX, DOC, PPT, HWP, and HWPX in addition to PDF and EPUB.

Example:

import os

from mistralai.search.toolkit.ingestion.extractors import PyMuPDFExtractor

extractor = PyMuPDFExtractor(
    pymupdf_pro_license_key=os.environ.get("PYMUPDF_PRO_LICENSE_KEY"),  # Optional
    content_type="application/pdf",               # Optional; inferred from the file otherwise
)
document = await extractor.extract(file)

import os

from mistralai.search.toolkit.ingestion.extractors import PyMuPDFExtractor

extractor = PyMuPDFExtractor(
    pymupdf_pro_license_key=os.environ.get("PYMUPDF_PRO_LICENSE_KEY"),  # Optional
    content_type="application/pdf",               # Optional; inferred from the file otherwise
)
document = await extractor.extract(file)

Parameters:

SmartPDFExtractor routes each PDF between local extraction and OCR based on the quality of its text layer. Most Office-exported PDFs carry a native text layer and do not need the expensive OCR route; scanned documents do. SmartPDFExtractor decides per document which delegate to use.

The extractor runs local extraction to completion and judges the finished document rather than a leading-pages sample. The local route is PyMuPDFExtractor; the OCR route is any DocumentExtractor you inject, so an air-gapped or gateway-fronted deployment can supply a self-hosted engine in place of MistralOCRExtractor. The chosen delegate's document is returned unchanged. Its own type, extractor_type, and metadata are preserved, so a caller that receives a PyMuPDFDocument gets the richer metadata PyMuPDF surfaces.

PDF only: any other input is rejected.

Installation: No extra of its own, but the local delegate needs the PyMuPDF extra and the OCR delegate needs its own:

uv add "mistralai-search-toolkit[extractor-pymupdf]"

uv add "mistralai-search-toolkit[extractor-pymupdf]"

Example:

import os

from mistralai.client import Mistral
from mistralai.search.toolkit.ingestion.extractors import (
    MistralOCRExtractor,
    PyMuPDFExtractor,
    SmartPDFExtractor,
)

ocr_extractor = MistralOCRExtractor(
    client=Mistral(api_key=os.environ["MISTRAL_API_KEY"]),
)
extractor = SmartPDFExtractor(
    pymupdf_extractor=PyMuPDFExtractor(content_type="application/pdf"),
    ocr_extractor=ocr_extractor,
    min_chars_per_page=200.0,  # Default; below this, route to OCR
)
document = await extractor.extract(file)

import os

from mistralai.client import Mistral
from mistralai.search.toolkit.ingestion.extractors import (
    MistralOCRExtractor,
    PyMuPDFExtractor,
    SmartPDFExtractor,
)

ocr_extractor = MistralOCRExtractor(
    client=Mistral(api_key=os.environ["MISTRAL_API_KEY"]),
)
extractor = SmartPDFExtractor(
    pymupdf_extractor=PyMuPDFExtractor(content_type="application/pdf"),
    ocr_extractor=ocr_extractor,
    min_chars_per_page=200.0,  # Default; below this, route to OCR
)
document = await extractor.extract(file)

Parameters:

Routing:

1. If force_route is set to ocr on the context, skip the assessment and use OCR.
2. Otherwise load local bytes. If they are unavailable and force_route is pymupdf, raise. If they are unavailable otherwise, route to OCR.
3. Run local extraction. If it fails and force_route is pymupdf, raise. If it fails otherwise, fall back to OCR.
4. Measure the finished document. If force_route is pymupdf or the mean chars/page meets min_chars_per_page, return the local document. Otherwise route to OCR.

A loader failure returns None rather than propagating: OCR resolves the file from file.path without raw, so the document is still extracted rather than dropped over an infrastructure fault the fallback is immune to. When OCR is reached and fails, the error propagates: the branches that reach it have already established that no usable local text exists.

Forcing a route: Subclass IngestContext and add a smart_pdf_options field to override routing per request. A single nullable force_route (rather than a pair of booleans) keeps the contradictory "both true" state unrepresentable.

from mistralai.search.toolkit.context import IngestContext
from mistralai.search.toolkit.ingestion.extractors.smart_pdf import (
    PdfRoute,
    SmartPDFExtractorRuntimeOptions,
)

class SmartPDFContext(IngestContext):
    smart_pdf_options: SmartPDFExtractorRuntimeOptions

# Skip the assessment and use a specific delegate
context = SmartPDFContext(
    smart_pdf_options=SmartPDFExtractorRuntimeOptions(force_route=PdfRoute.OCR),
)
document = await extractor.extract(file, context=context)

from mistralai.search.toolkit.context import IngestContext
from mistralai.search.toolkit.ingestion.extractors.smart_pdf import (
    PdfRoute,
    SmartPDFExtractorRuntimeOptions,
)

class SmartPDFContext(IngestContext):
    smart_pdf_options: SmartPDFExtractorRuntimeOptions

# Skip the assessment and use a specific delegate
context = SmartPDFContext(
    smart_pdf_options=SmartPDFExtractorRuntimeOptions(force_route=PdfRoute.OCR),
)
document = await extractor.extract(file, context=context)

force_route accepts PdfRoute.PYMUPDF ("pymupdf"), PdfRoute.OCR ("ocr"), or None to assess the document.

MistralAudioTranscriptionExtractor uses the Mistral audio transcription API to transcribe audio files into text. It supports speaker diarization and timestamp granularities.

Requirements:

• A Mistral API key

Installation: Core library (no extra required)

Example:

import os

from mistralai.client import Mistral
from mistralai.search.toolkit.ingestion.extractors import MistralAudioTranscriptionExtractor

extractor = MistralAudioTranscriptionExtractor(
    client=Mistral(api_key=os.environ.get("MISTRAL_API_KEY", "your-api-key")),
    model_name="voxtral-mini-latest",  # Default model
    language="en",          # Optional language hint
    diarize=True,           # Enable speaker diarization
    timeout_seconds=900,    # Transcription timeout (default)
)
document = await extractor.extract(file)

import os

from mistralai.client import Mistral
from mistralai.search.toolkit.ingestion.extractors import MistralAudioTranscriptionExtractor

extractor = MistralAudioTranscriptionExtractor(
    client=Mistral(api_key=os.environ.get("MISTRAL_API_KEY", "your-api-key")),
    model_name="voxtral-mini-latest",  # Default model
    language="en",          # Optional language hint
    diarize=True,           # Enable speaker diarization
    timeout_seconds=900,    # Transcription timeout (default)
)
document = await extractor.extract(file)

Parameters:

Features:

• Audio transcription with speaker diarization support
• Timestamp granularities for word and segment-level precision
• Language hint support for improved accuracy
• Support for raw file uploads and remote URL sources
• Customizable timeout for long audio files

PlainTextExtractor reads plain-text documents, such as TXT, MD, CSV, and source code files.

Installation: Core library (no extra required)

Example:

from mistralai.search.toolkit.ingestion.extractors import PlainTextExtractor

extractor = PlainTextExtractor(
    page_size=2000,    # Characters per page (default)
    encoding="utf-8",  # Default encoding
)
document = await extractor.extract(file)

from mistralai.search.toolkit.ingestion.extractors import PlainTextExtractor

extractor = PlainTextExtractor(
    page_size=2000,    # Characters per page (default)
    encoding="utf-8",  # Default encoding
)
document = await extractor.extract(file)

Configuration options:

Behavior:

• Files larger than page_size are split into multiple pages
• Each page becomes a document page with page_number metadata
• The entire file content is preserved in document.content
• Text is decoded with encoding, without auto-detection. Use decode_errors="replace" to tolerate invalid bytes. RTF files are converted to plain text.

HTMLExtractor parses HTML and HTM files, converting them to clean markdown for downstream chunking and retrieval. It strips boilerplate elements (navigation, footers, scripts) by default.

Installation:

For default markdown conversion:

uv add "mistralai-search-toolkit[html-converter-markdownify]"

uv add "mistralai-search-toolkit[html-converter-markdownify]"

Or use core library only if providing a custom converter.

Example:

from mistralai.search.toolkit.ingestion.extractors import HTMLExtractor

extractor = HTMLExtractor()
document = await extractor.extract(file)

from mistralai.search.toolkit.ingestion.extractors import HTMLExtractor

extractor = HTMLExtractor()
document = await extractor.extract(file)

The default converter is MarkdownifyConverter, backed by the markdownify library (MIT). It is tuned to produce clean markdown with ATX headings, consistent bullet styles, code fence language detection, and full subtree stripping for boilerplate tags.

Configuring the default converter:

MarkdownifyConverter accepts several options to customize its behavior:

from mistralai.search.toolkit.ingestion.extractors import HTMLExtractor, MarkdownifyConverter

# Use default settings
converter = MarkdownifyConverter()

# Or customize: only strip <script> and <style>, disable id/class filtering
converter = MarkdownifyConverter(
    ignore_tags=["script", "style"],
    ignore_ids=[],
    ignore_classes=[],
)

extractor = HTMLExtractor(converter=converter)
document = await extractor.extract(file)

from mistralai.search.toolkit.ingestion.extractors import HTMLExtractor, MarkdownifyConverter

# Use default settings
converter = MarkdownifyConverter()

# Or customize: only strip <script> and <style>, disable id/class filtering
converter = MarkdownifyConverter(
    ignore_tags=["script", "style"],
    ignore_ids=[],
    ignore_classes=[],
)

extractor = HTMLExtractor(converter=converter)
document = await extractor.extract(file)

MarkdownifyConverter options:

Default ignored tags: head, header, script, style, title, footer, form, button, nav, iframe.

Default ignored IDs: footer, sidebar, cookie, metadata.

Default ignored class patterns: footer, ^ad-, ^ad_, ^menu$, ^newsletter$, ^metadata$, ^muted$, vot(e|ing).

Custom converter:

You can inject any object that implements the HtmlToMarkdownConverter protocol:

from mistralai.search.toolkit.ingestion.extractors import HTMLExtractor, HtmlToMarkdownConverter

class MyConverter(HtmlToMarkdownConverter):
    def convert(self, html: str) -> str:
        return html  # your conversion logic

extractor = HTMLExtractor(converter=MyConverter())
document = await extractor.extract(file)

from mistralai.search.toolkit.ingestion.extractors import HTMLExtractor, HtmlToMarkdownConverter

class MyConverter(HtmlToMarkdownConverter):
    def convert(self, html: str) -> str:
        return html  # your conversion logic

extractor = HTMLExtractor(converter=MyConverter())
document = await extractor.extract(file)

Parameters:

Features:

• Converts HTML to clean markdown with customizable stripping
• Preserves structure (headings, lists, tables)
• Boilerplate removal (navigation, footers, and ads)
• Custom converter support for alternative HTML-to-markdown implementations

SpreadsheetExtractor reads spreadsheet documents (XLSX, XLS, ODS) and yields one CSV block for each sheet.

Installation:

uv add "mistralai-search-toolkit[extractor-spreadsheet]"

uv add "mistralai-search-toolkit[extractor-spreadsheet]"

Example:

from mistralai.search.toolkit.ingestion.extractors import SpreadsheetExtractor

extractor = SpreadsheetExtractor(
    include_sheet_name=True,  # Include sheet name in output
    row_limit=None,           # Optional row limit per sheet
    col_limit=None,           # Optional column limit per sheet
)
document = await extractor.extract(file)

from mistralai.search.toolkit.ingestion.extractors import SpreadsheetExtractor

extractor = SpreadsheetExtractor(
    include_sheet_name=True,  # Include sheet name in output
    row_limit=None,           # Optional row limit per sheet
    col_limit=None,           # Optional column limit per sheet
)
document = await extractor.extract(file)

Configuration options:

Behavior:

• Each sheet becomes a separate CSV block in the document
• Merged cells keep the value in the first cell only; empty cells are rendered as nan
• Formula cells yield their cached values
• Headers are inferred from first row

Extract email files (.eml and .msg) into a single markdown-backed document with subject, sender, recipients, date, and body. HTML bodies are converted to markdown when available.

Installation:

uv add "mistralai-search-toolkit[extractor-email]"

uv add "mistralai-search-toolkit[extractor-email]"

This extra adds eml-parser, extract-msg, and markdownify (for HTML bodies).

Example:

from mistralai.search.toolkit.ingestion.extractors import EmailExtractor

extractor = EmailExtractor()
document = await extractor.extract(file)

from mistralai.search.toolkit.ingestion.extractors import EmailExtractor

extractor = EmailExtractor()
document = await extractor.extract(file)

Features:

• Parses EML (RFC 822) and Outlook MSG formats
• Outputs structured markdown (subject, from, to, CC, date, body)
• Prefers plain-text body; falls back to HTML converted to markdown

Attachments: Use extract_email_attachments (or extract_eml_attachments / extract_msg_attachments) to process attachments from the same email file. They return a list of EmailAttachment (filename, content_type, data, extension) that you can pass to other extractors:

from mistralai.search.toolkit.ingestion.extractors import (
    EmailExtractor,
    EmailAttachment,
    extract_email_attachments,
)

document = await EmailExtractor().extract(file)
attachments = extract_email_attachments(file.raw, extension="eml")
for att in attachments:
    # Build a File from att.data and run through the right extractor
    ...

from mistralai.search.toolkit.ingestion.extractors import (
    EmailExtractor,
    EmailAttachment,
    extract_email_attachments,
)

document = await EmailExtractor().extract(file)
attachments = extract_email_attachments(file.raw, extension="eml")
for att in attachments:
    # Build a File from att.data and run through the right extractor
    ...

NumbersExtractor reads Apple Numbers documents (.numbers) and yields one CSV block for each table or sheet.

Installation:

uv add "mistralai-search-toolkit[extractor-spreadsheet]"

uv add "mistralai-search-toolkit[extractor-spreadsheet]"

Example:

from mistralai.search.toolkit.ingestion.extractors import NumbersExtractor, NumbersOptions

extractor = NumbersExtractor(
    row_limit=1000,           # Cap rows per table
    col_limit=50,             # Cap columns per table
    include_sheet_name=True,  # Prefix each block with sheet/table name
)
document = await extractor.extract(file)

from mistralai.search.toolkit.ingestion.extractors import NumbersExtractor, NumbersOptions

extractor = NumbersExtractor(
    row_limit=1000,           # Cap rows per table
    col_limit=50,             # Cap columns per table
    include_sheet_name=True,  # Prefix each block with sheet/table name
)
document = await extractor.extract(file)

To iterate over CSV strings without building a full Document, use extractor.iter_csv_pages(blob, options=NumbersOptions(...)).

Extract legacy Office formats (.doc, .ppt, .hwp, .hwpx) by converting them to PDF with PyMuPDF Pro, then running Mistral OCR. This extractor doesn't handle .xls. Use SpreadsheetExtractor for that.

Installation:

uv add "mistralai-search-toolkit[extractor-pymupdf]"

uv add "mistralai-search-toolkit[extractor-pymupdf]"

This extractor requires a PyMuPDF Pro license key, passed as pymupdf_license_key or set in PYMUPDF_PRO_LICENSE_KEY. Without it, the constructor raises an error.

Example:

from mistralai.client import Mistral
from mistralai.search.toolkit.ingestion.extractors import LegacyOfficeExtractor, MistralOCRExtractor

ocr_extractor = MistralOCRExtractor(
    client=Mistral(api_key="your-api-key"),
    include_image_base64=True,
    include_image_annotation=True,
)
extractor = LegacyOfficeExtractor(
    ocr_extractor,
    pymupdf_license_key="your-license-key",  # Required unless PYMUPDF_PRO_LICENSE_KEY is set
)
document = await extractor.extract(file)

from mistralai.client import Mistral
from mistralai.search.toolkit.ingestion.extractors import LegacyOfficeExtractor, MistralOCRExtractor

ocr_extractor = MistralOCRExtractor(
    client=Mistral(api_key="your-api-key"),
    include_image_base64=True,
    include_image_annotation=True,
)
extractor = LegacyOfficeExtractor(
    ocr_extractor,
    pymupdf_license_key="your-license-key",  # Required unless PYMUPDF_PRO_LICENSE_KEY is set
)
document = await extractor.extract(file)

Features:

• Converts legacy document to PDF in memory, then delegates to MistralOCRExtractor
• OCR options (for example, http_headers and image_limit) are set on the OCR extractor you pass in. Any DocumentExtractor works as the delegate.

Implement the DocumentExtractor protocol:

import base64
from mistralai.search.toolkit.context import IngestContext
from mistralai.search.toolkit.ingestion.extractors import DocumentExtractor
from mistralai.search.toolkit.document import (
    Document,
    DocumentChunk,
    DocumentFileMetadata,
    compute_char_locator,
    compute_id,
)
from mistralai.search.toolkit.ingestion import File

class Base64FileExtractor(DocumentExtractor):
    """Extract content from base64-encoded text files."""

    async def extract(self, file: File, context: IngestContext = IngestContext()) -> Document:
        decoded_content = base64.b64decode(file.raw).decode("utf-8")
        source_id = file.path or file.name

        return Document(
            source_id=source_id,
            content=decoded_content,
            chunks=[
                DocumentChunk(
                    source_id=source_id,
                    locator=compute_char_locator(0, len(decoded_content)),
                    start_offset=0,
                    end_offset=len(decoded_content),
                    parent_ref=compute_id(source_id),
                    content=decoded_content,
                )
            ],
            # File-backed documents use DocumentFileMetadata; extra keys (such as "encoding") are allowed.
            metadata=DocumentFileMetadata(
                extractor_type="text",
                filename=file.name,
                filepath=file.path,
                encoding="base64",
            ),
        )

extractor = Base64FileExtractor()
document = await extractor.extract(b64_file)

import base64
from mistralai.search.toolkit.context import IngestContext
from mistralai.search.toolkit.ingestion.extractors import DocumentExtractor
from mistralai.search.toolkit.document import (
    Document,
    DocumentChunk,
    DocumentFileMetadata,
    compute_char_locator,
    compute_id,
)
from mistralai.search.toolkit.ingestion import File

class Base64FileExtractor(DocumentExtractor):
    """Extract content from base64-encoded text files."""

    async def extract(self, file: File, context: IngestContext = IngestContext()) -> Document:
        decoded_content = base64.b64decode(file.raw).decode("utf-8")
        source_id = file.path or file.name

        return Document(
            source_id=source_id,
            content=decoded_content,
            chunks=[
                DocumentChunk(
                    source_id=source_id,
                    locator=compute_char_locator(0, len(decoded_content)),
                    start_offset=0,
                    end_offset=len(decoded_content),
                    parent_ref=compute_id(source_id),
                    content=decoded_content,
                )
            ],
            # File-backed documents use DocumentFileMetadata; extra keys (such as "encoding") are allowed.
            metadata=DocumentFileMetadata(
                extractor_type="text",
                filename=file.name,
                filepath=file.path,
                encoding="base64",
            ),
        )

extractor = Base64FileExtractor()
document = await extractor.extract(b64_file)