Skip to main content

Classifier Factory

In various domains and enterprises, classification models play a crucial role in enhancing efficiency, improving user experience, and ensuring compliance. These models serve diverse purposes, including but not limited to:

  • Moderation: Classification models are essential for moderating services and classifying unwanted content. For instance, our moderation service helps in identifying and filtering inappropriate or harmful content in real-time, ensuring a safe and respectful environment for users.
  • Intent Detection: These models help in understanding user intent and behavior. By analyzing user interactions, they can predict the user's next actions or needs, enabling personalized recommendations and improved customer support.
  • Sentiment Analysis: Emotion and sentiment detection models analyze text data to determine the emotional tone behind words. This is particularly useful in social media monitoring, customer feedback analysis, and market research, where understanding public sentiment can drive strategic decisions.
  • Data Clustering: Classification models can group similar data points together, aiding in data organization and pattern recognition. This is beneficial in market segmentation, where businesses can identify distinct customer groups for targeted marketing campaigns.
  • Fraud Detection: In the financial sector, classification models help in identifying fraudulent transactions by analyzing patterns and anomalies in transaction data. This ensures the security and integrity of financial systems.
  • Spam Filtering: Email services use classification models to filter out spam emails, ensuring that users receive only relevant and safe communications.
  • Recommendation Systems: Classification models power recommendation engines by categorizing user preferences and suggesting relevant products, movies, or content based on past behavior and preferences.

By leveraging classification models, organizations can make data-driven decisions, improve operational efficiency, and deliver better products and services to their customers.

For this reason, we designed a friendly and easy way to make your own classifiers. Leveraging our small but highly efficient models and training methods, the Classifier Factory is both available directly in la plateforme and our API.

Dataset Format​

Data must be stored in JSON Lines (.jsonl) files, which allow storing multiple JSON objects, each on a new line.

We provide two endpoints:

  • v1/classifications: To classify raw text.
  • v1/chat/classifications: To classify chats and multi-turn interactions.

There are 2 main kinds of classification models:

  • Single Target
  • Multi-Target

1. Single Target​

For single label classification, data must have the label name and the value for that corresponding label. Example:

{
    "text": "I love this product!",
    "labels": {
        "sentiment": "positive" // positive/neutral/negative
    }
}

For multiple labels, you can provide a list.

{
    "text": "I love this product!",
    "labels": {
        "sentiment": ["positive","neutral"]
    }
}

When using the result model, you will be able to retrieve the scores for the corresponding label and value.

Note that the files must be in JSONL format, meaning every JSON object must be flattened into a single line, and each JSON object is on a new line.

Raw .jsonl file example.
{"text": "I love this product!", "labels": {"sentiment": "positive"}}
{"text": "The game was amazing.", "labels": {"sentiment": "positive"}}
{"text": "The new policy is controversial.", "labels": {"sentiment": "neutral"}}
{"text": "I don't like the new design.", "labels": {"sentiment": "negative"}}
{"text": "The team won the championship.", "labels": {"sentiment": "positive"}}
{"text": "The economy is in a bad shape.", "labels": {"sentiment": "negative"}}
...
  • Label data must be a dictionary with the label name as the key and the label value as the value.

2. Multi-Target​

You can also have multiple targets and not only a single one. This is useful if you want to classify different aspects of the same content independently. Example:

{
    "text": "I love this product!",
    "labels": {
        "sentiment": "positive", // positive/neutral/negative
        "is-english": "yes" // yes/no, boolean
    }
}
  • Each target is independent of each other, meaning the scores for each label will also be independent.

Upload a file​

Once you have the data file with the right format, you can upload the data file to the Mistral Client, making them available for use in fine-tuning jobs.

from mistralai import Mistral
import os

api_key = os.environ["MISTRAL_API_KEY"]

client = Mistral(api_key=api_key)

training_data = client.files.upload(
    file={
        "file_name": "training_file.jsonl",
        "content": open("training_file.jsonl", "rb"),
    }
)  

validation_data = client.files.upload(
    file={
        "file_name": "validation_file.jsonl",
        "content": open("validation_file.jsonl", "rb"),
    }
)

Create a fine-tuning job​

The next step is to create a fine-tuning job.

  • model: the specific model you would like to fine-tune. The choice is ministral-3b-latest.
  • training_files: a collection of training file IDs, which can consist of a single file or multiple files.
  • validation_files: a collection of validation file IDs, which can consist of a single file or multiple files.
  • hyperparameters: two adjustable hyperparameters, "training_steps" and "learning_rate", that users can modify.
  • auto_start:
    • auto_start=True: Your job will be launched immediately after validation.
    • auto_start=False (default): You can manually start the training after validation by sending a POST request to /fine_tuning/jobs/<uuid>/start.
  • integrations: external integrations we support such as Weights and Biases for metrics tracking during training.
# create a fine-tuning job
created_jobs = client.fine_tuning.jobs.create(
    model="ministral-3b-latest", 
    job_type="classifier",
    training_files=[{"file_id": training_data.id, "weight": 1}],
    validation_files=[validation_data.id], 
    hyperparameters={
        "training_steps": 10,
        "learning_rate":0.0001
    },
    auto_start=False,
#   integrations=[
#       {
#           "project": "finetuning",
#           "api_key": "WANDB_KEY",
#       }
#   ]    
)

After creating a fine-tuning job, you can check the job status using client.fine_tuning.jobs.get(job_id = created_jobs.id).

Initially, the job status will be "QUEUED". After a brief period, the status will update to "VALIDATED". At this point, you can proceed to start the fine-tuning job:

# start a fine-tuning job
client.fine_tuning.jobs.start(job_id = created_jobs.id)

created_jobs

List/retrieve/cancel jobs​

You can also list jobs, retrieve a job, or cancel a job.

You can filter and view a list of jobs using various parameters such as page, page_size, model, created_after, created_by_me, status, wandb_project, wandb_name, and suffix. Check out our API specs for details.

# List jobs
jobs = client.fine_tuning.jobs.list()
print(jobs)

# Retrieve a jobs
retrieved_jobs = client.fine_tuning.jobs.get(job_id = created_jobs.id)
print(retrieved_jobs)

# Cancel a jobs
canceled_jobs = client.fine_tuning.jobs.cancel(job_id = created_jobs.id)
print(canceled_jobs)

Use a fine-tuned model​

When a fine-tuned job is finished, you will be able to see the fine-tuned model name via retrieved_jobs.fine_tuned_model.

classifier_response = client.classifiers.classify(
    model=retrieved_job.fine_tuned_model,
    inputs=["It's nice", "It's terrible", "Why not"],
)

Use classify_chat to classify chats and multiturn interactions.

Delete a fine-tuned model​

client.models.delete(model_id=retrieved_job.fine_tuned_model)

Cookbooks​

Explore our guides and cookbooks leveraging the Classifier Factory:

  • Intent Classification: Creating a single-target, single-label, intent classification model to predict user actions and improve customer interactions.
  • Moderation Classifier: Build a single-target, multi-label, simple moderation model to label public comments.
  • Product Classification: Create a multi-target, single-label and multi-label, food classification model to categorize dishes and their country of origin and compare to classic LLM solutions, enhancing recipe recommendations and dietary planning.

FAQ​

Q: Which models can we fine-tune to create our own classifiers?
A: Currently, the classifier factory utilizes ministral-3b.

Q: Where can I find the pricing?
A: You can find it on our pricing page in the Mistral Cloud section.