Predicted Outputs

Predicted Outputs optimizes response time by leveraging known or predictable content. This approach minimizes latency while maintaining high output quality. In tasks such as editing large texts, modifying code, or generating template-based responses, significant portions of the output are often predetermined. By predefining these expected parts with Predicted Outputs, models can allocate more computational resources to the unpredictable elements, improving overall efficiency.

Predicted Outputs shine in scenarios where you need to regenerate text documents or code files with minor modifications. The key parameter introduced is the prediction parameter, which enables users to define predicted outputs. For example, imagine you want your model to update the model used in a fine-tuning job. You can include the code snippet you'd like to modify as both the user prompt and the predicted output.

import os
from mistralai import Mistral

api_key = os.environ["MISTRAL_API_KEY"]
model = "mistral-large-2411"

client = Mistral(api_key=api_key)

code = """
created_jobs = client.fine_tuning.jobs.create(
model="open-mistral-7b",
training_files=[{"file_id": ultrachat_chunk_train.id, "weight": 1}],
validation_files=[ultrachat_chunk_eval.id],
hyperparameters={
"training_steps": 10,
"learning_rate":0.0001
},
auto_start=False
)
"""

prompt = "Change the model name from open-mistral-7b to open-mistral-nemo. Respond only with code, no explanation, no formatting."

chat_response = client.chat.complete(
  model= model,
  messages = [
    {
      "role": "user",
      "content": prompt,
    },
    {
      "role": "user",
      "content": code
    },
  ],
  prediction = {
    "type": "content",
    "content": code
  }
)

import os
from mistralai import Mistral

api_key = os.environ["MISTRAL_API_KEY"]
model = "mistral-large-2411"

client = Mistral(api_key=api_key)

code = """
created_jobs = client.fine_tuning.jobs.create(
model="open-mistral-7b",
training_files=[{"file_id": ultrachat_chunk_train.id, "weight": 1}],
validation_files=[ultrachat_chunk_eval.id],
hyperparameters={
"training_steps": 10,
"learning_rate":0.0001
},
auto_start=False
)
"""

prompt = "Change the model name from open-mistral-7b to open-mistral-nemo. Respond only with code, no explanation, no formatting."

chat_response = client.chat.complete(
  model= model,
  messages = [
    {
      "role": "user",
      "content": prompt,
    },
    {
      "role": "user",
      "content": code
    },
  ],
  prediction = {
    "type": "content",
    "content": code
  }
)