📄️ Interacting with Mistral 7B
Once you have deployed an the model with vLLM on a GPU instance, you can query it using the OpenAI-compatible REST API. This API is described on the API specification, but you can use any library implementing OpenAI API.
📄️ Guardrailing Mistral 7B
The ability to enforce guardrails when it comes to AI generation is crucial for front-facing applications. In this tutorial, we explain how to leverage system prompting to optionally enforce output constraints on top of our models. Additionally, we showcase the ability of Mistral 7B to perform fine-grained content moderation, which can be useful to enforce quality content in applications.