April 17, 2024Blog
v0.1-0.3
Mixtral 8x22B
A large open MoE model competing with much larger dense models.
Modalities
Context
64k
Retirement date3/30/2025
ReplacementMistral Small 3.2
Weights
| Weights | License | Parameters (B) | Active (B) | ≈ GPU RAM at bf16 - fp4 (GB) | Context Size (tokens) |
|---|---|---|---|---|---|
| 141 | 39 | 283 - 71 | 64k | ||
| 141 | 39 | 283 - 71 | 64k |
Other Models
