Mixtral 8x7B at a glance
Mixtral 8x7B used 12.9B of its 46.7B parameters per token, matched or beat GPT-3.5 on most standard benchmarks, and shipped under Apache 2.0 with a 32K context.
What changed with Mixtral 8x7B
Mistral AI released Mixtral 8x7B on December 11, 2023. It was a sparse mixture-of-experts model: at each layer a router sends each token to two of eight expert groups, so only 12.9B of its 46.7B total parameters are used per token. It handled a 32K-token context and English, French, Italian, German and Spanish.
Mistral said Mixtral beat Llama 2 70B on most benchmarks with about six times faster inference, and matched or outperformed GPT-3.5 on most standard tests. An instruction-tuned version scored 8.3 on MT-Bench. Weights were released under Apache 2.0.
Mixtral 8x7B in numbers
Values are copied from the linked source on the date shown. They are not IA Compa measurements.
- Context windowas of · Source for Context window: mistral.ai (opens in a new tab)
- 32K tokens
How it compares
Other models in the same category that we have covered, using each model's latest recorded value. (Models)
Where to compare Mixtral 8x7B
Independent leaderboards and official resources for comparing this model yourself. Rankings change often; each site shows its own current data.
General leaderboards
Why Mixtral 8x7B matters for businesses
For businesses, Mixtral was the first open model that could replace GPT-3.5 in many workflows, with good Spanish and other European languages, while running cheaper than its size suggested. It also made mixture-of-experts the standard design for efficient models.
Sources for this story
- Mixtral of experts (opens in a new tab)Official
Mistral AI · mistral.ai
- Mixtral-8x7B-v0.1 model card (opens in a new tab)Official
Mistral AI on Hugging Face · huggingface.co
- Mixtral 8x7B model page (opens in a new tab)Benchmark
Artificial Analysis · artificialanalysis.ai
Last verified: . IA Compa summarizes public announcements for business readers. We are not affiliated with the companies mentioned. Always confirm details, pricing, and availability with the official source.
- #llm
- #mistral
- #open-weights
- #mixture-of-experts
- #apache-2-0
