September 19, 2024: Microsofts GRIN-MoE AI: Efficient, Powerful Coding & Math - Microsofts GRIN-MoE (Gradient-Informed Mixture-of-Experts) AI model excels in coding and math, outperforming competitors in key benchmarks like MMLU and HumanEval. The models sparse computation via SparseMixer-v2 enables high efficiency, activating only 6.6 billion of its 163.8 billion parameters during inference. GRIN-MoEs scalability and performance make it ideal for enterprises with limited computational resources but come with limitations in multilingual and conversational tasks.
