Meta's New Llama 4 Models Stir Controversy

Llama 4 Scout and Maverick Face Scrutiny Over Benchmarking Practices Rashmi Ramesh (rashmiramesh_) • April 8, 2025

Meta released two new open-weight large language models that aim to scale artificial intelligence performance while lowering compute costs. Their debut drew criticism over how the company presented benchmark results, raising questions about transparency in AI model evaluation.

See Also: Fortinet Expands FortiAI Across Security Fabric Platform

Dubbed Llama 4 Scout and Llama 4 Maverick, both models are built on a mixture-of-experts architecture.

The launch comes at a time when researchers and developers are scrutinizing benchmark scores as they try to evaluate increasingly complex models. While Meta pitched Llama 4 as a major step forward, the use of non-public "experimental" versions for testing triggered debate about how representative those scores are of models available to users.

Scout and Maverick feature 17 billion active parameters but differ in how they route tasks to ...