OVHcloud introduces 7B AI model designed to run on lower hardware requirements

OVHcloud has introduced Olala mini 7A1B, a 7-billion-parameter large language model (LLM) designed to reduce computing and memory requirements by activating only a portion of its parameters for each request.

The model uses a hybrid architecture combining Differential-TPA and Mamba-3 attention mechanisms, alongside a mixture-of-experts (MoE) design. According to OVHcloud, Olala mini has one billion active parameters distributed across 256 experts per layer, allowing it to process requests without activating the entire model.

The company claims the model can run on a GPU with as little as 8GB of memory, delivering 160 tokens per second on an NVIDIA RTX A4500. It can also run on a standard CPU, with OVHcloud reporting speeds of up to 50 tokens per second. The model currently supports a context window of 32,000 tokens.

The model is scheduled to be released on Hugging Face under the Apache 2.0 licence, with its weights, pre-trained base model,...

Copyright of this story solely belongs to www.expresscomputer.in. To see the full text click HERE

Read more