Microsoft's New Coding AI Needs More Than 120GB of RAM for Best Performance
Microsoft's MAI-Code-1.1-Flash AI model can now run locally on developers' own hardware. The model uses 3-bit quantization to save memory and has a 256,000-token context window. However, there's a catch: Microsoft recommends more than 120GB of RAM for best performance, Neowin reports.
Microsoft's quantized model takes up roughly 53GB of space and peaked at 75.5GB of memory during testing with a 256K context. But that doesn't mean a computer with 76GB of RAM will run optimally, since the OS and other software also use memory.
The company tested two versions: The local version scored 70.80% on SWE-Bench, and the full version scored 72.6%. For Terminal-Bench 2.1, the local version scored 66.29%, and the full version scored 62.9%.
Microsoft plans to bring experimental access to the GitHub Copilot app, CLI, and Visual Studio Code by the end of October. Copilot will be able to route tasks between local and cloud...
Copyright of this story solely belongs to www.extremetech.com. To see the full text click HERE
