‘Those two jobs need different physics’: Rebellions CEO says training and inference need different chips
The AI race started off with a pretty clear direction – bigger and better. The first waves were characterized by building bigger models, but it’s all change in the world of artificial intelligence and with enterprises, SMBs and consumers all finding use cases for the technology, the focus has shifted.
Now, AI firms and model developers are looking to realize a much tougher goal. Efficiency. Cost per token, performance per watt, output per input, it’s all about driving maximum efficiency.
One clear divide is between training and inference. While training models still requires huge amounts of resources, inference efficiency is starting to improve, and one company (Rebellions) now believes an opening for inference-first hardware could create a new market.
The company’s racks are said to consume around 16-20kW, compared with around 120kW for leading GPU-based inference systems that, for many use cases, are sheer overkill.
Rebellions’ rack costs are also...
Copyright of this story solely belongs to techradar.com. To see the full text click HERE
