Modders unlock hidden AI power on Nvidia RTX 20-, 30-, and 40-series GPUs, nearly doubling some benchmarks
Serving tech enthusiasts for over 25 years.
TechSpot means tech analysis and advice you can trust.
Sounding off: A used RTX 3090 has become the go-to card for running AI models at home, mostly because 24GB of memory costs far less on a gaming card than on a workstation one. That bargain comes with a catch Nvidia prints in its own spec sheets, though. On GeForce cards, one of the most common kinds of AI math runs at exactly half speed. A few open-source projects now claim to remove that limitation on RTX 20-, 30-, and 40-series cards.
The limit lives in the tensor cores. When they multiply FP16 or BF16 numbers and add up the results in higher-precision FP32, GeForce cards issue those instructions at half the rate of the FP16-accumulate mode. That FP32 mode is the one most training and plenty of inference actually rely on. The...
Copyright of this story solely belongs to www.techspot.com. To see the full text click HERE
