Nvidia wants to stop AI costs skyrocketing with its new software router — but will it really make a difference?
- Nvidia's open source NeMo Switchyard 'smartly' routes each agent request to the cheapest model that can handle it
- The approach allows it to claim a 74% cost cut against a frontier-only model in return for a 6% reduction in accuracy
- NeMo Switchyard joins an increasingly well-established territory that already sees players such as RouteLLM, LiteLLM, and OpenRouter, in addition to in-house efforts at OpenAI and AT&T to save costs
Nvidia has released NeMo Switchyard, an open source model router that sits between an application and a pool of language models and decides, per request or per turn, which model should handle a particular task.
The aim is to maximize efficiency by picking the right model for the right task: pushing frontier-level models to do simple tasks that much smaller or cheaper models could handle is not only inefficient, but also costly for enterprise customers.
This comes at a time when...
Copyright of this story solely belongs to techradar.com. To see the full text click HERE
