Using Jev in Production

At work I have been playing with Jev by TypeSafe AI a lot, and the reason it got popular is pretty straightforward: it opened up a class of possibilities where you get an intelligent, LLM-like model that doesn't take forever, so you can use it as a quick classifier for actions -- typed choices, scores, or yes/no probabilities that your software can act on directly, instead of waiting on a long autoregressive reply. That speed is useful, and it also changes how features fail if you ship the score the wrong way.

Speed x quality

LLMs are slow, but they can reason in text. Jev is fast, and what you get back is uncertainty as numbers -- in spirit that sits closer to cross-encodersand other classifiers that read text and emit probabilities over labels, where the model understands the input and then gives you a distribution you can threshold....

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE