Babel Fish like ML model emerges after training on 4.5 million hours of multilingual spoken audio

Meta has developed a machine learning model its researchers claim offers near-instant speech-to-speech translation between around 36 languages.

Reminiscent of the Babel Fish from The Hitchhiker’s Guide to the Galaxy, the foundation model SEAMLESSM4T was trained on 4.5 million hours' of recorded human speech and takes a "savvy" approach that avoids onerous data annotation by exploiting snippets of internet audio.

Presenting the paper in the journal Nature today, the team from the Facebook parent company said that a relatively open model — on which other applications could be built — could support on-demand "streamlining multilingual exchange across various contexts."

In an accompanying article, Tanel Alumäe, professor of speech processing at Estonia's Tallinn University of Technology, said the model was pre-trained on a massive data set containing 4.5 million hours' worth of multilingual spoken audio to help establish patterns in the data, "making it easier to fine-tune the model ...