Meta has announced an AI-powered speech-to-speech translation system for unwritten languages.
As part of Meta’s Universal Speech Translator (UST) project, the company has built a translation system for Hokkien, which is a primarily oral language of the Chinese diaspora but lacks a standard written form, the company said in a blog post.
Since oral languages do not have standard written forms, producing transcribed text as the translation output does not work. So, the company has focused on speech-to-speech translation.
The company has created a number of techniques, such as speech-to-unit translation, which converts input speech into a series of sounds and generates waveforms from them.
Another technique is to use text from a closely related language to generate waveforms.
The Hokkien translation model is still in progress and can translate only one full sentence at a time, the company said.
The company is also releasing SpeechMatrix which is a large collection of speech-to-speech translations developed through a natural language processing toolkit called LASER.
The tools will allow other researchers to create their own speech-to-speech translation systems.
“Our AI research is helping break down language barriers in both the physical world and the metaverse to encourage connection and mutual understanding,” the company said.
