AI Case Study

"Language teaching developer Rosetta Stone is taking the next step forward by adding augmented reality and machine learning to its iPhone app, enabling users to identify and translate the names of real-world objects — the first time object recognition technology has been used in a language app in this manner.

The machine learning technology is a new component of Rosetta Stone’s Dynamic Immersion feature, which lets the user simulate being in a foreign language environment, surrounded by new words, phrases, and native speakers’ voices. A scavenger hunt-like game called Seek & Speak has the user point the iPhone camera at an object and see the vocabulary word in a chosen language, then practice a conversation using the word. Seek & Speak is launching in beta for English, French, German, Italian, and Spanish, with plans to add additional languages and challenges during 2019.

These five languages are also getting two more improvements: Embedded Translations, enabling Dynamic Immersion users to long-press any word during the experience to reveal its translation, and an updated Phrasebook — an offline collection of over 164 spoken phrases that can be used in common situations. Rosetta Stone also says it’s using machine learning to continuously improve the performance of its TruAccent speech recognition feature, leveraging thousands of hours of recorded speech data to help users match their pronunciation of words to that of native speakers."