The App Using Artificial Intelligence To Improve English Speaking Skills
Students look at books during a four-day book fair at a public park in downtown Hanoi on September 11, 2015
Vu Van had always been confident in her English speaking skills. Growing up in Nha Trang, Vietnam, she studied English in school and with a private tutor. Her schoolteachers were particularly strict about proper grammar, so by the time she was ready to move abroad after college, she had a strong grasp on the language. But when she began her MBA studies at Stanford University, she realized that her pronunciation of English words was holding her back. She struggled to communicate effectively with her professors and classmates, which negatively impacted her educational environment.
“You feel like you’re being ignored,” she said of the experience. Van quickly realized she wasn’t alone. After talking with other friends who were living and working in foreign countries, she found that they all shared a common frustration that stemmed from pronunciation hurdles. But hiring a private tutor is costly, especially for full-time students.
“That’s not really accessible for a lot of people,” Van said. She decided to find a better solution.
Van didn’t consider herself a technical person when she was studying at Stanford, but she had been following developments in machine learning and thought the technology could help solve the pronunciation problem. She also hadn’t seen much crossover between machine learning and language education up to that point.
Companies in the Bay Area “don’t pay attention to language learning [in this way] because they’re mostly American,” she said. But after surveying her friends, many of whom also grew up learning English in Vietnam, “I know that the market is there.”
In March 2016, Van debuted ELSA during SXSW, and the product won the SXSWedu launch competition. ELSA, which stands for English Language Speech Assistant, is a mobile app that uses AI to help people improve their pronunciation. Users can access different curriculum options based on their interests, such as travel or professional settings. They’re then lead through a series of vocabulary words and phrases, and ELSA’s algorithms analyze their speech. If ELSA detects a mispronunciation, it tells the user not only how the word should sound, but also how to move the tongue and lips to form the correct word.
This is essential to learning English effectively, Van said.
“When people can’t hear the difference themselves, hearing an American voice is a good step but it’s not enough unless you have an ear for languages,” Van said. That’s especially true for people whose native languages don’t include some of the most common sounds heard in English.
Van said she and the ELSA team have fed “hours and hours” of speech data into the system. They’ve also worked with speech therapists to identify key problem areas for people learning English in different parts of the world.
“It will never be 100% accurate, but the more data we have, the more accurate it will be,” she said. “If you look at Siri three to four years ago, it barely understood you. But it’s getting smarter and smarter.”
ELSA’s curriculums do not include lessons on grammar or vocabulary memorization because other apps, such as Duolingo, already do that well, Van said. She views ELSA as a tool for intermediate users to take their English skills to the next level.
An updated version of ELSA will launch in late September or early October, and it will allow users to declare their mother tongues so that the program can tailor the experience to their needs. The new version will also include an expanded curriculum.
Vietnamese users currently make up 30-35% of ELSA’s user base, Van said. Vietnam is serving as the company’s pilot market, although Van noted that it’s attracted users from Japan, Indonesia, Thailand, the Middle East and Europe.
But for now, Van’s primary focus is on helping expand English education in her native country, she said.
“I want people from home to learn English and have more opportunities in life.”