Challenges of AI Language Models in Understanding Vietnamese

Challenges of AI Language Models in Non-English Languages

Researchers at Stanford University recently explored the capabilities of a popular A.I. chatbot, Claude 3.5, developed by the artificial intelligence company Anthropic. The study involved testing the bot’s proficiency in Vietnamese, specifically asking it to compose a traditional poem in the “song thất lục bát” format. This poetic structure requires a specific pattern of lines containing seven, seven, six, and then eight words. However, when the bot produced an output, it failed to adhere to the required format, illustrating a significant shortcoming in its understanding of this cultural context.

In another instance, the team inquired about the proper Vietnamese term for a mother’s younger brother. Unfortunately, the bot incorrectly provided terms for a father’s younger and older siblings instead. These errors are not isolated to Claude 3.5; they highlight a broader issue where many A.I. systems struggle with languages outside of standard American English.

While the use of artificial intelligence has surged in Western countries, many regions around the globe remain underrepresented in this technological dialogue. A.I. experts express concern that this language disparity could deepen existing technological inequities, potentially leaving numerous cultures and communities behind.

Sang Truong, a Ph.D. candidate at Stanford’s Artificial Intelligence Laboratory and part of the research team, emphasized the stakes involved. He noted that even a brief delay in access to robust technology could result in significant economic setbacks, stating that “a few years of lag can lead to a few decades of economic delay.”

The tests conducted by Truong’s team revealed that A.I. tools, in general, struggle to deliver accurate facts and appropriate language when dealing with Vietnamese, which is categorized as a “low-resource” language by industry standards. This classification indicates a lack of sufficient datasets and online content for A.I. models to learn effectively.