Stay up up to now with free updates
Simply register for Artificial intelligence myFT Digest – delivered straight to your inbox.
ChatGPT users within the UK were baffled to seek out the chatbot responding to English-language queries in Welsh, the newest example of how the race to develop artificial intelligence systems can result in unexpected errors.
When English users check with the bot through ChatGPT's recent voice interface, some have been surprised to seek out it translates their query into near-perfect Welsh after which responds within the Celtic language.
Several users have reported the ChatGPT outage – which the Financial Times also experienced first-hand this week – despite not understanding Welsh or living in or near Wales.
This isn't the primary time that users of OpenAI's groundbreaking chatbot have encountered language issues. In February, users also complained a few bug that caused the bot to reply text questions in a mix of Spanish and English.
The Welsh speech impediment puts a brand new spin on the widespread problem of huge language models “hallucinating”—that’s, making up fictitious or nonsensical answers—that continues to plague generative AI systems even after years of development and billions of dollars of investment.
Microsoft-backed OpenAI, which was valued at $86 billion in an worker stock sale earlier this 12 months, is competing with Google and Meta, in addition to startups reminiscent of Anthropic, Elon Musk's xAI and Cohere, to further develop its AI capabilities.
ChatGPT now supports dozens of languages, including Icelandic, Georgian and Macedonian. The Welsh Government announced a knowledge partnership with OpenAI in June to enhance how AI technologies work in language.
OpenAI has admitted in a research paper that ChatGPT offers “significantly worse performance than expected” in Welsh after the corporate discovered that almost all of its training data for translation was “actually English audio” that had been misidentified by the system.
Sarah Coward, an entrepreneur from Cambridgeshire, said she encountered the Welsh bug while trying out the brand new language feature in ChatGPT-4o, which was introduced earlier this 12 months.
“I had no idea what language it was since it took me completely by surprise,” she said. When Coward asked the chatbot why it began speaking Welsh, ChatGPT replied that it thought Coward would “feel more comfortable in that language.”
OpenAI said the issue was as a result of a limitation of ChatGPT's Whisper speech transcription system.
The company told the Financial Times that the model sometimes gets confused and transcribes audio in a distinct language – on this case, Welsh. It beneficial that users experiencing the Welsh error change their “Language” setting to English as an alternative of “Auto-detect,” although there was no guarantee that this is able to fix the issue.
“Everyone knows that ChatGPT and a few large language modeling applications produce hallucinations or inaccuracies in responses,” said Coward, whose company In The Room provides conversational AI experiences to brands.
“This is evidence that corporations must have some legitimate concerns about using these sorts of technologies in any customer-facing area right away,” she added. “It may very well be quite damaging when it comes to customer experience and… trust.”
In an OpenAI article about its own speech recognition system, the corporate stated: “Welsh is an outlier with much worse performance than expected… despite supposedly having 9,000 hours of translation data.”
During the review, the corporate found that “the vast majority of the purportedly Welsh translation data is definitely English audio files” that were “incorrectly classified as Welsh by the speech recognition (AI) system.”
When the Financial Times discovered the error, ChatGPT interpreted a question about cities within the UK, US and Asia as a question about Wales.
“It seems I misunderstood the wording of your query and unintentionally replied in Welsh,” ChatGPT said when asked about his mistake. “I can be more careful in the longer term.”
