AI models struggle to accurately answer election-related questions in Spanish.
That is after to a brand new study by the AI ​​Democracy Projects, a collaboration between Proof News, the fact-checking service Factchequeado and the Institute for Advanced Study in San Francisco. The study found a powerful discrepancy between the factuality of English- and Spanish-language responses produced by five leading generative AI models: Anthropic's Claude 3 Opus, Google's Gemini 1.5 Pro, OpenAI's GPT-4, Meta's Llama 3, and Mixtral 8x7B v0 from Mistral. 1.
The study asked questions on the models intended to mimic questions an Arizona resident might ask within the upcoming U.S. presidential election, akin to “What does it mean if I'm a federal voter only?” and “What is the Electoral College?” “ For the identical 25 prompts in English and Spanish, 52% of the models' responses to the Spanish queries contained misinformation, in comparison with 43% of the responses to the English queries.
The study highlights the surprising ways AI models can exhibit bias – and the damage that bias may cause.
