Generative AI has a plethora of well-documented abuses, from the creation of scientific papers to Copy artists. And now it appears to be showing up in state influence operations.
A recent campaign was “very likely” supported by business AI speech generation products, including publicly released technology from hot startup ElevenLabs, based on a Current report from Massachusetts-based threat intelligence company Recorded Future.
The report describes a Russian-directed campaign called “Operation Undercut” aimed toward undermining Europe’s support for Ukraine, primarily using AI-generated voice-overs for fake or misleading “news” videos.
The videos, which were aimed toward a European audience, attacked, amongst other things, Ukrainian politicians as corrupt or questioned the advantages of military aid to Ukraine. For example, one video stated that “not even jammers can save American Abrams tanks,” referring to devices that US tanks use to fend off incoming missiles – reinforcing the purpose that there isn’t a point in using high-tech Send tanks to Ukraine.
The report said the video creators “very likely” used voice-generated AI, including ElevenLabs' technology, to make their content appear more legitimate. To confirm this, Recorded Future researchers submitted the clips to ElevenLabs AI speech classifier, This allows anyone to “tell if an audio clip was created with ElevenLabs” and get a match.
ElevenLabs didn’t reply to requests for comment. Although Recorded Future pointed to the likely use of several business AI speech generation tools, no others were mentioned aside from ElevenLabs.
The utility of AI voice generation was inadvertently demonstrated by the influencer campaign organizers themselves, who released – fairly sloppily – some videos with real human speakers who had “a recognizable Russian accent.” In contrast, the AI-generated voiceovers spoke in several European languages reminiscent of English, French, German and Polish, with none foreign-sounding accents.
According to Recorded Future, the AI also enabled the misleading clips to be published quickly in several languages spoken in Europe, reminiscent of English, German, French, Polish and Turkish (all languages, by the best way). supports from ElevenLabs.)
Recorded Future traced the activity to the Social Design Agency, a US government organization based in Russia sanctioned in March of this yr for “operating a network of over 60 web sites posing as real news organizations in Europe after which using fake social media accounts to amplify the fake web sites' misleading content.” All of this was ” on behalf of the Government of the Russian Federation,” the US State Department said on the time.
The overall impact of the campaign on public opinion in Europe was minimal, Recorded Future concluded.
This isn’t the primary time that ElevenLabs' products have been called to account for alleged misuse. The company's technology was behind a robocall that impersonated President Joe Biden and urged voters to not vote during a January 2024 primary, a voice fraud detection company concluded: based on Bloomberg. In response, ElevenLabs announced that it had introduced latest security measures reminiscent of robotically blocking votes from politicians.
ElfLabs Bans “unauthorized, harmful or misleading impersonation” and says To implement this, various tools are used, reminiscent of each automated and human moderation.
ElevenLabs has experienced explosive growth since its founding in 2022. ARR recently rose to $80 million from $25 million last yr and will soon be value $3 billion, TechCrunch previously reported. Its investors include Andreessen Horowitz and former Github CEO Nat Friedman.
