Most social users could have come across an influencer who looks a bit… off.
Maybe their facial expression are a bit too symmetrical, and their poses are a bit too rigid. Chances are, you’re not taking a look at a human in any respect – but an AI-generated forgery.
In some cases, these AI influencers are reasonably benign – just digital types of their real counterparts but not overtly attempting to deceive or manipulate.
However, this isn’t at all times the case. Disturbingly, there’s a network of Instagram accounts using artificial intelligence to create fake influencers with Down syndrome.
These bad actors steal content from real creators, then leverage AI to swap in computer-generated faces of individuals with Down syndrome. The goal? To exploit a vulnerable community for likes, shares, and ultimately, money.
But the deception doesn’t end there. Many of those accounts link out to shady adult web sites, where the AI-generated content is monetized.
View this post on Instagram
Sadly, that is just the most recent evolution of the “AI pimping” trend, where unscrupulous operators use machine learning to create counterfeit influencers for monetary gain. It’s not only Down syndrome, but fake amputee models, burn victims, and other types of AI-generated pornography.
AI image and video models at the moment are approaching a level of realism that makes them very viable substitutes for real humans. It’s affecting the style industry – where real models are facing substitute by the hands of the AI clones.
Even household names like H&M are wading into these murky waters. The fast fashion giant recently announced a campaign featuring AI-generated “digital twins” of real models. Back in 2023, an organization called lalaland.ai released tools for creating AI models for a subscription fee.
While H&M insists the models maintain control over their digital likenesses, many within the industry are skeptical. After all, in an era of cost-cutting and consolidation, why hire human talent when you possibly can license an inexpensive, infinitely replicable digital avatar?
The latest, most insidious twist here concerns the basic dignity and humanity of marginalized communities.
People with Down syndrome – or any disability – are usually not props to be manipulated for profit.
Moreover, the proliferation of AI-generated content threatens to erode public trust in media altogether. If we cannot trust the pictures we see online, the very foundation of digital discourse starts to erode.
So next time you’re scrolling through your feed and an influencer seems too good to be true, trust your gut.
