When you see the mythical Ouroboros, it's perfectly logical to think, “It's not going to remain that way.” A robust symbol – swallowing your personal tail – but difficult in practice. That could be the case with AI, too, which is liable to “model collapse” after being trained for just a few rounds on self-generated data, in accordance with a brand new study.
In an article published in Nature, British and Canadian researchers led by Ilia Shumailov in Oxford show that today’s machine learning models are fundamentally liable to a syndrome they call “model collapse.” In the introduction to the article they write:
We find that randomly learning from data created by other models results in “model collapse” – a degenerative process during which models forget the true underlying data distribution over time…
How does this occur and why? The process is definitely quite easy to know.
AI models are principally pattern recognition systems: They learn patterns from their training data, match those patterns to prompts, and fill within the almost certainly next items within the line. Whether you ask, “What's snickerdoodle recipe?” or “List the U.S. presidents so as of age at inauguration,” the model principally just returns the almost certainly continuation of that string of words. (Image generators are different, but similar in some ways.)
But here's the thing: models are inclined to go for probably the most common result. You won't get a controversial snickerdoodle recipe, but the most well-liked and customary one. And if you happen to ask a picture generator to generate a picture of a dog, it won't offer you a rare breed that it's only seen two images of in its training data; you'll probably get a golden retriever or a Labrador.
Now mix those two things with the incontrovertible fact that the web is flooded with AI-generated content and that latest AI models will likely ingest that content and train on it. That means they're going to see loads of gold!
And once they're trained to this proliferation of golden retrievers (or mediocre blog spam, fake faces, or generated songs), that's their latest ground truth. They'll think that 90% of dogs are really golden retrievers, and so when asked to generate a dog, they'll increase the proportion of golden retrievers even further – until they've principally lost track of what dogs even are.
This wonderful illustration from the accompanying Nature commentary article illustrates the method visually:
Something similar happens with language models and others that principally prefer probably the most frequent data of their training set for answers – which, to be clear, will likely be the precise thing to do. It's not likely an issue until it hits the ocean of bait that’s currently the general public web.
If the models proceed to steal data from one another, perhaps without knowing it, they may essentially get weirder and dumber until they break. The researchers provide quite a few examples and remedies, but go thus far as to call the breakdown of the models “inevitable,” a minimum of in theory.
While it might not prove the best way the experiments show, this possibility should scare everyone within the AI field. Diversity and depth of coaching data are increasingly seen as a very powerful consider the standard of a model. If data runs out but more data is generated, there’s a risk of model crashing. Does this fundamentally limit today's AI? If it does occur, how will we all know? And is there anything we will do to forestall or mitigate the issue?
At least the last query can probably be answered with “yes”, even when that mustn’t allay our concerns.
Qualitative and quantitative benchmarks for data provenance and variety would help, but we’re still removed from standardizing these. Watermarking AI-generated data would help other AIs avoid this, but thus far nobody has found an appropriate solution to mark images this manner (well… I actually have).
In fact, firms may even find it discouraging to share such information, and as a substitute hoard all of the highly worthwhile, original, human-generated data they will get their hands on to preserve what Shumailov et al. call their “first-mover advantage.”
(model collapse) have to be taken seriously if we’re to keep up the advantages of coaching with large-scale data crawled from the net. In fact, the worth of knowledge collected via real human interactions with systems is becoming increasingly worthwhile within the face of LLM-generated content in data crawled from the net.
… (I)t may turn into increasingly difficult to coach newer versions of LLMs without access to data crawled from the Internet before the mass adoption of the technology or without direct access to large-scale human-generated data.
Add it to the pile of probably catastrophic challenges for AI models – and to the arguments against today's methods for producing tomorrow's superintelligence.
