AI pioneer Yann LeCun began a energetic discussion today after telling the following generation of developers to not work on large language models (LLMs).
“It’s within the hands of huge corporations, there’s nothing you may usher in,” said Lecun VivaTech today in Paris. “They should work on next-generation AI systems that remove the constraints of LLMs.”
Meta's senior AI scientist and NYU professor's comments quickly sparked a flood of questions and sparked a discussion about the constraints of today's LLMs.
When confronted with query marks and head shaking, LeCun clarified on X (formerly Twitter) in a way: “I work on next-generation AI systems myself, not LLMs.” So technically I'm telling you: “Compete with me” or higher said: “Work on the identical thing as me, because that's the strategy to go, and the more the higher!”
In the absence of more concrete examples offered, many X users wondered what “next-generation AI” meant and what an alternative choice to LLMs might be.
Developers, data scientists, and AI experts offered quite a lot of options for X-threads and sub-threads: boundary-driven or discriminatory AI, multitasking and multimodality, categorical deep learning, energy-based models, more targeted small language models, area of interest use cases, custom fine-tuning and training, state-space models, and hardware for embodied AI. Some also suggested exploring Kolmogorov-Arnold networks (KANs), a brand new breakthrough in neural networking.
One user listed five next-generation AI systems:
- Multimodal AI.
- Reasoning and general intelligence.
- Embodied AI and Robotics.
- Unsupervised and self-supervised learning.
- Artificial General Intelligence (AGI).
Another said that “every student should start with the fundamentals,” including:
- Statistics and probability.
- Data processing, cleansing and transformation.
- Classic pattern recognition similar to Naive Bayes, decision trees, random forest and bagging.
- Artificial neural networks.
- Convolutional neural networks.
- Recurrent neural networks.
- Generative AI.
Dissenters, nonetheless, identified that now could be a perfect time for college students and others to work on LLMs, as the applying areas are still “barely explored.” For example, there remains to be loads to learn when it comes to command prompting, jailbreaking and accessibility.
Others pointed to Meta's own productive LLM structure and said LeCun was attempting to subversively suppress the competition.
“When the pinnacle of AI at a big company says, 'Don't even attempt to compete, you may have nothing to contribute,' it makes me wish to compete,” one other user commented dryly.
LLMs won’t ever reach the intelligence of a human
Lecun, an advocate of goal-oriented AI and open source systems, also said The Financial Times this week that LLMs have a limited understanding of logic and won’t achieve human-level intelligence.
They “don’t understand the physical world, don’t have any everlasting memory, cannot reason in any reasonable definition of the term, and can’t … plan hierarchically,” he said.
Meta recently introduced its Video Joint Embedding Predictive Architecture (V-JEPA), which may detect and understand highly detailed object interactions. The architecture is what the corporate calls the “next step toward Yann LeCun’s vision of advanced machine intelligence (AMI).”
Many share LeCun's feelings in regards to the setbacks of LLMs. The X account for AI chat app Wildlife called LeCun's comments a “great take” today, as closed-loop systems have “massive limitations” in relation to flexibility. “Whoever creates an AI with a prefrontal cortex and the flexibility to soak up information through open-ended self-training will probably win a Nobel Prize,” they claimed.
Others described the industry's “apparent fixation” on LMMs, calling them “a dead end on the road to real progress.” Even more noted that LLMs are nothing greater than “connective tissue that stitches systems together quickly and efficiently,” like telephone exchanges, before handing it off to the best AI.
Bringing old rivalries to the fore
LeCun has never been one to avoid debate, in fact. Many may remember the long, heated discussions between him and his AI colleagues Geoffrey Hinton, Andrew Ng and Yoshia Bengio in regards to the existential risks of AI (LeCun thinks that is overstated).
At least one industry observer recalled this drastic conflict of opinion, pointing to a recent interview with Geoffrey Hinton wherein the British computer scientist advised going all out on LLMs. Hinton has also argued that the AI brain is very near the human brain.
“It's interesting to see the basic disagreement here,” the user commented.
A matter that probably can’t be reconciled so quickly.
