Meta has released the highly anticipated Llama 3 series, with the primary two models, Llama 3-8B and Llama 3-70B, now generally available.
Days ago, Meta execs Nick Clegg and Yann LeCun said at an event in London that “Llama 3” was imminent this month.
The first two versions were released today and are the third and fourth major open models released this month, following xAI's Grok-1.5V and Mistral's 8x22B.
Llama 3 is pre-trained on a formidable 15 trillion tokens, a seven-fold increase in comparison with Llama 2. The pre-trained data also accommodates 4 times more code.
Under the hood, Llama 3 introduces architectural improvements akin to a more efficient tokenizer with a bigger vocabulary of 128,000 tokens.
Here's a fast overview of Llama 3's performance:
Performance of Lama 3 8B:
- Outperforms models like Mistral's 7B and Google's Gemma 7B in several benchmarks.
- Excellent for MMLU, ARC, DROP, GPQA (biology, physics and chemistry questions), HumanEval (code generation), GSM-8K (math problems), MATH (math benchmark), AGIEval (problem solving) and BIG-Bench Hard ( rational pondering).
70B comparison with other models:
- Llama 3 70B is competitive with top AI models like Google's Gemini 1.5 Pro.
- Beats Gemini 1.5 Pro in MMLU, HumanEval and GSM-8K.
- Performs higher than Anthropic's Claude 3 Sonnet (the mid-tier of the Claude 3 series) in five benchmarks: MMLU, GPQA, HumanEval, GSM-8K, and MATH.
These are excellent results for an open model (although the meta license has some limitations).
It makes Llama 3 the brand new strongest open source model free of charge (so to talk).
Llama 3 will even be more nice and fewer persistent to make use of – fewer non-responses and better accuracy on quizzes, historical facts and STEM-related questions.
Llama 3 is anticipated to be generally available on all major platforms, including cloud services and API providers.
Meta is already working on expanding Llama 3 to 400 billion parameters and adding latest features akin to multimodality, multilingual support and advanced context understanding.
Meta's rogue role in generative AI
In some ways, Meta has turn out to be the rebel of the generative AI industry.
Meta-Chief AI Scientist Yann LeCun, one in every of AI's most respected figureheads, represents what some see as divergent views on the direction of AI – Views criticizing closed source projects at Meta's big tech competitors.
Meanwhile, Nick Clegg, former UK deputy prime minister and head of worldwide affairs, has been called out for some at-times laissez-faire views on Meta's AI products, which can not come as a surprise to Brits on the market.
Last week, Clegg appeared to downplay the impact of AI on electioneering and deep fake manipulation. A view that completely contradicts the prevailing narrative that deep fakes may very well be (or already are) deeply destructive.
In fact, Meta's oversight board is currently actively investigating two cases of deep fake pornography. The board felt that Meta's content moderation efforts were too slow.
Meta also expressed optimism concerning the improved quality of its models. Joelle Pineau, vp of AI research at Meta, said: “In some ways, the models we now have today can be child’s play in comparison with the models that can come to market in five years.”
Pineau also warned: “If we make our model increasingly general and powerful without properly socializing it, we could have a giant problem on our hands.”
The release of Llama 3 also comes at a time when Meta's AI Facebook Agents are causing a stir on social media.
A meta-AI assistant — designed to present advice and answer questions — has emerged in a Facebook group for fogeys in New York City shocked people by claiming that a “gifted and disabled child” attends a selected school for the “gifted and talented.”
When confronted by the group members, the AI admitted, “I'm just a giant language model, I don't have any personal experiences or children,” in what some described as a Black Mirror-like incident.
The Llama 3, Grok-1.5 and Mistral models shift more power to open source communities while further diluting the generative AI market.
But that may very well be a very good thing, because it's now a matter of survival of the fittest and the ball is firmly in Microsoft's OpenAI camp, which is anticipated to make the following move on this fascinating game of genetic AI chess.
