Open source AI is finally catching up with the dominance of closed source AI. Today Metaone in every of the leaders within the open model category, released a mid-year update claiming that adoption of its Llama model family has reached recent heights, especially for the reason that release of the main Llama 3.1 last month.
The Mark Zuckerberg-led company announced that downloads of the Llama models on Hugging Face have reached 350 million, a greater than tenfold increase from last yr. It also noted that adoption of the models has grown significantly – each through Hugging Face and thru distribution partners – with major corporations akin to Zoom, Spotify, Infosys, AT&T and Goldman Sachs using them for internal and external use cases.
The update shows that open-source AI, which initially began slowly, can’t only sustain with closed-source by way of performance, but can also be gaining traction on the enterprise level and powering real-world applications. It also challenges the lead of OpenAI, which proclaimed since it has didn’t deliver groundbreaking AI products beyond announcements.
While OpenAI had a head start within the generative AI space, Meta was quick to leap on the bandwagon with its Llama model. The company launched the model three months after ChatGPT's. However, as a substitute of going closed source, it focused on an open approach, making its ecosystem – which incorporates Llama 2, Llama 3, and most recently Llama 3.1 – available through each Hugging Face and cloud partners.
“By making our Llama models publicly available, we’ve got created a vibrant and diverse AI ecosystem where developers have more selection and opportunity than ever before. Innovation has been broad and rapid, from startups pushing recent boundaries to enterprises of all sizes using Llama…,” the corporate wrote in a Blog post Today.
French startup Mistral followed the identical strategy for a lot of its models. This gave developers several powerful open models to construct on and create derivatives that achieved parity and even outperformed closed models on chosen metrics (see FinGPT, BioBert, Defog SQLCoder And Pind).
For Meta, the open strategy only began to achieve traction after the launch of Llama 2 in July last yr. Since the launch, downloads of the corporate's models on Hugging Face have increased greater than tenfold, reaching nearly 350 million. Last month, the corporate recorded greater than 20 million downloads on the platform – suggesting further increased traction with the recent release of Llama 3.1 405B.
“We see a growing preference for Llama within the developer community and powerful signs of continued growth. According to a survey by Artificial Analysis, an independent AI benchmarking website, Llama was the second most considered model and industry leader in open source,” the corporate added.
Companies using the Llama family of models for internal and external use cases include AT&T, DoorDash, Goldman Sachs, Niantic, Nomura, Shopify, Spotify, Zoom, Infosys and KPMG.
Many enterprise developers also use Llama models across Meta's extensive network of cloud and infrastructure providers, AWS, Microsoft Azure, Google Cloud, Groq, Nvidia, Databricks, and Snowflake. The company didn’t disclose partner-specific numbers, but confirmed that monthly Llama usage (by input/output token volume) increased 10x from January to July 2024 across select cloud service providers.
The statistics seem to indicate that hosted Llama usage is continually growing (with a peak in July when Llama 3.1 was released), however it's essential to notice that these numbers only confer with a few of the company's largest cloud service providers and never all partners.
This implies that actual monthly usage statistics may vary.
Pressure on OpenAI, Anthropic
Either way, the adoption of Llama by leading corporations like AT&T and Spotify shows that open source AI is catching up quickly. Recent performance improvements and the long-term cost advantages of the open approach are the largest aspects driving this shift. With further developments in the approaching months, we will expect open source AI to completely take over the dominance of closed models. This will put pressure on corporations offering closed models, forcing them to innovate more and further reduce the fee of using their models.
In particular, the impact of the open source movement is already visible. OpenAI, which began the wave of generative AI, has significantly reduced the costs of its existing models, including GPT-4o.
However, in relation to product innovation, the research lab led by Sam Altman appears to be lagging behind. All of the cutting-edge AI products announced to this point, including Sora and SearchGPT, are either still unreleased or only available to a select group of users.
