Nvidia's rivals and largest customers are rallying behind an OpenAI-led initiative to develop software that might make it easier for artificial intelligence developers to maneuver away from its chips.
Silicon Valley-based Nvidia has change into the world's Most worthy chipmaker since it has a near monopoly on the chips needed to develop large AI systems. But delivery bottlenecks and high prices are pushing customers to search for alternatives.
However, producing recent AI chips only solves a part of the issue. While Nvidia is best known for its powerful processors, industry insiders say its “secret sauce” is its Cuda software platform, which enables chips originally designed for graphics to speed up AI applications.
At a time when Nvidia is investing heavily in expanding its software platform, rivals like Intel, AMD and Qualcomm are targeting Cuda in hopes of poaching customers – with strong support from a few of Silicon Valley's biggest corporations.
Engineers from Meta, Microsoft and Google help develop Triton, software to efficiently execute code on a wide range of AI chips, OpenAI Approved in 2021.
Even as they proceed to spend billions of dollars on their latest products, major tech corporations are hoping Triton will help break the stranglehold that Nvidia has on AI hardware.
“Essentially it breaks the Cuda lock-in,” said Greg Lavender, Intel’s chief technology officer.
Nvidia dominates the marketplace for developing and deploying large language models, including the system behind OpenAI's ChatGPT. That has pushed the corporate's valuation past $2 trillion, and rivals Intel and AMD are struggling to catch up. Analysts expect Nvidia to announce this week that its revenue greater than tripled in its most up-to-date quarter in comparison with a 12 months earlier and profit rose greater than sixfold.
But Nvidia's hardware has only change into such a coveted commodity due to the accompanying software the corporate has developed over nearly 20 years, creating an enormous moat that competitors have struggled to beat.
“What Nvidia does for a living shouldn’t be (just) making the chip: we construct a whole supercomputer, from the chip to the system to the connections. . . however the software could be very necessary,” said CEO Jensen Huang on the GPU technology conference in March. He described Nvidia's software because the “operating system” of AI.
Nvidia was founded greater than 30 years ago to focus on video gamers, and its entry into AI was made easier by the Cuda software it developed in 2006 to enable general-purpose applications to run on its graphics processors.
Since then, Nvidia has invested billions of dollars developing lots of of software tools and services to make running AI applications on its GPUs faster and easier. Nvidia executives say the corporate is now hiring twice as many software engineers as hardware employees.
“I believe people underestimate what Nvidia has actually built,” said David Katz, partner at Radical Ventures, an investor specializing in AI.
“They have built a software ecosystem around their products that’s efficient, easy to make use of and truly works – making very complex things easy,” he added. “It’s something that has evolved over a really long time period with an enormous user community.”
Still, the high price of Nvidia's products and the long line to purchase its most advanced devices, equivalent to the H100 and the upcoming “superchip” GB200, have led a few of its biggest customers – including Microsoft, Amazon and Meta – to search for alternatives or develop your personal.
However, since most AI systems and applications already run on Nvidia's Cuda software, it’s time-consuming and dangerous for developers to rewrite them for other processors equivalent to AMD's MI300, Intel's Gaudi 3 or Amazon's Trainium.
“The thing is: if you should compete with Nvidia on this space, you not only must construct competitive hardware, but you furthermore may must make it easy to make use of,” said Gennady Pekhimenko, CEO of CentML, a startup. He develops software to optimize AI tasks and is an associate professor of computer science on the University of Toronto. “Nvidia's chips are really good, but in my view their biggest advantage is on the software side.”
Competitors like Google's TPU AI chips may offer comparable performance in benchmark tests, but “convenience and software support make a giant difference” in Nvidia's favor, Pekhimenko said.
Nvidia executives argue that its software work allows it to deploy a brand new AI model on its latest chips in “seconds” and provides continuous efficiency improvements. But these benefits have a catch.
“We see a heavy reliance on Cuda within the (AI) ecosystem, which makes it very difficult to make use of non-Nvidia hardware,” said Meryem Arik, co-founder of TitanML, a London-based AI startup. TitanML initially used Cuda, but last 12 months's GPU shortage caused the corporate to rewrite its products in Triton. Arik said this helped TitanML attract recent customers who desired to avoid what she called the “cuda tax.”
Triton, whose co-creator Philippe Tillet was hired by OpenAI in 2019, is open source, meaning anyone can view, adapt or improve its code. Proponents argue that this makes Triton more attractive to developers than Cuda, which is owned by Nvidia. Originally, Triton only worked with Nvidia's GPUs, but now it supports AMD's MI300. Support for Intel's Gaudi and other accelerator chips is planned soon.
Meta, for instance, has made the Triton software the center of its self-developed AI chip. MIA. When Meta released the second generation of MTIA last month, its engineers said Triton was “highly efficient” and “enough hardware independent” to work with a spread of chip architectures.
According to logs on GitHub and conversations in regards to the toolkit, developers from OpenAI competitors like Anthropic — and even Nvidia itself — have also been working on improving Triton.
Triton isn't the one try to challenge Nvidia's software advantage. Intel, Google, Arm and Qualcomm are among the many members of the UXL Foundation, an industry alliance that’s developing a Cuda alternative based on Intel's open source OneAPI platform.
Chris Lattner, a well known former senior engineer at Apple, Tesla and Google, has launched Mojo, a programming language for AI developers whose pitch is: “No Cuda required.” Only a small minority of software developers worldwide know learn how to program with Cuda, and it’s difficult to learn, he argues. With his startup Modular, Lattner hopes Mojo will make it “dramatically easier” to develop AI for “developers of every kind – not only the elite experts at the most important AI corporations.”
“Today’s AI software relies on software languages from over 15 years ago, just like a BlackBerry today,” he said.
Even if Triton or Mojo are competitive, it’ll take years for Nvidia's rivals to catch as much as Cuda's lead. Analysts at Citi recently estimated that Nvidia's share of the generative AI chip market will decline from about 81 percent next 12 months to about 63 percent in 2030, suggesting that Nvidia will remain dominant for a few years to come back.
“Building a competitive chip against Nvidia is a difficult but easier problem than constructing the complete software stack and getting people to make use of it,” Pekhimenko said.
Intel's Lavender stays optimistic. “The software ecosystem will proceed to evolve,” he said. “I believe the playing field will likely be leveled.”
