As generative AI touches a growing variety of industries, the businesses that make chips to run the models are benefiting enormously. Nvidia particularly has massive influence and has a estimated 70 to 95 percent of the marketplace for AI chips. Cloud providers from Meta To Microsoft are spending billions of dollars on Nvidia GPUs because they fear falling behind within the generative AI race.
It is subsequently comprehensible that generative AI vendors should not satisfied with the established order. Their success depends to a big extent on the whims of the dominant chip makers. And so that they, together with opportunistic enterprise capitalists, are on the hunt for promising newcomers that may compete with the established AI chip makers.
Etched is one among the many different chip firms vying for a seat on the table—nevertheless it's also one of the vital intriguing. Just two years old, Etched was founded by two Harvard dropouts, Gavin Uberti (formerly of OctoML and Xnor.ai) and Chris Zhu, who, together with Robert Wachen and former Cypress Semiconductor CTO Mark Ross, desired to create a chip that might do one thing: run AI models.
This is just not unusual. Many startups and tech giants are developing chips that exclusively run AI models, also called inference chips. Meta has MTIA, Amazon has Graviton and Inferentia, and so forth. But Etched's chips are unique in that they only run a single style of model: Transformers.
The Transformer, which was proposed by a team of Google researchers back in 2017, has change into by far the dominant architecture for generative AI models.
Transformers form the idea of OpenAI's video generation model Sora. They are at the center of text generation models like Anthropic's Claude and Google's Gemini. And they power art generators like the newest version of Stable Diffusion.
“In 2022, we bet that Transformers will take over the world,” said Uberti, CEO of Etched, in an interview with TechCrunch. “We've reached some extent within the evolution of AI where specialized chips that may perform higher than general-purpose GPUs are inevitable – and the world's engineering decision-makers realize it.”
Etched's chip, called Sohu, is an ASIC (application specific integrated circuit) – a chip tailored to a selected application and designed to run transformers. Manufactured using TSMC's 4nm processSohu can deliver significantly higher inference performance than GPUs and other general-purpose AI chips while consuming less power, Uberti claims.
“Sohu is over and over faster and cheaper than even Nvidia's next generation of Blackwell GB200 GPUs in terms of processing text, images and video,” Uberti said. “One Sohu server replaces 160 H100 GPUs. … Sohu will probably be a less expensive, more efficient and more environmentally friendly option for business leaders who need specialized chips.”
How does Sohu achieve all this? In various ways, but essentially the most obvious (and intuitive) is an optimized inference hardware and software pipeline. Because Sohu doesn't run non-Transformer models, the Etched team was in a position to eliminate hardware components that aren't relevant to Transformers and reduce the software overhead traditionally used to deploy and run non-Transformers.
Etched appears at an inflection point within the race for generative AI infrastructure. Aside from cost, the GPUs and other hardware components required today to run models at scale are dangerously power hungry.
Goldman Sachs predicts that AI will result in a 160% increase in data center electricity consumption by 2030, which is able to result in a big increase in greenhouse gas emissions. Researchers at UC Riverside have meanwhile treasure that the worldwide use of AI could lead to data centers consuming 1.1 to 1.7 trillion gallons of fresh water by 2027, Impact on local resources(Many data centers use water to chill servers.)
Uberti presents Sohu optimistically – or, depending in your interpretation, bombastically – as an answer to the industry’s consumption problem.
“In short, our future customers won’t give you the chance to afford not to change to Sohu,” Uberti said. “Companies are willing to bet on Etched because speed and value are of existential importance for the AI products they wish to develop.”
But can Etched – assuming it achieves its goal of bringing Sohu to the mass market in the following few months – succeed with so many others hot on its heels?
The company currently has no direct competitor, but AI chip startup Perceive recently Preview of a processor with hardware acceleration for transformers. Groq has also invested heavily in transformer-specific optimizations for its ASIC.
Competition aside, what if Transformers sooner or later exit of fashion? Uberti says in that case, Etched will do the apparent: develop a brand new chip. That's nice, but given how long it took to get Sohu to fruition, it's a fairly drastic fallback.
However, none of those concerns have stopped investors from pouring huge sums into Etched.
Etched announced today that it has closed a $120 million Series A funding round co-led by Primary Venture Partners and Positive Sum Ventures, bringing Etched's total raised to $125.36 million. The round included participation from major private investors including Peter Thiel (Uberti, Zhu and Wachen are former Thiel Fellows), GitHub CEO Thomas Dohmke, Cruise (and the Bot Company) co-founder Kyle Vogt and Quora co-founder Charlie Cheever.
These investors probably imagine that Etched has a very good probability of successfully growing its server sales business. Perhaps that’s the case – Uberti claims that unnamed customers have reserved “tens of hundreds of thousands of dollars” value of hardware to date. The upcoming launch of the Sohu Developer Cloud, which allows customers to pre-test Sohu through an interactive online playground, should boost sales, Uberti says.
Still, it seems too early to say whether this will probably be enough to propel Etched and its 35-person team into the long run its co-founders envision. The AI chip segment may be unforgiving even at the most effective of times – see the high-profile near-bankruptcies of AI chip startups like Mythic and Graphcoreand that declining investment in AI chip firms in 2023.
However, Uberti has a powerful selling point: “Video generation, audio-to-audio modalities, robotics and other future AI use cases will only be possible with a faster chip like Sohu. The entire way forward for AI technology will rely on whether the infrastructure can scale.”
