Determined to meet up with its competitors within the generative AI space, Meta is spending money billion on their very own AI efforts. Part of those billions will go towards the long run Recruiting AI researchers. But a fair larger portion is spent on developing hardware, particularly chips to run and train Meta's AI models.
Meta revealed the newest results of its chip development efforts today, conspicuously a day after Intel announced its latest AI accelerator hardware. Dubbed the “next generation” Meta Training and Inference Accelerator (MTIA), the chip is the successor to last yr's MTIA v1 and runs models for, amongst other things, rating and recommending display ads based on meta properties (e.g .Facebook).
Compared to MTIA v1, which is predicated on a 7nm process, the following generation MTIA is 5nm. (In chip manufacturing, “process” refers to the dimensions of the smallest component that could be built on the chip.) The next-generation MTIA is a physically larger design and features more processor cores than its predecessor. And while it uses more power – 90W vs 25W – it also has more internal memory (128MB vs 64MB) and runs at the next average clock speed (1.35GHz vs 800MHz).
According to Meta, next-generation MTIA is currently deployed in 16 of its data center regions and delivers as much as 3 times higher overall performance in comparison with MTIA v1. If that “3x” claim sounds a bit vague, you’re not flawed – we thought so too. But Meta only said the figure got here from testing the performance of “4 key models” of each chips.
“Because we control the whole stack, we will achieve greater efficiency in comparison with commercially available GPUs,” Meta writes in a blog post shared with TechCrunch.
Meta's hardware presentation – which comes just 24 hours after a press conference in regards to the company's various ongoing generative AI initiatives – is unusual for several reasons.
Firstly, reveals Meta in blog entry that the next-generation MTIA shouldn’t be currently getting used for generative AI training workloads, although the corporate says it has “multiple programs underway” exploring this. Second, Meta admits that next-generation MTIA is not going to replace, but complement GPUs for running or training models.
If you read between the lines, Meta moves slowly – perhaps slower than he would love.
Meta's AI teams are almost actually under pressure to chop costs. The company is predicted to issue one estimated By the top of 2024, $18 billion is predicted to be spent on GPUs to coach and run generative AI models, and—with training costs for state-of-the-art generative models within the tens of tens of millions—in-house hardware represents a horny alternative.
And as Meta's hardware declines, competitors move on, much to the dismay of Meta's leadership, I think.
Google this week made its fifth-generation custom chip for training AI models, TPU v5p, generally available to Google Cloud customers and introduced its first dedicated chip for running models, Axion. Amazon has several custom AI chip families available on the market. And Microsoft jumped into the fray last yr with the Azure Maia AI Accelerator and the Azure Cobalt 100 CPU.
In the blog entry, Meta says it took lower than nine months to go “from first silicon to production models” of the next-generation MTIA, which, to be fair, is shorter than the standard timeframe between Google TPUs. But Meta still has a variety of catching as much as do if it wants to realize a certain level of independence from third-party GPUs – and sustain with stiff competition.
