According to a report from The Information, Microsoft is working on a 500B parameter LLM called MAI-1 that would rival GPT-4 and Google Twins Models.
We recently reported on Microsoft's Phi-3 Mini family of small language models with parameters from 3.8B to 14B. With 500B parameters, MAI-1 is prone to be the most important model that Microsoft has deployed.
Due to its size, it’s on the identical level as GPT-4 and Google's larger one Twins Models. GPT-4 is rumored to have 1.76T parameters, however it is a Mixture of Experts (MoE) model, so only about 280B parameters are in play in inference.
There isn’t any information in regards to the architecture of MAI-1, but whether it is a dense model and never MoE, it can be quite powerful. Meta's expected Llama-3 model is predicted to have 400 billion parameters.
The development of MAI-1 is led by Mustafa Suleyman, co-founder and former head of applied AI at DeepMind.
Mustafa left DeepMind to co-found the AI startup Inflection in 2022. In March of this yr, Microsoft hired most of Inflection's employees and paid $650 million for the corporate's mental property rights.
MAI-1 is seemingly a totally latest Microsoft project and never a continuation of an existing Inflection project. There's no word on a release date yet, but we could see a preview of MAI-1 on May 16 at Microsoft's Build developer conference.
Microsoft is OpenAIis the most important investor, so the corporate is developing its own LLMs that may compete with those of OpenAI is just a little surprising for some. Is Microsoft playing it protected, pursuing multiple development strategies, or is it something else entirely?
Microsoft CTO Kevin Scott tried to downplay the issue. In a post on LinkedIn, Scott said: “I'm undecided why that is news, but to state the plain: We construct large supercomputers to coach AI models; Our partner Open AI uses these supercomputers to coach groundbreaking models; And then we each make these models available in services and products in order that many individuals can profit from them. We really like this arrangement.”
Scott could also be sincere on this statement, but when MAI-1 is released, it could put Microsoft in direct competition with the corporate by which it has invested billions of dollars.
Will MAI-1 be released on time? OpenAI to enhance it by releasing GPT-5? OpenAI has scheduled an event for this Thursday where the corporate was purported to unveil updates and product demonstrations, however the event has since been postponed.
As mysterious GPT-2 chatbots appear, disappear and now reappear, Microsoft is constructing massive models and OpenAI The AI drama keeps us guessing, it's relentless.
