According to an individual from this team, OpenAI's Superalignment team, which is chargeable for developing ways to regulate and control “superintelligent” AI systems, was promised 20% of the corporate's computing resources. But requests for a fraction of that computing power were often rejected, stopping the team from getting their work done.
This issue, amongst other things, led to several team members resigning this week, including co-lead Jan Leike, a former DeepMind researcher who helped develop ChatGPT, GPT-4, and ChatGPT's predecessor InstructGPT during his time at OpenAI.
Leike went public on Friday morning with some reasons for his resignation. “I had been disagreeing with OpenAI's leadership in regards to the company's core priorities for quite a while, until we finally reached a breaking point,” Leike wrote in a series of posts on “ought to be spent on the subsequent generations of models of security, surveillance, preparedness, protection, adversary resilience, (super)alignment, confidentiality, societal impact and related topics. It is sort of difficult to unravel these problems and I worry that we should not on the suitable path to get there.”
OpenAI didn’t immediately reply to a request for comment in regards to the resources promised and allocated to this team.
OpenAI founded the Superalignment team last July and was led by Leike and OpenAI co-founder Ilya Sutskever, who also left the corporate this week. It had the ambitious goal of solving the important thing technical challenges of controlling superintelligent AI over the subsequent 4 years. Along with scientists and engineers from OpenAI's former Alignment division, in addition to researchers from other organizations throughout the company, the team was expected to contribute research on the safety of internal and non-OpenAI models through initiatives including a research grant program that might recruit work from the broader AI industry and share together with her.
The Superalignment team managed to publish a lot of safety research results and funnel hundreds of thousands of dollars in grants to outside researchers. But as product launches increasingly stretched the bandwidth of OpenAI's leadership, the Superalignment team needed to fight for more upfront investments – investments they are saying are critical to the corporate's stated mission of developing superintelligent AI for the good thing about all humanity were .
“Building machines more intelligent than humans is an inherently dangerous endeavor,” Leike continued. “But in recent times, safety culture and processes have taken a back seat to shiny products.”
Sutskever's fight with OpenAI CEO Sam Altman created additional distraction.
Sutskever and OpenAI's old board decided to abruptly fire Altman late last 12 months over concerns that Altman had not been “consistently open” with board members. Under pressure from OpenAI investors, including Microsoft, and most of the company's own employees, Altman was eventually reinstated, much of the board resigned, and Sutskever allegedly never returned to work.
According to the source, Sutskever was instrumental within the Superalignment team – not only contributing to research, but in addition acting as a bridge to other departments inside OpenAI. He would also act as an envoy of sorts, illustrating the importance of the team's work to key OpenAI decision makers.
Following Leike's departure, Altman wrote in He hinted at a lengthy statement that co-founder Greg Brockman delivered on Saturday morning:
Although there’s little concrete about policies or commitments in Brockman's response, he said: “We need a really tight feedback loop, rigorous testing, careful consideration at every step, world-class security and a harmony of security and capabilities.”
Following the departure of Leike and Sutskever, John Schulman, one other co-founder of OpenAI, has stepped up to guide the Superalignment team's work, but there’ll now not be a dedicated team, but somewhat a loose group of researchers working in departments throughout firms are embedded. An OpenAI spokesperson described it as “deeper integration (of the team).”
There is a fear that because of this, AI development at OpenAI won’t be as safety-focused because it might have been.
