Jan Leike, a number one AI researcher who resigned from OpenAI earlier this month before publicly criticizing the corporate's approach to AI safety, has joined OpenAI competitor Anthropic to guide a brand new “superalignment” team.
In a post on X, Leike said his team at Anthropic will give attention to various facets of AI safety, particularly “scalable supervision,” “weak-to-strong generalization,” and research into automatic targeting.
A source acquainted with the matter told TechCrunch that Leike will report on to Jared Kaplan, Anthropic's chief scientific officer, and that Anthropic researchers currently working on scalable supervision – techniques for controlling the behavior of large-scale artificial intelligence in predictable and desirable ways – will move to Leike as Leike's team ramps up.
In some ways, the mission of Leike's team is analogous to that of OpenAI's recently disbanded Superalignment team. The Superalignment team, which Leike co-led, had an ambitious goal of solving the core technical challenges of governing superintelligent AI over the following 4 years, but was often stymied by OpenAI's leadership.
Anthropic has often tried to position itself as more security-focused than OpenAI.
Anthropic's CEO, Dario Amodei, was once VP of research at OpenAI and reportedly parted ways with OpenAI after disagreements over the corporate's direction – namely, OpenAI's growing industrial focus. Amodei brought quite a lot of former OpenAI employees to form Anthropic, including former OpenAI head of policy Jack Clark.
