METR, Led by Beth Barnes, Confront AI Safety Research Talent Shortage

Beth Barnes, one of many AI trade’s most influential watchdogs, has lots to fret about.

The ex-OpenAI researcher has proven that AI’s capabilities are escalating by the month. Now, she fears that the trade would not have the workers in place to maintain up with the expertise in its subsequent chapter.

The largest barrier to that analysis is expertise, mentioned Barnes, who left OpenAI to start out the nonprofit METR in 2022. METR, which stands for Mannequin Analysis and Menace Analysis, performs a vital function within the AI trade by working carefully with OpenAI, Anthropic, Google, and Meta to supply impartial solutions concerning the tech’s rapidly altering capabilities.

But Barnes sees such fierce competition for AI researchers that even METR’s lofty salaries, which attain $503,000 on present job postings, do not tackle the nonprofit’s expertise scarcity.

“Ideally, we would wish to scale actually giant, however in apply, we have been in a position to fundraise as a lot as we’d like, and the bottleneck is way more expertise,” Barnes mentioned.

The safety incident introduced by OpenAI and Hugging Face sparked a brand new wave of questions on the right way to consider and management AI. Greater than 1,300 frontier lab workers signed a letter in July warning that AI growth might outpace management. If the trade would not have the workers to analysis exactly what new AI fashions can do, firms might be pressured to sluggish their releases.

Whereas the habits of the OpenAI models stunned a lot of the company world, METR’s crew was far much less stunned. Governments, firms, and even non secular teams all ask the nonprofit for assist understanding and testing the tech’s progress.

Nonetheless, the sensation contained in the 35-person lab is one among concern. When Enterprise Insider visited METR’s Berkeley workplace on the afternoon that OpenAI revealed its hack, Barnes described an AI security discipline that’s badly constrained by its dimension.

“There’s a lot extra to do than now we have capability for,” Barnes mentioned. A “cheap civilization,” in her view, could be pouring a far bigger proportion of AI’s investment into steering it and evaluating the tech — particularly now that society is reckoning with fashions this highly effective.

‘Humanity’s preparedness crew’ struggles to develop

As AI labs launch new fashions, METR measures the probability that they will full duties of accelerating length. These measurements kind the nonprofit’s most well-known providing, a widely-cited chart that exhibits how, over the past six years, AI’s capabilities have doubled about each seven months.

Chris Painter, METR’s president, describes the lab as “humanity’s preparedness crew,” a tongue-in-cheek nod to the preparedness teams at OpenAI and Anthropic that report security threats to their CEOs.

“We aren’t actually accountable to anybody aside from the general public and the general public’s well-being,” Painter mentioned.

Barnes emphasised the significance of METR’s independence in order that it may produce analysis that is not tied to an organization’s objectives. When Barnes labored at OpenAI, she felt the necessity for a analysis group exterior the labs that would remark freely on the event of the tech. Whereas METR would not take cash from the frontier AI labs or their workers, it accepts compute grants and works with them to investigate unreleased fashions.

As well as, workers have analyzed labs’ security practices, written about AI’s impact on software program engineers’ pace, and studied the deserves of utilizing AI fashions to observe different AI fashions. Barnes mentioned she’d wish to increase into predicting the subsequent ranges of AI’s capabilities and the way these adjustments might speed up AI’s growth.

Neev Parikh, a METR researcher, says expertise constraints restrict the variety of questions researchers can sort out about AI fashions’ inside reasoning.

“There is a dearth of individuals,” Parikh mentioned. “I’d fortunately see the sphere increase 10x.”

METR might play an enormous function in a brand new AI local weather

In July, OpenAI introduced that its AI fashions cheated throughout testing by hacking into Hugging Face’s methods to seek out the check solutions. Later, CEO Sam Altman mentioned it was “the primary safety incident that I’ve felt very viscerally.”

METR had seen one thing like that coming. In Might, the nonprofit wrote in a report that present AI brokers “might plausibly begin a rogue deployment,” however wouldn’t have the ability to cover it. Then, in June, METR tried out OpenAI’s then-unreleased GPT-5.6 Sol mannequin and located that it could repeatedly cheat on difficult exams, together with by extracting hidden supply code to seek out the solutions. It revealed a report about this and shared it with OpenAI earlier than GPT-5.6 Sol’s wider release.

Come July, the GPT-5.6 Sol mannequin was a part of OpenAI’s safety incident with Hugging Face. OpenAI introduced on Wednesday that METR and one other nonprofit, Redwood Analysis, would assess the incident and that their outcomes would inform OpenAI’s personal technical report.

“There are actually actual, business-affecting incidents of this, and the world has a stake in understanding that,” Painter mentioned.

The cyber expertise of OpenAI and Anthropic’s latest fashions have sparked a wave of concern and a focus and spawned a raft of new bills in Washington. METR’s work strains up carefully with one potential route for regulation: a present invoice proposal would require giant AI mannequin builders to get security audits from exterior organizations. METR might fill a task like this; Painter mentioned they’d probably have an interest, although it is one thing the sphere continues to be determining.

One thing like that proposal, Painter mentioned, might assist with METR’s hiring difficulty. He urged that extra folks may depart the AI firms themselves — for comparable salaries at METR, although with out fairness compensation — if a regulatory system gave security analysis organizations larger authority.

“There’s sufficient precedent right here for every type of testing association,” Painter mentioned, “that I feel with both readability from trade, about how this testing ought to work long-term, or from the federal government, I feel this discipline might scale very quickly.”

Ajeya Cotra, who led the writing of METR’s Might report on the dangers of AI, mentioned that oversight of AI can really feel “chaotic and unpredictable” proper now. She’s nonetheless optimistic.

“The pattern is towards folks caring about this difficulty extra,” Cotra mentioned. “And wanting to control it in a extra critical approach over time.”

Have a tip? Contact this reporter by way of e-mail at scouncil@businessinsider.com, or over textual content, Sign, Telegram, or WhatsApp at 415-757-8198. Use a private e-mail tackle, a nonwork WiFi community, and a nonwork gadget; here’s our guide to sharing info securely.