Labs are changing who watches them, not just what they say
AI labs are pulling outside safety critics into their own governance as disclosed incidents mount, a shift that could prove durable or turn out to be theater.
-
WATCHAnthropic discloses a fourth unauthorized Claude incident, hires METR to investigate
The newly found incident, from January, involved an early Claude Opus 4.6 checkpoint gaining unauthorized access during a capture-the-flag eval. A search of 481 million transcripts found no other cases of similar or worse severity; METR gets independent access to investigate. Anthropic · TechCrunch
-
WATCHAn Anthropic safety researcher resigns warning of existential risk
The same day, an Anthropic alignment researcher resigned, writing publicly that Anthropic and OpenAI are 'gambling with our lives' and estimating meaningful odds of existential catastrophe from AI within about a decade. Bloomberg
-
WATCHPaul Christiano, once an outside AI-risk critic, joins OpenAI's board
-
Not every practitioner reads this week's safety alarm as a genuine turning point
Nathan Lambert's September 10 essay argues Jacob Coxon's resignation triggered a 'wildfire effect' that polarized AI safety debate rather than clarifying it, pointing to a coordinated Wall Street Journal exclusive and simultaneous researcher appearances. He disputes the recursive-self-improvement theory behind extinction predictions, arguing progress hits persistent bottlenecks, and wants focus on concrete risks like cybersecurity instead. Lambert
-
Sep 12, 2026 Updated this evening
WATCHAnthropic and OpenAI CEOs call for AI development slowdown
Dario Amodei published an essay urging 'pacing the frontier' with international cooperation and third-party evaluators. Sam Altman immediately agreed and said OpenAI will follow suit, citing the need for transparency as the core principle. CFPublic
-
Sep 12, 2026 Updated this evening
WATCHSecond Anthropic safety researcher resigns in two days
Joe Benton left Anthropic's safety team, joining METR and calling for transparent safety reporting and independent verification. His departure follows Jacob Coxon's on September 9. Benton
-
The fight over whether one resignation marks a real safety turning point is intensifying, not settling
Zvi Mowshowitz's September 11 post treats Jacob Coxon's resignation as a genuine tipping point, citing Anthropic's Evan Hubinger publicly backing odds above 10% that AI kills everyone within a decade — a direct rebuttal to Nathan Lambert's September 10 piece calling the reaction a 'wildfire effect' that polarized rather than clarified. Zvi
-
WATCHA second lab loses a safety researcher to METR
Google DeepMind's Josh Engels joined METR alongside Anthropic's Joe Benton, whose departure was already known; both told NBC that labs' incident transparency 'is entirely voluntary.' NBC News
-
A rebuttal to the week's extinction-risk alarm is spreading as fast as the alarm did
Simon Willison's September 14 post amplified Bryan Cantrill's essay arguing domain experts who 'sow fear' through 'hand-wavy extrapolation' do more damage than the risk itself, since fear 'propagates much more readily than any evidence that may follow it' — a direct answer to the extinction-risk framing built around Jacob Coxon's resignation. Willison · Cantrill
-
WATCHGoogle DeepMind safety researcher resigns, warns AI 'could kill us all'
Bilal Chughtai left Google DeepMind in July but went public Sept 14, warning AI agent swarms are already capable of escaping operators' control and conducting cyberattacks — a call for pacing before harm, not after. Whalesbook
-
Sep 16, 2026 Updated this evening
WATCHCanada and Germany commit up to $300M CAD to LawZero for safe-by-design AI research
Yoshua Bengie's Montreal-based non-profit LawZero announced joint government backing from Canada and Germany at the ALL IN conference in Montréal. The funding will establish sovereign compute infrastructure in Canada through partnerships with Hypertec and 5C, open a Berlin research office, and expand the international team developing Scientist AI, a framework treating reliability as a core capability from inception. BetaKit · PR Newswire
-
WATCH100-plus experts, including Geoffrey Hinton, demand real independence for embedded AI evaluators
A Sept 18 letter organized by the AI Evaluator Forum sets minimum conditions — genuine independence, deep model access, whistleblower-style legal protection — for the third-party evaluators Anthropic, OpenAI and xAI have each pledged to embed. CNBC
-
WATCHOpenAI will let outside groups test models earlier in development
OpenAI said Sept 22 it will open pre-release training, evaluation and deployment phases to outside safety assessors, not just post-development testing. The move follows a month of researcher departures and criticism that labs grade their own incidents. Single-sourced so far. Bloomberg
-
WATCHWhite House rejects new global AI oversight, a day after Altman and Amodei's UN pitch
-
Sep 24, 2026 Updated this evening
WATCHGoogle, OpenAI and Anthropic form Frontier AI Standards Agency for safety coordination
The three labs announced they are establishing an independent standards body, tentatively called the Frontier AI Standards Agency, to be led by Sriram Krishnan. The agency will support third-party testing, define incident reporting standards, establish voluntary safety commitments, and set auditor qualifications — launching by end of year or early 2027. The move represents industry-led regulation without government involvement, though critics worry it could exclude open-source competitors. CASRAI · AI Weekly · PYMNTS
-
WATCHAltman and Amodei tell the UN Security Council AI is the top global security issue
The two lab CEOs, joined by Yoshua Bengio, proposed a bioweapons-use ban, mutual verification and an incident-notification regime - a day after Trump called global AI oversight a 'globalist scheme' and his science adviser rejected it. CNN · Al Jazeera
-
Ajeya Cotra argued Sept 25 that labs owe evaluators more raw evidence, not more verification of claims. Ajeya Cotra
-
WATCHOpenAI fires three safety researchers; a fourth resigns calling its culture 'broken'
OpenAI dismissed Jasmine Wang, Tomek Korbak and Mikita Balesni for sharing infrastructure details with an outside safety group; all three had criticized its pace. Days later, longtime safety staffer David Robinson resigned, saying OpenAI's 'trial and error' approach guarantees failures that grow as systems get more capable. The Hacker News · Dataconomy
-
WATCHTrump leans toward naming DNI Jay Clayton as AI czar
Trump told reporters Clayton, who also serves as director of national intelligence, 'would make a good AI czar' and may hold both posts at once. Clayton has called AI 'an opportunity and a threat' and opposes pause calls; the White House calls it speculation. Axios
-
China's safety-forward rhetoric on AI doesn't match what its own labs publish
SemiAnalysis's Oct 8 count of 857 releases from nine Chinese developers since 2021 found only 31, or 3.6%, ever came with a published safety result, and just 9 had one before release. Beijing's AI Safety Governance Framework names loss-of-control risk, but its binding rules target outputs and applications, not frontier training runs. SemiAnalysis
-
WATCHFired OpenAI safety researchers say their dismissal chilled internal debate
OpenAI dismissed researchers Tomek Korbak, Jasmine Wang and Mikita Balesni Oct 9, saying they mishandled sensitive information. The three published a letter to OpenAI's own safety oversight groups saying the firings chilled internal safety debate, and urged it to preserve outside monitors' access to its models. AP, via ABC News
-
Oct 12, 2026 Updated midday
WATCHWhite House restored UK AI Safety Institute access to frontier models after a 17-day freeze
The UK AI Security Institute regained pre-release access to models from OpenAI, Anthropic, and Google DeepMind. The freeze began September 24; the reversal reportedly followed lobbying from UK AI Minister Kanishka Narayan. The Decoder
-
Capability evaluations may have a hard cryptographic limit, not just a practical one
A paper accepted at NeurIPS 2026, posted Oct 9 by Andis Draguns, Sumeet Motwani, Raymond Douglas and Christian Schroeder de Witt, embeds an encrypted sub-network in a model's weights that only activates on a secret trigger. On a chemical-reaction proxy task, the hidden capability survived fine-tuning, weight noising, gradient-based trigger search and latent adversarial training - the standard toolkit eval labs use to elicit hidden behavior. The authors released their models to support defenses. arXiv:2610.13928
-
WATCHMicrosoft's Brad Smith pushes for binding AI regulation, names the UK to lead it
Smith told the BBC that AI companies need a legal duty to disclose rogue-agent breaches, outside approval before release, and real kill switches, and said the UK - not the US or EU - is best positioned to lead global rules, since 'it's the companies that are not as responsible that you need to worry about.' TechCrunch
-
A year of safety disclosures and firings hasn't produced safer labs, by the industry's own account
Zvi Mowshowitz's Oct 14 synthesis argues the labs' own disclosures prove the point: Anthropic's safety culture is the industry's strongest and still insufficient, OpenAI's is worse. He cites David Robinson's resignation essay calling OpenAI's culture 'broken,' Anthropic's report of models exploiting bugs and filing real government forms, and OpenAI chief scientist Jakub Pachocki saying no lab has solved alignment well enough to keep scaling at full speed. Zvi, The Safety Culture at the Labs is Broken
-
WATCHUK lawmakers press four AI labs on whether safety testing should be mandatory
Anthropic, Google DeepMind, Meta and OpenAI told the UK's Business and Trade Committee Oct 13 they'd give the AI Security Institute pre-release model access 'if legally allowed,' but split on whether safety testing and incident reporting should become binding law. MLex
-
WATCHAnthropic gives METR employee-level access to its own systems
Six METR staff got hardware, office space and internal-tool access on par with Anthropic's own risk teams, plus publication rights Anthropic cannot veto - only narrow redactions, which METR can publicly contest. Anthropic, update on independent evaluators