![\"'We](\"https:\/\/static.toiimg.com\/thumb\/msid-134132453,imgsize-89311,width-400,height-225,resizemode-4\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg\" "\\"Joe")
<\/div>\n<\/div>\n<\/div>\nDays after Anthropic researcher Jacob Coxon resigned, warning that AI companies were \u201cgambling with our lives\u201d in their race to build self-improving superintelligence, another employee from the company\u2019s safety team has also stepped down, raising concerns over the risks posed by increasingly capable AI systems.Joe Benton, the former manager of Anthropic\u2019s Scalable Oversight team, said he left the company\u2019s safety team two weeks ago because he believes frontier AI companies are moving too quickly towards systems that could become far more capable than humans, without adequately addressing the risks.\u201cI left Anthropic’s safety team two weeks ago. Now feels like a good moment to explain why,\u201d Benton wrote in a post on X.<\/p>\n Benton said AI companies were racing to build machines \u201cmuch smarter than any human\u201d and warned that humanity may not survive if the development of such systems continues without stronger safeguards.\u201cAI companies are racing to build machines that are much smarter than any human, and we may not survive this,\u201d he wrote. Benton said he wanted to work outside the company to better inform the public about the risks and help navigate the transition responsibly.In a longer Substack post, Benton said frontier AI companies were racing towards \u201csuperintelligence\u201d, which he described as AI systems capable of recursively improving themselves. He warned that such systems could eventually have drives and desires that diverge from those of human overseers, while developing capabilities that cannot be effectively constrained.\u201cHumanity may not survive this transition,\u201d Benton wrote, arguing that greater preparation was needed to make the development of advanced AI safer.He also warned that AI companies could lose control of their systems or experience an \u201cintelligence explosion\u201d without the public ever knowing. According to Benton, competition between frontier AI firms creates pressure to spend less on safety because companies fear falling behind rivals.<\/p>\n Benton called for greater transparency from AI companies, including disclosure of progress towards recursive self-improvement and reporting of safety incidents and near-misses.He also said companies should have to meet minimum safety standards and undergo independent assessments to verify that they are meeting them.Benton is set to join METR, an independent organisation that evaluates the safety and risks of AI systems. He said independent checks could help change the incentives facing AI companies and encourage them to address safety concerns.His comments came after Anthropic researcher Jacob Coxon announced his resignation earlier this week.<\/p>\n Coxon, who previously worked at OpenAI before joining Anthropic, said he resigned because he believed neither company was acting responsibly in its pursuit of advanced AI.\u201cI resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly,\u201d Coxon said in an X post.He accused the companies of \u201cracing straight to self-improving superintelligence\u201d and \u201cgambling with our lives\u201d.Coxon also warned against underestimating the capabilities of increasingly advanced AI systems, saying they could soon be capable of hacking systems, transforming fields rapidly and acquiring real-world power and resources.Coxon\u2019s resignation drew attention from other Anthropic researchers. Samuel Marks, the company\u2019s scalable-oversight lead, said AI developers believe the technology could cause human extinction or similarly catastrophic outcomes, adding that such an event \u201ccould happen in the next few years\u201d. He also said that \u201cthe more senior the employee, the more concerned they are.\u201dBenton also cited his former manager Evan Hubinger, who has said he believes there is a greater than 10% chance that AI could kill humanity. Benton said many of his Anthropic colleagues were \u201cterrified\u201d by the risks posed by the systems they were building.Benton\u2019s exit adds to a list of AI safety researchers leaving major labs over concerns about safety and the direction of AI development. At OpenAI, former Superalignment co-lead Jan Leike resigned in May 2024, saying he disagreed with the company\u2019s priorities and that safety had taken a back seat to product development.Benton has called for greater transparency from AI companies, including disclosure of AI capability gains, safety incidents and near-misses, along with independent checks to ensure companies meet minimum safety standards.<\/div>\n Joe Benton calls for greater transparency from AI companies. Days after Anthropic researcher Jacob Coxon resigned, warning that AI companies were \u201cgambling with our lives\u201d in their race to build self-improving superintelligence, another employee from the company\u2019s safety team has also stepped down, raising concerns over the risks posed by increasingly capable AI systems.Joe Benton, […]<\/p>\n","protected":false},"author":1,"featured_media":23345,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[31],"tags":[],"class_list":["post-23344","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-31"],"magazineBlocksPostFeaturedMedia":{"thumbnail":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","medium":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","medium_large":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","large":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","1536x1536":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","2048x2048":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","blogsy-small":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","blogsy-small-tall":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","blogsy-small-square":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","blogsy-small-masonry":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","blogsy-medium":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","blogsy-medium-masonry":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","blogsy-large":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg","blogsy-wide":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg"},"magazineBlocksPostAuthor":{"name":"admin","avatar":"https:\/\/secure.gravatar.com\/avatar\/8709732a479614e7a8aa24d3eb1b239f30dc6d90c61464ed495001e7a469d856?s=96&d=mm&r=g"},"magazineBlocksPostCommentsNumber":"0","magazineBlocksPostExcerpt":"Joe Benton calls for greater transparency from AI companies. Days after Anthropic researcher Jacob Coxon resigned, warning that AI companies were \u201cgambling with our lives\u201d in their race to build self-improving superintelligence, another employee from the company\u2019s safety team has also stepped down, raising concerns over the risks posed by increasingly capable AI systems.Joe Benton, […]","magazineBlocksPostCategories":["\u0b26\u0b47\u0b36 \u0b2c\u0b3f\u0b26\u0b47\u0b36"],"magazineBlocksPostViewCount":1,"magazineBlocksPostReadTime":4,"magazine_blocks_featured_image_url":{"full":["https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg",400,225,false],"medium":["https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg",300,169,false],"thumbnail":["https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/joe-benton-calls-for-greater-transparency-from-ai-companies.jpg",150,84,false]},"magazine_blocks_author":{"display_name":"admin","author_link":"https:\/\/koshalsambada.in\/author\/admin"},"magazine_blocks_comment":0,"magazine_blocks_author_image":"https:\/\/secure.gravatar.com\/avatar\/8709732a479614e7a8aa24d3eb1b239f30dc6d90c61464ed495001e7a469d856?s=96&d=mm&r=g","magazine_blocks_category":"\u0b26\u0b47\u0b36 \u0b2c\u0b3f\u0b26\u0b47\u0b36<\/a>","_links":{"self":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/posts\/23344","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=23344"}],"version-history":[{"count":0,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/posts\/23344\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/media\/23345"}],"wp:attachment":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=23344"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=23344"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=23344"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\u2018We may not survive this\u2019<\/b><\/h3>\n<\/p>\n
Benton calls for independent AI safety checks<\/b><\/h3>\n<\/p>\n
Why did Jacob Coxon quit Anthropic?<\/b><\/h3>\n<\/p>\n
\n
Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"