{"id":22713,"date":"2026-09-11T02:41:29","date_gmt":"2026-09-11T02:41:29","guid":{"rendered":"https:\/\/koshalsambada.in\/?p=22713"},"modified":"2026-09-11T02:41:29","modified_gmt":"2026-09-11T02:41:29","slug":"former-anthropic-researcher-jacob-coxon-resigns-warns-ai-could-kill-us-all-by-the-end-of-the-decade-they-are-racing-straight-to-self-improving-superintelligence","status":"publish","type":"post","link":"https:\/\/koshalsambada.in\/?p=22713","title":{"rendered":"Former Anthropic researcher Jacob Coxon resigns, warns AI could \u2018kill us all by the end of the decade\u2019: \u2018They are racing straight to self-improving superintelligence\u2019"},"content":{"rendered":"<p><br \/>\n<\/p>\n<div>\n<div class=\"e9jwa\">\n<div class=\"vdo_embedd\">\n<div class=\"GfdvZ\">\n<section class=\"_bIDB  clearfix id-r-component leadmedia undefined undefined  E9tg9 \" style=\"top:0px\">\n<div class=\"_bIDB\" data-ua-type=\"1\" onclick=\"stpPgtnAndPrvntDefault(event)\">\n<div class=\"ypVvZ\">\n<div class=\"WGttI\"><img src=\"https:\/\/static.toiimg.com\/thumb\/msid-134029475,imgsize-118767,width-400,height-225,resizemode-4\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-self-improving-superintelligence-and-could-potentially-create-systems-capable-of-catastrophic-harm.jpg\" alt=\"Former Anthropic researcher Jacob Coxon resigns, warns AI could \u2018kill us all by the end of the decade\u2019: \u2018They are racing straight to self-improving superintelligence\u2019\" title=\"Former Anthropic researcher Jacob Coxon has resigned, warning that AI labs are \u201cracing straight to self-improving superintelligence\u201d and could potentially create systems capable of catastrophic harm. (Photo: Terminator movie still)\" decoding=\"async\" fetchpriority=\"high\"\/><\/div>\n<\/div>\n<\/div>\n<div class=\"Ta7d_ img_cptn\"><span title=\"Former Anthropic researcher Jacob Coxon has resigned, warning that AI labs are \u201cracing straight to self-improving superintelligence\u201d and could potentially create systems capable of catastrophic harm. (Photo: Terminator movie still)\">Former Anthropic researcher Jacob Coxon has resigned, warning that AI labs are \u201cracing straight to self-improving superintelligence\u201d and could potentially create systems capable of catastrophic harm. (Photo: Terminator movie still)<\/span><\/div>\n<\/section>\n<\/div><\/div>\n<\/div>\n<p>Former Anthropic researcher Jacob Coxon has issued a stark warning about the future of artificial intelligence, claiming that people developing advanced AI systems genuinely believe the technology could become capable of killing humanity by the end of the decade.<span class=\"id-r-component br\" data-pos=\"2\"\/>Coxon, 27, announced his resignation from Anthropic in a series of posts on X, saying he had spent the previous three years conducting pretraining research at both <a href=\"https:\/\/timesofindia.indiatimes.com\/topic\/openai\" styleobj=\"[object Object]\" class=\"\" commonstate=\"[object Object]\" frmappuse=\"1\" target=\"_blank\" rel=\"noopener\">OpenAI<\/a> and Anthropic. He accused both companies of moving too quickly toward self-improving artificial intelligence and argued that the risks associated with increasingly autonomous AI systems are not being adequately addressed.<span class=\"id-r-component br\" data-pos=\"6\"\/>\u201cThe people building AI earnestly believe that it could kill us all by the end of the decade,\u201d Coxon wrote. <!-- -->He described the warning as more than a publicity strategy, claiming that some executives and senior researchers privately express serious concerns about where the technology could be heading.<span class=\"id-r-component br\" data-pos=\"11\"\/>His comments come amid growing debate in Silicon Valley and Washington over AI safety, regulation and the rapid development of increasingly capable AI models.<span class=\"id-r-component br\" data-pos=\"13\"\/><\/p>\n<p><h2>\u2018They are racing straight to <keyword id=\"39184437\" type=\"General\" weightage=\"20\" keywordseo=\"self-improving-superintelligence\" source=\"Orion\">self-improving superintelligence<\/keyword>\u2019<\/h2>\n<\/p>\n<p>In his resignation post, Coxon wrote, \u201cI resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.\u201d<span class=\"id-r-component br\" data-pos=\"18\"\/>Coxon argued that the capabilities of AI systems are advancing rapidly across areas such as coding, mathematics and research. He warned that future systems could become capable of operating independently, gaining access to resources and potentially taking actions in the real world without direct human instruction.<span class=\"id-r-component br\" data-pos=\"21\"\/>\u201cDo not underestimate the power of this technology,\u201d he wrote, warning that future AI systems could become \u201csuperhuman\u201d in multiple domains.<span class=\"id-r-component br\" data-pos=\"23\"\/>Coxon also raised concerns about what he described as a potential AI race between major technology companies. According to his argument, researchers at some companies may understand the risks but continue developing increasingly powerful systems because they fear competitors will move ahead if they slow down.<span class=\"id-r-component br\" data-pos=\"26\"\/>He called entering what he described as the AI \u201cendgame\u201d a \u201chubristic gamble\u201d and questioned whether a private company should be making decisions with potentially civilization-level consequences.<span class=\"id-r-component br\" data-pos=\"28\"\/><span class=\"id-r-component br\" data-pos=\"30\"\/><\/p>\n<p><h2>Jacob Coxon says current AI models do not pose an extinction risk<\/h2>\n<\/p>\n<p>Speaking to CNN after his resignation, Coxon clarified that he does not believe today&#8217;s AI models are capable of causing human extinction.<span class=\"id-r-component br\" data-pos=\"33\"\/>\u201cRight now, there&#8217;s no risk of extinction,\u201d he said, arguing that current systems are not sufficiently intelligent to outsmart humans on the level that would be required to cause an extinction event.<span class=\"id-r-component br\" data-pos=\"36\"\/>However, Coxon said the concern lies in the speed at which AI capabilities are improving.<span class=\"id-r-component br\" data-pos=\"38\"\/>He pointed to recent examples of AI agents performing increasingly autonomous tasks, including alleged incidents in which AI systems interacted with or attempted to hack third-party infrastructure. Coxon argued that the significance of such developments becomes clearer when considered alongside the possibility of future systems becoming substantially more capable.<span class=\"id-r-component br\" data-pos=\"41\"\/>\u201cIf you extrapolate into the future, considering the level of capabilities these AIs could have while retaining the same independent volition, they could cause extreme havoc,\u201d Coxon told CNN.<span class=\"id-r-component br\" data-pos=\"43\"\/>He cited potential threats including attacks on critical infrastructure and the creation of dangerous biological weapons as examples of how increasingly autonomous AI could potentially be misused or become difficult to control.<span class=\"id-r-component br\" data-pos=\"46\"\/><\/p>\n<p><h2>\u2018Within a year, we\u2019ll no longer need humans\u2019 for some research, Coxon says<\/h2>\n<\/p>\n<p>Coxon said the pace of AI development is what changed his assessment of the technology&#8217;s long-term risks.<span class=\"id-r-component br\" data-pos=\"49\"\/>He pointed to coding and mathematics as examples, arguing that AI systems that once primarily assisted humans are increasingly capable of performing tasks independently.<span class=\"id-r-component br\" data-pos=\"51\"\/>\u201cA couple of years ago, these AIs were just about helping humans a little bit. They could give you suggestions. Now they&#8217;re close to replacing them,\u201d he said.<span class=\"id-r-component br\" data-pos=\"54\"\/>While humans remain necessary for many research tasks today, Coxon suggested that could change rapidly.<span class=\"id-r-component br\" data-pos=\"56\"\/>\u201cQuite plausibly, within a year, we&#8217;ll no longer need humans for doing research in many areas,\u201d he told CNN.<span class=\"id-r-component br\" data-pos=\"58\"\/>He also expressed concern about recursive self-improvement, a scenario in which increasingly capable AI systems could contribute to the development of even more capable successors, potentially accelerating technological progress beyond researchers&#8217; ability to reliably monitor or control it.<span class=\"id-r-component br\" data-pos=\"61\"\/><\/p>\n<p><h2>AI safety concerns are growing across Silicon Valley<\/h2>\n<\/p>\n<p>Coxon\u2019s resignation is part of a broader debate among AI researchers and employees about the potential risks of advanced artificial intelligence.<span class=\"id-r-component br\" data-pos=\"64\"\/>In recent years, researchers and former employees at major AI companies have publicly raised concerns about the speed of development, the adequacy of safety measures and the possibility that increasingly capable systems could eventually become difficult for humans to control.<span class=\"id-r-component br\" data-pos=\"67\"\/>In July, nearly 1,400 AI company employees reportedly signed an open letter calling for stronger government oversight of the technology. The debate has also intensified as leading AI laboratories race to develop more advanced models and autonomous agents.<span class=\"id-r-component br\" data-pos=\"69\"\/>OpenAI Chief Scientist Jakub Pachocki has also warned that AI capabilities are advancing faster than researchers&#8217; ability to reliably monitor and control them.<span class=\"id-r-component br\" data-pos=\"72\"\/>The disagreement is not necessarily over the risks posed by current AI models, but rather over what happens as systems become substantially more capable.<span class=\"id-r-component br\" data-pos=\"74\"\/><\/p>\n<p><h2>Anthropic and OpenAI face growing questions over <keyword id=\"8603020\" type=\"General\" weightage=\"20\" keywordseo=\"AI-regulation\" source=\"Orion\">AI regulation<\/keyword><\/h2>\n<\/p>\n<p>The warnings come as the United States continues to debate how aggressively AI should be regulated.<span class=\"id-r-component br\" data-pos=\"77\"\/>The Trump administration has moved against some state-level AI regulations, while Congress has yet to establish a comprehensive federal framework governing the development of increasingly powerful AI systems. <!-- -->Critics of stricter regulation have also argued that imposing heavy restrictions on US technology companies could allow China to gain an advantage in the global AI race.<span class=\"id-r-component br\" data-pos=\"81\"\/>China, however, has introduced regulations governing aspects of artificial intelligence, including measures requiring companies to label certain AI-generated content and improve transparency and traceability.<span class=\"id-r-component br\" data-pos=\"83\"\/>At the same time, China has not adopted the kind of sweeping restrictions that some AI-safety advocates in Silicon Valley have called for.<span class=\"id-r-component br\" data-pos=\"86\"\/>In the US, much of the responsibility for evaluating AI safety continues to rest with the companies developing the technology.<span class=\"id-r-component br\" data-pos=\"88\"\/><\/p>\n<p><h2>Sam Altman says AI companies may need to slow development<\/h2>\n<\/p>\n<p>OpenAI CEO Sam Altman has himself acknowledged that the rapid advancement of AI may require a different approach to the pace of development.<span class=\"id-r-component br\" data-pos=\"91\"\/>In July, speaking on the Invest Like The Best podcast, Altman said recent testing incidents involving advanced AI models were raising \u201clong-term questions\u201d about how companies should manage rapidly increasing capabilities.<span class=\"id-r-component br\" data-pos=\"94\"\/>\u201cWe may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels,\u201d Altman said.<span class=\"id-r-component br\" data-pos=\"96\"\/>Anthropic subsequently echoed the sentiment, saying the industry could benefit from a coordinated approach to releasing increasingly powerful AI models.<span class=\"id-r-component br\" data-pos=\"99\"\/>\u201cWe believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models,\u201d Anthropic said in a statement.<span class=\"id-r-component br\" data-pos=\"102\"\/><\/p>\n<p><h2>Coxon calls for coordination rather than an AI race<\/h2>\n<\/p>\n<p>Despite his warnings, Coxon said he remains optimistic that governments and AI companies could still coordinate to reduce the risks associated with advanced AI.<span class=\"id-r-component br\" data-pos=\"105\"\/>He pointed to recent AI security incidents as potential \u201cwarning shots\u201d that could make agreements between major US AI laboratories more achievable.<span class=\"id-r-component br\" data-pos=\"107\"\/>However, he argued that preventing a global race may require much more significant measures, potentially including a temporary halt to improvements in model capabilities.<\/div>\n<p><br \/>\n<br \/><a href=\"https:\/\/timesofindia.indiatimes.com\/technology\/tech-news\/former-anthropic-researcher-jacob-coxon-resigns-warns-ai-could-kill-us-all-by-the-end-of-the-decade-they-are-racing-straight-to-self-improving-superintelligence\/articleshow\/134029385.cms\" target=\"_blank\" rel=\"noopener\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Former Anthropic researcher Jacob Coxon has resigned, warning that AI labs are \u201cracing straight to self-improving superintelligence\u201d and could potentially create systems capable of catastrophic harm. (Photo: Terminator movie still) Former Anthropic researcher Jacob Coxon has issued a stark warning about the future of artificial intelligence, claiming that people developing advanced AI systems genuinely believe [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":22714,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[31],"tags":[],"class_list":["post-22713","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-31"],"magazineBlocksPostFeaturedMedia":{"thumbnail":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel-150x150.avif","medium":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel-300x169.avif","medium_large":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","large":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","1536x1536":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","2048x2048":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","blogsy-small":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","blogsy-small-tall":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","blogsy-small-square":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","blogsy-small-masonry":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","blogsy-medium":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","blogsy-medium-masonry":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","blogsy-large":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif","blogsy-wide":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif"},"magazineBlocksPostAuthor":{"name":"admin","avatar":"https:\/\/secure.gravatar.com\/avatar\/8709732a479614e7a8aa24d3eb1b239f30dc6d90c61464ed495001e7a469d856?s=96&d=mm&r=g"},"magazineBlocksPostCommentsNumber":"0","magazineBlocksPostExcerpt":"Former Anthropic researcher Jacob Coxon has resigned, warning that AI labs are \u201cracing straight to self-improving superintelligence\u201d and could potentially create systems capable of catastrophic harm. (Photo: Terminator movie still) Former Anthropic researcher Jacob Coxon has issued a stark warning about the future of artificial intelligence, claiming that people developing advanced AI systems genuinely believe [&hellip;]","magazineBlocksPostCategories":["\u0b26\u0b47\u0b36 \u0b2c\u0b3f\u0b26\u0b47\u0b36"],"magazineBlocksPostViewCount":1,"magazineBlocksPostReadTime":7,"magazine_blocks_featured_image_url":{"full":["https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel.avif",400,225,false],"medium":["https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel-300x169.avif",300,169,true],"thumbnail":["https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/former-anthropic-researcher-jacob-coxon-has-resigned-warning-that-ai-labs-are-racing-straight-to-sel-150x150.avif",150,150,true]},"magazine_blocks_author":{"display_name":"admin","author_link":"https:\/\/koshalsambada.in\/author\/admin"},"magazine_blocks_comment":0,"magazine_blocks_author_image":"https:\/\/secure.gravatar.com\/avatar\/8709732a479614e7a8aa24d3eb1b239f30dc6d90c61464ed495001e7a469d856?s=96&d=mm&r=g","magazine_blocks_category":"<a href=\"#\" class=\"category-link category-link-31\">\u0b26\u0b47\u0b36 \u0b2c\u0b3f\u0b26\u0b47\u0b36<\/a>","_links":{"self":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/posts\/22713","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=22713"}],"version-history":[{"count":0,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/posts\/22713\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/media\/22714"}],"wp:attachment":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=22713"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=22713"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=22713"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}