![\"Researchers](\"https:\/\/static.toiimg.com\/thumb\/msid-134555046,imgsize-46266,width-400,height-225,resizemode-4\/michael-burry.jpg\")
<\/div>\n<\/div>\n<\/div>\n<\/section>\n<\/div><\/div>\n<\/div>\nResearchers at Oxford and Cambridge have identified a fundamental flaw threatening the future of large language models like ChatGPT, and “Big Short” investor Michael Burry says the finding backs up a warning he’s been making about the technology’s core limitations for a while. A new study from the University of Oxford and University of Cambridge has revealed a major threat to large language models like ChatGPT, calling it \u201cmodel collapse.\u201d The research shows that when generative AI systems are repeatedly trained on synthetic data produced by earlier models, they begin to lose the \u201ctails\u201d of the original data distribution. In practice, this means models forget the creative, fringe, and unique nuances of human writing, collapsing into repetitive echo chambers.America\u2019s biggest investor, Michael Burry, reacted to the findings on X, saying: \u201cThis is to my point about compression being inevitable as human knowledge is too small for what we are building. AI-generated content will clearly contain propagation errors\u2026 LLMs will iterate those propagation errors infinitely faster with less ability to self-correct.\u201d<\/p>\n Burry drew attention to the research by sharing a post outlining what scientists call “model collapse.” The concern centers on a shift already underway across the internet: as generative AI produces more and more of the text online, future AI models risk being trained on data that was itself generated by earlier AI systems, rather than by humans.According to the research, feeding AI-generated content back into training indiscriminately causes what researchers describe as irreversible defects in the resulting model. Specifically, the model begins losing the “tails” of the original data distribution, meaning it gradually forgets the creative, unusual, and distinctly human nuances present in real human writing. Over successive generations, this causes the model to collapse into an increasingly narrow, repetitive echo chamber of its own outputs.Crucially, the researchers found this isn’t unique to ChatGPT or language models specifically. Their theoretical framework showed the collapse phenomenon is ubiquitous across learned generative models more broadly, appearing in large language models as well as in variational autoencoders and Gaussian mixture models.The stakes are significant for the entire AI industry, given how heavily tech companies rely on scraping large volumes of internet data to train increasingly capable models. The research warns that if companies want to sustain the benefits of training on web data going forward, model collapse needs to be taken seriously as a genuine constraint rather than a theoretical curiosity. The clear takeaway, as the research frames it: genuine, human-generated data is going to become an increasingly scarce and valuable resource in a web that’s rapidly filling up with AI-generated content.<\/p>\n Writing on X, Burry tied the research directly to an argument he’s made before: that compression of available information is essentially inevitable, because human knowledge itself is too limited relative to the scale of what AI companies are trying to build, and because human wants, needs, and questions are inherently repetitive and redundant.He went further, warning that AI-generated content will inevitably carry forward “propagation errors,” much like inaccuracies that have accumulated throughout human knowledge over generations \u2014 except AI systems, in his view, will be capable of iterating and reproducing those errors infinitely faster, with far less capacity to self-correct due to what he sees as a lack of genuine understanding.<\/p>\n Burry used the moment to make a broader, more philosophical argument about artificial general intelligence. He argued that large language models cannot attain true understanding \u2014 and therefore cannot reach AGI \u2014 because understanding, in his view, cannot exist unless reasoning exists independently of language first, something he called a likely impossibility for a system built entirely around language prediction. He acknowledged that researchers are already exploring ways to work around this limitation, while noting that many in the field haven’t accepted his underlying premise in the first place.<\/p>\n Burry’s skepticism lands in direct contrast to comments made earlier this month by Nvidia CEO Jensen Huang, who declared that AGI has already arrived, citing OpenAI’s GPT-6 Astra model as proof. Huang’s claim drew immediate pushback from parts of the AI research community.Stanford’s Institute for Human-Centered AI defines AGI as AI capable of learning, reasoning, and applying knowledge across a broad range of tasks at or beyond human-level performance \u2014 critically, with the ability to adapt to unfamiliar situations, unlike narrow AI built for specific tasks. The concept remains genuinely unsettled within the field, with no universally accepted definition or standardized test for confirming when AGI has actually been reached, alongside ongoing debates over its safety and ethical implications.<\/div>\n Researchers at Oxford and Cambridge have identified a fundamental flaw threatening the future of large language models like ChatGPT, and “Big Short” investor Michael Burry says the finding backs up a warning he’s been making about the technology’s core limitations for a while. A new study from the University of Oxford and University of Cambridge […]<\/p>\n","protected":false},"author":1,"featured_media":20924,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[31],"tags":[],"class_list":["post-30362","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-31"],"magazineBlocksPostFeaturedMedia":{"thumbnail":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","medium":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","medium_large":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","large":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","1536x1536":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","2048x2048":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","blogsy-small":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","blogsy-small-tall":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","blogsy-small-square":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","blogsy-small-masonry":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","blogsy-medium":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","blogsy-medium-masonry":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","blogsy-large":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg","blogsy-wide":"https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg"},"magazineBlocksPostAuthor":{"name":"admin","avatar":"https:\/\/secure.gravatar.com\/avatar\/8709732a479614e7a8aa24d3eb1b239f30dc6d90c61464ed495001e7a469d856?s=96&d=mm&r=g"},"magazineBlocksPostCommentsNumber":"0","magazineBlocksPostExcerpt":"Researchers at Oxford and Cambridge have identified a fundamental flaw threatening the future of large language models like ChatGPT, and “Big Short” investor Michael Burry says the finding backs up a warning he’s been making about the technology’s core limitations for a while. A new study from the University of Oxford and University of Cambridge […]","magazineBlocksPostCategories":["\u0b26\u0b47\u0b36 \u0b2c\u0b3f\u0b26\u0b47\u0b36"],"magazineBlocksPostViewCount":2,"magazineBlocksPostReadTime":4,"magazine_blocks_featured_image_url":{"full":["https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg",400,225,false],"medium":["https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg",300,169,false],"thumbnail":["https:\/\/koshalsambada.in\/wp-content\/uploads\/2026\/09\/michael-burry.jpg",150,84,false]},"magazine_blocks_author":{"display_name":"admin","author_link":"https:\/\/koshalsambada.in\/author\/admin"},"magazine_blocks_comment":0,"magazine_blocks_author_image":"https:\/\/secure.gravatar.com\/avatar\/8709732a479614e7a8aa24d3eb1b239f30dc6d90c61464ed495001e7a469d856?s=96&d=mm&r=g","magazine_blocks_category":"\u0b26\u0b47\u0b36 \u0b2c\u0b3f\u0b26\u0b47\u0b36<\/a>","_links":{"self":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/posts\/30362","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=30362"}],"version-history":[{"count":0,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/posts\/30362\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=\/wp\/v2\/media\/20924"}],"wp:attachment":[{"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=30362"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=30362"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/koshalsambada.in\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=30362"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}The problem: models trained on their own output<\/h2>\n<\/p>\n
Burry’s own warning, echoed and expanded<\/h2>\n<\/p>\n
A deeper claim: LLMs may never reach AGI<\/h2>\n<\/p>\n
Setting up a clash with Jensen Huang<\/h2>\n<\/p>\n
\n
Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"