The AI boom has turned the standard profit margin model on its head, according to Apollo Chief Economist Torsten Slok—and it’s making the industry’s growth unsustainable.
It is true there were massive strides in the last 8 years. But fundamentally, the tech is still the same large language model it was before, just bigger and better optimized.
It’s like going from an ancient, slow, Ford Model T that topped out at 45mph to a Bugatti that can do 260mph in 8 short years. It’s impressive, it boosts productivity, it is a marvel of modern technology, but that’s not my point of contention.
The issue is that AI companies have been funded on the promise that with enough advancements and upgrades this tech will achieve AGI. Which is an absurd statement to anyone in the field actually developing these things. That is the equivalent of promising that this 260mph Bugatti, with a few years of upgrades and advancements, will become a Harrier Jump Jet!
It’s just not happening, a fundamental shift in model architecture or technology used is needed. And from what we’ve seen so far, no one has discovered any.
They’re not even better optimized. Hallucination rates are up. Inference costs are up. There’s only an AI industry at all because they’re selling a highly subsidized product, but when they try to raise prices even a little the market collapses. Companies that were encouraging employees to up their AI use are now rationing tokens like chocolate in wartime. This isn’t like Uber where they can push out the old providers and then obtain market capture on something everyone needs. AI is not, and cannot be, essential, because you can always just get a human to do it.
I disagree. AI is enabling doing things that are not worth paying a human to do, yet still have value. the question is, whether the infrastructure costs can realign to hit that sweet spot.
I only have pretty basic machine learning knowledge, based on a few Master’s classes at college while I was doing gamedev, and even with that I can tell that the way they are expecting to get AGI by just feeding more data into a language model is simply not happening.
I remember a comparison from one of the AI-pilled tech-bros when AI was starting to get attention, and his metaphor about AGI was something along the lines of “Imagine a difference between a medieval commoner and Albert Einstein, that’s the difference in inteligence AI will soon have to the smartest people we know now”.
But that doesn’t make sense with the current approach. Imagine Einstein writing his cutting-edge theories, and the commoner is watching him behind his shoulder and vetting anything he does. Scratching his ideas, forcing him to redo it, if he doesn’t like it, pointing to a reddit thread about why. There’s no way he would ever finish anything new.
Unless they figure out a completely new way how to do AI reasoning, there is no way we’re getting anywhere near AGI. And that is also becoming more unlikely the longer we go with this approach, because every AI-pilled company is heavily outsourcing all of development to the current models, reducing their employee (and the whole worlds) innovation potential and skill. This is probably the last generation that can do serious academia, unless there are drastic measures done to limit access to AI in education. It’s fucked.
The issue is that AI companies have been funded on the promise that with enough advancements and upgrades this tech will achieve AGI. Which is an absurd statement to anyone in the field actually developing these things. That is the equivalent of promising that this 260mph Bugatti, with a few years of upgrades and advancements, will become a Harrier Jump Jet!
A better analogy is promising that a toddler will, with enough knowledge and training, eventually become a heart surgeon.
15 years ago the assumption with AGI was that we needed some paradigm shift in technology to create it, but after transformer technology was invented (the T in GPT), and it was trained on a lot of information, we discovered an emergent property that it could take natural language queries and answer them with its knowledge base-- which was unexpected and unintended.
While science can always end up going down the wrong path, the current mainstream stance is that we were wrong about needing new technology for AGI; it seems that AGI may be a function of information and training, on hardware we already have. Hence all the data centers being built.
Anyone who says with certainty that it will result in AGI is just as wrong as someone who says with certainty that it won’t.
That is where the evidence points. Now, I don’t want to oversell it: “where the evidence points” is wildly different than “exactly how it works”.
We have an emergent property that we don’t understand, but we can reliably increase the functionally and complexity of that emergent property as a function of training data and available compute. Does that mean that there isn’t some threshold where that stops working? No, there certainly could be a point where throwing more information and compute has no effect. We just don’t know. However, so far, there is no evidence such a barrier exists, and everyone is racing to find out.
I think that is where the hopes and hype point. The evidence, that which is gathered through controlled studies, points to an upper limit to this technology that does have emergent properties, but not ones that amount to cognition. That evidence also points to other hurdles, such as cognitive damage to the user and context windows nowhere near that of even a simpleminded creature, let alone a sapient one like a human.
The core problem is that these models are fixed; they are the same on day 1000 as they were on day 1. All their “learning”, as it were, happens in training before it is released. Everything it appears to learn after that date is contained in the rolling context window. Since they already have literally all the RAM they can get their hands on and are still at least an order of magnitude away from where they need to be on that, this technology either can’t do it or, at best, is so inefficient an approach that it can’t be done with all the planet’s resources.
Sometimes, especially in abstract constructions like software, you can start down the wrong path early and have to start over, because there is no path from where you are to where you need to get. In this case, they may have done that to the extreme, blinded by the lucrative prospects.
As far as I am aware, AGI does not imply sapience, consciousness, or whatever. You seem to believe it does. A high level Google search seems to suggest that I’m correct. Is there some reason you are lumping sapience with AGI?
I’m not sure if the rest of your comment presupposes this, so I’ll wait to comment on where I think you’re mistaken.
I dont believe it requires sapience; that is what the marketers are saying. The AI boom (and many historic boom cycles) is predicated on marketing and sentiment, not facts and data. That is why they always pop; the facts dont back up the hype.
The fact that your search results turn up results that align with the marketing is just the marketing working.
Look–neither of us are data scientists, but we do have eyes. If this is working, where’s the company with runaway success creating unimagined leaps in productivity and technology? We’re pouring a whole planet’s with of resources in and nothing much is coming out. If we spent this much on world hunger, everyone would be obese by now.
I dont believe it requires sapience; that is what the marketers are saying. The AI boom (and many historic boom cycles) is predicated on marketing and sentiment, not facts and data. That is why they always pop; the facts dont back up the hype.
I think there is some confusion. I do not believe AGI implies sapience, nor does Google, and now it seems that you don’t either. So who is discussing sapience?
If this is working, where’s the company with runaway success creating unimagined leaps in productivity and technology?
Hypothetically speaking, what do you think this would look like?
The marketers are not using the word “sapience”, as that word would require defining for the layman. They are claiming AI will usher in an era of human prosperity by taking over our work and big decisions to do them more efficiently–tasks that sound to almost everyone to require advanced decision-making and thought, the hallmarks of sapience and cognition. This is largely besides the point, though.
I think that if this were successful, you would see at least some early adopting companies coming up aces. Some company would have double the productivity with half the workforce, and use that to absolutely devour their market segment by undercutting everyone else on price because their operating costs are so much lower than the competition. I’d see a software company adopt breakneck software release cycles with substantial, material stability and performance improvements with each. I’d see some company anywhere suddenly begin to outperform their former selves and be able to convincingly point to AI as where their improvements came from. The only one like that I can think of is NVidia, who are rich strictly because they’re selling shovels in the gold rush.
Vishal Sikka, advisory board member of BMW. Recommended to Stanford by Marvin Minsky, one of his professors were John McCarty. And I must stand corrected, he has a PHD of computer sciences, not Math as I remembered it as.
Vishal has an AI based company himself, so there might be some personal reasons for why he’d advocate for using what AIs capable of rather than chasing an impossible (from his perspective) to hit milestone
That paper doesn’t seem to rule out AGI, only an single LLM model that can answer every arbitrarily difficult question on demand.
AGI does not necessarily mean one model acting alone, or being able to answer any question on demand. Humans are the same way: we often need time or collaboration to arrive at conclusions, but that doesn’t mean we don’t have “general intelligence”.
You’re not going to context hack complexity, every time you summarise something and pass it onto the next agent. You’re losing complexity.
We’re going to get some massive models with incredible context, but AGI requires models to never hallucinate. Even when the complexity requires more context than it has available, which is not feasible
We’ve already had a month now of Psychiatrists writing articles backpedaling over all the times they’d insisted AI couldn’t genuinely be thinking as we do and moving the goal posts to something else. Maybe there are researchers in the frontier labs who feel like they have a grip on what’s happening, but the ‘experts’ on forums have no idea.
The issue is that AI companies have been funded on the promise that with enough advancements and upgrades this tech will achieve AGI. Which is an absurd statement to anyone in the field actually developing these things.
You genuinely believe that the people in frontier labs under such severe NDAs we’ve had articles about how unusual they are believe it’s absurd to think that what they’re working on could lead to “AGI”?
What would the goal of that be in your mind? Getting as much investment as humanly possible before everyone figures out the leading specialists have been being paid insane amounts of money just to act like what they’re doing matters and the valuations all plummet to nothing over night? Why?
With all the money poured in to this from all the sources it’s coming from that would end in a lot of the most powerful companies, government bodies, and individuals in the world extremely unhappy and blaming the people running those frontier labs.
Getting a lot of news and attention and then saying “psych, lol” and being imprisoned for life or murdered doesn’t seem like a great long term plan.
I agree with all of those things, but for the last one I see a big example in all of the people not taking the news of a demonstrable global workspace in AI, including models that have been around for years now, as anything big.
People have been insisting that AI can’t possibly actually be thinking or have any of the most vaunted aspects of the human mind and that LLMs are a dead end that won’t go anywhere for years. We want to be different and special and unique. We always have.
Many of us here went to school when it was still taught that animals weren’t really conscious and couldn’t be self-aware. Being unwilling to look at new information and honestly assess is hubris and insistence on holding to the thing you’ve been insisting, it’s not a logical analysis.
Over the years we’ve pointed to several things to insist that AI wasn’t special like us, and that it couldn’t possibly be because it lacked aspects of the human mind needed for that. Somehow it kept turning out that every thing we pointed to in ourselves ended up being shown to have a very surprisingly similar to nearly direct analogue in those modern LLMs. Last month was jaw dropping, but the bulk of the world has been so focused on the entire world going to hell it hasn’t gotten near the attention it deserved yet.
What does it mean to think? This is a open-ended philosophical question and has any number of answers.
I completely agree with your 2nd and 3rd paragraph.
There are similarities and there are clear differences. The reason why we say LLMs are different from us is because we DO have general intelligence, LLMs have not displayed such capabilities despite the intense pressure to prove so. THIS is why we say LLMs couldn’t possibly be like us. Because evidence shows they aren’t, despite the similarities.
I would be interested to hear what developments in the last month you’re referring to, would be exciting to see a breakthrough.
Global workspace. The leading theory of consciousness. It’s a thing we can’t show empirically in ourselves, but can now see and experiment on something that matches it’s description in AI. Not just the proprietary latest models, the years old models anyone can download from the internet. No one built it, it was something else that emerged somehow through training and went unnoticed.
Here’s someone trying to backpedal into saying the thing that really matters actually isn’t genuinely thinking, now it’s… living with your mistakes. That’s how far the goal posts have been kicked over the last 6 months ending in this one. Not genuinely being able to think, rational self-awareness. Just remembering when you messed up a few months ago. And it’s such a weird pivot because AI memory is a design aspect that can be changed, something external frameworks to enhance with databases already exists for, a large part limited in current design because the more you send in the more it costs to process so it’s always capped, and also… sort of unnatural.
Running AI models the way we do with “frozen weights” isn’t mandatory. It takes a lot more hardware to do it, but it’s possible to run AI in a way more like they run during it’s training. The model files themselves would be unfrozen and allowed to change as you communicate. It’s reportedly something that’s had issues with the models forgetting things even as big as the language you’re talking in, but also a thing that is sometimes used during the alignment process. And one of the major reasons the big companies don’t care to look in to it is because if you have 10,000 people communicating with an AI over the internet and all telling it to do different things and act different ways and all of those things can be learning on the level of the model files themselves instead of confined to external temporary context windows it’s going to go crazy.
Memory on on the actual model level isn’t nonexistent, it’s what the entire training process is based on. It just wouldn’t make a good consumer product for sale, so parts of the models are removed before they’re made available online for the ones that are.
Everything you described would be a small improvement, not a leap towards AGI that is needed. That is why it isn’t being researched further. Massive investment for minimal gain.
There isn’t some massive leap needed. That’s what I’m saying. Everything we’ve used to try to show it’s not possible for the LLMs to have the most important aspects of human cognition have been shown to be present in some near or direct analogue. I linked the article in Psychology Today because memory is the sort of thing that has become the last hold out over all of those other words, and memory is a pure design choice that can be altered in many ways.
The insistence that LLMs of today could never lead to AGI seems based on out dated and incomplete understanding of them than people have failed to update as research has advanced. There are very real reasons for the frontier researchers who likely know more on these topics than has become available publicly in research to continue to focus on them.
Disagree.
It is true there were massive strides in the last 8 years. But fundamentally, the tech is still the same large language model it was before, just bigger and better optimized.
It’s like going from an ancient, slow, Ford Model T that topped out at 45mph to a Bugatti that can do 260mph in 8 short years. It’s impressive, it boosts productivity, it is a marvel of modern technology, but that’s not my point of contention.
The issue is that AI companies have been funded on the promise that with enough advancements and upgrades this tech will achieve AGI. Which is an absurd statement to anyone in the field actually developing these things. That is the equivalent of promising that this 260mph Bugatti, with a few years of upgrades and advancements, will become a Harrier Jump Jet!
It’s just not happening, a fundamental shift in model architecture or technology used is needed. And from what we’ve seen so far, no one has discovered any.
They’re not even better optimized. Hallucination rates are up. Inference costs are up. There’s only an AI industry at all because they’re selling a highly subsidized product, but when they try to raise prices even a little the market collapses. Companies that were encouraging employees to up their AI use are now rationing tokens like chocolate in wartime. This isn’t like Uber where they can push out the old providers and then obtain market capture on something everyone needs. AI is not, and cannot be, essential, because you can always just get a human to do it.
I disagree. AI is enabling doing things that are not worth paying a human to do, yet still have value. the question is, whether the infrastructure costs can realign to hit that sweet spot.
Exactly this, especially the last sentence.
I only have pretty basic machine learning knowledge, based on a few Master’s classes at college while I was doing gamedev, and even with that I can tell that the way they are expecting to get AGI by just feeding more data into a language model is simply not happening.
I remember a comparison from one of the AI-pilled tech-bros when AI was starting to get attention, and his metaphor about AGI was something along the lines of “Imagine a difference between a medieval commoner and Albert Einstein, that’s the difference in inteligence AI will soon have to the smartest people we know now”.
But that doesn’t make sense with the current approach. Imagine Einstein writing his cutting-edge theories, and the commoner is watching him behind his shoulder and vetting anything he does. Scratching his ideas, forcing him to redo it, if he doesn’t like it, pointing to a reddit thread about why. There’s no way he would ever finish anything new.
Unless they figure out a completely new way how to do AI reasoning, there is no way we’re getting anywhere near AGI. And that is also becoming more unlikely the longer we go with this approach, because every AI-pilled company is heavily outsourcing all of development to the current models, reducing their employee (and the whole worlds) innovation potential and skill. This is probably the last generation that can do serious academia, unless there are drastic measures done to limit access to AI in education. It’s fucked.
A better analogy is promising that a toddler will, with enough knowledge and training, eventually become a heart surgeon.
15 years ago the assumption with AGI was that we needed some paradigm shift in technology to create it, but after transformer technology was invented (the T in GPT), and it was trained on a lot of information, we discovered an emergent property that it could take natural language queries and answer them with its knowledge base-- which was unexpected and unintended.
While science can always end up going down the wrong path, the current mainstream stance is that we were wrong about needing new technology for AGI; it seems that AGI may be a function of information and training, on hardware we already have. Hence all the data centers being built.
Anyone who says with certainty that it will result in AGI is just as wrong as someone who says with certainty that it won’t.
That analogy makes the assumption that LLMs are dormant baby AGIs. There is no evidence that AGI is just a bigger and better curated LLM.
That is where the evidence points. Now, I don’t want to oversell it: “where the evidence points” is wildly different than “exactly how it works”.
We have an emergent property that we don’t understand, but we can reliably increase the functionally and complexity of that emergent property as a function of training data and available compute. Does that mean that there isn’t some threshold where that stops working? No, there certainly could be a point where throwing more information and compute has no effect. We just don’t know. However, so far, there is no evidence such a barrier exists, and everyone is racing to find out.
I think that is where the hopes and hype point. The evidence, that which is gathered through controlled studies, points to an upper limit to this technology that does have emergent properties, but not ones that amount to cognition. That evidence also points to other hurdles, such as cognitive damage to the user and context windows nowhere near that of even a simpleminded creature, let alone a sapient one like a human.
The core problem is that these models are fixed; they are the same on day 1000 as they were on day 1. All their “learning”, as it were, happens in training before it is released. Everything it appears to learn after that date is contained in the rolling context window. Since they already have literally all the RAM they can get their hands on and are still at least an order of magnitude away from where they need to be on that, this technology either can’t do it or, at best, is so inefficient an approach that it can’t be done with all the planet’s resources.
Sometimes, especially in abstract constructions like software, you can start down the wrong path early and have to start over, because there is no path from where you are to where you need to get. In this case, they may have done that to the extreme, blinded by the lucrative prospects.
As far as I am aware, AGI does not imply sapience, consciousness, or whatever. You seem to believe it does. A high level Google search seems to suggest that I’m correct. Is there some reason you are lumping sapience with AGI?
I’m not sure if the rest of your comment presupposes this, so I’ll wait to comment on where I think you’re mistaken.
I dont believe it requires sapience; that is what the marketers are saying. The AI boom (and many historic boom cycles) is predicated on marketing and sentiment, not facts and data. That is why they always pop; the facts dont back up the hype.
The fact that your search results turn up results that align with the marketing is just the marketing working.
Look–neither of us are data scientists, but we do have eyes. If this is working, where’s the company with runaway success creating unimagined leaps in productivity and technology? We’re pouring a whole planet’s with of resources in and nothing much is coming out. If we spent this much on world hunger, everyone would be obese by now.
I think there is some confusion. I do not believe AGI implies sapience, nor does Google, and now it seems that you don’t either. So who is discussing sapience?
Hypothetically speaking, what do you think this would look like?
The marketers are not using the word “sapience”, as that word would require defining for the layman. They are claiming AI will usher in an era of human prosperity by taking over our work and big decisions to do them more efficiently–tasks that sound to almost everyone to require advanced decision-making and thought, the hallmarks of sapience and cognition. This is largely besides the point, though.
I think that if this were successful, you would see at least some early adopting companies coming up aces. Some company would have double the productivity with half the workforce, and use that to absolutely devour their market segment by undercutting everyone else on price because their operating costs are so much lower than the competition. I’d see a software company adopt breakneck software release cycles with substantial, material stability and performance improvements with each. I’d see some company anywhere suddenly begin to outperform their former selves and be able to convincingly point to AI as where their improvements came from. The only one like that I can think of is NVidia, who are rich strictly because they’re selling shovels in the gold rush.
We have math PHDs with proofs current LLMs can’t become AGI, because they’ll always have a context issue
I would love it if you’d point me at these mathematical proofs.
This is the primary paper I reference.
https://arxiv.org/pdf/2507.07505
Vishal Sikka, advisory board member of BMW. Recommended to Stanford by Marvin Minsky, one of his professors were John McCarty. And I must stand corrected, he has a PHD of computer sciences, not Math as I remembered it as.
Varin Sikka is his son, co author of the paper and based on Stanford’s site an undergraduate. https://profiles.stanford.edu/363374
Vishal has an AI based company himself, so there might be some personal reasons for why he’d advocate for using what AIs capable of rather than chasing an impossible (from his perspective) to hit milestone
That paper doesn’t seem to rule out AGI, only an single LLM model that can answer every arbitrarily difficult question on demand.
AGI does not necessarily mean one model acting alone, or being able to answer any question on demand. Humans are the same way: we often need time or collaboration to arrive at conclusions, but that doesn’t mean we don’t have “general intelligence”.
You’re not going to context hack complexity, every time you summarise something and pass it onto the next agent. You’re losing complexity.
We’re going to get some massive models with incredible context, but AGI requires models to never hallucinate. Even when the complexity requires more context than it has available, which is not feasible
We’ve already had a month now of Psychiatrists writing articles backpedaling over all the times they’d insisted AI couldn’t genuinely be thinking as we do and moving the goal posts to something else. Maybe there are researchers in the frontier labs who feel like they have a grip on what’s happening, but the ‘experts’ on forums have no idea.
You genuinely believe that the people in frontier labs under such severe NDAs we’ve had articles about how unusual they are believe it’s absurd to think that what they’re working on could lead to “AGI”?
What would the goal of that be in your mind? Getting as much investment as humanly possible before everyone figures out the leading specialists have been being paid insane amounts of money just to act like what they’re doing matters and the valuations all plummet to nothing over night? Why?
With all the money poured in to this from all the sources it’s coming from that would end in a lot of the most powerful companies, government bodies, and individuals in the world extremely unhappy and blaming the people running those frontier labs.
Getting a lot of news and attention and then saying “psych, lol” and being imprisoned for life or murdered doesn’t seem like a great long term plan.
I believe the people in frontier labs are doing their best to push in that direction.
I believe the investors believe they will have AGIs in a few years.
I believe the human species has throughout history repeatedly proven the height of its hubris and the depth of its stupidity.
I agree with all of those things, but for the last one I see a big example in all of the people not taking the news of a demonstrable global workspace in AI, including models that have been around for years now, as anything big.
People have been insisting that AI can’t possibly actually be thinking or have any of the most vaunted aspects of the human mind and that LLMs are a dead end that won’t go anywhere for years. We want to be different and special and unique. We always have.
Many of us here went to school when it was still taught that animals weren’t really conscious and couldn’t be self-aware. Being unwilling to look at new information and honestly assess is hubris and insistence on holding to the thing you’ve been insisting, it’s not a logical analysis.
Over the years we’ve pointed to several things to insist that AI wasn’t special like us, and that it couldn’t possibly be because it lacked aspects of the human mind needed for that. Somehow it kept turning out that every thing we pointed to in ourselves ended up being shown to have a very surprisingly similar to nearly direct analogue in those modern LLMs. Last month was jaw dropping, but the bulk of the world has been so focused on the entire world going to hell it hasn’t gotten near the attention it deserved yet.
What does it mean to think? This is a open-ended philosophical question and has any number of answers.
I completely agree with your 2nd and 3rd paragraph.
There are similarities and there are clear differences. The reason why we say LLMs are different from us is because we DO have general intelligence, LLMs have not displayed such capabilities despite the intense pressure to prove so. THIS is why we say LLMs couldn’t possibly be like us. Because evidence shows they aren’t, despite the similarities.
I would be interested to hear what developments in the last month you’re referring to, would be exciting to see a breakthrough.
Global workspace. The leading theory of consciousness. It’s a thing we can’t show empirically in ourselves, but can now see and experiment on something that matches it’s description in AI. Not just the proprietary latest models, the years old models anyone can download from the internet. No one built it, it was something else that emerged somehow through training and went unnoticed.
Here’s someone trying to backpedal into saying the thing that really matters actually isn’t genuinely thinking, now it’s… living with your mistakes. That’s how far the goal posts have been kicked over the last 6 months ending in this one. Not genuinely being able to think, rational self-awareness. Just remembering when you messed up a few months ago. And it’s such a weird pivot because AI memory is a design aspect that can be changed, something external frameworks to enhance with databases already exists for, a large part limited in current design because the more you send in the more it costs to process so it’s always capped, and also… sort of unnatural.
Running AI models the way we do with “frozen weights” isn’t mandatory. It takes a lot more hardware to do it, but it’s possible to run AI in a way more like they run during it’s training. The model files themselves would be unfrozen and allowed to change as you communicate. It’s reportedly something that’s had issues with the models forgetting things even as big as the language you’re talking in, but also a thing that is sometimes used during the alignment process. And one of the major reasons the big companies don’t care to look in to it is because if you have 10,000 people communicating with an AI over the internet and all telling it to do different things and act different ways and all of those things can be learning on the level of the model files themselves instead of confined to external temporary context windows it’s going to go crazy.
Memory on on the actual model level isn’t nonexistent, it’s what the entire training process is based on. It just wouldn’t make a good consumer product for sale, so parts of the models are removed before they’re made available online for the ones that are.
Everything you described would be a small improvement, not a leap towards AGI that is needed. That is why it isn’t being researched further. Massive investment for minimal gain.
There isn’t some massive leap needed. That’s what I’m saying. Everything we’ve used to try to show it’s not possible for the LLMs to have the most important aspects of human cognition have been shown to be present in some near or direct analogue. I linked the article in Psychology Today because memory is the sort of thing that has become the last hold out over all of those other words, and memory is a pure design choice that can be altered in many ways.
The insistence that LLMs of today could never lead to AGI seems based on out dated and incomplete understanding of them than people have failed to update as research has advanced. There are very real reasons for the frontier researchers who likely know more on these topics than has become available publicly in research to continue to focus on them.