In a letter to the CEOs of OpenAI, Anthropic, and Meta, Sanders writes that almost every day brings a new story about how these companies are losing...
Any pause will have to be a global treaty; the unfortunate reality is that we’re in an arms race now, and despite Sanders’ threats otherwise, I don’t think there is going to be much will to pause in the US while China keeps developing.
And to be clear, I think it would be every wise to pause development and, afterwards, keep the pace slow. We know how to make “AI” more capable, but we don’t know how to make it stay aligned with human goals. The technology is useful and isn’t going anywhere, but that doesn’t mean we have to develop it as quickly as possible without regard to the real dangers it brings with it.
We’re not in an arms race. That’s part of the bullshit Anthropic and like want you to believe. LLMs are not nukes. LLMs are a fad with most benefits having been to coders, presuming they were already good coders.
25 years from now we’re either going to still be in this “arms race,” or we’ll have better clarity on how the billionaire class misled us all.
Stopping development will do us more good than the harm of perusing a perpetual wealth transfer in the name of a fake arms race.
Even if it were an arms race. Slowing down means you eventually speed back up in a better direction. Why would you want to be so gun ho about what’s clearly an argument for progressing with arrogance toward the technology, because someone else might discover something first? What?
I would summarize it to conclude: The “AI arms race” is not an objective technological fact so much as a politically useful narrative that has helped align government policy with the interests of large AI companies.
My opinion is that we should be a lot more critical of this narrative.
I am definitely going to read that link, from the start at 1.0, but now right now. Thanks, though.
Accepting your summary (and sorry if I ask a question answered in the link) but what would an “objective technological fact” regarding any technology being an arms race look like? As in, it seems obvious that a technology that can perform complex exploits of software vulnerabilities without human assistance could be used as a weapon-- so why wouldn’t that be a national concern to ensure America has the better technology? How is this different than missiles or bombs, from a technological standpoint?
To be fair, an AI system capable of independently discovering and exploiting serious software vulnerabilities would be a legitimate national-security concern. The US would have a strong interest in possessing and defending against that capability.
But a technology is not itself an arms race. An arms race is a pattern of behavior between adversaries involving that technology. That requires observable political, military and economic facts.
I would point these out as red flags:
Developing identifiable autonomous cyberweapons.
Each side expands its program substantially because it believes the other is expanding.
Established military units, budgets, infrastructure and deployment plans around those systems.
I would point these out as not counting:
AI can be weaponized.
China is developing AI.
AI could confer military advantages.
American companies are spending heavily on general-purpose models.
The United States wants to lead commercially.
In table format, the comparison between AI and those other technologies would ideally look something like this:
An AI arms race would look very different than what we see right now.
With the AI bubble, the communities being disrupted by the environmental impacts of data centers (including infranoise), the constant fear of “use AI or be replaced by AI,” the risky financials on all of this, … all the people that lost retirement value to the SpaceX IPO, … this stuff is a serious public nuisance.
We don’t need tech CEOs scaring us with talk of an “arms race.” It’s fucked up.
They discuss a lot of the same points I would, right there. The incentives of AI leaders, Chinas current position, the lies from AI companies to communities, the way their stories change, the fact that they’re the ones calling it an “arms race.”
I would alternatively ask this: what expertise claims we are in an arms race? AI companies don’t count as expertise on global arms races, in what I think is a reasonable opinion.
There are several problems here. If you address them, I’m happy to answer your question faithfully. But as it stands, I don’t think it’s a fair comparison.
First and foremost, we’re specifically discussing an “arms race,” not just a “race.” I fail to understand what qualifies an AI company to be an expert in national security, weapons, and foreign affairs. To me, an AI company is an expert in what makes them more profit. That just so happens to involve AI here, but not “arms races.”
Second, doctors are probably not necessarily experts on vaccines. They’re likely to be experts on vaccine benefits, recommendation, procurement, administering, storage, … but that’s just administration and some operational knowledge. A vaccine expert (IMO) is a specialist for vaccines. They’re probably not working in a hospital.
A pharmaceutical company might be an expert on vaccines. But what’s the parallel to AI companies? Pharmaceutical companies research and develop actual vaccines. AI companies do not research and develop arms. The only argument I understand for this would be to say that an LLM is a weapon.
I don’t buy that an LLM is a weapon. Real competition exists, and certain military applications could become arms races, but AI as a whole is not one. Building out swaths of data centers to power more AI without a concrete public benefit isn’t something we should put up with.
You don’t benefit from the narrative that this is an arms race, not as an individual or a citizen. The benefactors here are the AI companies.
The technology can certainly be used as a weapon; it can hack systems already, and defend against hacks. This capability will only increase, and logically the better AI affords a strategic benefit. Hence, an arms race.
The rest of your questions aren’t questions or presuppose that AI can’t be used as a weapon.
In case it isn’t obvious, there are experts in this field that aren’t employed by AI.
The US and China are in an economic race. We aren’t racing to develop a particular military capability. Nobody can even name what that capability would be… “mass hacking?” Is that it? Sounds scary… I don’t buy it. It gives me the impression of fear mongering.
If it were about AI driven military capabilities, we could have probably worked with the compute we had years ago.
I think what should be obvious is the fact that “can be used as a weapon” doesn’t justify it being an “arm.” The fact that they can “hack” now is largely irrelevant. It’s a nice talking point that was parroted around the AI circle jerk recently, but largely unsubstantiated. I think they all caught on to the hype Mythos was getting, so they took the narrative for another spin. But even Mythos was a load of propaganda.
How do you know it wasn’t conveniently allowed to happen, to further fuel the propaganda machine? How do you know they didn’t exaggerate or leave blank important details, particularly because they’re financially incentivized to maintain the hype?
State actors have always hacked one another. Now with a fancy tool to do automated discovery, they might be a little faster. But that speed is also arguable, because there are reports on how AI led development actually shifts the development time burden rather than resolving it.
In case it isn’t obvious, there are experts in this field that aren’t employed by AI.
Those experts in the field (that aren’t employed by AI) mostly agree with this stance, from what I can tell.
I ran the report through AI (the irony is not lost on me) and while it’s really very clear this is closer to a manifesto than a unbiased report (not that it claims to be unbiased), I don’t strictly disagree with any specific point.
I have several thoughts, however:
If you don’t think having a legion of skilled hackers that never sleep is a weapon, fine-- hell, even if you tighten your tinfoil hat and reject the claim that they can even do that, if the American government and the Chinese government both believe it has this capability, and they both believe it’s advantage to have the better technology, then this is an arms race. Even if they’re both wrong about the capabilities of AI or the plausibility of AGI, its still an arms race. If the it turned out that atomic bombs weren’t feasible or possible to create, that wouldn’t have made the race to build one less of an arms race, without hindsight.
If not clear, I don’t believe a conspiracy on this scale is a realistic assumption without more data than “I don’t believe it”. It would have to spread across several competing companies throughout most, if not all, of the employees.
Like I said in the first comment, Sanders can threaten all he wants, but I just don’t believe Congress is going to take a risk on “its not really that important” and impose restrictions on AI development. (Side note that no restrictions on AI development is not the same as no restrictions on AI use or deployment.)
I think, as I said before, that we will need a global treaty on pausing development before any one country or company pauses development, or even slows it down.
It’s beyond “an arms race.” AI has become a commodity that’s everywhere and it’s being used and developed by zillions of completely different groups all over the world with wildly varying use cases and completely different goals.
At this point, trying to ban it would be like trying to ban dandelions from growing.
Except when you think about if a ban were actually implemented it would absolutely be pro-entrenched companies like OpenAI and Anthropic and open weight models could be banned outright, I’ve already seen well-intentioned but misguided “experts” saying exactly that, BAN open weight models and pause further development, who would that benefit directly?
Woah, we’re talking LLMs here, this is not a secret military technology. And China is very open about the developments made there - you can freely study and reverse engineer plenty of Chinese models, and see all the weights out in the open.
This whole race about them is, in my opinion, nothing but trying to pump the bubble even faster, churning the economy in the process and playing on gullible or self-interested politicians. Countries will be just fine assessing risks and then moving on.
It’s a one participant race. They’re throwing fears about Chinese LLM because of the fact that many of their models end up being open-source, and they know if they don’t get people cemented into their models before some open-sourced Chinese alternative manages to meet(or even exceed ) their performance. No company is going to pay them money because they’re just gonna run it themselves for way less than subscription costs.
right now, they acknowledge that the non-US alternatives for the models are gaining traction and are getting pretty dang good, and that scares them, so they’ve moved over to customer lock-in hoping that the effort it would take to move from a subscription model over to one of the alternative models wouldn’t be worth it for a company. A US only pause would almost certainly leave them dead in the water.
Any pause will have to be a global treaty; the unfortunate reality is that we’re in an arms race now, and despite Sanders’ threats otherwise, I don’t think there is going to be much will to pause in the US while China keeps developing.
And to be clear, I think it would be every wise to pause development and, afterwards, keep the pace slow. We know how to make “AI” more capable, but we don’t know how to make it stay aligned with human goals. The technology is useful and isn’t going anywhere, but that doesn’t mean we have to develop it as quickly as possible without regard to the real dangers it brings with it.
We’re not in an arms race. That’s part of the bullshit Anthropic and like want you to believe. LLMs are not nukes. LLMs are a fad with most benefits having been to coders, presuming they were already good coders.
25 years from now we’re either going to still be in this “arms race,” or we’ll have better clarity on how the billionaire class misled us all.
Stopping development will do us more good than the harm of perusing a perpetual wealth transfer in the name of a fake arms race.
Even if it were an arms race. Slowing down means you eventually speed back up in a better direction. Why would you want to be so gun ho about what’s clearly an argument for progressing with arrogance toward the technology, because someone else might discover something first? What?
The chatbot labs don’t have 25 years. They might have 2.
What expertise are you drawing from to come to this conclusion?
Here’s a decent resource if you care to read about, rather than the podcast: https://ainowinstitute.org/publications/research/1-3-ai-arms-race-2-0-from-deregulation-to-industrial-policy
I would summarize it to conclude: The “AI arms race” is not an objective technological fact so much as a politically useful narrative that has helped align government policy with the interests of large AI companies.
My opinion is that we should be a lot more critical of this narrative.
I am definitely going to read that link, from the start at 1.0, but now right now. Thanks, though.
Accepting your summary (and sorry if I ask a question answered in the link) but what would an “objective technological fact” regarding any technology being an arms race look like? As in, it seems obvious that a technology that can perform complex exploits of software vulnerabilities without human assistance could be used as a weapon-- so why wouldn’t that be a national concern to ensure America has the better technology? How is this different than missiles or bombs, from a technological standpoint?
To be fair, an AI system capable of independently discovering and exploiting serious software vulnerabilities would be a legitimate national-security concern. The US would have a strong interest in possessing and defending against that capability.
But a technology is not itself an arms race. An arms race is a pattern of behavior between adversaries involving that technology. That requires observable political, military and economic facts.
I would point these out as red flags:
I would point these out as not counting:
In table format, the comparison between AI and those other technologies would ideally look something like this:
An AI arms race would look very different than what we see right now.
With the AI bubble, the communities being disrupted by the environmental impacts of data centers (including infranoise), the constant fear of “use AI or be replaced by AI,” the risky financials on all of this, … all the people that lost retirement value to the SpaceX IPO, … this stuff is a serious public nuisance.
We don’t need tech CEOs scaring us with talk of an “arms race.” It’s fucked up.
Here’s a really good podcast on the subject: https://podcasts.apple.com/us/podcast/the-ezra-klein-show/id1548604447?i=1000779843294
They discuss a lot of the same points I would, right there. The incentives of AI leaders, Chinas current position, the lies from AI companies to communities, the way their stories change, the fact that they’re the ones calling it an “arms race.”
I would alternatively ask this: what expertise claims we are in an arms race? AI companies don’t count as expertise on global arms races, in what I think is a reasonable opinion.
Do doctors and pharmaceutical companies count as experts when discussing vaccines?
There are several problems here. If you address them, I’m happy to answer your question faithfully. But as it stands, I don’t think it’s a fair comparison.
First and foremost, we’re specifically discussing an “arms race,” not just a “race.” I fail to understand what qualifies an AI company to be an expert in national security, weapons, and foreign affairs. To me, an AI company is an expert in what makes them more profit. That just so happens to involve AI here, but not “arms races.”
Second, doctors are probably not necessarily experts on vaccines. They’re likely to be experts on vaccine benefits, recommendation, procurement, administering, storage, … but that’s just administration and some operational knowledge. A vaccine expert (IMO) is a specialist for vaccines. They’re probably not working in a hospital.
A pharmaceutical company might be an expert on vaccines. But what’s the parallel to AI companies? Pharmaceutical companies research and develop actual vaccines. AI companies do not research and develop arms. The only argument I understand for this would be to say that an LLM is a weapon.
I don’t buy that an LLM is a weapon. Real competition exists, and certain military applications could become arms races, but AI as a whole is not one. Building out swaths of data centers to power more AI without a concrete public benefit isn’t something we should put up with.
You don’t benefit from the narrative that this is an arms race, not as an individual or a citizen. The benefactors here are the AI companies.
The technology can certainly be used as a weapon; it can hack systems already, and defend against hacks. This capability will only increase, and logically the better AI affords a strategic benefit. Hence, an arms race.
The rest of your questions aren’t questions or presuppose that AI can’t be used as a weapon.
In case it isn’t obvious, there are experts in this field that aren’t employed by AI.
The term “arms race” has a definition and I’m pretty sure this still doesn’t apply. So in my opinion, even at best, this is political propaganda.
The US and China are in an economic race. We aren’t racing to develop a particular military capability. Nobody can even name what that capability would be… “mass hacking?” Is that it? Sounds scary… I don’t buy it. It gives me the impression of fear mongering.
If it were about AI driven military capabilities, we could have probably worked with the compute we had years ago.
I think what should be obvious is the fact that “can be used as a weapon” doesn’t justify it being an “arm.” The fact that they can “hack” now is largely irrelevant. It’s a nice talking point that was parroted around the AI circle jerk recently, but largely unsubstantiated. I think they all caught on to the hype Mythos was getting, so they took the narrative for another spin. But even Mythos was a load of propaganda.
How do you know it wasn’t conveniently allowed to happen, to further fuel the propaganda machine? How do you know they didn’t exaggerate or leave blank important details, particularly because they’re financially incentivized to maintain the hype?
State actors have always hacked one another. Now with a fancy tool to do automated discovery, they might be a little faster. But that speed is also arguable, because there are reports on how AI led development actually shifts the development time burden rather than resolving it.
Those experts in the field (that aren’t employed by AI) mostly agree with this stance, from what I can tell.
I linked a paper in one of my earlier replies. Here’s another by a different author: https://tnsr.org/2021/06/debunking-the-ai-arms-race-theory/
… and, I just found a table. Here’s 7 more I guess.
I ran the report through AI (the irony is not lost on me) and while it’s really very clear this is closer to a manifesto than a unbiased report (not that it claims to be unbiased), I don’t strictly disagree with any specific point.
I have several thoughts, however:
If you don’t think having a legion of skilled hackers that never sleep is a weapon, fine-- hell, even if you tighten your tinfoil hat and reject the claim that they can even do that, if the American government and the Chinese government both believe it has this capability, and they both believe it’s advantage to have the better technology, then this is an arms race. Even if they’re both wrong about the capabilities of AI or the plausibility of AGI, its still an arms race. If the it turned out that atomic bombs weren’t feasible or possible to create, that wouldn’t have made the race to build one less of an arms race, without hindsight.
If not clear, I don’t believe a conspiracy on this scale is a realistic assumption without more data than “I don’t believe it”. It would have to spread across several competing companies throughout most, if not all, of the employees.
Like I said in the first comment, Sanders can threaten all he wants, but I just don’t believe Congress is going to take a risk on “its not really that important” and impose restrictions on AI development. (Side note that no restrictions on AI development is not the same as no restrictions on AI use or deployment.)
I think, as I said before, that we will need a global treaty on pausing development before any one country or company pauses development, or even slows it down.
It’s beyond “an arms race.” AI has become a commodity that’s everywhere and it’s being used and developed by zillions of completely different groups all over the world with wildly varying use cases and completely different goals.
At this point, trying to ban it would be like trying to ban dandelions from growing.
The groups doing frontier research with the appropriate funding are few enough that a global pause would be possible, and useful.
I agree that a ban would be impossible, and counterproductive even if it were possible.
Except when you think about if a ban were actually implemented it would absolutely be pro-entrenched companies like OpenAI and Anthropic and open weight models could be banned outright, I’ve already seen well-intentioned but misguided “experts” saying exactly that, BAN open weight models and pause further development, who would that benefit directly?
Woah, we’re talking LLMs here, this is not a secret military technology. And China is very open about the developments made there - you can freely study and reverse engineer plenty of Chinese models, and see all the weights out in the open.
This whole race about them is, in my opinion, nothing but trying to pump the bubble even faster, churning the economy in the process and playing on gullible or self-interested politicians. Countries will be just fine assessing risks and then moving on.
It’s a one participant race. They’re throwing fears about Chinese LLM because of the fact that many of their models end up being open-source, and they know if they don’t get people cemented into their models before some open-sourced Chinese alternative manages to meet(or even exceed ) their performance. No company is going to pay them money because they’re just gonna run it themselves for way less than subscription costs.
right now, they acknowledge that the non-US alternatives for the models are gaining traction and are getting pretty dang good, and that scares them, so they’ve moved over to customer lock-in hoping that the effort it would take to move from a subscription model over to one of the alternative models wouldn’t be worth it for a company. A US only pause would almost certainly leave them dead in the water.
Order Nvidia and AMD to stop selling AI GPUs or whatever they’re called.
Which country is doing the ordering?