No worries, it seems companies are very eager to make this happen sooner than later 👍
Wait, so its not the actual event and ensuing casualties that have AI researchers spooked, but the fact that it might cause the public to turn against AI?
Let me put this in perspective:
In many countries its against the law to freely distribute plans for making neurotoxins or bombs, because the democratization of such knowledge would lower the threshold for people to commit acts of terror.
Likewise the plans for making a hydrogen bomb are a close kept government secret, because nuclear proliferation increases the likelihood of radiological accidents or even nuclear war.
How is it then that AI companies freely publish their AI models to any and all actors willing to pay them? Even though they know that this technology lowers the threshold for bad actors to commit cybercrime, engage in cyberwarfare, spread misinformation, commit fraud, manipulate markets and whatnot? The unregulated democratization of AI exposes societies to unprecedented risks.
Is it any wonder the public holds a negative view on AI?
I can’t wait until AI eats the entire stock market. Not by trading on it, but by being used to manage the software and deciding but the most efficient way to do its job is to delete everything.
The thing about Chernobyl was that it was, ultimately, an unwanted mistake.
The thing about AI is that the shitty mass casualty outcome seems like the intended outcome.
Chernobyl: A massive catastrophic accident during testing and shift changes.
AI: A massive catastrophic accident intentionally caused by corporate greed and capitalism
This take is a denial of current reality.
They’re already using “AI” for genocide in Palestine, etc.
This is how capitalism always develops and uses “technology”.
Yeah, but the thing is…there is absolutely nothing stopping Iran, Hezbollah or Hamas from developing their own AI.
The technology needed to develop AI that can sabotage infrastructure, spread disinformation, develop autonomous weapons etc is already out there.
Whatever advantage the US and Israel had when it comes to military applications of AI, is quickly disappearing.
I think they meant some unintended catastrophe like Chernobyl was where things went out of control. The use of AI in this example is intentional and controllable.
Top AI researchers should go fuck themselves right to hell for handwaving away and ignoring the inherent problems with their work while plowing ahead full steam so they can get personally rich.
Them admitting this is possible should already concern people.
Delusional take. We, the people pretty much hate AI already. We hate how it‘s utilized against us.
People weren’t wildly supportive of nuclear power before Chernobyl / TMI either, but after? 40 years of virtual moratorium on new construction in most of the world, absolute rollback in Germany. We’ve continued to poision ourselves with coal and wreck the climate with CO2 instead of learning to do nuclear right. If building of new plants weren’t so difficult, older and less stable plants like Fukushima could have been decomissioned before having major problems.
Except in this case, we’re turning down the advancement of a technology that hurts humanity in every conceivable way and poisons the environment.
a technology that hurts humanity in every conceivable way and poisons the environment.
That’s how I feel about Bitcoin, which, by the way, STILL consumes more energy in its datacenters than AI.
Most relevant paragraph for the tldr crowd:
Flavors of AI doomsaying vary dramatically, ranging from Skynet-style scenarios to mass unemployment. But more recently, as it’s become clear that one of AI’s most practical applications is generating code, experts have been sounding the alarm on AI’s potential to disrupt cybersecurity. Hackers could easily abuse AI agents and coding tools to orchestrate devastating cyberattacks, both increasing the scale of these attacks and lowering the skill needed to carry them out.
It goes on to talk about how claude has yet to release a particular model and China has closed-sourced some of its over cybersecurity concerns.
The AI threat to cybersecurity isn’t (so much) people accepting vulnerable AI code, it’s AI finding the vulnerabilities in widespread legacy code.
As for “lowering the bar” - we’ve had skript kiddiez since forever, the availability of AI to skript kkiddiez just raises the bar for finding and fixing zero days by the White Hats before the Black Hats find and exploit them.
This “AI is inevitable” mindset is the most rapey shit ever.
That’ll happen when they turn over some piece of critical infrastructure or something over to AI. Power, stock market, something that will affect tens or hundreds of millions of people and wreck lives or even kill people.
Nonetheless these clowns will push for it because profit, and some fool politicians will OK it because campaign money.
They AI owners wont care if it kills millions or releases the formula to recreate smallpox or anything… Only the stock market or their cash flow will matter to them.
stock market
Algorithmic traders flash-crashed the market over 15 years ago. AI in trading has been a thing since before then. In that field the latest LLMs are an evolution, not a revolution.
At this moment, I am sitting in a Amazon Kiro course. We have spent more than 2 hours and half the tokens of our test account just to fucking connect to an MCP server.
The process is so convoluted and Byzantine it has to be some kind of cult to enjoy this pain.
I find that the AI tools worth using are the ones that the AI agents themselves configure properly for you.
Learning how to connect an MCP server myself feels like an extremely ephemereal skill - one that will become nearly worthless several days before you start to learn how to do it.
Chernobyl blew up because they were dicking around with an unsafe design.
This analogue is very apt.
An unsafe design that wasnt even fully tested.
Still bang on.
“Top AI researcher” I can no longer read those words without Doofy voice 😂
Probability of unhinged techbro bs is around 115%
They’re always going on about how dangerous their AI is.
“Our hey our AI found 3,000 0-day security vulnerabilities in top software, we’re only giving it to top software companies so that they can patch their products before we release”
It could already be happening. The likelihood could even be higher than not.
It doesn’t necessarily need to be a single event. There are already AI systems being sold in healthcare. Even pre-LLMs. Who’s to say these system are being closely monitored and studied. There could statistically be patients who have died or been maimed who otherwise wouldn’t if they had real human professionals instead of someone who used a system that was sold on the doctors office.
Hold on a minute… the field of AI and in general algorithms and models is quite old. LLM is relatively new in there. For science and healthcare we’ve been using specialist ai for a long while for things like image processing, sequencing and what’s not. Those are absolutely monitored have have generally high standards because they are healthcare related already. Don’t mix those things.
There were lots of “expert systems” developed and deployed in the 1990s, and they have been undergoing modest development improvements since then.
Who’s to say these system are being closely monitored and studied.
Mostly: the FDA, EUMDR and similar agencies and regulatory systems around the world.
There could statistically be patients who have died or been maimed who otherwise wouldn’t if they had real human professionals instead of someone who used a system
Works both ways, and, statistically, the data says the machines are improving outcomes overall. Hotshot doctor I knew had a childhood friend die rather suddenly in his 60s in the late 1990s. Really bothered him so he dug in to the case post mortem, did all the research using then present state of the art computer search tools, and determined what really happened. His friend went to the best possible ER given the circumstances, was seen by the best available human doctors in that site, who made the best possible calls based on the available information, and they made the wrong calls - wrong diagnosis led to interventions that made a bad situation worse and within less than 3 days after initial presentation his friend was dead. It took my hotshot doctor using keyword search tools he was very skilled with months to untangle all the information, and his conclusion was: the only thing that would have saved his friend’s life would be blind luck that a specialist (who didn’t practice in that hospital) might have been called to his case at random, because the standard of care didn’t trigger bringing in the right specialists in this situation, and if the right Dx were made and the right intervention was done, his friend could have lived another 30 years. But, if they did nothing, he would have still been dead within a week, maybe sooner, so they made the (wrong) call, did the (well meaning, but actually harmful) intervention, and made him die faster. LLMs could have saved his friend, turned those months of research using 1990s state of the art search tools into a few hours, timely enough to have made the right interventions when they were needed.
Machines make mistakes too, just not as often as people. Since tthe 1970s and even before, the depth of medical knowledge available in libraries far exceeds the capacity of any human, or even reasonably sized teams of humans, to access quickly on demand. Machine assistance in searching that information has been improving dramatically over the past 30 years. I’d much rather have a doctor that uses it than one that just gives his opinion based on what he learned in school and practice. But, on the flip side, I definitely don’t want a doctor who soaks his brain in gin at dinner and expresso at breakfast and relies 100% on what the machine says rather than engaging his own brain. Unfortunately, doctors like that, and others who are more concerned with filling their high paying surgery schedules than actually caring for patients best outcomes are all too common.
