It is significant because a random teenager can’t google “download exploits” and have them available 5mins later.
Powerful AI models and agents though are on your fingertips without you even asking.
Sure, people can buy guns. But what if every person could materialize a chainsaw instead regardless of their skill, maturity, age, or criminal record? 🤔
Full disclosure was a thing once upon a time, where exploits and proofs of concept were dumped publicly, forcing companies to fix the issue or be compromised. That’s mostly been moved away from in favor of responsible disclosure, giving companies time to patch the issue before it’s known publicly.
Maybe we should be moving back to full disclosure to force these companies to take data security seriously. Or at least then we could point to a known vulnerability as proof the company is shitty and is neglecting their infrastructure.
Didn’t think I’d ever side with no script kiddie but at this point fuck it.
If your company can’t even be bothered to do the bare minimum in security then yeah I hope the least skilled hacker ever comes along and wrecks it.
Thing is, with the latest frontier models, the least skilled person can find a crack in the most secure company around, as long as they can string a few sentences together.
It isn’t about “bare minimum” anymore. All it takes is a single lapse in vigilance from a single employee, and they’re in… and the LLM doesn’t have to pause to figure out what to do next.
some hacker unleashes malicious AIs to the internet, breaking it apart cause AI keeps finding vulnerabilities in everything and break things faster than humans can fix
corporates build corporate internet and the blackwall, which is AI to fight malicious AIs
Yeah, but an LLM’s arms race isn’t “doing the bare minimum in security”, which is what the poster before was saying.
This is a genuine concern, where whoever has access to the best/most recent/most expensive models can unleash chaos - I’m talking state-sponsored attacks, mega-corp espionage, bored billionaires,…
Most people on lemmy seem to condemn use of LLMs in any way for anything, I wonder what those folks opinion of this stance is - should companies use the tools or not?
Finding holes in software has employed “fuzzing”, where you send completely random payloads, as a research tactic for quite a while (and it has found exploits). LLMs just seem like “educated” fuzzing, I don’t see why anyone would complain about updating your suite with them.
Cybersecurity is actually one of the few fields that can benefit from AI. There are companies like Horizon3 who are using it alongside their other threat models to do continuous pen testing.
Gonna take a guess here that what is used in cybersecurity is not LLMs but one of the more useful machine learning applications. Just a nitpick cause today “ai” and “LLM” are sadly synonymous.
No, LLMs can definitely be useful for cyber too. It’s the whole reason the US government banned Claude Fable for export.
An LLM can not just try existing exploits like a script kiddy, but with iteration it can try variations and if you know what runs on the server, inspect the source for potential exploits.
They can also look at your setup and say what issues they see (reverse proxy config, etc).
Doesn’t replace an expert, but can be useful for a first pass before you get the highly paid people involved.
Well the problem is that for example curl got flooded with generated security reports where only 5% had some true security potential. So your llm will basically flood you with false positives
If 5% of the reports are genuine security vulnerabilities that they wouldn’t have found otherwise, that’s looking like a big win to me, not sure how you see it differently.
script kiddies wrecking corporate security is funny
prompt kiddies doing it is just depressing
And no-skilled attackers can buy exploits.
Claude helping is insignificant to the story.
The real headline should be:
At least 14 companies’ IT security is practically non-existent
It is significant because a random teenager can’t google “download exploits” and have them available 5mins later.
Powerful AI models and agents though are on your fingertips without you even asking.
Sure, people can buy guns. But what if every person could materialize a chainsaw instead regardless of their skill, maturity, age, or criminal record? 🤔
Random teenagers can absolutely google “download exploits” and have them available, that’s pretty much always been the case…
https://www.exploit-db.com/
Full disclosure was a thing once upon a time, where exploits and proofs of concept were dumped publicly, forcing companies to fix the issue or be compromised. That’s mostly been moved away from in favor of responsible disclosure, giving companies time to patch the issue before it’s known publicly.
Maybe we should be moving back to full disclosure to force these companies to take data security seriously. Or at least then we could point to a known vulnerability as proof the company is shitty and is neglecting their infrastructure.
Didn’t think I’d ever side with no script kiddie but at this point fuck it.
If your company can’t even be bothered to do the bare minimum in security then yeah I hope the least skilled hacker ever comes along and wrecks it.
Thing is, with the latest frontier models, the least skilled person can find a crack in the most secure company around, as long as they can string a few sentences together.
It isn’t about “bare minimum” anymore. All it takes is a single lapse in vigilance from a single employee, and they’re in… and the LLM doesn’t have to pause to figure out what to do next.
Pentesters have access to LLMs too
Gooooood morning Night City!
Yeah, but an LLM’s arms race isn’t “doing the bare minimum in security”, which is what the poster before was saying.
This is a genuine concern, where whoever has access to the best/most recent/most expensive models can unleash chaos - I’m talking state-sponsored attacks, mega-corp espionage, bored billionaires,…
The people you listed were already doing this. The problem is Darrell, the guy who thinks Earth is flat, can also do this.
Only for a year or so. Any company still vulnerable after these tools have been out long enough deserve it.
Most people on lemmy seem to condemn use of LLMs in any way for anything, I wonder what those folks opinion of this stance is - should companies use the tools or not?
Finding holes in software has employed “fuzzing”, where you send completely random payloads, as a research tactic for quite a while (and it has found exploits). LLMs just seem like “educated” fuzzing, I don’t see why anyone would complain about updating your suite with them.
Cybersecurity is actually one of the few fields that can benefit from AI. There are companies like Horizon3 who are using it alongside their other threat models to do continuous pen testing.
Yeah imo the one thing ai is legitimately useful for is finding answers to difficult problems that can be trivially verified as correct.
In this case hallucinations actually help…
Gonna take a guess here that what is used in cybersecurity is not LLMs but one of the more useful machine learning applications. Just a nitpick cause today “ai” and “LLM” are sadly synonymous.
No, LLMs can definitely be useful for cyber too. It’s the whole reason the US government banned Claude Fable for export.
An LLM can not just try existing exploits like a script kiddy, but with iteration it can try variations and if you know what runs on the server, inspect the source for potential exploits.
They can also look at your setup and say what issues they see (reverse proxy config, etc).
Doesn’t replace an expert, but can be useful for a first pass before you get the highly paid people involved.
You know what, fair enough. I don’t know enough about that particular one.
Well the problem is that for example curl got flooded with generated security reports where only 5% had some true security potential. So your llm will basically flood you with false positives
If 5% of the reports are genuine security vulnerabilities that they wouldn’t have found otherwise, that’s looking like a big win to me, not sure how you see it differently.
The problem is identifying which 5%. Nobody wants to filter that much AI slop.
If you’re working for a company’s cybersec, that’s your job. And a much preferable one to waiting for an attacker to do it for you.
Only if you’re stuck in the past.