It has become impossible to tell managers mesmerised by artificial intelligence that the tools are not, in fact, helpful. So employees just play along with the fiction to keep their jobs, writes our tech columnist
Completely disagree. The difficult part of software development was never writing code OR speed of delivery. It was understanding requirements and problem solving. LLMs still can not do either of those things and there is no evidence they ever will be able to.
An example of how harmful LLMs actually are to development can succinctly be described with an issue I had a few weeks ago. I found an issue in an open source project, code was fine if a bit hard to understand. I came up with a PR to fix the problem.
In the time from me checking out the code to submitting the PR, a little less than 24 hours, the maintainer had completely rewritten the entire project with Claude. It was complete nonsense. Incredibly difficult to understand. Abstracting things that didn’t need abstracting. My PR was useless, because the entire project was new. The maintainer definitely didn’t understand the changes either. If a bug came up there’s no way AI would be able to solve it (the bug was still there even though the code was entirely new).
LLMs don’t understand the code. They just make things that look like they will work. And then a human has to maintain it (or keep paying billions of dollars for Claude to try to fix it).
I’m not a kid doing some vibe coding, of course I know what I want and basically know how to implement it, the AI is just helping me writing it, auto-complete and all.
the person in your scenario is just fkin stupid. When people say that llms are helpful for coding, they aren’t talking about telling an llm to rewrite an entire codebase.
You are right, but right now we are living in a very messy reality where it’s hard to know who’s being stupid and who is using it well. One person I know that, well, I was never a huge fan of his work but at least it was somewhat serviceable is now all-in on AI and his code has been rewritten in a similar manner as the parent poster comments. He’s got no idea how it works or how it should worked, the AI decided to rewrite it in an entirely different language, and it’s a buggy mess and it never fixes the bugs without making new bugs. Then he hit his token quota 3 weeks early and basically said he was going to stop working on it because he no longer could manually work the codebase. He didn’t ask for a rewrite, but AI advised him that his language choice was a poor fit and reworked things in another language, one that none of us use to that level of seriousness. It also made it largely based on super convoluted regular expressions.
The problem is that the leadership is singing the praises of these people, they were on their ‘leaderboards’ of AI adoption and much like the craze of praising “lines of codes”, we are neck deep in the most stupid non-technical evaluation of technical work you could imagine.
In other words, you’re arguing that their usage without discernment is detrimental.
Cory agrees on this very same article, and it also includes the same nuance this chain is trying to give voice to:
The other question Suresh implicitly raises is: “How can you reconcile the failure of AI in the enterprise with the individual claims of skilled technologists who insist that AI is helping them do great work?” The answer is that these AI users are “centaurs” – experienced workers who are assisted by automation on terms that they set for themselves.
Thanks to their skill and experience, these workers possess discernment, the ability to tell good code from bad, and (more importantly) good uses of code-generation tools from bad. They demonstrate the adage that worker-driven automation improves quality, while capital-driven automation improves throughput.
We wouldn’t be having this conversation if LLMs had been given the chance to grow into being the same way the web did. Whereas we would be having this same conversation with the letters swapped if corporations were the ones to spawn the WWW instead of the way it came about.
It’s not just “usage without discernment”, it’s that AI flattens some costs which are very obvious and very measurable (coding) in such a way that it introduces or amplifies other costs which are much more diffused and hard to measure (code and design reviewing, bug fixing, maintenance, adding new requirements), plus AI is totally incapable of doing the higher level tasks that shape what code needs to be done (technical analysis, requirements analysis and in bigger companies technical architecture).
People who are non-experts, aren’t really senior domain experts or have some kinds of unbalanced expertise (they’ve never really progressed beyond being a coder, or they don’t have full life-cycle experience with big projects or they’re in an industry or position where they just make the code, shove it out the door and it’s not their problem anymore) just look at the one thing they in their ignorance think is THE cost in programming - coding - and go “hey, this AI thing is amazing” even while AI is creating all sorts of much more time consuming problems which they don’t really understand formally (they think those things are just “bad luck”, “there’s nothing we can do about avoiding this” and “it’s just the way things are in programming”) that require the time of people with higher expertise levels (i.e. who are more costly) to solve and AI doesn’t even help with the kind of stuff which if done wrongly or not at all can condemn a software project to fail before it even starts like just half-way decent Technical and Requirements Analysis.
That’s why you get some programmers going “this AI shit is amazing” whilst the really senior software development types are just nodding their heads and thinking “these people are ignorant as fuck juniors”.
It was understanding requirements and problem solving. LLMs still can not do either of those things and there is no evidence they ever will be able to.
I don’t know… I just made a scheduling / timesheet creation app. Multi-user, overlapping clients and providers, multiple funding sources. Took 10 calendar days to make the initial app working part time, maybe 2-3 hours a day. Initially written in Python, decided at that point I’d rather have it in Go. Because the initial app had robust requirements and design docs, the translation to Go happened in less than 5 calendar days, with almost zero human involvement beyond telling the agent “continue” at each stopping point. After the Go translation was done (and debugged by the LLM to a flawless translation - only difference is that it runs faster), I was given a new timesheet to use for some of the workers, weekly instead of bi-weekly. Pay weeks start on Monday instead of Thursday. Various wrinkles about how the employees and clients and services are identified, weird sub-totals by service. All I told the LLM was: “Here’s a new timesheet that we’ll be using for some workers, design the necessary modifications and extensions to accomodate it.” It did, independently. It highlighted three shortcuts it took and I told it not to take those shortcuts, it adjusted.
That’s not quite rocket science, but it’s still impressive: to dissect the given .pdf, determine what data goes in what fields, in what formats, with what calculations, based on just reading the page, then adapt the existing app to fill it out automatically.
In my professional work, I have watched AI code reviews catch 10x more dumb slop human errors than human reviews used to the same time a year earlier, consistently for about 8 months now.
Completely disagree. The difficult part of software development was never writing code OR speed of delivery. It was understanding requirements and problem solving. LLMs still can not do either of those things and there is no evidence they ever will be able to.
An example of how harmful LLMs actually are to development can succinctly be described with an issue I had a few weeks ago. I found an issue in an open source project, code was fine if a bit hard to understand. I came up with a PR to fix the problem.
In the time from me checking out the code to submitting the PR, a little less than 24 hours, the maintainer had completely rewritten the entire project with Claude. It was complete nonsense. Incredibly difficult to understand. Abstracting things that didn’t need abstracting. My PR was useless, because the entire project was new. The maintainer definitely didn’t understand the changes either. If a bug came up there’s no way AI would be able to solve it (the bug was still there even though the code was entirely new).
LLMs don’t understand the code. They just make things that look like they will work. And then a human has to maintain it (or keep paying billions of dollars for Claude to try to fix it).
I’m not a kid doing some vibe coding, of course I know what I want and basically know how to implement it, the AI is just helping me writing it, auto-complete and all.
Dumb people use their tools incorrectly. That’s all you’re describing.
the person in your scenario is just fkin stupid. When people say that llms are helpful for coding, they aren’t talking about telling an llm to rewrite an entire codebase.
You are right, but right now we are living in a very messy reality where it’s hard to know who’s being stupid and who is using it well. One person I know that, well, I was never a huge fan of his work but at least it was somewhat serviceable is now all-in on AI and his code has been rewritten in a similar manner as the parent poster comments. He’s got no idea how it works or how it should worked, the AI decided to rewrite it in an entirely different language, and it’s a buggy mess and it never fixes the bugs without making new bugs. Then he hit his token quota 3 weeks early and basically said he was going to stop working on it because he no longer could manually work the codebase. He didn’t ask for a rewrite, but AI advised him that his language choice was a poor fit and reworked things in another language, one that none of us use to that level of seriousness. It also made it largely based on super convoluted regular expressions.
The problem is that the leadership is singing the praises of these people, they were on their ‘leaderboards’ of AI adoption and much like the craze of praising “lines of codes”, we are neck deep in the most stupid non-technical evaluation of technical work you could imagine.
In other words, you’re arguing that their usage without discernment is detrimental.
Cory agrees on this very same article, and it also includes the same nuance this chain is trying to give voice to:
We wouldn’t be having this conversation if LLMs had been given the chance to grow into being the same way the web did. Whereas we would be having this same conversation with the letters swapped if corporations were the ones to spawn the WWW instead of the way it came about.
The reason is the same. There is only one war.
It’s not just “usage without discernment”, it’s that AI flattens some costs which are very obvious and very measurable (coding) in such a way that it introduces or amplifies other costs which are much more diffused and hard to measure (code and design reviewing, bug fixing, maintenance, adding new requirements), plus AI is totally incapable of doing the higher level tasks that shape what code needs to be done (technical analysis, requirements analysis and in bigger companies technical architecture).
People who are non-experts, aren’t really senior domain experts or have some kinds of unbalanced expertise (they’ve never really progressed beyond being a coder, or they don’t have full life-cycle experience with big projects or they’re in an industry or position where they just make the code, shove it out the door and it’s not their problem anymore) just look at the one thing they in their ignorance think is THE cost in programming - coding - and go “hey, this AI thing is amazing” even while AI is creating all sorts of much more time consuming problems which they don’t really understand formally (they think those things are just “bad luck”, “there’s nothing we can do about avoiding this” and “it’s just the way things are in programming”) that require the time of people with higher expertise levels (i.e. who are more costly) to solve and AI doesn’t even help with the kind of stuff which if done wrongly or not at all can condemn a software project to fail before it even starts like just half-way decent Technical and Requirements Analysis.
That’s why you get some programmers going “this AI shit is amazing” whilst the really senior software development types are just nodding their heads and thinking “these people are ignorant as fuck juniors”.
I don’t know… I just made a scheduling / timesheet creation app. Multi-user, overlapping clients and providers, multiple funding sources. Took 10 calendar days to make the initial app working part time, maybe 2-3 hours a day. Initially written in Python, decided at that point I’d rather have it in Go. Because the initial app had robust requirements and design docs, the translation to Go happened in less than 5 calendar days, with almost zero human involvement beyond telling the agent “continue” at each stopping point. After the Go translation was done (and debugged by the LLM to a flawless translation - only difference is that it runs faster), I was given a new timesheet to use for some of the workers, weekly instead of bi-weekly. Pay weeks start on Monday instead of Thursday. Various wrinkles about how the employees and clients and services are identified, weird sub-totals by service. All I told the LLM was: “Here’s a new timesheet that we’ll be using for some workers, design the necessary modifications and extensions to accomodate it.” It did, independently. It highlighted three shortcuts it took and I told it not to take those shortcuts, it adjusted.
That’s not quite rocket science, but it’s still impressive: to dissect the given .pdf, determine what data goes in what fields, in what formats, with what calculations, based on just reading the page, then adapt the existing app to fill it out automatically.
I truly don’t want to try your app… I don’t want all my data leaked or my PC, laptop or phone bricked just because a dumb AI-slop code error.
It’s not for you, anyway.
In my professional work, I have watched AI code reviews catch 10x more dumb slop human errors than human reviews used to the same time a year earlier, consistently for about 8 months now.