The funny thing is that if you never understand the codebase then you will keep thinking Claude is doing a great work delivering all this incredible software, when all it has done is created unnecessary tech debt.
But this is leaving out the part where the developers that clean up or rewrite... will do it using LLMs.
Have you tried refactoring or porting codebases larger than a million lines of code pre-gen-AI and again post-gen-AI? It's night and day difference. One would be insane to schedule a team on 8 months worth of grunt work porting from one language or framework to another which can now be done by 1 person in 4 weeks.
Of course the person driving it has to tell it exactly what to do and has to have the requisite knowledge to understand how to effectively structure or fix the software. Maybe new developers don't build this skill so easily anymore. But I don't see why a strong developers skills would atrophy in this case though unless they just never use their knowledge and never give instructions to the AI.
To developers speaking of skill atrophy: are you still making sure that when using LLMs you are actively exercising skills like system design, debugging, reviewing for clean code and just in general doing effective code review? If you are doing that, why do you feel skill atrophy? And if you aren't doing it, why not? What about LLMs prevents us from exercising these skills?
> A twenty line for loop. It overengineers most things.
Anecdote I like to tell.. I was working on a financial planning software, intentionally purely vibe coded as an experiment.
I eventually discovered AI had implemented seven duplicate copies of tax calculation functions. All of them different. All of them wrong. All of them giving different answers for same input.
Not even the most junior of newbie junior engineers would do something this crazy. But AI was happy to do it. It will solve the immediate problem, efficiently. Even if the most efficient solution is something ridiculous like this.
I have also a weird story to tell that a human did and it is as crazy as this. It happen in 2019 so no LLMs at all.
A person that was hired as an expert in our startup spent more than one week full time working on implementing his solution to the problem we were having. I checked the code after one week to see the progress and was curious how they are implementing an already crazy sounding idea.
I found that the whole week was spent re-implementing in python, python's built-in "float" function. That was it, the whole code was just that.
Our problem was related to financial services and their implementation of "float" was not even correct.
They would not do it in the span of a day or a week. But I’ve definitely seen something like that happen over a period of multiple months.
The llm just allows to generate faster.
We can feel smug about that but all it means is that we need to be clearer on our requirements and preferences up front.
State that similar functions should be in one place and there should be only one. Today there has to be compelling reason why that function is different from others. Normalise the function name based on what it does. Why are there different ones?
Then there are all the other guard rails in place.
Better guidance from mentors, reviewers, and automated project tooling helps everyone. Juniors, seniors, and engineers.
> The jump in capabilities in the last 6 months has been substantial.
What I would like to see is a chart graphing the model size against some objective measure of capabilities, specifically for coding.
It's easy to see gains when you're doubling the effort. What I want to know is if the extra effort is opening up more capabilities over time or fewer capabilities over time.
I have literally been hearing that, over and over and over, since '22.
And whilst it is obvious things are growing... Saying that, sounds almost entirely like the person saying it cannot objectively look at the environment. If everything has changed in the last six months, why has the industry not radically changed to match it?
Everything really did change with the Pentium II. It did with 3dfx. It did with Damerau's taken on Levenshtein. Hell, everything changed with React. The AI leap with seq2seq completely revolutionised the entire industry. But... Its kid, the LLM? Really?
Its different, in that when you teach that engineer, they either leave because now they hate you, or they grow. They change to meet the standards of a project, rather than inventing their own.
We don't get seniors, without juniors. I'd say more than half the job, is just... Learning. People grow.
Exactly. People ask how we get seniors with juniors using llm. The answer is the same. Review the code. Analyse write down what is wrong. What you expect to have been better. Force every change to be documented and explained enough.
It just takes forever now. The understanding is lower, the effort is lower, and frankly, I think the interest is lower too. I might be in the last generation who truly had fun working on a 'shrodinger' bug.
Yes. So find someone with intrinsic desire to engineer. Mentor them.
In the mean time put a plethora of guardrails in place to make sure the AI Train doesn’t derail production.
Oh. And keep showing your value. In the end every org can do with less low paid overeager uninterested juniors. Might as well let agents Do those tasks.
> Its different, in that when you teach that engineer, they either leave because now they hate you, or they grow.
Or they just have their own hubris and ignore your (provably better) suggestions because their way is "better/easier/how we've always done things".
And then you end up with someone sprinkling N+1 issues throughout the system and making systems with bad architectures throughout the years, not thinking about backpressure etc., as well as shoving ALL the dependencies into a single codebase cause they're not used to creating new ones, turning patches into eventual month long version upgrades because everything keeps breaking with anything newer than JDK 8 and some of the packages are deprecated and gahhhh I should pick up woodworking as a hobby.
Though, to address the original claim:
>> If it is a better engineer than you... You need practice.
This feels like a thought terminating cliche. Like, it will spit out bullshit every now and then, and make assumptions that I don't think that many engineers would (e.g. since a lot of each app is environment-specific), but at the same time when you guide it and give it examples, it can really be quite good! So not that unlike humans at all, even competent devs might not necessarily know about every pattern in any given codebase, especially when one has been around for 10 years and grown quite a bit.
It can be quite good if you have something like ArchUnit or your own tools for linting project architecture and patterns, alongside proper documentation that doesn't assume that you're a team member with X years of experience on system Y. AI just forces people to be less lazy and ignorant about knowledge transfer, which they should have also been for the sake of other humans!
I have never had it be every now and then. It is always bullshit, the first time around. Usually followed by, "You ignored the first three rules, and all the examples, try again."
I have worked with some amazingly incompetent devs. Some promoted into place to become someone else's problem, and some parachuted in through connections, and never once have they brought the continuous and unevolving frustration of modern LLMs.
I taught one of our "React SME"s, what Typescript even was. And they, were less likely to throw a ten thousand line fix, where twenty would do. They didn't see that the DB had a list of validation rules, and copy and paste those into seven different files, instead of just querying the DB, so the code would be kept up to date. And whilst I might have had to repeat that the DD is our source of truth, they never argued that implementing the DD wasn't aligned to the intended design.
AI isn't forcing people to be better with knowledge transfer. It's taking conversations that happened in person, that were back and forth and gave both people a better understanding of a thing, and turning it into a half-assed and out-of-date wiki. Without all the things that make a wiki actually useful to onboard the next newcomer.
I do get what some of these frustrations are, and where they come from. I'm the official documentation maintainer, because nobody else ever wrote anything down on expectations. Its fairly thankless. But... All I've seen AI do in that same realm, is exacerbate misunderstanding.
Like when it presented an Apex script (Salesforce) guaranteed to exceed governor limits to our junior, who took it and ran with it. And when it aborted, the AI spent half a day leading the junior around and telling them to change config settings - that the environment was the problem, not the code.
The fix? Toss the three layers of Queueables, and use Database.insert on a list. Job done. PR for 150LoC, instead of the offered 780LoC.
I am former java enterprise dev, so yes I often code this way. Unit testing, decomposition... Some projects CI refuse to merge commits with 20 line loop and duplicated code...
But that is not a point. Claude can code tight compact loops, it just needs to be instructed to do so! If it does "enterprise code", it means it had no instructions about code style.
If your documentation, spec, agent.md does not have proper guidance on coding style... yet another red flag!
Overabstracting, deduplicating things that don't need to be. Building metaclasses because it saw a single orchestrator in the whole codebase.
If it is a better engineer than you... You need practice.