Drivers for laptops. Do all the sound cards work flawlessly? Is the power usage/battery life similar? Sadly this is a big part of what holds it back.

> By the time you’ve sorted out a complicated idea into little steps that even a stupid machine can deal with, you’ve certainly learned something about it yourself.

I love these quotes. I got a much deeper, more elegant understanding of the grammar of a human language as I wrote a phrase generator and parser for it. Writing and refactoring it gave me an understanding of how the grammar works. (And LLMs still confidently fail at really basic tasks I ask them for in this language.)

Except your team is full of occasionally insane "people" who hallucinate, lie, and cover things up.

We are trading the long term benefits for truth and correctness for the short term benefits of immediate productivity and money. This is like how some cultures have valued cheating and quick fixes because it's "not worth it" to do things correctly. The damage of this will continue to compound and bubble up.

I agree. The further I have progressed into my career the more I have been focused on the stability, maintainability and "supportability" of the products I work on. Going slower in order to progress faster in the long run. I feel like everyone is disregarding the importance of that at the moment and I feel quite sad about it.

Not only that, there’s this immense drive for “productivity” so they have more time to… Do more work. It’s insanity.

This is a fair argument but it’s rapidly becoming a non-argument.

LLMs have come a long way since ChatGPT 4.

The idea that they’ll always value quick answers, and always be prone to hallucination seems short-sighted, given how much the technology has advanced.

I’ve seen Claude do iterative problem solving, spot bad architectural patterns in human written code, and solve very complex challenges across multiple services.

All of this capability emerging from a company (Anthropic) that’s just five years old. Imagine what Claude will be capable of in 2030.

> The idea that they’ll always value quick answers, and always be prone to hallucination seems short-sighted, given how much the technology has advanced.

It’s not shortsighted, hallucinations still happen all the time with the current models. Maybe not as much if you’re only asking it to do the umpteenth React template or whatever that should’ve already been a snippet, but if you’re doing anything interesting with low level APIS, they still make shit up constantly.

> All of this capability emerging from a company (Anthropic) that’s just five years old. Imagine what Claude will be capable of in 2030.

I don't believe VC-backed companies see monotonic user-facing improvement as a general rule. The nature of VC means you have to do a lot of unmaintainable cool things for cheap, and then slowly heat the water to boil. See google, reddit, facebook, etc...

For all we know, Claude today is the best it will ever be.

The current models had lots and lots of hand written code to train on. Now stackoverflow is dead and github is getting filled with AI generated slop so one begins to wonder whether further training will start to show diminishing returns or perhaps even regressions. I am at least a little bit skeptical of any claim that AI will continue to improve at the rate it has thus far.

If you don't really understand how LLMs of today are made possible, it is really easy to fall into the trap of thinking that it is just a matter of time and compute to attain perpetual progress..

I have not found that to be true on a personal level, but in fairness it does seem to be a widely reported problem. At its core, I think it is an issue of alignment. That is something different than skill.

I agree with you, but considering the state of modern software, I think the values "truth and correctness" have been abandoned by most developers a long time ago.

Be that as it may, we shouldn’t be striving to accelerate the decline, and be recruiting even more people who never learned those values.

It’s the Eternal September of software (lack of) quality.

> Except your team is full of occasionally insane "people" who hallucinate, lie, and cover things up.

Wait.. are we talking about LLMs or humans here?

Humans are accountable, an LLM subscription is not..

The humans operating the LLM are accountable.

That is the point. It is nonsense to delegate your responsibility to something that is neither accountable nor reliable if you care about not tanking your reputation..

Tests cannot show the absence of bugs.

These are fundamentals of CS that we are forgetting as we dismantle all truth and keep rocketing forward into LLM psychosis.

> I care about this. I don't want to push slop, and I had no real answer.

The answer is to write and understand code. You can't not want to push slop, and also want to just use LLMs.

I hate to pile on the criticism here but this gives me uneasy futuristic vibes.

My dad recently passed away but some of the sweetest things we remember as kids was how he would always tell us "make up stories." They were silly little stories that were probably lame, but we could feel his love for us as he took the time to spin up some silly little story. I would never trade that for the best LLM creativity.

> introducing an error or two in formal proof systems often means you’re getting exponentially further away from solving your problem

I wish people understood that this is pretty much true of software building as well.

Now shudder at the thought that people are pushing towards building more and more of the world's infrastructure with this kind of thinking.

Now shudder at the fact that the error rate for hunan-written software isn't much better: https://xkcd.com/2030/

That is a great xkcd comic, but it doesn't show that the error rate "isn't much better." But are there are sources that have measured things and demonstrated this? If this is a fact I am genuinely interested in the evidence.

The problem is that the LLM's sources can be LLM generated. I was looking up some health question and tried clicking to see the source for one of the LLMs claim. The source was a blog post that contained an obvious hallucination or false elaboration.

Excellent observation. I get so frustrated every time I hear the "we have test-suites and can test deterministically" argument. Have we learned absolutely nothing from the last 40 years of computer science? Testing does not prove the absence of bugs.

Don't worry, the LLM also makes the tests. /s

Trading pesky things like accuracy and employees for compute/shell games; money-concentrating machine go brrrrrt

It seems a better and fuller solution to a lot of these problems is to just stop using AI.

I may be an odd one but I'm refusing to use agents, and just happily coding almost everything myself. I only ask a LLM occasional questions about libraries etc or to write the occasional function. Are there others like me put there?