Easy to forget but there was a ton of industry+investor excitement around computer vision from ~2015-2021, to the extent that the "MLops" niche sprung up around it. This was called AI at the time, and mostly went out the window when general-pupose pretrained models arrived.
We’re reaching a saturation threshold where older models are good enough for many tasks, certainly at 100x faster inference speeds. Llama3.1 8B might be a little too old to be directly useful for e.g. coding but it certainly gets the gears turning about what you could do with one Opus orchestrator and a few of these blazing fast minions to spit out boilerplate…
Remember when GPT-2 was “too dangerous to release” in 2019? That could have still been the state in 2026 if they didn’t YOLO it and ship ChatGPT to kick off this whole race.
I was just thinking earlier today how in an alternate universe, probably not too far removed from our own, Google has a monopoly on transformers and we are all stuck with a single GPT-3.5 level model, and Google has a GPT-4o model behind the scenes that it is terrified to release (but using heavily internally).
Before ChatGPT was even released, Google had an internal-only chat tuned LLM. It went "viral" because some of the testers thought it was sentient and it caused a whole media circus. This is partially why Google was so ill equipped to even start competing - they had fresh wounds of a crazy media circus.
My pet theory though is that this news is what inspired OpenAI to chat-tune GPT-3, which was a pretty cool text generator model, but not a chat model. So it may have been a necessary step to get chat-llms out of Mountain View and into the real world.
> some of the testers thought it was sentient and it caused a whole media circus.
Not "some of the testers." One engineer.
He realized he could get a lot of attention by claiming (with no evidence and no understanding of what sentience means) that the LLM was sentient and made a huge stink about it.
He was unfairly labelled as a lunatic early on. I'd implore anyone reading this thread to see what he had to say for yourself and form your own opinion: https://youtube.com/watch?v=kgCUn4fQTsc
Is that really the case in the last few years/decades?
My understanding is that any company that can (read: has enough money for good lawyers), will prefer to use trade secrets for a combination of reasons, a big one being that competitors cannot use that technology after 10 years/when the patent expires.
Admittedly this was from my entrepreneurship classes in a European uni, so I'm not sure how it is in different places in the world.
Patents in the US are 20 years. Given how short sighted modern companies are, I can’t imagine anyone at any large company is even planning for something 20 years in the future, much less placing much value in an outcome that far out.
They didn't YOLO ChatGPT. There were more than a few iterations of GPT-3 over a few years which were actually overmoderated, then they released a research preview named ChatGPT (that was barely functional compared to modern standards) that got traction outside the tech community because it was free, and so the pivot ensued.
In 2019 the technology was new and there was no 'counter' at that time. The average persons was not thinking about the presence and prevalence of ai in the way we do now.
It was kinda like a having muskets against indigenous tribes in the 14-1500s vs a machine gun against a modern city today. The machine gun is objectively better but has not kept up pace with the increase in defensive capability of a modern city with a modern police force.
That's rewriting history. What they said at the time:
> Nearly a year ago we wrote in the OpenAI Charter : “we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research,” and we see this current work as potentially representing the early beginnings of such concerns, which we expect may grow over time. This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas. -- https://openai.com/index/better-language-models/
Then over the next few months they released increasingly large models, with the full model public in November 2019 https://openai.com/index/gpt-2-1-5b-release/ , well before ChatGPT.
> Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT‑2 along with sampling code (opens in a new window).
"Too dangerous to release" is accurate. There's no rewriting of history.
> Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper.
I wouldn't call it rewriting history to say they initially considered GPT-2 too dangerous to be released. If they'd applied this approach to subsequent models rather than making them available via ChatGPT and an API, it's conceivable that LLMs would be 3-5 years behind where they currently are in the development cycle.
I'm also looking for "proof of pulse" somewhere and can't find it on the linked site, on Amazon, in his HN bio, or with a Google search. Unfortunately a necessity in the AI era.
What is proof of pulse? That I'm alive? This is my first time posting here i just signed up. This is also my first book that I have attempted to publish myself.
Yes, proof that you are a human. With so much AI slop on the internet you have to be defensive about your attention to avoid wasting it on low-effort LLM outputs.
I understand 100 percent! Too many people just copy paste blindly and I can't even look at social media feeds anymore. I spend a lot of my time thinking about where Ai is going to take us in the next few years.
It's also more than a little misleading to compare to the 2022 peak. Anybody who was hiring software engineers in 2020-2022 or being hired as one knows that was a wild and unsustainable period.
What, you mean a person who has only previous interacted with computers via smartphones taking a 6 month "JavaScript bootcamp" and getting a $150k/year salary on the other side isn't sustainable?
> This is supported out of the box by any agent worth its salt, including OpenCode and CC.
I thought Claude Code didn't support AGENTS.md? At least according to this open issue[0], it's still unsupported and has to be symlinked to CLAUDE.md to be automatically picked up.
You're right, for CC it's "nested CLAUDE.md files". The support I meant was about the "automatic inclusion in context upon sibling-or-child file touch" feature, rather than the name of the file.
Fair, I was hoping there was a feature that I was missing. Minor papercut to have to include harness-specific files/symlinks in your repo but it's probably a temporary state until the tools and usage patterns are more settled.
Nah, this is intentional by Anthropic, out of the top 20 coding agents 19 support AGENTS.md (fake numbers but I've seen someone else go through them). It's just a dumb IE6-style strategy.
Exactly, it's the same documentation any contributor would need, just actually up-to-date and pared down to the essentials because it's "tested" continuously. If I were starting out on a new codebase, AGENTS.md is the first place I'd look to get my bearings.
Kind of looks like vibecoding is doing to SaaS what Chinese mass manufacturing did to physical products two decades ago. Only the marketing and distribution matter in a world where it's very easy for others to clone something and sell it at a lower price.
> Only the marketing and distribution matter in a world where it's very easy for others to clone something and sell it at a lower price
Great point. AI remixes and rips-off existing code-bases in a manner that is impossible to attribute copyright violation making it legal. ie, Perfect cloning. In a world where cloning is legal, the engineering cost of product drops to zero. That is where software production could be headed. What remains is marketing/distribution/sales.
There will remain niches solving "hard problems" which cant be cloned, but those will be rare. Hard problems are where a lot of engineering complexity resides, involving interacting components for which there are no examples in training datasets to copy from. For example, a complex distributed system or hardware with multiple nuanced tradeoffs.
And yet people can still make money producing and selling high quality physical products. It's a smaller market but there are people who don't want mass produced chinese crap and they go out of their way to find it.
There will be people who will pay for "human coded" software if it is better. Quality is always a differentiator that some people will pay for.
reply