I’m bullish on AI as tech but folks are starting to sniff out that the financials of everything going on at the moment aren’t sustainable for much longer.
I hope we have more of a “reality correction” than full blown bubble bursting, but the data is increasingly looking like we’re about to have a massive implosion that wipes out a generation of startups and sets the VC ecosystem back a decade.
The tech is way underpriced right now. It's basically a subsidized market right now, with the money flowing in coming from the private sector.
The problem here is that it remains to be seen who is willing to pay for the service once it's priced at cost or even with a margin. And based on valuations of AI companies one would expect a huge margin.
It's hard for me to imagine paying real money for something that gives me a maybe-hallucinated answer that I need to check every single time. A flaky test is worse than a failing test.
Plus I can run a reasonable LLM on my own hardware, so I don't even need to pay anyone else. And what I can run locally is only going to get better and better.
This is true, but this is also true for on-premise hosting vs cloud. And cloud has been booming for at least a decade before LLMs appeared. I suspect AI will follow a similar trajectory, i.e. companies don't move their AI deployments on-prem until they hit a certain scale.
This is very true, but I think the other point is that AI doesn't have much "moat". If a competitor can take a pre-trained Chinese LLM, fine tune it a bit, fiddle with the prompt, and ship a product which is not as good but way cheaper, then you've (or Oracle's) got a problem.
Actually, in that scenario the AI labs (OpenAI, Anthropic, etc) have a problem. The cloud providers (including Oracle!) will do with the models what they've been doing with open source software: just take it and run it on their infra and charge money for providing it as-a-service.
This is why you're seeing the AI labs now try to build their own data centers.
Yes LLMs hallucinate, no it's no longer 2022 and ChatGPT (gpt-3.5) is the pinnacle of LLM tech. Modern LLMs in an agentic loop can self correct, you still need to be on guard but if used correctly (yes, yes, holding it wrong etc. etc.) can do many, many tasks that do not suffer from "need to check every single time".
I must be holding it wrong then, because in my ChatGPT history I've abandoned 2/3rds of my conversations recently because it wasn't coming up with anything useful.
Granted, most of that was debugging some rather complicated typescript types in a custom JSX namespace, which would probably be considered hard even for most humans as well as there being comparatively few resources on it to be found online, but the issue is that overall it wasted more of my time than it saved with its confidently wrong answers.
When I look at my history I don't see anything that would be worth twenty bucks - what I see makes me think that I should be the one getting paid.
I think the reason people talk past each other on this is that some of them are using LLMs for every little question they have, and others are using them only for questions that they can't trivially answer some other way. Sure, if all your questions have straightforward, uncontroversial answers then the LLMs will often find them on the first try, but on the other hand you'd also find them on the first try on wikipedia, or the man page, or a google search. You'll only think the ChatGPT is useful if you've forgotten how to use the web.
If you're only asking genuinely difficult questions, then you need to check every single time. And it's worse, because for genuinely difficult questions, it's often just as hard to check whether it's giving garbage as it would have been to learn enough to answer the question in the first place.
If the problem as stated is "Performing an LLM query at newly inflated cost $X is an iffy value proposition because I'm not sure if it will give me a correct answer" then I don't see how "use a tool that keeps generating queries until it gets it right" (which seems like it is basically what you are advocating for) is the solution.
I mean, yeah, the result will be more correct answers than if you just made one-off queries to the LLM, but the costs spiral out of control even faster because the agent is going to be generating more costly queries to reach that answer.
Apologies that you're taking on the chin here. Generally, I'll just skip fantastical HN threads with a critical mass of BS like this, with pity, rather than an attempt to share (for more on that c.f. https://news.ycombinator.com/item?id=45929335)
Been on HN 16 years and never seen anything like the pack of people who will come out to tell you it doesn't work and they'll never pay for it and it's wrong 50% of the time, etc.
Was at dinner with an MD a few nights back and we were riffing on this, came to the conclusion is was really fun for CS people when the idea was AI would replace radiologists, but when the first to be mowed down are the keyboard monkeys, well, it's personal and you get people who are years into a cognitive dissonance thing now.
I want AI to be as strong as possible. I want AGI, I especially want super intelligence. I will figure out a new and better job if you give me super intelligence.
The problem is not cognitive dissonance, the problem is we don't have what we are pretending we have.
We have the dot com bubble but with a bunch of Gopher servers and the web browser as this theoretical idea yet to be invented and that is the bull case. The bear case is we have the dot com bubble but still haven't figured out how to build the actual internet. Massive investment in rotary phone capacity because everyone in the future is going to be using so much phone dial up bandwidth when we finally figure out how to build the internet.
Yeah, it really pulled the veil away, didn't it? So much dismissiveness and uninformed takes, from a crowd that had been driving automation forward for years and years and you'd think they'd get more familiar with these new class of tools, warts and all.
Say what exactly? Driving automation of all kind with Claude Code level tools has been incredibly fruitful. And once you spent sufficient time with them you know when and where they fall on their faces and when they provide real tangible reproducible benefits. I could not care less for the AI hype or bubble or whatever, I just use what I see works as I'm staring these tools down for 10h+/day.
The problem is that these conversations are increasingly drifting apart as everyone has different priors and experiences with this stuff. Some are stuck in 2023, some have so very specialized tasks that it's more work whipping the agent in line that it saves, others found a ton of automation cases where this stuff provides clear net benefits.
Don't care for AGI, AI girlfriends or LLM slop, but strap 'em in a loop and build a cage for them to operate in without lobotomizing themselves and there's absolutely something to be gained there (for me, at least).
really?
>>many tasks that do not suffer from "need to check every single time"
like which tasks?
How do you decide whether you need to check or not?
If you're asking it to complete 100 sequences, and if the error rate is 5%, which 5% of the sequences do you think it messed up or _thought_ otherwise? if the 5% is in the middle, would the next 50 sequences be okay?
I wonder if a price correction would be a boon for open source, with the economics of smaller / self hosted models making a lot more sense when API prices have to surge.
It's not actually subsidized and the economics of smaller/self-hosted models are a much, much, worse nightmare (source: guy who spent last 2 years maintaining llama.cpp && any provider you can think of) (why is it bad? same reason why 20 cars vs. 1 bus is bad. same reason why only being able to use transportation if you own a car would be bad)
Source on it being subsidized? :)
(there isn't one, other than an aggro subset of people lying to eachother that somehow literally everyone is losing money, while posting record profit margins)
(https://en.wikipedia.org/wiki/Hitchens%27s_razor)
And really the reason that it would be like that is that the models don't learn, per se, within their lifetime.
I'm told that each model is cashflow positive over its lifetime, which suggests that if the companies could just stop training new models the money would come raining down.
If they have to keep training new models though to keep pace with the changes in the world though then token costs would be only maybe 30% electricity and 70% model depreciation -- i.e. the costs of training the next generation of model so that model users don't become stranded 10 years in the past.
via govt relationships, long term irreplaceable services, debt or convictions.. Also don't forget the surveillance budgets and the best spigots there, win.
Generally, I worry HN is in a dark place with this stuff - look how this thread goes, ex. descendant of yours is at "Why would I ever pay for this when it hallucinates." I don't understand how you can be a software engineer and afford to have opinions like that. I'm worried for those who do, genuinely, I hope transitions out there are slow enough, due to obstinance, that they're not cast out suddenly without the skills to get something else.
It's subsidised by VC funding. At some point the gravy train stops and they have to pivot to profit so that the VCs deliver return-on-investment. Look at Facebook shoving in adverts, Uber jacking up the price, etc.
> I don't understand how you can be a software engineer and afford to have opinions like that
I don't know how you can afford not to realise that there's a fixed value prop here for the current behaviour and that it's potentially not as high as it needs to be for OpenAI to turn a profit.
OpenAI's ridiculous investment ability is based on a future potential it probably will never hit. Assuming it does not, the whole stack of cards falls down real quick.
(You can Ctrl-C/Ctrl-V OpenAI for all the big AI providers)
This is all about OpenAI, not about AI being subsidized...with some sort of directive to copy/paste "OpenAI" for all the big AI providers? (presumably you meant s/OpenAI/$PROVIDER?)
If that's what you meant: Google. Boom.
Also, perhaps you're a bit new to industry, but that's how these things go. They burn a lot of capital building it out b/c they can always fire everyone and just serve at cost -- i.e. subsidizing business development is different from subsiziding inference, unless you're just sort of confused and angry at the whole situation and it all collapses into everyone's losing money and no one will admit it.
You're replying to a story about a hyperscaler worrying investors about how much they're leveraging themselves for a small number of companies.
From the article:
> OpenAI faces questions about how it plans to meet its commitments to spend $1.4tn on AI infrastructure over the next eight years.
Someone needs to pay for that 1.4 trillion, that's 2/3 of what Microsoft makes this year. If you think they'll make that from revenue, that's fine. I don't. And that's just the infra.
I'm a big fan and user of AI but I don't see how you can say it's not subsidized. You can't just ignore the costs of training or staff or marketing or non-model software dev. The price charged for inference has to ultimately cover all those things + margin.
Also, the leaked numbers being sent to Ed Zitron suggest that even inferencing is underwater on a cost basis, at least for OpenAI. I know Anthropic claims otherwise for themselves.
Which isn't the same as saying LLMs and related technology aren't useful... they are.
But as you mentioned the financials don't make sense today, and even worse than that, I'm not sure how they could get the financials to make sense because no player in the space on the software side has a real moat to speak of, and I don't believe its possible to make one.
People have preferences over which LLM does better at job $XYZ, but I don't think the differences would stand up to large price changes. LLM A might feel like its a bit better of a coding model than LLM B, but if LLM A suddenly cost 2x-3x, most people are going to jump to LLM B.
If they manage to price fix and all jump in price, I think the amount of people using them would drop off a cliff.
And I see the ultimate end result years from now (when the corporate LLM providers might, in a normal market, finally start benefiting from a cross section of economies of scale and their own optimizations) being that most people will be able to get by using local models for "free" (sans some relatively small buy-in cost, and whatever electricity they use).
The most sobering statistic I've seen is that the entire combined amount of consumer spending on AI products is currently less than the revenue of Genshin Impact.
It certainly does but B2B revenue can also be much more "fake", in a sense. i.e. if Microsoft spends $500 million on OpenAI, which makes OpenAI spends $500 million on Azure... where does the profit come from? There have been a few interesting articles (which I unfortunately can't look up right now) recently describing how incestuous a lot of the B2B AI spend is, which is reminiscent of the dot-com bubble.
Well, Genshin Impact is at the forefront of predatory B2C business practice. It is a gacha game, engineered to extract as much money from its prey as possible.
On the other end, most AI company can afford to be generous with their user/consumer right now because they are being bankrolled by magic money.
The real test will be when they have to start the enshitification. Will the product still be enough to convince consumer to spend an amount of money guarantying a huge margin for the service provider ? Will they have to rely on whale desperately needing to talk to their IA girlfriend ? Or company and people who went deep into the whole vibe coding thing, and can't work without an agent ?
I think it is hard to say right now. But considering the price of the hardware and running it, I don't think they will have to price the service insanely to at least be profitable. To be as profitable as the market seems to believe, that's another story.
Regardless of your feelings on Genshin/gacha (which I agree is predatory), the point is that the revenue of a single game developed by a few hundred people is currently making more money than an entire industry which is "worth" trillions of dollars, according to the stock market, and is, according to Sam Altman, so fundamentally important to the US economy that the US government is an insurer of last resort who will bail out AI companies if their stock price falls too much.
Isn't AI just as bad if not worse here? I'd bet there are far more people who have been duped by ChatGPT (and others) to think it's their friend, lover, or therapist than people who are addicted to Genshin Impact.
I would be curious to see how it compares to the combined revenue of gay furry gacha games and VNs. Are we talking parity, multiples, or orders of magnitude? Anything other than the latter would be a bucket of cold water.
The money is in business licenses. Why only look at consumer? Consumers are mostly still using the free version which exists to convince employers to pay.
I don't know about a decade... the dotcom bubble bursting was pretty close to normal within 5 years or so. Still a long time, and from personal experience the 50% pay cut from before and a year later was anything but fun.
The market as a whole always recovers. But individual companies, or even entire industries can vanish without a trace. So betting on the entire market is a fairly safe bet, long-term. Betting on OpenAI is much more risky.
I hope we have more of a “reality correction” than full blown bubble bursting, but the data is increasingly looking like we’re about to have a massive implosion that wipes out a generation of startups and sets the VC ecosystem back a decade.