Not quite, it's got quite a bit more complicated with agentic use cases.
Prefill (input tokens) is heavily compute bound. And the ratio of input to output continues to rise, as typically in agentic sessions you have a few tokens output for a tool call and (many) thousands of input from the tool result.
Then you have cached input tokens, which is a totally different issue, system RAM or NVMe bound.
Obviously output tokens is VRAM memory bandwidth bound, but this is less and less of the bottleneck these days for overall agentic speed.
This is why I was careful to specify tokens per second not time to first token which is the prefill step you’re talking about. Clearly to run these models well you need both but as I said adding more chips or compute units can give you more latency where as overall memory bandwidth (throughput) is limited by access to fast memory.
I can easily use close to 100 million input tokens a day. A few million output tokens but at the end of the day maybe a thousand or so lines of code get written.
According to SemiAnalysis, both inference and post-training (RLVR) is mostly memory bandwidth bound. Only pre-training is compute bound, but it now only takes a small share of overall data center capacity.
There isA study about fire fighting. If you shit down every fire quickly then eventually you’ll have a major black swan. If you let fires burn a bit more then you’ll avoid black swan effect that burns down everything
I’m not sure if “let fires burn a bit more” is exactly what we’re after but hazard reduction burns are absolutely a thing and are an essential part of fire risk management. I don’t think your post deserved to get slammed like that.
The region can build up resistance to big fires by having small fires.
Why can the Chinese build models and Europe cannot? The algorithms behind this stuff are not that complicated, are they? Is it the cost of energy? The illegality and difficulty of obtaining all the data in the world? Lack of capital to start moonshot labs? Lack of optimism?
The Chinese just seem to have an ability to get it done without anywhere near the GPUs of the US and Europe can buy these GPUs.
I think relying on the US and China for AI is probably not ideal? For example I think Qwen have not released the Omni models as open weights in the past, it’d be good to know if they’re doing this here?
It's not that they can't, but why would they? It's a race to the bottom commodity wise, much smarter to build on top of the model layer like Legora and Lovable than make the models.
So the grand plan is to run US or Chinese models on hardware that you buy from US or China, and hope they'll keep selling us the hardware forever and giving away the models for free forever? Your moat is what? The prompts? The JavaScript for loop calling the model? Neat.
How many times will we do this? This line of thought is precisely why we can't build almost anything, and China can built almost everything. We could do X, but we're too smart - let's offload the actual work to China and we'll run our economy on IP and B2B deals.
In Europe AI is seen as evil that must be stopped. Unlike in the US, AI doomers have Commission well manned. Ursula addressed this in her State of Union speech yesterday. The EU rather chooses consumer protection, even if it means killing AI and maintaining status quo.
Because any AI company would be hit by hate and regulation derived from this, no VC invests in the EU. It's more state and large enterprise investments, good old East Germany style. And historically it has not been that efficient.
I agree with this, the rich actually are oddly MORE greedy in Europe and take double the equity for half the money (roughly) so founders get a lot less upside or they move their businesses to the US and get US capital at better rates.
The "Attention is all you need" paper authors where four (!) Europeans and one from US, amongst others. The "Top 10" lists are just complete rubbish, not worth the paper they are written on.
Europeans do not have the hustle mentality to break the law like Uber so you wont see them compete in anything data heavy. They also are quite risk adverse, probably from having a lower Gini coefficient.
ETH Zürich is up there high, particularly for computer science.
The LLM they were involved in last year (Apertus) still was a letdown.
As someone who actually went there, my impression is that Europe in general is complacent when it comes to computers, and any bright eyed student will get their motivation choked out of them in academia here.
If you want to do things with AI in Europe, you can have a bigger effect by working for a consulting company than being at a university. That's … not a good sitatution.
Simpler view for me: this is one of the most capital intense technologies to exist. Europe does not have enough capital to compete. China is building its own chips. It’s own everything. Silicon up. How do you compete with that. Only Google and maybe Aamazon is doing it domestically.
I feel people are too focused on US/China and don't pay attention to what is going on in the world.
ASML in the Netherlands for example was the only company in the world that makes EUV lithography machines, which all the major chip companies depend on. China recently reverse engineered their work to create machines since late 2025, but not sold commercially.
Ireland has chip production facilities.
EU might not be in the top 2, but it is not out of the running at all.
I believe the that the companies who claim to not train on my data are more likely to not train on my data than the companies who refuse to even claim they won't.
Also why Meta gets a +1, just charge less money on the training path.
I’m not sure that follows. You’re assuming that all those claims have the same weight, without considering the size, jurisdiction, reputation or even the general vibe of the company making that claim.
If you factor that in, then there are clearly different tiers: one you can trust, and one that may well just be saying that to increase market share with little reputational or legal consequences if they are found to be lying.
Yes I sometimes think the "don't train on my data" is actually a good signal for "this data/person is probably better to train on because they want to keep something private". The whole copyright system should have stopped these guys from training on everyone's data and it did not, if you think they care about the privacy checkbox I think you're dreaming personally, based on their past behavior.
Unfortunately, during that time is when they screwed around with the gold confiscations. So the best option would have been to illegally hoard gold until they struck the statute down?
I'm shocked that more people don't know about this. It seems insane that the "land of the free" (yeah I know, not really, but that's the pitch anyway) would have such an obviously command-economy policy through the heart of the cold war.
The US is going to debase itself endlessly through spend-print-spend-print. At some point they may load up enough debt that the economy suffers a gradual heat death, in the style of Japan, wherein too much of your national capital is going to debt maintenance, sitting in a low yield blackhole sucking the dynamism out of your system (instead of going to productive use, business expansion, R&D, et al).
There's absolutely nothing particularly interesting or special about the direction the US is going. It's very, very, very easy to see what's coming and has been for ~20 years (since Bush nearly doubled the size of the Federal Government and blew up our finances with simultaneous tax cuts + massive spending expansion, we've never turned back from the bleed).
Gold has gone up ~10x since the early Bush years precisely because of the USD debasement, that's the reduction in value in the dollar being represented in the ultimate store of value. All of it has been remarkably predictable. I've been chirping about it forever here and there's nothing special about my insight either, this stuff is plain as day national econ 101.
i know what you mean. 2008 and covid taught me to stop underestimating the abilities/nerve of our financial schemers and their political representatives to kick the can down the road. but i am too anxious about it to think straight. what to do?
> We were 10 years from paying off the national debt when Clinton left office. 10 years!
To be fair, if the US government had continued paying off the debt, the only way the (global) books would've balanced would've been for the US populace and businesses to take on waaaaayyyyyy more debt. Not sure if that would've been better.
(For a much better explanation of this, read Trade Wars are Class Wars).
The turning point is approaching: Interest rates will gradually overtake all other gov't expenditures. All politics will revolve around shoring up the parasitic drain on the rest of absolutely everything.
Why? Seems the US doesn't like democracy anymore and the EU for all its faults still does. I can see Japan joining too and potentially South Korea, Philippines, and others who are concerned about the state of the world and want to work together rather than drive the world apart.
If I’d said to you last year “Hey! I bet Canada is going to join the EU! No, of course not as charter-violating full member, as a newly invented ‘associate member’!” You’d probably have said “That’s both crazy and oddly specific, the chances of that have to be approaching zero”.
Further cooperation between the EU and Canada has been talked about for a decade or so (see CETA). Just because it is news to you, it doesn't mean it is news.
Yes but also "associate member" is, despite specificity, less surprising than full membership would have been.
Same as economic gravity meant that Brexit didn't disconnect UK from EU it only made trade harder, neither this kind of deal nor full membership can be much more than a band-aid for Trump's BS, it won't fully compensate for the harm he and his administration are doing to Canada.
I think the world would be jumping into alignments like this if the US hadn't spent the last 80 years building a "nuclear umbrella" for the "western alliance" with itself in the centre.
I bet many countries would prefer to disassociate with US foreign policy right now if they weren't security dependent.
I expect after this destructive episode in history has played out, if we're all still alive and free in another decade then there will be alternative umbrellas.
I live here. I talk to older people. They tell me about the horrors of WW2 and how glad they are the EU succeeded as a peace project. Before the EU, we had centuries of cruelties between what are now members of the same block.
It might be good to expand your perspective a little. Standard of living isn't just the economy.
(Intended as a reply to parent post, which is now dead)
I live in the EU, and its nothing like you describe. In my country life has improved sharply since it joined the EU. It's improved things in a way that would never of happened otherwise. It's gone from nearly a third world country to an advanced economy in a very short period. And I've enjoyed the ability to live work and travel around the EU freely. Europe is an incredible place.
Yes, it was nice in the 99s snd 2000s when they were building a better EU. And then it devolved into regulation after regulation, strings attached to everything and so on.
Now we’re in 2026 where I can’t buy a platic fork and knife, I have to drink everything with a paper straw and so on.
The dream was nice but it was hijacked along the way.
I thank you for this very high level analysis that gives absolutely zero actual content or details because big funny words, but I think we have to agree to disagree on that one.
And on the climate policies in particular, my personnal belief is that we haven't gone far enough fast enough, limit access to our market to product made using climate respecting or restoring ways should have been the rule decades ago already.
I know some people look at the way the world climate is changing and think "ehh so what / it's not really us doing it", but that's not how we see things here.
Have you talked to the US, China or India about the climate? You also have not provided any actual content, sorry. 20% of people in the EU are at risk of poverty or exclusion, but sure let's mown down the economy. And as for the lack of investments you can see what is happening to the automotive industry. Go ask for any amount of money in the EU vs the US and come back to me. I'm not against green tech but very much against the austerity pushed on us by the ruling bureaucracy.
How funny, they chose very beautiful old architecture and very ugly modern architecture and invented a result - why not choose things like the Bahai temple in Delhi or Fallingwater by Frank Lloyd Wright? I'm sure there are some old ugly buildings and the ones that people hated probably got torn down so there's survivorship bias in play here too!
People here are assuming that current CEOs are not actively destroying their companies, maybe the agent presented here is actually a very accurate simulation!
You know there was a lab that was cutting edge in Europe and it is still there - Deepmind - but the UK let it be sold to the US for a fraction of what it's worth today - $400m. We just allow US companies to asset strip the EU and then people complain about us never having big companies here...
Deepmind would likely be worth conservatively $200 BILLION today (probably more without the overheads of joining Google to be honest) or around 500 times the sale price in 2014, not a bad return.
How much of the current valuation is due to Google's resources and support, though?
"Sell company now and it's worth more a decade later" is the entire point; acquisitions are an investment and the investment is supposed to appreciate.
It’s impossible to know for sure but if they’d stayed private I think they’d be more not less successful, just because startups can move more nimbly. Although we’d need to define success, we’d be here forever…
It wouldn't have. They had over a billion in debt, which Google casually fixed. Google then very conveniently supplied them with GCP server power. And remember all of this was in like 2014, way before the AI hype. They wouldn't have had access to capital to survive long enough to make it to today.
reply