I agree with your fundamental point. However, I don't think steady erosion of mastery is the only way that these next years have to go, even if it looks the most likely at present. Supposing LLMs or whatever future architecture surpass even the greatest human minds in intelligence, why is that situation fundamentally different to living in a world with Einstein, i.e. a level of mastery I'll never reach before the end of my life? As one interested in the depths, I prefer to live in a world with peaks ever greater than myself---it doesn't prevent me from going as deep as I can, inspired by where they've reached, and doing the things that matter to me.
Turing's view, in fact, is similar: "There would be great opposition [to AI] from the intellectuals [read programmers in the context of this thread] who were afraid of being put out of a job. It is probable though that the intellectuals would be mistaken about this. There would be plenty to do, i.e. in trying to keep one’s intelligence up to the standard set by the machines, for it seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers. There would be no question of the machines dying, and they would be able to converse with each other to sharpen their wits."
[0] Thomas Bernhard's The Loser is a fantastic account of the opposite standpoint---of the second best piano student, who cannot stand existing in a world with Glenn Gould.
Amazon's free cash flow rises year over year (apart from the post-COVID period) [0] while Walmart's doesn't [1] and price multiples are largely determined by expected FCF over time not directly revenue/EBITDA. FCF rain or shine maps roughly onto possibility of paying out dividends/buybacks, which determines the value of an equity in capital markets ("discounted present value of a company's future cash flows").
Honestly, I don't think it's irrational: the car industry is just horrible from a business perspective, which is why Tesla had to be financed for so long by crypto scams and most investors wouldn't touch it. Historically (if of course briefly/crudely), it was always a debt-backed gamble on overproduction hoping you could expand forever globally without competition (Ford) or into new market segments through financing (GM).
It's paywalled unfortunately, but [1] is an illustrative Financial Times article discussing car manufacturer behavior in relation to Covid shutdowns and strikes. Many firms found the manufacturing shutdowns to be a boon: the winning strategy to accept it as a cost cut and just raise prices on existing inventory for above average financial performance.
My sense is that Tesla is now just taking that a step further by getting rid of their Fordist aspirations and applying the unarguably successful Apple model to the automotive industry. They don't want to mass produce cars and hope for X% conversion rate to software and services over time: they literally don't want customers who are not able or not going to pay for recurring software services. Software is where free cash flow comes from and free cash flow is where dividends/buybacks come from, which determines the value of an equity. That, of course, is why we get paid well.
I end with the disclaimer that obviously I don't believe the world should be meticulously and exclusively organized for the production of free cash flow, but I do think it's important to understand the logic.
I don't know what it looks like on the ground now, but Scala was the defacto language of data infrastructure across the post-Twitter world of SV late stage/growth startups. In large part, this was because these companies were populated by former members of the Twitter data team so it was familiar, but also because there was so much open source tooling at that point. ML teams largely wrote/write Python, product teams in JS/whatever backend language, but data teams -- outside of Google and the pre-Twitter firms -- usually wrote Scala for Spark, Scalding etc in the 2012-2022ish era.
I worked in Scala for most of my career and it was never hard to get a job on a growth stage data team or, indeed, in finance/data-intensive industries. I'm still shocked at how the language/community managed to throw away the position they had achieved. At first I was equally shocked to see the Saudi Sovereign Wealth fund investing in the language, but then realized it was just 300k from the EU and everything made sense.
It's still my favorite language for getting things done so I wouldn't be upset with a comeback for the language, but I certainly don't expect it at this point.
Mostly the latter. Scala 3 is almost completely irrelevant to the big data space so far. Databricks took six years before upgrading their proprietary Spark runtime to Scala 2.13. Flink dropped the Scala API before even moving to 2.13. I don't know if Scio will seriously attempt the move to Scala 3. All of them suffer from Twitter libraries being abandoned, which isn't insurmountable, but an annoyance still.
And I don't think it matters anymore. I predict that the JVM will eventually be out of the equation. We're already seeing query engines being replaced by proprietary or open source equivalents in C++ or Rust. Large scale distribution is less of a selling point with modern cloud computing. Do you really need 100 executors when you can get a bare metal instance with 192, 256 or 384 cores?
People want a dataframe API in Python because that's what the the ML/DS/AI crowd knows. Queries and processing will be done in C++ or Rust, with little or even zero need for a distributed runtime. The JVM and Scala solve a problem that simply won't exist anymore.
Yeah, this is certainly the correct take. There's an alternate timeline where the Scala community focused during the peak on making it a better language for numeric computing/ML rather than the Nth category theoretic framework, but here we are. At a job almost a decade ago, we made some progress on an open source dataframes (and unfortunately proprietary data visualization) library for Scala, but we didn't get far enough before the company closed and the project died [1].
Still my favorite language I had the privilege to work with professionally for over a decade. However, in this post-JVM world, I'm actually excited to see a lot more OCaml discussion on here lately. The Jane Street work on OxCaml is terrific after a long period of stagnation with the language. I'm using it for most of my projects these days.
Yeah, I've lived in London, New York and San Francisco for work and the first had the lowest cost of living in absolute terms. Local developer salaries are, however, shockingly low outside of finance and consulting to US eyes.
The public transportation point is definitely key: London is just so unspeakably large spatially and it's all more or less well-connected that there isn't the same scarcity of commutable apartments as in NYC/SF. It wasn't uncommon for older colleagues to even commute in from Kent or elsewhere in the English countryside -- and often their morning train wasn't much longer than my own.
It seems to all be debt financed, i.e. just a private equity model slightly specialized for tech. The "innovation" is that Bending Spoons has an in-house engineering team it seems they try to keep constant yet scale out to all the acquisitions. I hadn't looked into them much before, but https://www.colinkeeley.com/blog/bending-spoons-operating-ma... is an interesting report -- though not focused on the finance side.
Really fantastic work! Can't wait to play around with your library. I did a lot of work on this at a past job long ago and the state of JS tooling was so inadequate at the time we ended up building an in-house Scala visualization library to pre-render charts...
More directly relevant, I haven't looked at the D3 internals for a decade, but I wonder if it might be tractable to use your library as a GPU rendering engine. I guess the big question for the future of your project is whether you want to focus on the performance side of certain primitives or expand the library to encompass all the various types of charts/customization that users might want. Probably that would just be a different project entirely/a nightmare, but if feasible even for a subset of D3 you would get infinitely customizable charts "for free." https://github.com/d3/d3-shape might be a place to look.
In my past life, the most tedious aspect of building such a tool was how different graph standards and expectations are across different communities (data science, finance, economics, natural sciences, etc). Don't get me started about finance's love for double y-axis charts... You're probably familiar with it, but https://www.amazon.com/Grammar-Graphics-Statistics-Computing... is fantastic if you continue on your own path chart-wise and you're looking for inspiration.
Thanks - and great question about direction. My current thinking: Focus on performance-first primitives for the core library. The goal is "make fast charts easy" not "make every chart possible." There are already great libraries for infinite customization (D3, Observable Plot) - but they struggle at scale.
That said, the ECharts-style declarative API is intentionally designed to be "batteries included" for common cases. So it's a balance: the primitives are fast, but you get sensible defaults for the 80% use case without configuring everything. Double y-axis is a great example - that's on the roadmap because it's so common in finance and IoT dashboards. Same with annotations, reference lines, etc. Haven't read the Grammar of Graphics book but it's been on my list - I'll bump it up. And d3-shape is a great reference for the path generation patterns. Thanks for the pointers!
Question: What chart types or customization would be most valuable for your use cases?
Most of my use cases these days are for hobby projects, which I would bucket into the "data science"/"data journalism" category. I think this is the easiest audience to develop for, since people usually don't have any strict disciplinary norms apart from clean and sensible design. I mention double y-axes because in my own past library I stupidly assumed no sensible person would want such a chart -- only to have to rearchitect my rendering engine once I learned it was one of the most popular charts in finance.
That is, you're definitely developing the tool in a direction that I and I think most Hacker News readers will appreciate and it sounds like you're already thinking about some of the most common "extravagances" (annotations, reference lines, double y-axis etc). As OP mentioned, I think there's a big need for more performant client-side graph visualization libraries, but that's really a different project. Last I looked, you're still essentially stuck with graphviz prerendering for large enough graphs...
Ha - the double y-axis story is exactly why I want to get it right. Better to build it in properly than bolt it on later.
"Data science/data journalism" is a great way to frame the target audience. Clean defaults, sensible design, fast enough that the tool disappears and you just see the data.
And yeah, graphviz keeps coming up in this thread - clearly a gap in the ecosystem. Might be a future project, but want to nail the 2D charting story first and foremost.
Thanks for the thoughtful feedback - this is exactly the kind of input that shapes the roadmap.
Well, even the idea of "diagnosis" in this case implies that there is something wrong. I saw the whole idea of aphantasia/variations in mental imagery enter the mainstream over the past ~decade, it's really disheartening how people just can not ever accept that there are differences between people without immediately branding one type as good and the other as bad.
So true! I was well into my thirties until I learned that people actually can "see" images. I was totally perplexed by this revelation. After some research I realized that this also applies to taste, smell, sounds.. and none of them I can "imagine".
In hindsight this explained a lot of things. One example would be that I always was bad at blindfold chess even though I was a decent chess player. Before, I never understood how people can do this.
Still I am absolutely fine. I can recognize all these things. I can describe them. I just can "imagine" them.
After the first shock you understand that everything has pros and cons. E.g. I never have trouble sleeping. I close my eyes and turn the world around me off. My wife can see images very vividly and always has trouble going to sleep.
In the end we just need to accept that the brain is very complex and each of us has developed / adapted the best way, allowed by our biology.
That's so funny: I also first started to realize I had aphantasia during a period when I was taking chess very seriously during university. Unlike even lesser skilled peers, it was so difficult for me to understand games written out in chess books without playing them out on the board and I couldn't understand why...
Experiences like that are how I understand the question of 'shame' relating to aphantasia and the importance of 'diagnosis'/understanding how your mind actually works. 'Diagnosis' just helps you understand how to adapt and prevents you from slamming your head against approaches that won't work no matter how hard you try.
Similarly on sleep, I can sleep anywhere anytime with little effort and always tell my wife, who often has insomnia, "just close your eyes until you sleep" to her frustration.
What's really remarkable is how similar the life experiences are of most who have aphantasia...
I definitely have memories linked to smell, but I can't imagine or remember and pull them up on demand, I am reminded of them when I detect that scent. I can make myself imagine/remember sourness though, but not other flavors. Just thinking of lemon, citrus, pickles, etc. makes my mouth water and start tasting sourness.
I hear your argument, but short of major algorithmic breakthroughs I am not convinced the global demand for GPUs will drop any time soon. Of course I could easily be wrong, but regardless I think the most predictable cause for a drop in the NVIDIA price would be that the CHIPS act/recent decisions by the CCP leads a Chinese firm to bring to market a CUDA compatible and reliable GPU at a fraction of the cost. It should be remembered that NVIDIA's /current/ value is based on their being locked out of their second largest market (China) with no investor expectation of that changing in the future. Given the current geopolitical landscape, in the hypothetical case where a Chinese firm markets such a chip we should expect that US firms would be prohibited from purchasing them, while it's less clear that Europeans or Saudis would be. Even so, if NVIDIA were not to lower their prices at all, US firms would be at a tremendous cost disadvantage while their competitors would no longer have one with respect to compute.
All hypothetical, of course, but to me that's the most convincing bear case I've heard for NVIDIA.
People will want more GPUs but will they be able to fund them? At what points does the venture capital and loans run out? People will not keep pouring hundreds of billions into this if the returns don't start coming.
There is a real chance that the Japanese carry trade will close soon the BoJ seeing rates move up to 4%. This means liquidity will drain from the US markets back into Japan. On the US side there is going to be a lot of inflation between money printing, refund checks, amortization changes and a possible war footing. Who knows?
Doesn't even necessarily need to be CUDA compatible... there's OpenCL and Vulkan as well, and likely China will throw enough resources at the problem to bring various libraries into closer alignment to ease of use/development.
I do think China is still 3-5 years from being really competitive, but still even if they hit 40-50% of NVidia, depending on pricing and energy costs, it could still make significant inroads with legal pressure/bans, etc.
OpenCL is chronically undermaintained & undersupported, and Vulkan only covers a small subset of what CUDA does so far. Neither has the full support of the tech industry (though both are supported by Nvidia, ironically).
It feels like nobody in the industry wants to beat Nvidia badly enough, yet. Apple and AMD are trying to supplement raster hardware with inference silicon; both of them are afraid to implement a holistic compute architecture a-la CUDA. Intel is reinventing the wheel with OneAPI, Microsoft is doing the same with ONNX, Google ships generic software and withholds their bespoke hardware, and Meta is asleep at the wheel. All of them hate each other, none of them trust Khronos anymore, and the value of a CUDA replacement has ballooned to the point that greed might be their only motivator.
I've wanted a proper, industry-spanning CUDA competitor since high school. I'm beginning to realize it probably won't happen within my lifetime.
The modern successor to OpenCL is SYCL and there's been some limited convergence with Vulkan Compute (they're still based on distinct programming models and even SPIR-V varieties under the hood, but the distance is narrowing somewhat).
I suspect major algorithmic breakthroughs would accelerate the demand for GPUs instead of making it fall off, since the cost to apply LLMs would go down.
> The proposition that technological progress that increases the efficiency with which a resource is used tends to increase (rather than decrease) the rate of consumption of that resource.
There will always be an incentive to scale data centers. Better algorithms just mean more bang per gpu, not that “well, that’s enough now, we’ve done it”.
Even if LLMs didn't advance at all from this point onward, there's still loads of productive work that could be optimized / fully automated by them, at no worse output quality than the low-skilled humans we're currently throwing at that work.
inference requires a fraction of the power that training does. According to the Villalobos paper, the median date is 2028. At some point we won't be training bigger and bigger models every month. We will run out of additional material to train on, things will continue commodifying, and then the amount of training happening will significantly decrease unless new avenues open for new types of models. But our current LLMs are much more compute-intensive than any other type of generative or task-specific model
Run out of training data? They’re going to put these things in humanoids (they are weirdly cheap now) and record high resolution video and other sensor data of real world tasks and train huge multimodal Vision Language Action models etc.
The world is more than just text. We can never run out of pixels if we point cameras at the real world and move them around.
I work in robotics and I don’t think people talking about this stuff appreciate that text and internet pictures is just the beginning. Robotics is poised to generate and consume TONS of data from the real world, not just the internet.
While we may run out of human written text of value, we won't run out of symbolic sequences of tokens: we can trivially start with axioms and do random forward chaining (or random backward chaining from postulates), and then train models on 2-step, 4-step, 8-step, ... correct forward or backward chains.
Nobody talks about it, but ultimately the strongest driver for terrascale compute will be for mathematical breakthroughs in crypography (not bruteforcing keys, but bruteforcing mathematical reasoning).
Yeah, another source of "unlimited data" is genetics. The human reference genome is about 6.5 GB, but these days, they're moving to pangenomes, wanting to map out not just the genome of one reference individual, but all the genetic variation in a clade. Depending on how ambitious they are about that "all", they can be humongous. And unlike say video data, this is arguably a language. We're completely swimming in unmapped, uninterpreted language data.
Inference leans heavily on GPU RAM and RAM bandwidth for the decode phase where an increasingly greater amount of time is being spent as people find better ways to leverage inference. So NVIDIA users are currently arguably going to demand a different product mix when the market shifts away from the current training-friendly products. I suspect there will be more than enough demand for inference that whatever power we release from a relative slackening of training demand will be more than made up and then some by power demand to drive a large inference market.
It isn’t the panacea some make it out to be, but there is obvious utility here to sell. The real argument is shifting towards the pricing.
> We will run out of additional material to train on
This sounds a bit silly. More training will generally result in better modeling, even for a fixed amount of genuine original data. At current model sizes, it's essentially impossible to overfit to the training data so there's no reason why we should just "stop".
You'd be surprised how quickly improvement of autoregressive language models levels off with epoch count (though, admittedly, one epoch is a LOT). Diffusion language models otoh indeed keep profiting for much longer, fwiw.
"On the other hand, training on synthetic data has shown much promise in domains where model outputs are relatively easy to verify, such as mathematics, programming, and games (Yang et al., 2023; Liu et al., 2023; Haluptzok et al., 2023)."
With the caveat that translating this success outside of these domains is hit-or-miss:
"What is less clear is whether the usefulness of synthetic data will generalize to domains where output verification is more challenging, such as natural language."
The main bottleneck for this area of the woods will be (X := how many additional domains can be made easily verifiable). So long as (the rate of X) >> (training absorption rate), the road can be extended for a while longer.
How much of the current usage is productive work that's worth paying for vs personal usage / spam that would just drop off after usage charges come in? I imagine flooding youtube and instagram with slop videos would reduce if users had to pay fair prices to use the models.
The companies might also downgrade the quality of the models to make it more viable to provide as an ad supported service which would again reduce utilisation.
For any "click here and type into a box" job for which you'd hire a low-skilled worker and give them an SOP to follow, you can have an LLM-ish tool do it.
And probably for the slightly more skilled email jobs that have infiltrated nearly all companies too.
Is that productive work? Well if people are getting paid, often a multiple of minimum wage, then it's productive-seeming enough.
Exactly, the current spend on LLMs is based on extremely high expectations and the vendors operating at a loss. It’s very reasonable to assume that those expectations will not be met, and spending will slow down as well.
Nvidia’s valuation is based on the current trend continuing and even increasing, which I consider unlikely in the long term.
> Nvidia’s valuation is based on the current trend continuing
People said this back when Folding@Home was dominated by Team Green years ago. Then again when GPUs sold out for the cryptocurrency boom, and now again that Nvidia is addressing the LLM demand.
Nvidia's valuation is backstopped by the fact that Russia, Ukraine, China and the United States are all tripping over themselves for the chance to deploy it operationally. If the world goes to war (which is an unfortunate likelihood) then Nvidia will be the only trillion-dollar defense empire since the DoD's Last Supper.
China is restricting purchases of H200s. The strong likelihood is that they're doing this to promote their own domestic competitors. It may take a few years for those chips to catch up and enter full production, but it's hard to envision any "trillion dollar" Nvidia defense empire once that happens.
> short of major algorithmic breakthroughs I am not convinced the global demand for GPUs will drop any time soon
>> Or, you know, when LLMs don't pay off.
Heh, exactly the observation that a fanatic religious believer cannot possibly foresee. "We need more churches! More priests! Until a breakthrough in praying technique will be achieved I don't foresee less demand for religious devotion!" Nobody foresaw Nietzsche and the decline in blind faith.
But then again, like an atheist back in the day, the furious zealots would burn me at the stake if they could, for saying this. Sadly no longer possible so let them downvotes pour instead!
They aren’t yet because the big providers that paid for all of this GPU capacity aren’t profitable yet.
They continually leap frog each other and shift around customers which indicates that the current capacity is already higher than what is required for what people actually pay for.
Yeah but OpenAI is adding ads this year for the free versions, which I'm guessing is most of their users. They are probably hedging on taking a big slice of Google's advertising monopoly-pie (which is why Google is also now all-in on forcing Gemini opt-out on every product they own, they can see the writing on the wall).
Google, Amazon, and Microsoft do a lot of things that aren't profitable in themselves. There is no reason to believe a company will kill a product line just because it makes a loss. There are plenty of other reasons to keep it running.
Do you think it's odd you only listed companies with already existing revenue streams and not companies that started with and only have generative algos as their product?
Algorithmic breakthroughs (increases in efficiency) risk Jevons Paradox. More efficient processes make deploying them even more cost effective and increases demand.
I definitely agree on the importance of personalized benchmarks for really feeling when, where and how much progress is occurring. The standard benchmarks are important, but it’s hard to really feel what a 5% improvement in X exam means beyond hype. I have a few projects across domains that I’ve been working on since ChatGPT 3 launched and I quickly give them a try on each new model release. Despite popular opinion, I could really tell a huge difference between GPT 4 and 5 , but nothing compared to the current delta between 5.1 and Gemini 3 Pro…
TLDR; I don’t think personal benchmarks should replace the official ones of course, but I think the former are invaluable for building your intuition about the rate of AI progress beyond hype.
Turing's view, in fact, is similar: "There would be great opposition [to AI] from the intellectuals [read programmers in the context of this thread] who were afraid of being put out of a job. It is probable though that the intellectuals would be mistaken about this. There would be plenty to do, i.e. in trying to keep one’s intelligence up to the standard set by the machines, for it seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers. There would be no question of the machines dying, and they would be able to converse with each other to sharpen their wits."
[0] Thomas Bernhard's The Loser is a fantastic account of the opposite standpoint---of the second best piano student, who cannot stand existing in a world with Glenn Gould.