Different types of DRAM can literally be made from the same already-etched wafer. The DRAM bits themselves don't change at all. What's different between DDR4, DDR5, and HBM is the IO interface to the chip. Changing this does not require significant retooling or relearning.
> The DRAM bits themselves don't change at all. What's different between DDR4, DDR5, and HBM is the IO interface
That's not completely accurate - since the bw between these are different, the routing and therefore propagation delays for DDR4 won't allow it to magically be used as DDR5 or HBM.
If you design for the most strict timings, then sure.
> Different types of DRAM can literally be made from the same already-etched wafer.
The assumption here is that you would stop making DDR5 but continue to make DDR4 so that you could start making DDR5 again without too much trouble. But the older chips have even lower margins than the newer ones. Most of the fabs and equipment for making DDR4 were created when it was current and then they stay in operation as long as there is still enough demand for it.
If you don't make DDR5 and DDR6, what happens to your DDR4 fabs when DDR4 is where DDR2 is now? They close because nobody wants it anymore. And then you're not trying to get to DDR6 from DDR4, you're trying to get to DDR6 from an empty desert.
They're still at somewhat of a process disadvantage, but they have demonstrated an ability to produce DDR4 on older processes than it's typically been produced on. So it stands to reason that their process disadvantage will not stop them from producing DDR5 at scale. Their DDR5 will just use a little more silicon, and squeeze the jigglyness from a few more electrons, but in this market, who cares if RAM cost 15% more to make and was 15% less efficient to run, if it's available to purchase at all.
They will eventually eat everything while they laugh at us. Why would you build a rail network if it isn't profitable? Why build anything if it isn't profitable? Why would you even house people if the profit isn't guaranteed to be as big as other sectors?
Everyone wanted denarius then escudos then guilders then pounds then dollars and soon yuan. They make stuff over there, you can buy it with yuan.
I think India might come after that but Africa is sure to follow. Give it a few hundred years.
> it’s not like companies are making a bunch of product just to throw them in the trash
Sure they do. You even mention one in the venerable plastic bag. Is it the best bag? No, of course not. Is it a good bag? Absolutely not. Is it the cheapest bag to produce? You betcha.
> Is it the best bag? No, of course not. Is it a good bag? Absolutely not
Plastic is absolutely the best packaging material ever created, it's so good, it feels like magic. It's light, it's cheap, it's waterproof, it's durable and doesn't just decompose, it comes in a miriad shapes and forms and so on. There is a reason it's everywhere
One of those adjectives describes the plastic bag I'm familiar with. Sometimes it lasts long enough to get the food in the house without spilling through a hole which spontaneously appeared in the bag.
I remember when that switch at grocers from paper to plastic was taking hold, and you could choose. "Paper or plastic?" was the question asked. Some comedian (probably) had a good one liner: "That'll be 42.39. Kill a tree or choke a fish?"
The plastic bag is sold to businesses! If every supermarket in the world decided to never buy another plastic bag then they would no longer be produced!
There can be a futility to it all in that the “ideal option” simply isn’t produced of course.
I find boots theory is often a bit too convenient in this topic though. There is unlikely to be magic structural solutions that allow every part of your life to remain as convenient. At one point our lives will have to change in structure.
EDIT: to be extra clear, I think systemic coordinated changes is needed. I just think the “it’s the corporations doing this!” narrative to obscure the needs for reorganization of daily life on top of systemic change
Many cities have banned plastic bags, and the results have been miraculous for waterways and wetlands. It turns out that shore animals don't benefit as much from "hope a few customers choose the better thing, but otherwise let them take home single-use crap that immediately blows off into natural settings."
Would you be ok if stores offered the option between a cheap plastic bag and a more expensive non-plastic one? (All the stores here already do it, btw).
I think the externalities of plastic recycling must be internalized economically by requiring all manufacturers of items to pre-pay for the recycling of said items up front, as part of the manufacturing cost. Similar to how bottle returns are managed, which has been very successful. Items which are provably biodegradable or designed to facilitate repair may be exempt.
Plastic bags are already taxed where I live. Consumers pay that tax, obviously. Other costs imposed on producers of plastic items will just be passed down to consumers.
Exactly. The point of this sort of tax should not be to collect revenue, it should be to ensure that non-biodegradable bags are being disposed of correctly. To the extent that this is not happening, any bag tax is malfunctioning. Such a tax is either insufficient or poorly-designed. (Our city just banned chain stores from giving out plastic bags under 4 mils thick, and stores now give out paper and sell re-usable bags.)
Even if that state is just straight up burning all the tax income from single-use plastic bags, by taxing them you incentivize consumers and distributors towards untaxed, ideally more sustainable alternatives, like single use paper bags or robust multi-use bags.
> Our city just banned chain stores from giving out plastic bags under 4 mils thick, and stores now give out paper and sell re-usable bags
I don't see how this is not a massive win? Paper bags are significantly more sustainable, and multi-use bags are more durable and thrown aways less simply from being more expensive alone.
People are much more wasteful with things they didn't pay for, regardless of "inherent" value.
We’re not disagreeing. I’m saying that the tax should be set high enough that it creates the desired behavior, which is to disincentivize the widespread use of polluting plastic bags AND/OR ensure that they’re recycled and don’t wind up in the environment. If you’re charging $.05 per bag and people are just eating the tax and the bags are winding up in wetlands in similar amounts, that means your tax regime isn’t effective. You should either increase the tax or improve the system. My city’s absolute ban is equivalent to setting the tax to infinity, which is one solution that seems to work well.
That ain't working. A plastic bag discarded in a ditch by the side of the road, or blown by the wind from the landfill, is still going to end up into the ocean. No amount of prepay recycling is going to take back that plastic from the ocean.
Term limits incentivize a deep state exactly one layer removed from those to which the limits apply, as a repository of institutional knowledge about how things actually get done.
This seems rational. We on't have term limits int he US Congress and it doesn't seem any the better for it.
Japan, a heavily bureaucratized country, systematically moves junior and mid-tier staff around in some departments to minimize the possibility of nest-feathering and empire-building, although I would not say it's perfect by a long way.
We do have term limits for positions like the presidency, and what we see is a perpetual power structure one layer removed, in the party system, which effectively chooses who we're permitted to vote for.
Introducing term limits only forces the wealth and power to change it's face periodically. It is addressing a symptom, not the cause.
> Yes, and the made up words of kilo and kibi were given specific definitions by the people who made them up
Good for them. People make up their own definitions for words all the time. Some of those people even try to get others to adopt their definition. Very few are ever successful. Because language is about communicating shared meaning. And there is a great deal of cultural inertia behind the kilo = 2^10 definition in computer science and adjacent fields.
Inability to communicate isn't what we observe because as I already stated, meaning is shared. Dictionaries are one way shared meaning can be developed, as are textbooks, software source codes, circuits, documentation, and any other artifact which links the observable with language. All of that being collectively labeled culture. The mass of which I analogized with inertia so as to avoid oversimplifications like yours.
My point is that one person's definition does not a culture, make. And that adoption of new word definitions is inherently a group cultural activity which requires time, effort, and the willingness of the group to participate. People must be convinced the change is an improvement on some axis. Dictation of a definition from on high is as likely to result in the word meaning the exact opposite in popular usage as not. Your comment seems to miss any understanding or acknowledgement that a language is a living thing, owned by the people who speak it, and useful for speaking about the things which matter most to them. That credible dictionaries generally don't accept words or definitions until widespread use can be demonstrated.
It seems like some of us really want human language to work like rule-based computer languages. Or think they already do. But all human languages come free with a human in the loop, not a rules engine.
That's why Obama gave him a medal. Homan is great at what he does. Too bad for the selective outrage over this. He was doing his job non-politically in the background for years until TDS brain rot made the left so easily manipulated into hysteria.
It seems like, in the course of calling out a perceived assumption, you may have made an assumption yourself. I'm aware of the difference between casualties and deaths. My chosen terminology applies equally to both. And I think both are relevant to a number of related stats like lifetime earnings, mental and physical health, family prospects, etc.
War is hell. And I don't think anyone comes out untouched by it. The stats on vets are brutal.
As state of the art machines continue to chase the latest node, capacity for older nodes has become much less expensive, more openly documented, and actually accessible to individuals. Open source FPGA and ASIC synthesis tools have also immensely improved in quality and capability. The Raspberry Pi Pico RP2350 contains an open source Risc-V core designed by an individual. And 4G cell phones like the https://lilygo.cc/products/t-deck-pro are available on the market built around the very similar ESP32. The latest greatest will always be behind a paywall, but the rising tide floats all boats, and hobbyist projects are growing more sophisticated. Even a $1 ESP32 has dual 240mhz 32bit cores, 8Mb ram, and fast network interfaces which blow away the 8bit micros I grew up with. The state of the open-source art may be a bit behind the state of the proprietary arts, but is advancing as well.
It's really fun to have useful hardware that's easy to program at the bare metal.
chipfoundry.io charges $14,950 for packaged 100 chips. As far as small batch manufacturing goes, that's reasonably affordable. $149 ea. Occasionally I see better deals crop up as part of group buys or for bare dies. Presumably, one would prototype their design on an inexpensive FPGA board first, to verify functionality. So as to be reasonably sure the first batch of chips worked. Folks like Sam Zeloof are working to build new tools for one-off and small batch designs as well, which may further reduce small quantity prices.
Even with MoE, holding the model in RAM while individual experts are evaluated in VRAM is a bit of a compromise. Experts can be swapped in and out of VRAM for each token. So RAM <-> VRAM bandwidth becomes important. With a model larger than RAM, that bandwidth bottleneck gets pushed to the SSD interface. At least it's read-only, and not read-write, but even the fastest of SSDs will be significantly slower than RAM.
> Experts can be swapped in and out of VRAM for each token.
I've often wondered how much it happens in practice. What does the per-token distribution of expert selection actually look like during inference? For example does it act like uniform random variable, or does it stick with the same 2 or 3 experts for 10 tokens in a row? I haven't been able to find much info on this.
Obviously it depends on what model you are talking about, so some kind of survey would be interesting. I'm sure this must but something that the big inference labs are knowledgeable about.
Although, I guess if you are batching things, then even if a subset of experts is selected for a single query, maybe over the batch it appears completely random, that would destroy any efficiency gains. Perhaps it's possible to intelligently batch queries that are "similar" somehow? It's quite an interesting research problem when you think about it.
Come to think of it, how does it work then for the "prompt ingestion" stage, where it likely runs all experts in parallel to generate the KV cache? I guess that would destroy any efficiency gains due to MoE too, so the prompt ingestion and AR generation stages will have quite different execution profiles.
The model is explicitly trained to produce as uniform a distribution as possible, because it's designed for batched inference with a batch size much larger than the expert count, so that all experts are constantly activated and latency is determined by the highest-loaded expert, so you want to distribute the load evenly to maximize utilization.
Prompt ingestion is still fairly similar to that setting, so you can first compute the expert routing for all tokens, load the first set of expert weights and process only those tokens that selected the first expert, then load the second expert and so on.
But if you want to optimize for single-stream token generation, you need a completely different model design. E.g. PowerInfer's SmallThinker moved expert routing to a previous layer, so that the expert weights can be prefetched asynchronously while another layer is still executing: https://arxiv.org/abs/2507.20984
I thought paging was so inefficient that it wasn't worth doing vs using CPU inference for the parts of the model that are in system memory. Maybe if you have a good GPU and a turtle of a CPU, but still somehow have the memory bandwidth to make shuffling data in and out of the GPU worthwhile? I'm curious to know who is doing this and why.
With a non-sequential generative approach perhaps the RAM cache misses could be grouped together and swapped on a when available/when needed prioritized bases.
reply