A AMD Ryzen AI Max Pro 396 will get 50t/s with Qwen3.5 35B.
In addition, the these local models are very, very, very sensitive to the template used. Make sure it is correct. I was using the wrong template and it would answer but felt like it had a brain worm.
The parameters must also be what is recommended, otherwise they go off the rails.
I get great results now after messing with it for a while. I prefer the 35B model because I enjoy how fast tokens appear at 50t/s, but at around 20-25t/s with the 122B model, it is also completely usable. And that one is very smart.
I have been running nearly the same experiments for the same reasons. It has been a lot of tweaking and patching because I am on a Strix Halo system with amd gfx1191, but the LLM component is working nicely.
The evaluation I use is having it one shot a 3d scene using threejs. After everything, the output was comparable to Claude Sonnet (Opus actually does worse on this task strangely).
For my local setup, I have settled on the qwen3.5 family after testing most of the local usable models. Here are the models I use ranked by intelligence:
The 122B model is actually very, very smart. But I have found that token speed is more important, and 35B is smart enough. At 50t/s I can get a lot more done, and I am going to build a mechanism for it to escalate intelligence if needed.
GPT-OSS119B failed at my evals.
MistralSmall4 is too buggy to use (I believe it is too new, the templating is messed up, and agentic use has too many issues). That said I evaluated it directly via copy and paste and the results were not comparable to Qwen. But it is very very fast.
I am running a patched build of llamacpp to get these results. There are a few changes that need to be made to increase prompt processing speeds (about a 30% increase) and be able to use rocm. It took a lot of setup but my flake in nixos is stable now.
Long story short, I can confirm a lot of what was shared in his blog.
*this was written at 4:30am on my phone when I wake up, apologies for typos.
Are you running NixOS on the Strix as a main OS? Would be interesting to read how that's working and what your config is like. Any containers involved or hand building outside of Nix?
I am using nixos unstable pkgs. The published rocmPackages were recently updated and now include kernels for gfx1151, which I surprisingly found out this morning. Before, you would have to set a flag to use the older kernels because they were not available.
My flake modules with rocm config are a bit messy, but maybe I can find time to throw a repo up with it. It contains all necessary packages, flags, boot options, llamacpp patches, and some hacks to get pytorch working smoothly with rocm.
What this means: no, I do not need containers anymore for replicating working configs. The flake configures the system with appropriate libs. I build llamacpp with patches for rocm, and I can run comfyui for generative processes. I have generative image and video working as of today, and next I will get generative 3d modeling working. I'd like to have Trellis2 running this week.
None of this would be possible for me without NixOS, as an aside. It keeps track of configuration for me so I do not have scatter shell scripts and unpredictable deps anymore. I used to build Zed from source with scripts, for example. Now it is a module with patches. Llamacpp is the same. Very clean and requires no working memory, when something needs adjusting I just go refresh myself with the module in one place.
Sounds very neat. Thanks for the explanation. I'm not too familiar with Nix, though I've done a few installs. This sounds like a very interesting setup.
Use of an app is not necessarily the problem. Requiring Google Play or the App Store is. We should be able to use apps without being in walled gardens.
Recursive self-questioning predates external tools and is already well known. What is new is broad access to a low cost, non retaliatory dialogic interface that removes many social, sexual, and status pressures. LLMs do not make people think. They reduce interpersonal distortions that often interfere with thinking. That reduction in specific social biases (while introducing model encoded priors) is what can materially improve cognition for reflective and exploratory tasks.
Simply, when thinking hits a wall, we can now consult a machine via conversation interface lacking conventional human social biases. That is a new superpower.
Yeah nice in theory but the reality is far from this.
In order to implement tolls, you need several components involving middlemen. This includes frontend software, backend, payment processing, transponder management, all the hardware involved, support staff, sometimes toll station staff, among other things.
These toll companies are often owned by foreign companies that are in it for the long haul, offering sweet deals up front then gradually charging more and more with no end in sight, as roads diminish in quality and rest stops fall into disarray.
Toll roads are a scam, a regressive tax on the working class, and downright immoral. We should not limit the mobility of people.
Would American companies treat an average motorist better than foreign companies? Are you insinuating that these good, law-abiding, American companies are COMMUNIST?
Road management should be administered by federal and state agencies, including the administration of tolls when they are foolishly utilized. It should not be a for profit venture, it is a mechanism of taxation for public logistics.
It should not be possible to offload management to private orgs outside of very specific subcontracting / purchasing of components.
Instead, often times full road management is given to private orgs. They are not given a robust legal incentive to act in the interests of the road system, and how could they? They are interesting in using the road to maximize their profit, at the loss of everyone needing the road.
Tolls are a regressive tax on the working class. The rich don't even need to use the roads as much because they have other people delivering for them. When they need the road system, the tolls are nothing to them.
The working class, which are generally required to be driving to survive, are left holding the bag for tolls. In places with bad public transit, tolls are just a forced wealth transfer from working class to private firms managing the tolls.
The people who use something should pay for its upkeep. It doesn't matter if that makes it a "regressive" tax. If you are a daily user of a road, you should pay more for its upkeep than someone who doesn't use the road.
Why should a delivery driver pay the toll for the road to my house, and not me? Why should I be able to exploit flat-rate product pricing like that and skim some money from all customers of the delivery service?
Why should I pay the toll to drive to a friend's house? They're the one getting the benefit out of having easy access to transportation.
What if my taxes pay for all the roads in my town, while the neighboring town chooses to implement tolls instead? Why should I get double-taxed? Prisoner's dilemma and race-to-the-bottom?
Why should I have to deal with having my license plate stolen, and police time wasted (who are paid out of taxes), because of people who don't pay the tolls?
Which may already be a sign of ability to pay? Not that I will argue against the right of US Americans to have a country that gets more and more divided by "class" defined by money, an interesting if not very ethical experiment for sure.
The very well-known in Germany satiric news website "Der Postillion" had an interesting provocative piece just yesterday (German, but auto-translate takes care of that): https://www.der-postillon.com/2023/12/weihnachtsmann-ungerec... -- "Schlimmer Verdacht: Bevorzugt der Weihnachtsmann die Kinder reicher Eltern?" ("A disturbing suspicion: Does Santa Claus favor the children of wealthy parents?")
Being able to get to places by car is one of the most basic needs in the US. I think it leads to cementing the monetary status quo and monetary class affiliation when that becomes even more dependent on how much money one can spend on it. A nicer car being more expensive is fine in that regard, it does not get you from A to B much or any faster than the cheap one. Being able to choose roads or lanes that will take you there much faster is different.
It removes one's personal "hard work" contribution to success if more and more of it is out of your control - after all, how much money you start the game of life with is nothing one has control over. Maybe making that kind of mechanism worse is not the best idea in the long term. Unless we are really aiming for what all the dystopia movies and anime have been showing us.
There are also tons and tons of indirect effects. For example, I would make the claim that wealthy shareholders benefit a lot more from roads than poor people, even when they don't drive, since the companies they own and the entire economy needs them. The poorer people driving to work "paying their share" does not look so clearly justified to me, unless one believes that their salaries are perfect indications of their role in value creation.
> The very well-known in Germany satiric news website "Der Postillion" had an interesting provocative piece just yesterday (German, but auto-translate takes care of that): https://www.der-postillon.com/2023/12/weihnachtsmann-ungerec... -- "Schlimmer Verdacht: Bevorzugt der Weihnachtsmann die Kinder reicher Eltern?" ("A disturbing suspicion: Does Santa Claus favor the children of wealthy parents?")
Canadian stand-up comedian Casually Explained (I don't actually know if he stands up to record his videos) had basically the same joke a few days before them.
Because the cost is not an issue for higher income people. The poor either sacrifice something else to pay the toll, or they take a (likely longer, slower, or more congested) alternate route to avoid the tolls. This ends up costing them more time, which of course is a fixed quantity per day, so they again end up sacrificing. In a way it's regressive even if they avoid it.
We have removed all tolls here in Nova Scotia,including for small car ferry's ,
were not rich or populous,but are building out our infrastructure bit by bit to facilitate ease of transport and the prevention of accidents and traffic jams.
The other thing they added are info signs accross the main hyways comming in, giving
times for the main transit routes, making it easy to redirect , 45 MIN!, yikes! sounds like coffee and grocerie shopping to me!
It has realy made a huge difference getting around the city and has opened up options for travelling rural routes that have ferries.
This is a strange argument that leaves out some important considerations. You could easily say that because the rich don’t need to use public transit the fares charged for riding public transit are a regressive tax on the working class that use it. Shouldn’t you also argue against public transportation ride fares enriching the private companies that build turnstiles and ticketing machines?
Every single lifestyle item of a modern life, whether you have a car or not, depends on the road system.
If you want food, products, or services, you depend on the roads. This means it should be taxed universally and equitably. We should all contribute our fair share to maintain the roads.
Tolls are a regressive tax on low-income people who do the most to make society work, and it is unfortunate that more people do not see this. What's more, they are generally administered by corrupt and inefficient private for-profit orgs. This creates even more overhead which then costs more money.
These orgs generally have slimy deals with city and state governments, while directly profiting from public works that built the road system to begin with.
There are much better ways to fund the road system. Tolls are among the worst.
I don't agree with this perspective. A tax on negative externalities doesn't have to be regressive. It depends on what the tax money is spent on. This is an extreme example, but if you added a congestion tax and then spent the money on a tiny UBI, you might generate $10/person/month, which would be a major uplift to the poorest in our society who don't drive at all. The argument against congestion pricing is further weakened by the fact that those harmed (drivers, pay the tolls) are also those who benefit (drivers, who enjoy less congestion). The ones who are harmed the most are those displaced from driving, who have to find something else to do and don't enjoy the benefits of reduced convention. That's using congestion pricing as an example, but the same argument applies to taxing vehicles in proportion to the wear they impose on roads.
Business owners who pay the tax are free to raise their prices, which is how it's supposed to work. They're currently raising their prices because their drivers waste time in congested traffic and because they pay taxes to the government for road maintenance.
For an analogy, it also makes sense to tax companies who dump their waste in rivers, to the extent that their waste dirties the rivers. If there is some ultra-valuable product that could only be made by dirtying a river (idk, let's say that for some reason insulin had to be made that way), it would be a good that it could still be made, while discouraging people from dirtying rivers for little reason. No one would say "polluting the river should be free because we all use products that are made by polluting rivers." If polluting rivers were free and the government just taxed everyone to clean them up afterwards, we probably all really would use products made by polluting rivers! but that doesn't mean we would be worse off by taxing it directly.
That said, I agree that there's no reason for tolls to fund the road system. Hypothecated taxes are generally not a good idea, despite the fact that they're very intuitively appealing.
While what you're saying does seem like a direct solution (congestion), it is the wrong solution.
The solution to congestion is robust public transit. Full stop.
If a light rail is more comfortable and a faster experience than a car, people will use that instead. Public transit has been traditionally so atrocious, for reasoning we can attribute to many factors, that most people don't use it even if it existed.
If public transit was actually done right, people would be happy to use it. It is more energy efficient, more cost efficient, less of a mental burden, and I believe can be significantly more comfortable.
This is the fundamental issue for me. Society keeps taking these horrible shortcuts that cost all of us instead of just doing the right thing to begin with.
No matter how good public transit is, it will never fully replace the need or desire for cars. Especially now that self-driving cars are hitting the scene. There will be routes poorly served by transit, people hauling stuff that can't be easily taken on a bus or train, people who prefer privacy, stuff being transported by truck, and people who just like road trips. These people are all still imposting a cost on other drivers by driving (when there is congestion). I haven't yet heard a good reason for why that should be free, no matter how good public transit is.
A group of very mentally ill, insecure people with a lot of material wealth control the internet and media.
They get to write the narrative.
We can analyze just one small tool in the belt of narrative control: censoring. If you've been warned or banned on Reddit, you can imagine how this works. If you've said something against the mold of what they allow, you will get censored. With so many people commenting, some subset of people will always say what you want to see. You censor or derank opinions you don't want, and boost opinions you want. This is a defensible form of writing a narrative without actually having to artificially write anything.
Of course with AI, you can now just write anything and seed ideas.
Give such sick people the reigns, and you get a false reality has little connection to what's really happening.
OK, but applying the idea from critical legal theory that "the purpose of the law is the protect status quo power" to mental health to infer that diagnoses must similarly reinforce archetypes with social/economic/political utility for the system - how does that gel with the idea that people capable of aquiring great wealth (a measure of 'system utility') are highly mentally ill?
Aside from that, I'm not saying you're wrong or right about that theory, I'm just wondering how it falls down around that idea.
On this topic of interenet behavior, maybe I'm not really sure or maybe I am, but my view is it's less about some sort of diempowering imposition of external/elite evil upon a innocent and good mass population, but rather about the medium itself enabling latent negativities in the populus to surface. Which doesn't mean the population is itself not good and innocent - it is also multifaceted. Thus, such dynamics might operate in a "Stanford Prison Experiment" kind of "cover and permission" way.
My view of many of these dynamics are its more about emergent self-regulating properties of a system than it is about top-down control. In a sense, that's a lot more liberating and empowering for people, because then they are not cast as victims of some evil from on high, they are the architects of their experience, for good or bad.
The view you espouse, while seeming to empower the downtrodden by taking aim at hidden sources of evil power, I feel in fact disempowers by playing up the fake victim narratives that disempower and confuse people. In other words, your idea, while seemingly edgy and incisive, may in fact be what any such extant "evil elites" would want you to think, if they hope to have control! Haha :)
Anyway, I'm not trying to cut down your idea here in this topic - personally I believe people are very much in charge of their experiences, that's what I've found in my life - but in this kind of mass topic, who knows? Anywa, thanks for responding. Just some food for thought and maybe discussion. Have a good one :)
> diagnoses must similarly reinforce archetypes with social/economic/political utility for the system
Unless extreme wealth is part of the diagnostic criteria, this model says the diagnostic criteria would be designed to reinforce archetypes in the general populace, and that the status quo powerful would simply not receive such diagnoses. That doesn't stop other people from reviewing the checklists and drawing their own conclusions. (I, myself, haven't done this, so I'm not sure whether the "powerful people are diagnosable as mentally ill" conclusion is valid.)
> Thus, such dynamics might operate in a "Stanford Prison Experiment" kind of "cover and permission" way.
The Stanford Prison Experiment is actually a good example: Philip Zimbardo had his thumb very firmly on the scales, and excluded that information from his write-up. The claim that "people are just like that" has been fabricated enough times that I'm deeply suspicious of it.
Fair enough, because it wouldn't apply (as part of the power of the system in this very model), to the powerful themselves, means the criteria would only need to shape the rest, not reinforce the elite traits, such as they may be. In other words it could serve its purpose of protecting their power and hierarchy even if it was only ever applied to everyone but them. Makes sense. Thanks.
Re your point about the SPE, I'm not saying I disbelieve you -- but I don't know -- (seems plausible that a big ticket "objective experiment" was infact non-scientifically reproducible or even used as a psyop to gaslight people into accepting their "original sin" - or whatever) but can you show some evidence of this?
Highly functioning sociopaths. And this diagnosis never goes alone in otherwise perfectly balanced individuals, does it. Most of them have missing/broken father figure syndrome which manifests in various bad and rather unfixable personality traits.
The societies we humans build always allow such persons to rise to the top - it doesn't matter if market democracy or brutal communism, fascism etc. The last type that didn't work well was some sort of feudal kingdom style where power was shared among elite across generations, inherited and rarely claimed by more competent, ambitious and vicious folks from lower ranks. But this is also how we got most of the progress in past 150 years, so its a double-edged sword. I wish I had a solution, maybe some Deus Ex-style of neutral AGI, but who would build such an AGI when everybody competent wants more power and manipulate others to their favor.
Heck, we often celebrate them by looking at their achievements, conveniently ignoring what utter piece of shit they are as humans (Ford is a prime example - a great inspiration for Hitler among others, and musk doesn't go far and look how uncritically he was celebrated also here for a long time and often still is... but the list is very long, basically almost all billionaires and high power folks).
With great power comes great impact even if they don't try, and who doesn't like some ego boost. People imitate them, follow them, subconsciously accept their values more easily. They literally imprint their values on rest of the world and we allow it due to our laziness, convenience and inherent sheepish mentality of masses which we are part of whether we like it or not - just look at how most folks need some form of a role model.
Intresting. I'm not saying (to pick some well-known execs/founders/leaders at random) Jobs, Musk, Zuck, Bezos, Huang, Trump, Xi are "high functioning sociopaths" but Jobs and Bezos both had missing biological fathers. Musk had a violent one. Zuck, not sure - but something seems weird with the dad, it's never spoken of tho. Huang was raised without parents present (only communicating via casette tape shipped on boat - wow!), living overseas from age 9, in a violent type of environment. Trump's dad was a disciplinarian tough on his brother, but Trump found ways to stand up to him. Xi's father was purged/rehabilitated by the Communists and they had to live in caves, farming dust and being bitten by lice, etc for years. I don't know any of them personally and I'm not speaking to their actual stories, as I don't know.
All this tho -- can the mother have no impact? I don't think so. Children are raised by their mothers. Why put the blame on dads, if solely? Seems not fair. A bifurcation in blame in society that can only cause a fracture that leads to greater wrongs later.
Also, while such questions are intriguing -- much of this talk of what's wrong with the internet, points the blame at a few rich people. This seems misguided, and misses the point that the internet is largely "us" - all of us. If we are doing something "wrong" but deflect, we're never going to get better. Even if some bad people are trying to push buttons, we're the ones that have to take responsibility for how we act and to do good.
When I'm chatting online, I'm sure as hell not talking with Bezos - he can't text that much, least of all in the hot-tub. I'm talking with some random. And we each have to take resopnsibility for our behavior. If the rando I'm talking with says, "Why am I bad? Because Jeff Bezos made me this way." It sounds totally ridiculous. And it is, of course. I think the hijacking of a question about "why is the internet negative sometimes" into a 2-minutes-hate on rich-elite is the wrong approach to solutions and understanding.
Most people have significantly less than what we are spoon fed by media and the internet at large.
Just as in history we learn of emperors and kings instead of the common person, most digital content is about the modern day lords, barons, emperors, and kings. They call them billionaires, presidents, CEOs, prime ministers, etc now, but they are the exact same as they always have been.
If you turn the screen off and take a walk, start talking with real people that actually provide value to society, the world is much kinder than we've all been made to believe.
The real people are a good people, as they long have been. Their stories may not be written, but the Earth itself carries their memories.
In addition, the these local models are very, very, very sensitive to the template used. Make sure it is correct. I was using the wrong template and it would answer but felt like it had a brain worm.
The parameters must also be what is recommended, otherwise they go off the rails.
I get great results now after messing with it for a while. I prefer the 35B model because I enjoy how fast tokens appear at 50t/s, but at around 20-25t/s with the 122B model, it is also completely usable. And that one is very smart.
reply