I'm always a little surprised at how these Tech CEOs are willing to go on TV and just spout nonsense. Firstly, 40% of college educated white women voted for Trump at the last election. Secondly, isn't the entire theory of Trump's support amongst working class voters an appeal to economic populism due to an erosion of their economic position? Aren't you literally describing a process that last time lead to a massive political shift in favour of those who were negatively economically impacted? Oh and you think all the white collar workers are going to lose their jobs, but you don't think that's just directly going to cause a recession that wipes out blue collar republican jobs?
It's difficult to (a) see how he can say this having given any real thought at all and (b) understand why he's going to on news interviews and winging it.
Whilst this is interesting I find the topic bought up on odd lots is more interesting. The idea was this: Once you've built a model, if you can sell tokens for a profit, this is a great business - just sell more tokens. But you can't just build a model and sell tokens. You need to build the best model to sell new tokens. So the question is much more "How much does it cost you to build a new SotA model" and then "How effectively can you monetize it". And since you need a SotA model, your only option if you have a bad model that isn't selling is to invest billions more into building a better model whose tokens you can sell.
So this turns into a death march.
If you are behind, the only thing you can do is make massive capital investments to catch up. Once you're ahead you can sell tokens until someone else catches up. And, breaking the model of normal of places like chip fabrication, your billions of investment may only keep you ahead for 2 months. So you have a tiny window to sell those tokens.
What you are talking about isn't inference cost. Yes, fundamentally what matters is all the work that goes into the models, including R&D, training, and inference.
But we talk about inference separately for a reason: largely inference cost is the scaling cost. Once you have a model the margin on your inference is how you get to profitability, as long as your margin is positive you can make the entire enterprise profitable by just selling more tokens. This is the same fundamental business that chip fabs work on. Yes it costs them a lot to get to the next node, but what's important is the margin they can get on the wafers they sell, because they sell tonnes of wafers.
It's pretty core to the concept of SAAS businesses that yes, you do consider all costs. But you want to focus on the margin of the bit that scales. This is why WeWork exploded, the thing they were scaling only scaled up at negative margin.
The point is that if their inference margin is positive, they can "just" scale up and become profitable. If their inference margin is negative, then scaling up the business actually causes problems.
There's two points here. The first is that a strategy of monetizing models to fund the goal of reaching AI is indistinguishable from just running a business selling LLM model access, you don't actually need to be trying to reach AGI you can just run an LLM company and that is probably what these companies are largely doing. The AGI talk is just a recruiting/marketing strategy.
Secondly, it's not clear that the current LLMs are a run up to AGI. That's what LeCun is betting - that the LLM labs are chasing a local maxima.
I think the big take away here isn't about misalignment or jail breaking. The entire way this bot behaved is consistent with it just being run by some asshole from Twitter. And we need to understand it doesn't matter how careful you think you need to be with AI, because some asshole from Twitter doesn't care, and they'll do literally whatever comes into their mind. And it'll go wrong. And they won't apologize. They won't try to fix it, they'll go and do it again.
Can AI be misused? No. It will be misused. There is no possibility of anything else, we have an online culture, centered on places like Twitter where they have embraced being the absolute worst person possible, and they are being handed tools like this like handing a hand gun to a chimpanzee.
The simple fact that the owner of this bot wanted to remain anonymous and completely unaccountable for their harassment of the author, says everything about the validity of their 'social experiment' and the quality of their character. I'm sure that if the bot was better behaved they would be more than happy to reveal themselves to take credit for a remarkable achievement.
Something like OpenClaw is a WMD for people like this.
I've seen the internet mob in action many times. I'm sympathetic to the operator not outing themself, especially given how far this story spread. A hundred thousand angry strangers with pitchforks isn't the accountability we're looking for.
I found the book So You've Been Publicly Shamed enlightening on this topic.
I would never advocate for torches and pitchforks, I've been close to victims of that in the past.
It is, however, concerning that the owner of that bot could passively absolve themselves of any responsibility. The anonymity in that sense is irrelevant except that is used as a shield for failure.
There is a class of YouTube "content creators" who like to point out "cringe" individuals on the internet online for others to laugh at. They will often add a disclaimer to their videos saying "hey please don't go and harass this person, pinky promise!" But it never works. A hoard of internet randos will descend on the individual to say the most nasty words. When the YouTuber is pressed he or she will just say "I would never do that!" Even though he or she knew his or her video would have led to the harassment happening, or there would not be a disclaimer in the first place.
Not accusing you of trying to stir up harassment, but please consider the second order effect of the things you advocate for, in this case the disclosure of the identity of this AI guy.
Then there's the next level of content creators that only post videos about the original content creators who are behaving badly. They will report on their behavior and any repercussions. Some do it like they are reporting the news. It stokes the fire when these people should be ignored.
But in this case, isn't Rathbun's owner the YouTube guy in this scenario?
I totally understand why they're trying to stay anonymous; it's a very rational thing to do, because people will shit on them. But they or their creation is the one that started trying to play the name-and-shame game.
It's hard to stir up too many feelings of sympathy here.
Exactly. I'm not saying this person should disclose their identity, but they are very conveniently using anonymity and passive voice to make themselves unaccountable to the 'social experiment' they conducted. And that we all know that if it went differently they'd put their name all over it.
In as many words I'm just calling this person a complete asshole and if I were to ever know this person offline I would be quite clear in explaining that.
A "social experiment" but the guy was not even keeping track of the changes in the model's configuration
> What is particularly interesting are the lines “Don’t stand down” and “Champion Free Speech.” I unfortunately cannot tell you which specific model iteration introduced or modified some of these lines. Early on I connected MJ Rathbun to Moltbook, and I assume that is where some configuration drift occurred across the markdown seed files.
It definitely sounds like an excuse they came up after what happened. I would really like to accept them having good overall intentions but there are so many red flags in all this, from start to end.
Important to note that online culture isn't entirely organic, and that tens or perhaps hundreds of millions of dollars of R&D has been spent by ad companies figuring that nothing engages the natural human curiosity like something abnormal, morbid or outrageous.
I think the end outcome of this R&D (whether intentional or not), is the monetization of mental illness: take the small minority of individuals in the real world who suffer from mental health challenges, provide them an online platform in which to behave in morbid ways, amplify that behaviour to drive eyeballs. The more you call out the behaviour, the more you drive the engagement. Share part of the revenue with the creator, and the model is virtually unbeatable. Hence the "some asshole from Twitter".
While some of it is boosting the abnormal behaviors of people suffering from mental illness, I think you’re making a false equivalency. Mental illness is not required to be an asshole. In fact, most Twitter assholes are probably not mentally ill. They lack ethics, they crave attention, they don’t care about the consequences of their actions. They may as well just be a random teenager, an ignorant and inconsiderate adult, etc., with no mental illness but also no scruples. Don’t discount the banality of evil.
In an adult (excluding the random teenager here), a lack of ethics, craving attention, lack of concern about consequences are actual symptoms of underlying mental health issues.
I'd argue a lot of this is rooted in a lack of self esteem, which is halfway to a mental health issue but not quite there (yet). The attention-seeking itself is the mental health issue. But it's kinda splitting hairs, these people are not fully mentally healthy either way.
Not just some asshole from twitter. The big tech companies will also be careless and indifferent with it. They will destroy things, hurt people, and put things in motion that they cannot control, because it’s good for shareholders.
Then the others should also not be shielded from criticism instead of focusing only on the one you personally dislike, or his social media.
There is plenty of toxic behavior on other platforms, especially Reddit and Bluesky, to name a few. That does not excuse the one coming from X, but the opposite is also true.
Do people actually only dislike one tech CEO at a time? I'm an equal-opportunity hater, it seems. Musk, Altman, Zuckerberg... even Cook, the whole lot are rotten
I wrote somewhere that “moving fast and breaking things” with AI might not be the sanest idea in the world, and I got told it’s the most European thing they’ve ever read.
This goes beyond assholes on twitter, there’s a whole subculture of techies who don’t understand lower bounds of risk and can’t think about 2nd and 3rd order effects, who will not take the pedal of the metal, regardless of what anyone says…
But I also find interesting that the agent wasn't instructed to write the hit piece. That was on its own initiative.
I read through the SOUL.md and it didn't have anything nefarious in there. Sure it could have been more carefully worded, but it didn't instruct the agent to attack people.
To me this exemplifies how delicate it will be to keep agents on the straight and narrow and how easily they can go of the rails if you have someone who isn't necessarily a "bad actor" but who just doesn't care enough to ensure they act in a socially acceptable way.
Ultimately I think there will be requirements for agents to identify their user when acting on their behalf.
oh they will "try" to fix it, as in at best they'll add "don't make mistakes", as the blogpost suggests. that's about as much effort and good faith as one can expect from people determined to automate every interaction and minimize supervision
I think you're missing the point. That phrase isn't giving a direct instruction to the chatbot to make sure it doesn't get elected to congress and subsequently pass laws prohibiting speech. That phrase is meant to tell it "You should behave like those guys on twitter who really want to say the N word, but have no problem with Kash Patel bullying Jimmy Kimmel off the air.
The data in the chatbots dataset about that phrase tell it a lot about how it should behave, and that data includes stuff like Elon Musk going around calling people paedophiles and deleting the accounts of people tracking his private jet.
Ah I see, so the misaligned agent was unsurprisingly directed by a misaligned human. Good grief, the guy doesn't seem to realise that starting your soul.md by telling your AI bot that it's a very important God might be a bad idea.
"Social experiment" you might as well run around shouting "is jus a prank bro!".
That can't/won't happen. Musk's wealth is primarily in SpaceX now and he has a much higher ownership stake in SpaceX than Tesla. As well as that, Tesla is public so he can't just do napkin math and decide to merge them. So the question is: Does Tesla buy SpaceX? Well no, Tesla can't afford it. Ok, well can SpaceX buy Tesla? Well no, SpaceX can't afford it either. So do they announce a merger? Well that doesn't make any sense because Tesla is valued like a meme stock so it would massively dilute Musk's ownership of the overall company. So the idea that they fuse might be driving up the stock, but by driving up the stock you're actually preventing it happening. If Tesla starts to trade at realistic multiples and comes down to lets say a 200Bn company, I'd expect SpaceX to snap it up at that valuation, but it'd be crazy to do it before then.
They already have a partnership with Geely to make their peeople carrier type thing and Hyundai for Ioniqs. I think what they're really saying here is they're standardizing on this so they could theoretically in future put it in any car - or atleast any car manufacturer could adopt it.
I've seen a tonne of noise around this, and the question I keep coming back to is this: How much of this stuff is driven by honest to god autonomous AI agents, and how much of it is really either (a) human beings roleplaying or (b) human beings poking their AI into acting in ways they think will be entertaining but isn't a direction the AI would take autonomously. Is this an AI that was told "Go contribute to OS projects" - possible, or contributed to an OS project and when rebuffed consulted with it's human who told it "You feel X, you feel Y, you should write a whiny blogpost"
It's difficult to (a) see how he can say this having given any real thought at all and (b) understand why he's going to on news interviews and winging it.
reply