I really doubt this actually. To me, Jev is a great example ofcounter positioning. When you consider just how hyper optimized the labs are around auto regressive LLMs, and just how much money they have already invested and are pre committed to investing in an entire stack for auto regressive transformers... then responding to Jev becomes nearly impossible actually. They would just be giving up too much.
Just think, everything from their current sources of revenue, the sales use cases they tout, the marketing on the websites, the messaging to customers, then technically to the APIs, their internal batching and scheduling algos, their GPU configs, the chips themselves. ALL OF IT is designed with generative text models in mind. Jev breaks all of it.
I think basically no chance of a response any time soon.
I don’t understand how you’ve reasoned your way here.
How could Jev have possibly built something out of reach of a frontier lab providing the same or 5x as much resourcing to one of their teams to achieve? Which they can do because Jev has only received $40M of funding recently, so a round that is approximately what OpenAI is spending per math problem they try cracking.
In addition to that, these frontier labs have got extremely good at generating synthetic data and running generalised training pipelines. I can only imagine how easy it would be for them to build this internally vs Jev building it from scratch.
And then the final thing: one of the best places you might apply Jev is within a harness, behind layers that customers increasingly have abstracted from them. Frontier labs have huge incentives to do this as it could make their offering much better and cheaper. And whoever gets this first wins another big attraction for users.
My take on this is Jev is either acquired almost immediately for the benefit of the next 1-3 months head start for whichever lab acquires them or we get a similar model offered from all labs in 3-6 months or sooner.
Many comments are missing the point or just totally backwards. I have been both an IC eng and a founder / CTO (though me and my cofounder ran it more like co-CEOs). AI's will not be replacing CEOs (or other great execs) because being CEO (especially of a startup) is all about a few things, and all of those things are almost structurally what AI can not do. It's all about...
* Setting a strong, non-consensus vision. -> AI's are trained to produce the median / mode answer. So this almost disqualifies them by default.
* Having courage to do the correct, obvious thing even when your team or investors don't think it's the right call. (This might be firing someone everyone likes but who's role is no longer needed, it might be changing strategies away from the product everyone just spent 6 months on, etc. etc. the list goes on). -> Ben Horowitz has called this kind of decision basically the entire value of a CEO. This is also EXACTLY where today's LLMs keep getting worse and worse. They are utterly spineless.
* Having unique skills and credibility to build a team -> This can come from many sources (track record, vision, charisma, specific achievements, network, etc.) , but people need to want to join YOUR team, not someone else's. They have to believe that YOU are going to win against the market. There is no way you're going to have an edge by using a commercially available AI API with an open source harness at the helm. That is just so utterly uncompelling to top candidates.
* Fundraising -> On the surface, very ammenable to AI, but again, as an investor, would you seriously invest in a CEO that is literally replicatable by any other company?
The value I can see for something like this is a human to get a "2nd pair of eyes". But humans have tons of advisors already and generally speaking, getting takes on what you should do as CEO is not the hard part. It's seeing through the fog of war, finding clarity, and then implementing that clarity across the org, which is surprisingly hard.
> Setting a strong, non-consensus vision. -> AI's are trained to produce the median / mode answer. So this almost disqualifies them by default.
Most human CEOs are disqualified then. Whenever I join a company these days, they stress how important the company's values are, and they are somehow basically the same as every other company.
That rhymes with the reasons why AI can't replace engineers.
VCs likely won't demand that you be replaced with a claude subscription. But shareholders might wonder why they should approve a fat package for a meatbag. Private equity, picking over dinosaurs, won't hesitate.
Lindy Fiorina hacking up HP? That job is now mechanised.
I am in nearly the exact same boat as the OP. I'll observe a few things after having 2 kids and running a startup at the same time for a few years...
* Your amount of "lost" time with kids is often less than you think. On one side, you very likely socialized more, traveled more, were less disciplined about what you worked on, etc. On another side, some things that you are going to do anyway (dishes, maybe calling your parents, groceries), you can often do with kids, and so this time isn't really "lost"
* If you are fortunate enough to have some means, you might consider not outsourcing parenting, but outsourcing "life admin" work. Specifically things like laundry, dishes, buying groceries, picking up around the house, etc. I recently got someone to come to our house 3x / week for 3 hours and it's a game changer for me and my wife (who also works). (I know my point above says you can do some of this with kids, and that's kind of true, but not always, and it still lets you do other things with your time)
* Assuming you have a partner, and the premise here is you still want to be a very active parent, then I recommend being very deliberate about almost never parenting together. As in, one of you take all the kids, and the other can do what they want for 1-4 hours at a time.
Lastly, while I'm a pretty active dad myself, I reject the premise put forth here that your job is to maximize the time spent with your kids, as if this is the main metric by which parenting success should be judged. Just no. Actively no. Kids having time alone and with their own friends outside of the eye or ear of their parents is a really important part of growing up. Kids having time with various caregivers who are not their parents (be that grandparents, aunts, uncles, friend's parents, camp counselors, babysitters) is also good and healthy.
The struggle is real though. There's no easy answer. Hope these thoughts help!
>Lastly, while I'm a pretty active dad myself, I reject the premise put forth here that your job is to maximize the time spent with your kids, as if this is the main metric by which parenting success should be judged. Just no. Actively no.
Kinda disagree. My Kids 3 mind, so apples:oranges. But something I keep reminding my wife about, whenever she feels like shes failed, is that childhood outcomes are strongly correlated to parental involvement and interest at this age. 2 active, interested parents is a huge advantage over 1, and 1 is a huge advantage over 0. Just the time we both spend with him, engaged in conversation or reading books, has already paid massive dividends, and we can see that he just soaks it all up like a sponge. Wont be that way forever, he will eventually find better interaction with peers. But right now its a massive force multiplier, and we only get one shot at it.
Has anyone seen any tools for enforcing that people understand each PR? It feels like there's a new missing piece here that is like CI for humans, where you as the code owner need to basically pass a test (auto generated by AI) about wtf is going on with this PR and why. If that quiz isn't green then you can't ship it...
It's not simple weights and numbers all the way down. The available output is pre-set by the tokens we allow it to predict.
There was a whole bit in there about not having a language module or using words. But it does. We tell it.
Humans do not come pre programmed with a set of possible "tokens". We just figure it out and I believe that fact captures something very essential. Maybe the missing piece of AGI. The fact that humans can just be awash in pure sense data, and somehow just figure out what is important and what to do. Never ceases to amaze me.
The set of tokens is learned, more or less. So I don’t get what point you’re trying to make here. There’s not a human manually deciding what tokens make up the token dictionary.
But focusing on production cost is silly. The cost to consumers is what matters. Software is already free or dirt cheap because it can be served at zero marginal cost. There was only a market for cheap industrial clothes because tailor made clothes were expensive. This is not the case in software and that's why this whole industrialization analogy falls apart upon inspection
One thing that has become clearer to me over the years is that reasoning by analogy (like this article does) sounds a lot smarter than it is. If you look from first principles, it's clear that physical goods and software don't share the same properties and thus the analogy falls apart.
Physical goods like clothes or cars have variable costs. The marginal unit always costs > 0, and thus the price to the consumer is always greater than zero. Industrialization lowered this variable cost, while simultaneously increasing production capacity, and thus enabled a new segment of "low cost, high volume" products, but it does not eliminate the variable cost. This variable cost (eg. the cost of a hand made suit) is the "umbrella" under which a low cost variant (factory made clothes) has space to enter the market.
Digital goods have zero marginal cost. Many digital goods do not cost anything at all to the consumer! Or they are as cheap as possible to actively maximize users because their costs are effectively fixed. What is the "low value / low cost" version of Google? or Netflix for that matter? This is non-sensical because there's no space for a low cost entrant to play in when the price is already free.
In digital goods, consumers tend to choose on quality because price is just not that relevant of a dimension. You see this in the market structure of digital goods. They tend to be winner (or few) take all because the best good can serve everyone. That is a direct result of zero marginal cost.
Even if you accept the premise that AI will make software "industrialized" and thus cheaper to produce, it doesn't change the fact that most software is already free or dirt cheap.
The version of this that might make sense is software that is too expensive to make at all because the market size (eg. number of consumers * price they would pay) is less than the cost of the software developer / entrpreneurs time. But by definition those are small markets, and not anything like the huge markets that were enabled by physical good industrialization.
Digital goods do have a marginal cost. It's a lot lower than with physical goods, but there is a cost: at the very minimum, a digital good takes up storage space. A streamed digital good requires bandwidth and electricity (and in most of the world, both are metered resources).
Also, most consumers don't choose on quality; they choose on price. This is why free mobile games became huge and paid mobile games are a dying breed. In the physical world, it's why shein and alibaba nearly became trillion-dollar companies.
Sure there is some minimal marginal cost, but it's so close to zero that it's usually negligible, and the incentive is to basically give it away and "monetize" something else. Your point about games actually just makes my original point. Software is already usually free or dirt cheap, which is why reducing the cost to make the software can't create some "low cost / low value" quadrant. Unless your talking about bespoke software that has such a small market size it isn't worth making today. I could maybe see that area opening up, but even that software would not fit the OP's description of software that "has no owner and is not meant to be maintained"
I feel like we can round down fractions of a cent to zero. In practice, it's basically zero.
And, I think, consumers would like to balance both cost and quality. The problem is cost is obvious, quality is purposefully obfuscated. You really can't tell what is or is not quality software without spending an unreasonable amount of time and requiring an unreasonable amount of knowledge. Same with most modern physical goods.
Analogies are useful for adding new possibilities to the list of ideas you consider. They're not good for ruling anything out; you need other forms of reasoning for that.
This video was fascinating. I didn't know about "open endedness" as a concept but now that I see it, of course it's an approach.
One thought... in the video, Ken makes the observation that it takes way more complexity and steps to find a given shape with SGD vs. open-endedness. Which is certainly fascinating. However...
Intuitively, this feels like a similar dynamic is at play with the "birthday paradox". That's where if you take a room of just 23 people, there is a greater than 50% chance that two of them have the same birthday. This is very surprising to most people. It seems like you should need way more people (365 in fact!). The paradox is resolved when you realize that your intuition is asking how many people it takes to have your birthday. But the situation with a room of 23 people is implicitly asking for just one connection among any two people. Thus you don't have 23 chances, you have 23 ^ 2 = 529 chances.
I think the same thing is at work here. With the open-ended approach, humans can find any pattern at any generation. With the SGD approach, you can only look for one pattern. So it's just not an apples to apples comparison and sort of misleading / unfair to say that open-endedness is way more "efficient", because you aren't asking it to do the same task.
Said another way, I think with the open-endedness, it seems like you are looking for thousands (or even millions) of shapes simultaneously. With SGD, you're kinda flipping that around, and looking for exactly 1 shape, but giving it thousands of generations to achieve it.
I did a chat with Gemini about the paper, and tldr is...
* They introduce a loop at the beginning between Q, K, and V vectors (theoretically representing "question", "clues" and "hypothesis" of thinking)
* This loop contains a non linearity (ReLU)
* The loop is used to "pre select" relevant info
* They then feed that into a light weight attention mechanism.
They claim OOM faster learning, and robustness acro domains. There's enough detail to probably do your own PuTorch implementation, though they haven't released code. The paper has been accepted into AMLDS2025. So peer reviewed.
At first blush, this sounds really exciting and if results hold up and are replicated, it could be huge.
I really doubt this actually. To me, Jev is a great example ofcounter positioning. When you consider just how hyper optimized the labs are around auto regressive LLMs, and just how much money they have already invested and are pre committed to investing in an entire stack for auto regressive transformers... then responding to Jev becomes nearly impossible actually. They would just be giving up too much.
Just think, everything from their current sources of revenue, the sales use cases they tout, the marketing on the websites, the messaging to customers, then technically to the APIs, their internal batching and scheduling algos, their GPU configs, the chips themselves. ALL OF IT is designed with generative text models in mind. Jev breaks all of it.
I think basically no chance of a response any time soon.
reply