> How parallelism changed the agent’s research strategy
> With a single GPU, the agent is stuck doing greedy hill-climbing: try one thing, check the result, pick a direction, try the next thing. With 16 GPUs, the strategy shifts. ...skip... 12 experiments in a single 5-minute wave. This makes it much harder to get stuck in local optima and much easier to find interaction effects between parameters.
The agent can theoretically come up with a protocol to run those same 12 experiments one-by-one and only then decide which branch to explore next - which I think would lead to the same outcome?
But in this case, it just happened to have stumbled on this particular outcome only because it didn't get a chance to execute a greedy strategy after the first 1 or 2 results.
> The agent can theoretically come up with a protocol to run those same 12 experiments one-by-one and only then decide which branch to explore next - which I think would lead to the same outcome?
At least in theory, adaptiveness should save samples and in this case, compute. (As noted, you can always turn the parallel into serial and so the serial approach, which gets information 'from the future', should be able to meet or beat any parallel approach on sample-efficiency.)
So if the batch only matches the adaptive search, that suggests that the LLM is not reasoning well in the adaptive setting and is poorly exploiting the additional information. Maybe some sort of more explicit counterfactual reasoning/planning over a tree of possible outcomes?
Yeah, assuming there's no active monitoring during the training runs, you can trivially give the agent an abstraction which turns "1 GPU" into "16 GPUs" that just so happens to take 16x the wall-clock time to run.
In fact, looking at the blog post, the agent orchestrating 16 GPUs is half as efficient as the agent using 1 GPU in GPU-time. Since it uses 16 GPUs to reach the same result as 1 GPU in 1/8 of the time.
Very cool. I'm imagining using this with Claude Code, allowing it to wire this up to MCP or to CLI commands somehow and using that whole system as an interactive dashboard for administering a kubernetes cluster or something like that - and the hypothetical first feature request is to be able to "freeze" one of these UI snippets and save it as some sort of a "view" that I can access later. Use case: it happens to build a particularly convenient way to do a bunch of calls to kubectl, parse results and present them in some interactive way - and I'd like to reuse that same widget later without explaining/iterating on it again.
Did you see that Claude Code just came out with "channels" that allows for messages to be injected into the session/sent out by claude via hooks and MCP server [1]? I had CC code an integration between fenced and CC using channels and it actually worked - a little clunky since there is no streaming, but very interesting nevertheless.
Grassroots-level adoption was key for things like Dropbox, IMHO - it solved a real need for individual people and it worked well and was easy to use. Same for Docker - developers adopted it in droves, and then enterprises followed.
Inrupt is trying to bootstrap a two-sided marketplace of sorts: product builders won't care until enough potential customers demand support for the "data pods", and regular people won't care until "data pods" solve real everyday problems for them.
Hopefully Inrupt's team has enough business-savvy people on it to find ways to gain traction to slog through some of the tough early stages of the product adoption cycle.
> "In many ways, 2017 marked the year that cryptocurrency stopped being about technologically innovative peer-to-peer cash and instead essentially became a new, unregulated penny stock market."
I don't think it stopped being about tech innovation - there is a ton of stuff happening around proof-of-stake, layer 2 networks, state channels etc. It's just that the stories around the speculative aspects of the cryptocurrency assets have been dominating in 2017.
My toddler is allowed about a half hour of cartoons on Youtube on the big TV per day. He's played with the iPad a few times in his life. Lots of books and Legos and physical toys in our house. Waiting for longitudinal studies to come out, not taking chances with the little brain.
Every other month I go on a media diet: no social media, no news, no unnecessary browsing of any kind - only books and a small number of podcasts are allowed (plus whatever Internet usage is necessary for work). I've been doing this for more than a year now, and it has helped me feel a lot less "fragmented". I highly recommend it.
What do you do when you need to know something? (recent example: my toilet stopped refilling, I had no idea what was wrong and found the information on the internet). Also, which podcasts do you prioritize?
> but what's to stop us from cloning big chunks of our civilizations... simulating them, but using vastly less power/resources for each simulation? Then writing software to have them pass knowledge among the civilizations? We have just a few hundred countries, what if we had a trillion communicating at the speed of computer circuitry?
The principle of computational irreducibility [1] is what will stop us from "cloning" civilizations. That and chaos theory - any tiny deviation in initial conditions of such a simulation or cloning process could produce unusable results.
"simulating them, but using vastly less power/resources" is a pipe dream.
'Better' as in achieves more results per unit of time. Which is a fundamental problem for humans too. Many do have the intellectual capacity to invent something like general relativity, but few would be capable of doing so in the very limited timeframe we have available and even fewer actually end up doing that instead of dedicating their thinking time somewhere else. More thinking and more output per timeframe should lead to significant improvements for both human and AI in terms of results, which is generally the meaningful part
Hardware that has more memory, more processing speed, faster access to memory, and more parallelism is better than hardware without those characteristics.
The exact same software running on better hardware will run faster and can tackle larger problems.
We can't possibly build a human with twice the memory that thinks twice as fast. However once we have an AI which is roughly equivalent to a human, having an AI with twice as much memory that thinks twice as fast is just 2-5 years. (How long depends on where the bottleneck is.)
Yes. Nowhere near as good as AlphaGo, but yes it would do better.
When Deep Blue beat Kasparov at chess, the program was not significantly better than what had been state of the art for the previous decade. They just threw enough hardware at it.
For chess programs there is an almost linear relationship between your search depth and effective ELO rating, and search depth went up by a constant with each generation of Moore's law.
But we, humans, aren't going at anything like the speed of light. What if we tweaked our DNA to produce human beings with the working memory capacity of 50 items instead of the normal 7-ish [1]? One such researcher would be able to work faster, on more problems at once, and to consider more evidence and facts. The next bottleneck for that person, of course, would be the input/output capacity (reading, writing, typing, communicating), but even with those limitations, I bet they would be a lot more efficient than the average "normal" human. The question is - would you call such a person more "intelligent"?
Or we get more humans and then it's a coordination problem right? I mean there is a point in comparing individual vs collective intelligence. This is a bit like communist systems. They work in theory because you get to plan the economy centrally, but in fact more chaotic systems (unplanned) do better (check growth of capitalist countries vs communist ones).
The agent can theoretically come up with a protocol to run those same 12 experiments one-by-one and only then decide which branch to explore next - which I think would lead to the same outcome?
But in this case, it just happened to have stumbled on this particular outcome only because it didn't get a chance to execute a greedy strategy after the first 1 or 2 results.
Worse experiment design + parallelism = better experiment design + serialized execution ?
reply