The early batch of Anthropic employees were mostly rationalist-adjacent AI safety folk that were almost uniformly claiming P_DOOM > .10 three years ago, so I believe them to be earnest.
It's very interesting to me that besides the other small safety labs that don't actually produce frontier models, Anthropic manages to keep such a good reputation within that subculture compared to OpenAI. Despite having as crazy internal politics as OpenAI, they have converged quite a bit from the original vision of safety first through Darwinistic pressures.
At least, it seems this way from the outside. I'm curious if the view from the inside is that different.
edit: to be clear, my reading as an outsider is that Anthropic is seen as relatively better in the AI safety community, but has definitely dropped in absolute reputation too. This recent thread and the references show some of that: https://www.lesswrong.com/posts/6j3kBHdowGLCeqobg/dear-god-p...
Aren't the route restrictions that this data was collected on a large confounder? They are specifically placed to avoid difficult or dangerous areas. For example in the bay area only recently were they allowed to go on the freeway, and only in certain areas.
The authors of AI 2027 have already said they would need to extend by two years. AI 2040 is a different type of document but probably easier to read to get a sense of their thinking. My understanding is the main difference between knowledgeable 'normies' and them is they expect progress to continue at a fast rate, and that economic diffusion issues are not as bad.
From this they get rapid growth, namely > 100% GDP growth around 2031
The early batch of Anthropic employees were mostly rationalist-adjacent AI safety folk that were almost uniformly claiming P_DOOM > .10 three years ago, so I believe them to be earnest.
It's very interesting to me that besides the other small safety labs that don't actually produce frontier models, Anthropic manages to keep such a good reputation within that subculture compared to OpenAI. Despite having as crazy internal politics as OpenAI, they have converged quite a bit from the original vision of safety first through Darwinistic pressures.
At least, it seems this way from the outside. I'm curious if the view from the inside is that different.
edit: to be clear, my reading as an outsider is that Anthropic is seen as relatively better in the AI safety community, but has definitely dropped in absolute reputation too. This recent thread and the references show some of that: https://www.lesswrong.com/posts/6j3kBHdowGLCeqobg/dear-god-p...
Then it’s not a benchmark. I think his thesis is solid: if neither pelicans nor bicycles stick out, it follows that there isn’t special attention being given to them by the labs
That huge impromptu 2023 party at the Exploratorium Hugging Face did had such amazing covid-unthawing AI spring energy, and I keep thinking about it. It was clear then they had the momentum to do a lot of things, and more so now. So I was surprised by this news.
The party is a is a relatively small concern compared to the openness and github comparisons, but since no one else mentioned it, I hope something like it happens again. Or maybe that moment has passed.
The protein folding solutions like ESM/Alphafold were not due to this LLM/agentic coding or autoresearch type approaches though. They were designed by bio ML researchers.
It's hard to keep track of the frontier on bio ML, but it seems that we're going slower than what Demis Hassabis said in 2024 with 5 years to full cell molecular simulation. We still aren't able to reliably model a tiny surface of the cell membrane.
And of course there's Derek Lowe's takes on the drug discovery pipeline waiting for the proof in the pudding.
To me the only reasonable bullish position is that there is a very non-linear AGI threshold for accelerating progress that we haven't hit yet.
For me personally, I'm looking at other more tractable fields as a proxy to measure this kind of progress. The best modest evidence is from the agentic coding area, (modest because these kinds of gains may not translate to bio progress). Other soft-ish fields to like legal/law/tax are also interesting to watch, as a small amount of people are now trusting AI for these areas that were considered totally unusable a year ago. Another proxy is being able to generate generally entertaining media.
SRE is a 'say no to powerful people' job as well. I think for this to work the leadership needs to show support for it. The friction is still there, but in more tolerable areas. I bet for ethics this isn't the case at OAI, but everyone values security and stability.
For an SRE there can be more directed hate received from the junior employees, that want to release new features they developed. Especially because there is less accountability across orgs. Security is an interesting one because it seems to have less of this friction, maybe because it's more clear cut what is an issue.
The early batch of Anthropic employees were mostly rationalist-adjacent AI safety folk that were almost uniformly claiming P_DOOM > .10 three years ago, so I believe them to be earnest.
It's very interesting to me that besides the other small safety labs that don't actually produce frontier models, Anthropic manages to keep such a good reputation within that subculture compared to OpenAI. Despite having as crazy internal politics as OpenAI, they have converged quite a bit from the original vision of safety first through Darwinistic pressures.
At least, it seems this way from the outside. I'm curious if the view from the inside is that different.
edit: to be clear, my reading as an outsider is that Anthropic is seen as relatively better in the AI safety community, but has definitely dropped in absolute reputation too. This recent thread and the references show some of that: https://www.lesswrong.com/posts/6j3kBHdowGLCeqobg/dear-god-p...
reply