I actually built this analysis while I worked at Microsoft so I 100% agree. Doing the work at the platform level is the way to go and you can actually make a significant impact with this kind of approach.
The other value of this that's not obvious is that doing it client side ends up touching all the grids/generators in the world outside of the market based accounting that tends to drive the datacenter carbon impact analysis.
There have been a few questions about the state of Show HN lately. Was actually interested in this post but I see all the OPs responses to questions are Dead? I do see its a new account but I don't really see anything egregious or against policy for these.
That is a pretty good article although the one factor not mentioned that we see that has a huge impact on energy is batch size but that would be hard to estimate with the data he has.
We've only launched to friends and family but I'll share this here since its relevant: we have a service which actually optimizes and measures the energy of your AI use: https://portal.neuralwatt.com if you want to check it out. We also have a tools repo we put together that shows some demonstrations of surfacing energy metadata in to your tools: https://github.com/neuralwatt/neuralwatt-tools/
Our underlying technology is really about OS level energy optimization and datacenter grid flexibility so if you are on the pay by KWHr plan you get additional value as we continue to roll new optimizations out.
DM me with your email and I'd be happy to add some additional credits to you.
To add a bit more to what @scottcha is saying: overall GPU load has a fairly significant impact on the energy per result. Energy per result is inversely related, since the idle TDP of these servers is significant the more the energy gets spread the more efficient the system becomes. I imagine Anthropic is able to harness that efficiency since I imagine their servers are far from idle :)
You can infer the discount from the pricing of the batch API, which is presumably arranged for minimum inference costs. Anthropic offers a 50% discount there, which is consistent with other model providers.
Neuralwatt | https://neuralwatt.com | REMOTE (US – Seattle/Denver/Boulder metros only) | Full-time | $180k–$220k DOE
Energy is the #1 constraint in new datacenter buildouts. Neuralwatt is reshaping AI compute around energy efficiency to maximize revenue per kilowatt. We’re a VC-backed, early-stage startup building optimization tools for AI, HPC, and datacenter workloads.
We're hiring 2 experienced founding engineers to help architect our core systems and work directly with customers.
What you'll do:
Technically: Architect critical datacenter infrastructure - Write Rust and Python - Measure real-world energy impact - Design state-of-the-art AI-led optimizations
Non-technically: Help build the business and win customers - Present at conferences - Develop marketing and company materials
Requirements:
- 5–10+ years of software development experience
- Thrive in ambiguous, outcome-driven environments
- Experience working closely with customers
- Clear communication and strong leadership
- Familiarity with LLM/AI infrastructure
Location: Remote-first, but we meet regularly in Seattle/Denver metro areas.
To apply: Email: scott@neuralwatt.com Subject: HN Hiring Include: - Resume - GitHub profile - A short note on why you're interested.
Please note: At this time, we are unable to offer visa sponsorship.
Neuralwatt | https://neuralwatt.com | REMOTE (US – Seattle/Denver/Boulder metros only) | Full-time | $180k–$220k DOE
Energy is the #1 constraint in new datacenter buildouts. Neuralwatt is reshaping AI compute around energy efficiency to maximize revenue per kilowatt. We’re a VC-backed, early-stage startup building optimization tools for AI, HPC, and datacenter workloads.
We're hiring 2 founding engineers to help architect our core systems and work directly with customers.
What you'll do:
Technically:
Architect critical datacenter infrastructure
- Write Rust and Python
- Measure real-world energy impact
- Design state-of-the-art AI-led optimizations
Non-technically:
Help build the business and win customers
- Present at conferences
- Develop marketing and company materials
Requirements:
- 5–10+ years of software development experience
- Thrive in ambiguous, outcome-driven environments
- Experience working closely with customers
- Clear communication and strong leadership
- Familiarity with LLM/AI infrastructure
Location:
Remote-first, but we meet regularly in Seattle/Denver metro areas.
To apply:
Email: scott@neuralwatt.com
Subject: HN Hiring
Include:
- Resume
- GitHub profile
- A short note on why you're interested
I’ve asked that question on linked in to the Cerebras team a couple times and haven’t ever received a response. There is system max tdp values posted online but I’m not sure you can assume the system is running in max tdp for these queries. If it is the numbers are quite high (I just tried to find the number but couldn’t find it but I had it in my notes as 23kw).
If someone from Cerebras is reading this feel free to dm me as optimizing this power is what we do.
Maybe I’m a statistical anomaly or maybe I just don’t know the baseline occurrence rate for this stuff but I have 3 close acquaintances two of which are this persons age or younger with similar symptoms (tachycardia, though to a lesser degree) going on. Is there data on the incidence rates for this stuff and has it been increasing since 2021?
Not tachycardia, but I had frequent heart palpitations for a few months after contracting Covid the first time in late ‘21. Did not notice anything in subsequent times. Maybe it was a placebo effect, as I had knowledge of Covid’s effects on the heart, maybe not.
The are many great things about Aurora, here are a few as I've been using it since it came out.
1. Its open source & open weights and free to use non-commercially.
2. Its configurable to easily fit on my local gpu for development purposes.
3. I've also gotten great engagement from the repo owners.
Shameless plug . . . I run a startup who is working to help this https://neuralwatt.com We are starting with an os level (as in no model changes/no developer changes required) component which uses RL to run AI with a ~25% energy efficiency improvement w/out sacrificing UX. Feel free to dm me if you are interested in chatting either about problems you face with energy and ai or if you'd like to learn more.
reply