ATOM: American Truly Open Models

Author here! Just wanted to say that this is indeed in a good place to share, some very useful stuff, but is also very work in progress. I'm may 60% or so to my first draft. Said progress is coming every day and I happily welcome fixes or suggestions on GitHub.

Thanks. Is there a PDF version? I kind of feel difficulty switching links.

Yes

https://rlhfbook.com/book.pdf

As the other commenter said, R1 required very standard RLHF techniques too. But a fun way to think about it is that reasoning models are going to be bigger and uplift the RLHF boat.

But we need a few years to establish basics before I can write a cumulative RL for LLMs book ;)

This is a GREAT book, if you decide to write it in a rolling fashion you'd have at least one reader from the start :)

thx, you're right. I've fixed it!

Author here, you can find more on my website: https://natolambert.com/cv Have been building RLHF systems at HuggingFace since ChatGPT, with some other experience before.

Why stay with HF instead of going OpenAI?