Which data uses for training? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		sbassi 63 days ago \| parent \| context \| favorite \| on: NanoChat – The best ChatGPT that $100 can buy Which data uses for training?

simonw 63 days ago | [–]

karpathy/fineweb-edu-100b-shuffle: https://huggingface.co/datasets/karpathy/fineweb-edu-100b-sh...

Which is derived from HuggingFaceFW/fineweb-edu: https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu

HuggingFaceTB/smol-smoltalk: https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk

And extra fine-tuning on portions of:

cais/mmlu: https://huggingface.co/datasets/cais/mmlu

openai/gsm8k: https://huggingface.co/datasets/openai/gsm8k

allenai/ai2_arc: https://huggingface.co/datasets/allenai/ai2_arc

eranation 63 days ago | [–]

I think he mentioned somewhere he used fineweb (I assume this one https://huggingface.co/datasets/HuggingFaceFW/fineweb)

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact