the models are huge, so not a single (latest gen) one can fit on a single GPU. I... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		make3 3 months ago \| parent \| context \| favorite \| on: Alibaba Cloud says it cut Nvidia AI GPU use by 82%... the models are huge, so not a single (latest gen) one can fit on a single GPU. It's likely that these are small unpopular (non flagship) models, or that they only pack eg one layer of each model.

svachalek 3 months ago [–]

Per the very short article, the solution was to pack multiple models per GPU.

make3 3 months ago | [–]

yes but that could mean a layer per model

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact