The model is known to be MXFP4 according to Kimi's release blog post, so the model weights will be less than 1536GB: https://www.kimi.com/blog/kimi-k3
Also, their previous models were native INT4, so it would be weird if they went larger now.
* Sparse Experts: 1481.4 GB
* Dense Experts: 1.9 GB
* Self-Attention: 72.4 GB
* LLM Head: 2.4 GB
* Embeddings: 2.4 GB
* Vision Encoder: 0.35 GB (surprisingly small)
plus some miscellaneous parameters.
Most importantly, we now know that the model has 104B active parameters, which is quite a lot and will make it difficult to self-host efficiently.
reply
The model is known to be MXFP4 according to Kimi's release blog post, so the model weights will be less than 1536GB: https://www.kimi.com/blog/kimi-k3
Also, their previous models were native INT4, so it would be weird if they went larger now.