Hi mark, Wanted to know if the float4 training is possible with torchao as we trying to fit a large model on a single GPU for training.

we have experimental support for float4 training with the mx formats https://github.com/pytorch/ao/tree/main/torchao/prototype/mx...

But that's waiting for Blackwell to be released so we get the hardware support. SO recommendation for now would be to use either fp8 training or int8 training

Hi mark, Wanted to know if the float4 training is possible with torchao as we trying to fit a large model on a single GPU for training