8-bit precision training

[rasbt]

Might be worthile to look into

from torchao.float8 import convert_to_float8_training
convert_to_float8_training(model)

some time

[mtasic85]

I watched this week this video: https://www.youtube.com/watch?v=Br07GsnnvWc

This performance is expected:

About Embedding layer, it should be investigated if they use appropriate class like this one, or we need to take care of it: https://github.com/pytorch/ao/blob/e7b33bc91c831d10249c1222c8b4b667f18f28b7/torchao/quantization/prototype/qat/embedding.py#L27

[rasbt]

Thanks for sharing this. Need to check out the video some time. It's long (but good explanations of tricky things are naturally long) but looks really informative.