How do I not cache text embeddings for Sana?

[OhioT]

The Sana training is working, but there is a >200GB of disk space used for the text cache. How can I keep Gemma in memory and have it make embeddings during training instead of pre-caching?

[bghira]

this isn't supported, and that would add more than 9GB of VRAM to the consumption

[OhioT]

GH200s are $1.49 now so I have VRAM but disk space isn't cheap.

Fine, time to hack it in by messing with compute_embeddings_for_sana_prompts()

[bghira]

if you're using GH200s you're already wasting money like an idiot