Quantized GGUF Model Not Recognized as Language Input in Futo Keyboard

[goranskular]

I fine-tuned the base model for the Croatian language using the finetune.ipynb notebook and then quantized with (from candle):

$ tensor-tools quantize --quantization q8_0 base-1m.hr-futo/model.safetensors --out-file base-1m.hr-futo-q80.bin

When importing the quantized .bin file into the Futo keyboard, the process of loading appears as it's working, but it is not set as the language under "Voice input." Instead, it shows (None) for the language.

Steps Taken:

• Fine-tuned the base model for Croatian: GoranS/whisper-base-1m.hr
• Fine-tuned using finetune.ipynb: GoranS/whisper-base-1m.hr-futo
• Quantized the fine-tuned model with the tensor-tools command shown above
• Imported the resulting .bin file into the Futo keyboard

Issue:

• The Futo keyboard does not recognize the imported quantized model as a valid language input for voice recognition. The language remains (None) in the settings.

Questions:

• Is there a specific requirement for producing the quantized .bin file that I may have missed?
• Should the quantization process or any other pre-import step be handled differently to ensure compatibility with the Futo keyboard?

Any guidance or recommendations would be greatly appreciated. Thank you!

[abb128]

Please use whisper.cpp's convert-h5-to-ggml.py to convert the model to work with the app, which uses whisper.cpp. Candle is not the right tool

[goranskular]

Thank you. It is working. It is in GoranS/whisper-base-1m.hr-futo repo on hf. File base-1m.hr-futo-q80.bin

Solved: whisper.cpp's convert-h5-to-ggml.py and whisper.cpp's quantize:
$ ./whisper.cpp/build/bin/quantize ggml-model.bin ggml-model-q80.bin q8_0