Can I serve speechbrain trained model whisper with faster whisper?

[cod3r0k]

Can I serve speechbrain trained model whisper with faster whisper?

[MahmoudAshraf97]

You have to convert it to CT2 first, there are several converters available, you can check CT2 documentation for more information

[cod3r0k]

Great, can you help me more? What is CT2? @MahmoudAshraf97

[MahmoudAshraf97]

The backend of Faster Whisper
https://github.com/OpenNMT/CTranslate2/

[cod3r0k]

Great. You mean that I do as below:

from transformers import WhisperProcessor, WhisperForConditionalGeneration

# Load the Whisper model from Hugging Face
processor = WhisperProcessor.from_pretrained("openai/whisper-large")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-large")

# Save the model in Hugging Face format
model.save_pretrained("whisper_huggingface")
processor.save_pretrained("whisper_huggingface")

Then

ctranslate2-converter --model whisper_huggingface --output_dir whisper_ctranslate2

Do I do it correctly?

[MahmoudAshraf97]

Exactly, if the model you have in not in huggingface format, you need to convert it first to that format then to CT2