Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

重建效果是否符合预期 #61

Open
zzchust opened this issue Dec 11, 2024 · 3 comments
Open

重建效果是否符合预期 #61

zzchust opened this issue Dec 11, 2024 · 3 comments

Comments

@zzchust
Copy link

zzchust commented Dec 11, 2024

测试了一条ucg的音频, 使用75token-large模型重建音频,效果很差,可以帮忙看看不?
测试音频-原始音频+重建音频.zip

@zzchust
Copy link
Author

zzchust commented Dec 11, 2024

`from encoder.utils import convert_audio
import torchaudio
import torch
from decoder.pretrained import WavTokenizer

device=torch.device('cuda')

config_path = "configs/wavtokenizer_smalldata_frame75_3s_nq1_code4096_dim512_kmeans200_attn.yaml"
model_path = "WavTokenizer-large-speech-75token/wavtokenizer_large_speech_320_24k.ckpt"
audio_path = "souce_audio.mp3"

audio_outpath = "reconstruct_audio.wav"

wavtokenizer = WavTokenizer.from_pretrained0802(config_path, model_path)
wavtokenizer = wavtokenizer.to(device)

torchaudio.set_audio_backend('soundfile')

wav, sr = torchaudio.load(audio_path)
wav = convert_audio(wav, sr, 24000, 1)
bandwidth_id = torch.tensor([0])
wav=wav.to(device)
features,discrete_code= wavtokenizer.encode_infer(wav, bandwidth_id=bandwidth_id)
audio_out = wavtokenizer.decode(features.cuda(), bandwidth_id=bandwidth_id.cuda())
torchaudio.save(audio_outpath, audio_out.cpu(), sample_rate=24000, encoding='PCM_S', bits_per_sample=16)
`

@jishengpeng
Copy link
Owner

jishengpeng commented Dec 11, 2024

测试了一条ucg的音频, 使用75token-large模型重建音频,效果很差,可以帮忙看看不? 测试音频-原始音频+重建音频.zip

large版本中的训练中文数据也很少,得用干净中文重新训练,issue区里有很多人已经试过了。

@zzchust
Copy link
Author

zzchust commented Dec 16, 2024

好的,感谢回复!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants