Skip to content
/ VOCAL-TRACK-EXTRACTION-USING-NEURAL-NETWORKS Public

There are four models in this project: Deep Clustering Model, Hybrid Deep Clustering Model, U-net Model and UH-net Model. Models are trained on DSD100 dataset. The project is based on PyTorch.

3 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

MortadhaMannai/VOCAL-TRACK-EXTRACTION-USING-NEURAL-NETWORKS

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
1_jhYv-BI-dEQe85I7B4qjcQ.png
1_jhYv-BI-dEQe85I7B4qjcQ.png
 
 
3-Figure1-1.png
3-Figure1-1.png
 
 
41467_2021_22008_Fig1_HTML.png
41467_2021_22008_Fig1_HTML.png
 
 
41598_2021_2790_Fig1_HTML.png
41598_2021_2790_Fig1_HTML.png
 
 
42256_2021_297_Fig1_HTML.png
42256_2021_297_Fig1_HTML.png
 
 
459898_1_En_4_Fig1_HTML.png
459898_1_En_4_Fig1_HTML.png
 
 
A Survey of Audio-Based Music Classification and Annotation.pdf
A Survey of Audio-Based Music Classification and Annotation.pdf
 
 
Architecture-of-the-shallow-U-Net-model-for-feature-extraction.png
Architecture-of-the-shallow-U-Net-model-for-feature-extraction.png
 
 
Build_Dataset.ipynb
Build_Dataset.ipynb
 
 
DEEP CLUSTERING AND CONVENTIONAL NETWORKS FOR MUSIC SEPARATION STRONGER TOGETHER.pdf
DEEP CLUSTERING AND CONVENTIONAL NETWORKS FOR MUSIC SEPARATION STRONGER TOGETHER.pdf
 
 
DEEP CLUSTERING FOR SINGING VOICE SEPARATION.pdf
DEEP CLUSTERING FOR SINGING VOICE SEPARATION.pdf
 
 
Deep clustering Discriminative embeddings for segmentation and separation.pdf
Deep clustering Discriminative embeddings for segmentation and separation.pdf
 
 
John R. Hershey, Zhuo Chen, Jonathan Le Roux, and Shinji Watanabe, Deep clustering- Discriminative embeddings for segmentation and separation.pdf
John R. Hershey, Zhuo Chen, Jonathan Le Roux, and Shinji Watanabe, Deep clustering- Discriminative embeddings for segmentation and separation.pdf
 
 
Neural Network-based Approaches for Vocal Track Extraction.pdf
Neural Network-based Approaches for Vocal Track Extraction.pdf
 
 
Objective Function.pdf
Objective Function.pdf
 
 
README.md
README.md
 
 
Report.pdf
Report.pdf
 
 
Revolutionizing Vocal Track Extraction Innovative Hybrid Neural Network Approaches with Deep Clustering, Unet and UHnet Models.pdf
Revolutionizing Vocal Track Extraction Innovative Hybrid Neural Network Approaches with Deep Clustering, Unet and UHnet Models.pdf
 
 
SINGING VOICE SEPARATION WITH DEEP U-NET CONVOLUTIONAL NETWORKS.pdf
SINGING VOICE SEPARATION WITH DEEP U-NET CONVOLUTIONAL NETWORKS.pdf
 
 
The-illustration-of-our-UH-BDNN-D-4-L-2-In-our-proposed-network-design-the.png
The-illustration-of-our-UH-BDNN-D-4-L-2-In-our-proposed-network-design-the.png
 
 
The-structure-of-deep-convolutional-embedded-clustering-DCEC-It-is-composed-of-a.png
The-structure-of-deep-convolutional-embedded-clustering-DCEC-It-is-composed-of-a.png
 
 
U-netc.pdf
U-netc.pdf
 
 
applsci-10-01727-g005.png
applsci-10-01727-g005.png
 
 
architecture_unetV2.png
architecture_unetV2.png
 
 
bascial_dsp_items.pdf
bascial_dsp_items.pdf
 
 
binary_mask.png
binary_mask.png
 
 
cluster_model.py
cluster_model.py
 
 
config.py
config.py
 
 
data_loader.py
data_loader.py
 
 
dc_loss.png
dc_loss.png
 
 
dc_mask.png
dc_mask.png
 
 
dc_model.json
dc_model.json
 
 
dc_model.png
dc_model.png
 
 
dc_train.py
dc_train.py
 
 
decode.ipynb
decode.ipynb
 
 
deep-clustering-banner.jpeg
deep-clustering-banner.jpeg
 
 
evaluation.py
evaluation.py
 
 
final_empty_mask.png
final_empty_mask.png
 
 
final_mask.png
final_mask.png
 
 
front-end.png
front-end.png
 
 
hc_mask.png
hc_mask.png
 
 
hd_model.json
hd_model.json
 
 
hd_train.py
hd_train.py
 
 
hdcluster.wav
hdcluster.wav
 
 
hybrid_model.py
hybrid_model.py
 
 
hybridmodel.png
hybridmodel.png
 
 
main.py
main.py
 
 
mask_rcnn.pdf
mask_rcnn.pdf
 
 
mel_dealer.py
mel_dealer.py
 
 
mix.wav
mix.wav
 
 
music_decoder.py
music_decoder.py
 
 
newimage12.png
newimage12.png
 
 
result analysis.ipynb
result analysis.ipynb
 
 
scale_mask.png
scale_mask.png
 
 
system_matching.png
system_matching.png
 
 
train_dc.ipynb
train_dc.ipynb
 
 
train_hybrid.ipynb
train_hybrid.ipynb
 
 
train_unet.ipynb
train_unet.ipynb
 
 
u-net.png
u-net.png
 
 
u_net_loss.png
u_net_loss.png
 
 
uhnet.wav
uhnet.wav
 
 
uhnet_loss.png
uhnet_loss.png
 
 
uhnet_mask.png
uhnet_mask.png
 
 
uhnet_model.json
uhnet_model.json
 
 
unet.wav
unet.wav
 
 
unet_mask.png
unet_mask.png
 
 
unet_model.json
unet_model.json
 
 
unet_model.py
unet_model.py
 
 
unet_train.py
unet_train.py
 
 
utils.py
utils.py
 
 
vocal.wav
vocal.wav
 
 

Repository files navigation

Vocal Track Extraction

Author: Mortadha Manai

Report Link :https://github.com/MortadhaMannai/VOCAL-TRACK-EXTRACTION-USING-NEURAL-NETWORKS/blob/main/Report.pdf

Paper links :

1- zendo.org : https://zenodo.org/record/8274725

2- OpenAir.com : https://explore.openaire.eu/search/publication?pid=10.5281%2Fzenodo.8267702&fbclid=IwAR13OfUARkpyVk1jzk2fFoqaxVeNz2xbDwNySsu8vCV0FxwslG0eI8hqx90

Introduction

There are four models in this project: Deep Clustering Model, Hybrid Deep Clustering Model, U-net Model and UH-net Model. Models are trained on DSD100 dataset. The project is based on PyTorch.

Scripts

  • Data preprocess:

    • Build_Dataset.ipynb: generate dataset from DSD100
    • config.py: define project-level parameters
    • data_loader.py: define torch loader
    • mel_dealer.py: convert music file to melspectrogram and convert spectrogram back

  • Model defination:

    • unet_model.py: define U-net Model and UH-net Model
    • cluster_model.py: define Deep Clustering Model
    • hybrid_model.py: define Hybrid Deep Clustering Model

  • Model training:

    • utils.py: define loss functions
    • unet_train.py: train functions for u-net / uh-net model
    • hd_train.py: train functions for hybrid deep clustering model
    • dc_train.py: train functions for deep clustering model
    • train_dc.ipynb, train_hybrid.ipynb and train_unet.ipynb: train models

  • Model evaluation:

    • evaluation.py: define evaluation functions
    • music_decoder.py: retrieve audio file from model outputs

Current Sample Outputs

Audios

Original Music ( Vocal Track)
==> Hybrid Deep Clustering Model
==> U-net Model
==> UH-net Model

Masks

  • Masked Power Spectrograms:

  • Generated Masks:

About

There are four models in this project: Deep Clustering Model, Hybrid Deep Clustering Model, U-net Model and UH-net Model. Models are trained on DSD100 dataset. The project is based on PyTorch.

Resources

Readme
Activity

Stars

3 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages