the gpu memory usage of finetuning dualstylegan on 8 gpus

[1787648106]

Hi, thx for your excellent work! When I finetune the model on a 8 gpus per node device , I found that the gpu memory usage of the first card is about 2 times higher than that of the other seven cards. Is this normal?

[williamyang1991]

This is because some of the pre-trained model is saved on GPU rather than CPU. So when you load these models before running the multi-GPU process, all models are loaded into GPU0, making GPU0 has high memory usage.
You can manually check each of your loaded pre-trained models, and make sure they are saved and loaded into CPU before running the multi-GPU process.

[1787648106]

Thanks for your reply, I'll try it later.