How to assign GPU for training

[okingjo]

Hello there, just curious where can I manually set the GPU to be used for training.
I got two graphic cards on my rig. At the beginning I answered "yes" to DistributedDataParallel during the installation process, and it fails to start training every single time. At first it was the NCCL issue, and I have manually replace with GLOO to solve it. Then it was the attribute error of DistributedDataParallel, and I could not kill it. So I decide to reinstall and answered No this time and use 1 GPU only. Then the problem is, where can I assign the GPU?

[okingjo]

Solved by myself. just add:
set CUDA_VISIBLE_DEVICES=1
to the gui.bat file and all done.
I was expecting something much more complicated, you know, since multi-GPU support on the SD webUI was pretty nasty

[okingjo]

I guess you`d have to go to the nvidia control panel to check the GPU ID first, or the CUDA_VISIBLE_DEVICES command
My GPUs are all inside so they are labeled 0,1 etc. pretty ordinary
for a EGPU, I don't know how system would assign that number

[Deph1]

by GPU ID are we talking the number or a larger device ID I can see. Sorry to pester you with this nonsense, I wish it was clearer on the guide....

[Deph1]

Only work around I've managed is disabling the Laptop GPU in device manager LOL

[okingjo]

not the official “device ID” in the system info, I mean the number assigned by system as recognized GPU, e.g. 0, 1, 2 etc.
to check it in nvidia control panel - 3d settings - manage 3d settings - CUDA-GPUs, you should be able to see all GPUs there. In your case, it should be something like 3060 1of2, 3090 2of2, if that is the case, 3060 is 0 and 3090 is 1, CUDA_VISIBLE_DEVICES=0 foo 3060 and CUDA_VISIBLE_DEVICES=1 for 3090

[LilaPferd01]

Didn´t work for me.

C:\Users\user1\stable-diffusion\Kohya\kohya_ss>set CUDA_VISIBLE_DEVICES=1
System Information:
System: Windows, Release: 10, Version: 10.0.19045, Machine: AMD64, Processor: Intel64 Family 6 Model 63 Stepping 2, GenuineIntel

Python Information:
Version: 3.10.6, Implementation: CPython, Compiler: MSC v.1932 64 bit (AMD64)

Virtual Environment Information:
Path: C:\Users\user1\stable-diffusion\Kohya\kohya_ss\venv

GPU Information:
 MiB: NVIDIA GeForce GTX 980, VRAM: 4096
←[33mWarning: GPU VRAM is less than 8GB and will likelly result in proper operations.←[0m

Validating that requirements are satisfied.
All requirements satisfied.
Load CSS...
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

My gui.bat looks like this. I am also on the newest commit.

@echo off

:: Activate the virtual environment
call .\venv\Scripts\activate.bat
set PATH=%PATH%;%~dp0venv\Lib\site-packages\torch\lib
set CUDA_VISIBLE_DEVICES=1
:: Debug info about system
python.exe .\tools\debug_info.py

:: Validate the requirements and store the exit code
python.exe .\tools\validate_requirements.py

:: If the exit code is 0, run the kohya_gui.py script with the command-line arguments
if %errorlevel% equ 0 (
    python.exe kohya_gui.py %*
)