-
|
After watching recent video, I want to try LR-Free branch. How to install and run this branch? |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 7 replies
-
|
I just published a new version of the GUI that has the dadaptation optimizer built in. You can use it by selecting dadaptation as the optimizer and making sure to set the Learning, Text Encoder Learning rate and Unet learning rate as shown in the image below:
But the best I have found so far is Adafactor... produced the best results for me. Just set the Learning Rate value to 1 and give it a try. |
Beta Was this translation helpful? Give feedback.
-
|
I will copy those values, Default values in the GUI were these... Learning rate: 0.0001, Text Encoder learning rate: 5e-5, Unet learning rate: 0.0001. But We should use Learning rate: 1, Text Encoder learning rate: 0.5 and Unet learning rate: 1 But Just a request... (I am sorry if I shouldn't be asking this question.) |
Beta Was this translation helpful? Give feedback.
-
|
Hummmm... Technically I could... But some users will complain that switching optimizer mangles the LR values they had manually set... So there is not one solution please all for this... I will see if I can think of something... Like maybe a reset default values button? Those apparently small thing can be daunting to implement because different options will require different defaults... And maintaining this will be a lot of work. Maybe I will think of something... |
Beta Was this translation helpful? Give feedback.
-
|
:) I agree. default button is also great feature. But you should keep track of all best/workable/accepted values for that. or just add these values in your Help section and add a message/tool tip in the UI. Where can we find documentation for all the optimizers available in kohya? |
Beta Was this translation helpful? Give feedback.
-
|
Do we still have to set these default values if Dadaptation is selected? |
Beta Was this translation helpful? Give feedback.
-
|
@bmaltais could you screen shot the settings to use with adafactor please? |
Beta Was this translation helpful? Give feedback.
-
|
With adafactor you put 1 for the LR. You can put 0.5 in the Text Encoder to
reduce the learning for it in half.
…On Thu, Mar 16, 2023, 7:57 a.m. Jon Wong ***@***.***> wrote:
@bmaltais <https://github.com/bmaltais> could you screen shot the
settings to use with adafactor please?
—
Reply to this email directly, view it on GitHub
<#219 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABZA34UFMX5PKIZI4VSF4UDW4L52DANCNFSM6AAAAAAVDQBNS4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
so the same as shown above just with adafactor changed in the optimizer? |
Beta Was this translation helpful? Give feedback.
-
|
Correct
…On Thu, Mar 16, 2023, 12:43 p.m. Jon Wong ***@***.***> wrote:
so the same as shown above just with adafactor changed in the optimizer?
[image: image]
<https://user-images.githubusercontent.com/41051927/225691221-db036cd8-b58d-4cde-af1f-34fe2efb159e.png>
—
Reply to this email directly, view it on GitHub
<#219 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABZA34XXNPPXXXKUHR7QTJDW4M7KFANCNFSM6AAAAAAVDQBNS4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
I just published a new version of the GUI that has the dadaptation optimizer built in. You can use it by selecting dadaptation as the optimizer and making sure to set the Learning, Text Encoder Learning rate and Unet learning rate as shown in the image below:
But the best I have found so far is Adafactor... produced the best results for me. Just set the Learning Rate value to 1 and give it a try.