-
Notifications
You must be signed in to change notification settings - Fork 4
Dare merge
Won-Kyu Park edited this page Jun 30, 2024
·
1 revision
In the Model-Mixer, Dare merge is not exactly the same implementation of the original code.
Dare merge in the model-mixer is based on the following: https://github.com/martyn/safetensors-merge-supermario/blob/main/merge.py#L9-L18 https://github.com/yule-BUAA/MergeLM/blob/main/model_merging_methods/mask_weights_utils.py#L9C5-L35 https://github.com/yule-BUAA/MergeLM/blob/main/model_merging_methods/merging_methods.py#L418 (See also https://github.com/wkpark/sd-webui-model-mixer/discussions/120)
but simplified.
- no sign consideration. etc.
(fixed droprate=0.5) density = 1 - droprate
in this case, lambda is the variable alpha
(fixed lambda=1.0) density = 1 - droprate
in this case, density (or droprate) is the variable alpha