You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thank your for your paper and work, but I meet some wired problem in training. Can you help me?
It's my training process curve. Notice Lion will increase loss sometimes but decease later. It seems not stability? Can you give me some advises how to tune Lion's parameters?
Below is my setting:
Batchsize = 8, GPU = 64 A100, with syncBN
Network: PointPillars
Optimizer setting:
The text was updated successfully, but these errors were encountered:
Thanks for the data point. Can you please tell us your baseline settings? Like which optimizer, which learning rate and weight decay?
I think for instability problem, you can either reduce the learning rate a little bit or try to lower the beta2, for example, we used (0.95, 0.98) in our llm training.
Thank your for your paper and work, but I meet some wired problem in training. Can you help me?
It's my training process curve. Notice Lion will increase loss sometimes but decease later. It seems not stability? Can you give me some advises how to tune Lion's parameters?
Below is my setting:
Batchsize = 8, GPU = 64 A100, with syncBN
Network: PointPillars
Optimizer setting:
The text was updated successfully, but these errors were encountered: