Loss NaN when training on NYUv2

[hit421]

I have reproduced the model trained on NYUv2 but met with problems. The loss and all evaluation metrics came to NAN after training with 26 epochs.
My setting is one 3090 GPU, pytorch == 1.9.1 and use default settings for all parameters . Do you have any idea to solve this problem?
Thanks for your help.