https://arxiv.org/abs/2104.04946
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost (Zhen Wu, Lijun Wu, Qi Meng, Yingce Xia, Shufang Xie, Tao Qin, Xinyu Dai, Tie-Yan Liu)
여러 형태의 dropout을 조합해서 트랜스포머를 regularize하기. 흥미로운 레시피네요.
#transformer #regularization